Go 语言开发 API 实践

garguntua 的个人博客 / 6 / 8 / 创建于 7年前

一点感想

现阶段,我个人特别倾向于,API使用Go语言开发,后台管理系统使用Laravel开发.兼顾并发性能和开发效率.

CPU密集型的任务,可以通过队列交给静态语言(Go)来处理.这样可以结合各种语言的优势来解决问题.

业务实测

我最近开发了一套超高并发场景下的任务调度系统,管理后台使用Laravel开发(感谢laravel-admin),管理数据存入mysql,每5分钟系统自动读取mysql数据生成任务数据存入redis,并同步汇总数据到mysql.

业务接口始终只跟Redis通信.等于所有操作全部在内存中完成.

整套代码综合起来,只花了20多个小时,就完成了后台管理+高性能api,开发效率非常棒,感觉Go开发api,开发速度确实比php要慢,大概慢一倍的样子,但所有可能会出错误的地方都给处理了错误,运行起来更可靠,性能也更好,还是很值得的.

Go这边采用gin框架

Redis包采用(https://github.com/garyburd/redigo)

Mysql包采用(https://github.com/jinzhu/gorm)

阿里云2核4G的机器,ab测试:

直接跑gin框架,输出字符串,5000并发qps在1.3W

ab -c 100 -n 1000 http://127.0.0.1:9000/ping

Concurrency Level:      100
Time taken for tests:   0.064 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      384000 bytes
HTML transferred:       4000 bytes
Requests per second:    15620.85 [#/sec] (mean)
Time per request:       6.402 [ms] (mean)
Time per request:       0.064 [ms] (mean, across all concurrent requests)
Transfer rate:          5857.82 [Kbytes/sec] received

ab -c 5000 -n 20000 http://127.0.0.1:9000/ping

Concurrency Level:      5000
Time taken for tests:   1.564 seconds
Complete requests:      20000
Failed requests:        0
Total transferred:      7680000 bytes
HTML transferred:       80000 bytes
Requests per second:    12790.56 [#/sec] (mean)
Time per request:       390.913 [ms] (mean)
Time per request:       0.078 [ms] (mean, across all concurrent requests)
Transfer rate:          4796.46 [Kbytes/sec] received

ab -c 10000 -n 20000 http://127.0.0.1:9000/ping

Concurrency Level:      10000
Time taken for tests:   2.173 seconds
Complete requests:      20000
Failed requests:        0
Total transferred:      7680000 bytes
HTML transferred:       80000 bytes
Requests per second:    9201.88 [#/sec] (mean)
Time per request:       1086.734 [ms] (mean)
Time per request:       0.109 [ms] (mean, across all concurrent requests)
Transfer rate:          3450.71 [Kbytes/sec] received

ab -c 20000 -n 20000 http://127.0.0.1:9000/ping

Concurrency Level:      20000
Time taken for tests:   2.121 seconds
Complete requests:      20000
Failed requests:        0
Total transferred:      7680000 bytes
HTML transferred:       80000 bytes
Requests per second:    9428.29 [#/sec] (mean)
Time per request:       2121.276 [ms] (mean)
Time per request:       0.106 [ms] (mean, across all concurrent requests)
Transfer rate:          3535.61 [Kbytes/sec] received

我们可以看到Go在1w,2w的并发下也能工作,qps并没有出现太大的下降,这种能力是PHP不具备的.

Go连接Redis,操作Redis数据,并输出响应,5000并发qps在4.5K

ab -c 100 -n 5000 http://127.0.0.1:9000/get_task

Concurrency Level:      100
Time taken for tests:   0.251 seconds
Complete requests:      5000
Failed requests:        0
Non-2xx responses:      5000
Total transferred:      1955000 bytes
HTML transferred:       90000 bytes
Requests per second:    19925.64 [#/sec] (mean)
Time per request:       5.019 [ms] (mean)
Time per request:       0.050 [ms] (mean, across all concurrent requests)
Transfer rate:          7608.32 [Kbytes/sec] received

ab -c 5000 -n 5000 http://127.0.0.1:9000/get_task

Concurrency Level:      5000
Time taken for tests:   1.119 seconds
Complete requests:      5000
Failed requests:        0
Total transferred:      2220000 bytes
HTML transferred:       285000 bytes
Requests per second:    4467.63 [#/sec] (mean)
Time per request:       1119.162 [ms] (mean)
Time per request:       0.224 [ms] (mean, across all concurrent requests)
Transfer rate:          1937.14 [Kbytes/sec] received

Go连接Redis,计算数据并存入Mysql,这个没什么参考性,主要是mysql的写锁问题

ab -c 100 -n 1000 http://127.0.0.1:9000/syn-task

Concurrency Level:      100
Time taken for tests:   0.380 seconds
Complete requests:      1000
Failed requests:        0
Total transferred:      393000 bytes
HTML transferred:       12000 bytes
Requests per second:    2630.01 [#/sec] (mean)
Time per request:       38.023 [ms] (mean)
Time per request:       0.380 [ms] (mean, across all concurrent requests)
Transfer rate:          1009.37 [Kbytes/sec] received

Go语言这种在高并发环境下的稳定性能是很有价值的,我们拿go操作redis的接口,5000并发,qps4.5K,为例,单个请求在1S以下,勉强能接受.

一个小时能处理的请求数达到了1千6百万.这个能力已经能应付一个颇具规模的业务了.

gin框架

我对比了几个主流的go语言框架,beego是国人开发的,自带了很多组件,文档丰富,不过性能相对差些.gin这个框架比较轻量,可以结合其他优秀的开源库做开发.

既然是做api,轻量高性能的gin自然成了最佳选择.

https://github.com/gin-gonic/gin

代码大概长这样

//初始化gin
r := gin.Default()
gin.SetMode(gin.ReleaseMode)

// 心跳检测
r.GET("/ping", func(c *gin.Context) {
    c.String(200, "pong")
})
//获取任务接口
r.GET("/get_task", getTask)

go语言操作Mysql

可以使用 https://github.com/jinzhu/gorm 也可以使用 https://github.com/go-sql-driver/mysql
反正我是用原生sql写语句,都一样

常用方法

//链接mysql
db, err := gorm.Open("mysql", DSN)
defer db.Close()
if err != nil {
    log.Panic("mysql db connect faild --- "+err.Error())
}

//执行语句
db.Exec("update tasks set v1=?,v2=?,v3=? where id ="+id,v1,v2,v3)

//查询语句
db.Raw("SELECT tasks.* from tasks where date='?'",today).Scan(&tasks)

//遍历查询结果
for _, task := range tasks {

}

go语言操作Redis

需要注意的是,redis当中的数据全部是以字符串类型存储的,从redis中取到的数据注意类型转换

//链接redis
c, err := redis.Dial("tcp", "127.0.0.1:6379")
if err != nil {
    log.Panic("get task api,connect redis server faild --- "+err.Error())
}

//获取key-value
value,err := redis.Int(c.Do("GET",key))

//获取key列表
keys,err := redis.Strings(c.Do("KEYS","*"))

//获取hash对象
object,_ := redis.StringMap(c.Do("HGETALL",key))

//写入一个hashmap的字段
c.Do("HSET",key,"filedName",filedValue)

//删除key-value
c.Do("DEL",key)

//写入key-value
c.Do("SET",key,value)

我挺喜欢这种redis封装方式的,基于原生命令的封装,不需要记忆库自身的方法,挺实用的