使用SQL参数绑定方式查询,内存oom排查
场景
查询阿里云多模数据库lindorm文档,使用database/sql 通过绑定参数的方式查询数据内存持续上升
使用pprof排查
Time: Apr 4, 2023 at 2:09pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 416.45MB, 98.74% of 421.77MB total
Dropped 42 nodes (cum <= 2.11MB)
Showing top 10 nodes out of 42
flat flat% sum% cum cum%
235.53MB 55.84% 55.84% 235.53MB 55.84% reflect.New
67.50MB 16.00% 71.85% 67.50MB 16.00% google.golang.org/protobuf/internal/impl.consumeStringValidateUTF8
36.91MB 8.75% 80.60% 36.91MB 8.75% database/sql.(*DB).addDepLocked (inline)
33.50MB 7.94% 88.54% 416.45MB 98.74% database/sql.(*DB).prepareDC
24MB 5.69% 94.23% 335.54MB 79.55% github.com/apache/calcite-avatica-go/v5.(*conn).prepare
10.50MB 2.49% 96.72% 346.04MB 82.04% database/sql.(*driverConn).prepareLocked
8.50MB 2.02% 98.74% 8.50MB 2.02% google.golang.org/protobuf/internal/impl.pointer.AppendPointerSlice (inline)
0 0% 98.74% 416.45MB 98.74% database/sql.(*DB).Prepare
0 0% 98.74% 416.45MB 98.74% database/sql.(*DB).PrepareContext
0 0% 98.74% 416.45MB 98.74% database/sql.(*DB).PrepareContext.func1
- prepare相关函数内存占用最高
解决方案
- “database/sql 包在准备 SQL 语句时会缓存一部分信息,已优化下次的查询。在某些情况下你不想要缓存的这部分数据,你可以直接销毁他。db.Stmt.Close 方法将销毁与此相关的缓存数据。”
- 直接查询使用 db.Query方法