Go 教程:使用 Go 来构建一个类 Google Analytics 应用

Go

当您考虑构建谷歌分析工具时,会想到什么?臆想巨大的平台将无济于事,您会失败的。 但是,坚持下去,走近一点,丢弃所有装饰品,看看核心,您看到了什么?

谷歌分析工具的核心是一个非常简单的应用程序,它从用户访问网页时生成的HTTP请求中提取有用的数据,以支持其附加功能,并进一步通过异步HTTP调用从JavaScript代码中发送更多数据。

在这篇博文中,我们将在Go中构建谷歌分析工具的基本版本,我将其称为go-gal-analytics(可根据需要发音)。

这篇博文的目的是展示在Go中构建酷炫的东西是多么简单。要遵循本教程,您必须至少了解Go和willpower的基本知识才能到达终点。

在我们开始之前,请查看 go-gal-analytics 面板

我们将在页面视图跟踪以下参数-

  • Country
  • City
  • Device
  • Platform
  • OS
  • Browser
  • Language
  • Referral

如何工作

网站包含许多资源,如HTML,CSS,JS,图像等,所有这些实质上都是存储在服务器上的文件。当用户键入网站地址并按Enter键时,浏览器将为每个资源创建一个HTTP请求并将其发送到服务器。然后,服务器评估这些请求中的每一个,并将所请求的资源发送回去,我们称之为HTTP响应。

HTTP请求和响应都包含称为HTTP标头的有用信息。

我们将使用请求的HTTP标头来跟踪以下参数-

  • 国家和城市-从X-Forwarded-ForX-Real-Ip标头中检索用户的IP地址,然后可用于从Maxmind Geo数据库中查找国家/城市
  • 设备、平台、操作系统、浏览器-解析User-AgentHTTP标头
  • 语言-解析Accept-LanguageHTTP标头
  • 引用-解析RefererHTTP标头

然后我们将一个虚拟的透明图像像素(尺寸为1x1)放置在我们要跟踪分析的网站上。这样用户看不到图像,但浏览器会将其解释为资源,并将带有所有令人惊叹的HTTP标头的HTTP请求发送到我们的服务器,我们提取标头,跟踪数据并将虚拟透明图像像素作为HTTP响应发送回去。

这是我们的图像像素 -

<img src="http://localhost/knock-knock" border="0" width="1" height="1" />

项目目录 -

$ go-gal-analytics
├── gogal
│   ├── server.go
│   ├── assets
│   │   └── GeoLite2-City.mmdb
│   ├── handler
│   │   └── event.go
│   ├── model
│   │   └── event.go
│   ├── repository
│   │   └── event.go
│   ├── route
│   │   └── route.go
│   ├── service
│   │   └── event.go
│   ├── utils
│   │   └── counter
│   │       └── counter.go
│   └── web
│       ├── css
│       │   └── style.css
│       ├── index.html
│       └── js
│           └── script.js
├── go.mod
├── go.sum
└── main.go 

注意:

在我之前的教程中,很少有人会在关注该博客文章时遇到困难,因为他们经常不得不查找每个.go文件要使用的软件包名称和导入。

为简单起见,我始终使用.go文件的立即父文件夹名称作为程序包名称。例-

-event.go文件位于服务文件夹内,因此使用包服务
-server.go文件位于gogal文件夹中,因此使用软件包gogal
-counter.go文件位于计数器文件夹内,因此使用包计数器
-main.go是根目录,是应用程序的入口点,因此使用软件包main

至于 import,使用 VSCode 或其他编辑器安装 Go 插件,您的导入将自动完成。

1: 初始化项目

新建 go.mod 文件:

$ go mod init

2: 计数器计数页面视图

counter 类型为 map[string]uint64, 因为我们特定参数的页面视图计数将保存为

Country:

{
    "USA": 83723,
    "UK": 2323
}

OS:

{
    "Linux": 4324,
    "Windows": 958
}

和其他参数相同。

接下来在 counter 目录下新建 counter.go

package counter

import (
    "sync"
)

// Counter is go routine safe counter used to count events.
// We add Read/Write Mutex to prevent race conditions.
type Counter struct {
    sync.RWMutex
    counter map[string]uint64
}

// NewCounter creates and returns a new Counter
func NewCounter() *Counter {
    return &Counter{
        counter: make(map[string]uint64),
    }
}

// Incr increments counter for specified key
func (c *Counter) Incr(k string) {
    c.Lock()
    c.counter[k]++
    c.Unlock()
}

// Val returns current value for specified key
func (c *Counter) Val(k string) uint64 {
    var v uint64
    c.RLock()
    v = c.counter[k]
    c.RUnlock()
    return v
}

// Items returns all the counter items
func (c *Counter) Items() map[string]uint64 {
    c.RLock()
    items := make(map[string]uint64, len(c.counter))
    for k, v := range c.counter {
        items[k] = v
    }
    c.RUnlock()
    return items
}

3: 事件 Model

一个Event是一个具有我们将跟踪的包含所有参数的单页视图。

Event

package model

type Event struct {
    Location EventLocation `json:"location"`
    Device   EventDevice   `json:"device"`
    Referral string        `json:"referral"`
}

type EventLocation struct {
    Country string `json:"country"`
    City    string `json:"city"`
}

type EventDevice struct {
    Type     string `json:"type"`
    Platform string `json:"platform"`
    OS       string `json:"os"`
    Browser  string `json:"browser"`
    Language string `json:"language"`
}

func (e *Event) Valid() bool {
    if e.Location.City != "" ||
        e.Location.Country != "" ||
        e.Device.Type != "" ||
        e.Device.Platform != "" ||
        e.Device.OS != "" ||
        e.Device.Browser != "" {
        return true
    }
    return false
}

4: 事件储存库 repository

EventRepository 是我们将使用的存储库,为了简单起见,我们不使用持久数据存储。

repository 目录中新建 event.go 文件:

package repository

import (
    "github.com/erybz/go-gal-analytics/gogal/model"
    "github.com/erybz/go-gal-analytics/gogal/utils/counter"
)

// Stats are constants for which event stats can be retrieved
type Stats string

const (
    // StatsLocationCountry is stats for Country
    StatsLocationCountry Stats = "country"
    // StatsLocationCity is stats for City
    StatsLocationCity = "city"
    // StatsDeviceType is stats for Device Type
    StatsDeviceType = "deviceType"
    // StatsDevicePlatform is stats for Device Platform
    StatsDevicePlatform = "devicePlatform"
    // StatsDeviceOS is stats for OS
    StatsDeviceOS = "os"
    // StatsDeviceBrowser is stats for Browser
    StatsDeviceBrowser = "browser"
    // StatsDeviceLanguage is stats for Language
    StatsDeviceLanguage = "language"
    // StatsReferral is stats for Referral
    StatsReferral = "referral"
)

// EventRepository is storage repository for Events.
// We are not using a persistent datastore like SQL.
// Once the application is exited all stats will be gone.
type EventRepository struct {
    locationCountry *counter.Counter
    locationCity    *counter.Counter
    deviceType      *counter.Counter
    devicePlatform  *counter.Counter
    deviceOS        *counter.Counter
    deviceBrowser   *counter.Counter
    deviceLanguage  *counter.Counter
    referral        *counter.Counter
}

// NewEventRepository creates and returns new EventRepository
func NewEventRepository() *EventRepository {
    return &EventRepository{
        locationCountry: counter.NewCounter(),
        locationCity:    counter.NewCounter(),
        deviceType:      counter.NewCounter(),
        devicePlatform:  counter.NewCounter(),
        deviceOS:        counter.NewCounter(),
        deviceBrowser:   counter.NewCounter(),
        deviceLanguage:  counter.NewCounter(),
        referral:        counter.NewCounter(),
    }
}

// AddEvent adds an event to the repository
func (tr *EventRepository) AddEvent(ev *model.Event) {
    tr.locationCountry.Incr(ev.Location.Country)
    tr.locationCity.Incr(ev.Location.City)
    tr.deviceType.Incr(ev.Device.Type)
    tr.devicePlatform.Incr(ev.Device.Platform)
    tr.deviceOS.Incr(ev.Device.OS)
    tr.deviceBrowser.Incr(ev.Device.Browser)
    tr.deviceLanguage.Incr(ev.Device.Language)
    tr.referral.Incr(ev.Referral)
}

// Events returns stats for the specified event query
func (tr *EventRepository) Events(d Stats) map[string]uint64 {
    m := make(map[string]uint64)
    switch d {
    case StatsLocationCountry:
        m = tr.locationCountry.Items()
    case StatsLocationCity:
        m = tr.locationCity.Items()
    case StatsDeviceType:
        m = tr.deviceType.Items()
    case StatsDevicePlatform:
        m = tr.devicePlatform.Items()
    case StatsDeviceOS:
        m = tr.deviceOS.Items()
    case StatsDeviceBrowser:
        m = tr.deviceBrowser.Items()
    case StatsDeviceLanguage:
        m = tr.deviceLanguage.Items()
    case StatsReferral:
        m = tr.referral.Items()
    }
    return m
}

5: 创建 EventService

EventService 用以连接 EventRepository.

它从 HTTP 请求中构建Event并将其存储在EventRepository中以统计信息。

创建事件.go服务文件夹中。

service 目录下创建 event.go 文件:

package service

import (
    "log"
    "net"
    "net/http"
    "net/url"

    "github.com/avct/uasurfer"
    "github.com/erybz/go-gal-analytics/gogal/model"
    "github.com/erybz/go-gal-analytics/gogal/repository"
    "github.com/oschwald/geoip2-golang"
    "github.com/tomasen/realip"
    "golang.org/x/text/language"
)

// EventService is service for event logging and stats
type EventService struct {
    eventRepo   *repository.EventRepository
    geoIPReader *geoip2.Reader
}

// NewEventService returns new EventService
func NewEventService() *EventService {
    return &EventService{
        eventRepo:   repository.NewEventRepository(),
        geoIPReader: initGeoIPReader("gogal/assets/GeoLite2-City.mmdb"),
    }
}

// BuildEvent builds a trackable event from the request
func (ts *EventService) BuildEvent(r *http.Request) (*model.Event, error) {
    clientIP := net.ParseIP(realip.FromRequest(r))
    userAgent := uasurfer.Parse(r.UserAgent())
    referrerURL, _ := url.Parse(r.Referer())
    langTags, _, _ := language.ParseAcceptLanguage(r.Header.Get("Accept-Language"))

    userLanguage := ""
    if langTags != nil && len(langTags) >= 1 {
        userLanguage = langTags[0].String()
    }

    geoData, err := ts.geoIPReader.City(clientIP)
    if err != nil {
        return nil, err
    }

    if userAgent.IsBot() {
        return nil, nil
    }

    event := &model.Event{
        Location: model.EventLocation{
            Country: geoData.Country.Names["en"],
            City:    geoData.City.Names["en"],
        },
        Device: model.EventDevice{
            Type:     userAgent.DeviceType.StringTrimPrefix(),
            Platform: userAgent.OS.Platform.StringTrimPrefix(),
            OS:       userAgent.OS.Name.StringTrimPrefix(),
            Browser:  userAgent.Browser.Name.StringTrimPrefix(),
            Language: userLanguage,
        },
        Referral: referrerURL.Hostname(),
    }
    return event, nil
}

// LogEvent logs the event to repository
func (ts *EventService) LogEvent(event *model.Event) {
    ts.eventRepo.AddEvent(event)
}

// Stats retrieves event statistics from the repository
// Since we are storing the events as-
// {
//     "USA": 83723,
//     "UK": 2323
// }
// we need to convert it as follows for the Stats API-
// [
//   {
//     "country": "USA",
//     "pageViews": 83723
//   },
//   {
//     "country": "UK",
//     "pageViews": 2323
//   }
// ]

func (ts *EventService) Stats(dim repository.Stats) []map[string]interface{} {
    allStats := make([]map[string]interface{}, 0, 1)
    for k, v := range ts.eventRepo.Events(dim) {
        stat := map[string]interface{}{
            string(dim): k,
            "pageViews": v,
        }
        allStats = append(allStats, stat)
    }
    return allStats
}

func initGeoIPReader(path string) *geoip2.Reader {
    db, err := geoip2.Open(path)
    if err != nil {
        log.Fatal(err)
    }
    return db
}

6. 创建处理器 Handler

处理器 EventHandler 使用 EventService.

它处理 HTTP 请求用以跟踪事件和提供统计信息。

接下来在 handler 目录下创建 event.go

package handler

import (
    "encoding/json"
    "log"
    "net/http"

    "github.com/erybz/go-gal-analytics/gogal/repository"
    "github.com/erybz/go-gal-analytics/gogal/service"
    "github.com/julienschmidt/httprouter"
)

// EventHandler is handler for Events
type EventHandler struct {
    eventService *service.EventService
}

// NewEventHandler creates and returns new EventHandler
func NewEventHandler() *EventHandler {
    return &EventHandler{
        eventService: service.NewEventService(),
    }
}

// Track accepts analytics request and builds event from it
func (h *EventHandler) Track(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
    if r.Method != http.MethodGet {
        http.Error(w, "Request method is not GET", http.StatusNotFound)
        return
    }
    event, err := h.eventService.BuildEvent(r)
    if err != nil {
        log.Println(err)
    }

    if event != nil && event.Valid() {
        h.eventService.LogEvent(event)
    }

    w.Header().Set("Cache-Control", "no-cache, no-store, must-revalidate")
    w.Header().Set("Content-Type", "image/gif")
    w.Write(createPixel())
}

// Stats retrieves stats for the specified query
func (h *EventHandler) Stats(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {
    if r.Method != http.MethodGet {
        http.Error(w, "Request method is not GET", http.StatusNotFound)
        return
    }

    urlVals := r.URL.Query()
    query := urlVals.Get("q")

    stats := h.eventService.Stats(
        repository.Stats(query),
    )

    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(stats)
}

// This is an image pixel, specifically GIF89a
// Check the references at the end for more info. 
func createPixel() []byte {
    return []byte{
        71, 73, 70, 56, 57, 97, 1, 0, 1, 0, 128, 0, 0, 0, 0, 0,
        255, 255, 255, 33, 249, 4, 1, 0, 0, 0, 0, 44, 0, 0, 0, 0,
        1, 0, 1, 0, 0, 2, 1, 68, 0, 59,
    }
}

7. Create routes

请在 route 目录下创建 route.go 文件:

package route

import (
    "net/http"

    "github.com/erybz/go-gal-analytics/gogal/handler"
    "github.com/julienschmidt/httprouter"
)

// Routes initializes the routes
func Routes() http.Handler {
    rt := httprouter.New()

    eventHandler := handler.NewEventHandler()
    rt.GET("/knock-knock", eventHandler.Track)
    rt.GET("/stats", eventHandler.Stats)

    rt.ServeFiles("/dashboard/*filepath", http.Dir("./gogal/web"))

    return rt
}

8. 初始化 Server

gogal目录下添加 server.go

package gogal

import (
    "fmt"
    "log"
    "net/http"
)

// Server struct containing hostname and port
type Server struct {
    Hostname string `json:"hostname"`
    HTTPPort string `json:"httpPort"`
}

// NewServer creates new instance of server
func NewServer(host, port string) *Server {
    return &Server{
        Hostname: host,
        HTTPPort: port,
    }
}

// Run starts the server at specified host and port
func (s *Server) Run(h http.Handler) {
    fmt.Println(s.Message())

    log.Printf("Listening at %s", s.Address())
    log.Fatal(http.ListenAndServe(s.Address(), h))
}

// Address returns formatted hostname and port
func (s *Server) Address() string {
    return fmt.Sprintf("%s:%s", s.Hostname, s.HTTPPort)
}

// Message is the server start message
func (s *Server) Message() string {
    m := `
                                      .__   
   ____   ____             _________  |  |  
  / ___\ /  _ \   ______  / ___\__  \ |  |  
 / /_/  (  <_> ) /_____/ / /_/  / __ \|  |__
 \___  / \____/          \___  (____  |____/
/_____/                 /_____/     \/      
                                     Analytics

`
    return m
}

9. 创建应用入口

在根目录下新建 main.go 文件:

package main

import (
    "flag"

    "github.com/erybz/go-gal-analytics/gogal"
    "github.com/erybz/go-gal-analytics/gogal/route"
)

func main() {
    hostname := flag.String(
        "h", "0.0.0.0", "hostname",
    )
    port := flag.String(
        "p", "8000", "port",
    )
    flag.Parse()

    s := gogal.NewServer(*hostname, *port)
    r := route.Routes()
    s.Run(r)
}

10. 开始测试

$ go run main.go -p 80

                                      .__   
   ____   ____             _________  |  |  
  / ___\ /  _ \   ______  / ___\__  \ |  |  
 / /_/  (  <_> ) /_____/ / /_/  / __ \|  |__
 \___  / \____/          \___  (____  |____/
/_____/                 /_____/     \/      
                                     Analytics

2020/06/05 00:00:50 Listening at 0.0.0.0:80

以图片 src 的形式在页面添加跟踪器:

<img src="http://localhost/knock-knock" border="0" width="1" height="1" />

请求 stats 接口以查看统计数据:

curl -s -X GET "https://go-gal.herokuapp.com/stats?q=country"
[
  {
    "country": "United States",
    "pageViews": 5
  },
  {
    "country": "Canada",
    "pageViews": 3
  }
]

以下是 go-gal-analytics API 提供的功能:

  • /knock-knock - 使用<img> 标签来统计事件
  • /stats - 使用以下参数来获取统计数据
    • ?q=country
    • ?q=city
    • ?q=deviceType
    • ?q=devicePlatform
    • ?q=os
    • ?q=browser
    • ?q=language
    • ?q=referral

注意:

运行此程序你需要前往 MaxMind 注册并下载 GeoIP 数据库 GeoLite2-City.mmdb 并放置于 assets 目录下。

未来优化

  • 目前只支持单站点,你可以让其支持多站点;
  • 添加一个真实的 SQL/NoSQL 数据库,用以存储数据;
  • 支持除了页面统计的其他统计信息;
  • 优化代码中对错误的处理。

完整的源码 —— go-gal-analytics

小惊喜

你还可以邮件中添加图片并使用此统计 API,以统计邮件的打开率和浏览器、地理位置等用户信息。

本文中的所有译文仅用于学习和交流目的,转载请务必注明文章译者、出处、和本文链接
我们的翻译工作遵照 CC 协议,如果我们的工作有侵犯到您的权益,请及时联系我们。

原文地址:https://eryb.space/2020/06/05/build-goog...

译文地址:https://learnku.com/go/t/45966

本文为协同翻译文章,如您发现瑕疵请点击「改进」按钮提交优化建议
讨论数量: 0
(= ̄ω ̄=)··· 暂无内容!

讨论应以学习和精进为目的。请勿发布不友善或者负能量的内容,与人为善,比聪明更重要!