2021年8月26日 星期四

grpc 心得

前言

wiki 上,grpc最常見的應用場景是 微服務 框架下,多種語言服務之間的高效互動。
因為官方文檔php不能做服務端,所以從golang開始研究

golang

(使用windows 10 環境)

事前準備

安裝golang

檢查

$ go version
go version go1.17 windows/amd64

下載Protocol buffer編譯器(protoc)

https://developers.google.com/protocol-buffers/docs/downloads  
https://github.com/protocolbuffers/protobuf/releases/latest  
https://github.com/protocolbuffers/protobuf/releases/download/v3.17.3/protoc-3.17.3-win64.zip  
下載後解壓縮,將 protoc 設到環境變數

檢查

$ protoc --version
libprotoc 3.17.3

安裝Go plugins(protocol compiler plugins)

$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.26
go: downloading google.golang.org/protobuf v1.26.0

$ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.1
go: downloading google.golang.org/grpc/cmd/protoc-gen-go-grpc v1.1.0
go: downloading google.golang.org/protobuf v1.23.0

會安裝到 ~/go/pkg/mod/google.golang.org/ 和 ~/go/bin/ 目錄

下載範例程式

$ git clone -b v1.35.0 https://github.com/grpc/grpc-go
$ cd grpc-go/examples/helloworld

執行範例

服務端

/project_path/grpc-go/examples/helloworld ((v1.35.0))
$ go run greeter_server/main.go
go: downloading github.com/golang/protobuf v1.4.2
go: downloading google.golang.org/protobuf v1.25.0
go: downloading google.golang.org/genproto v0.0.0-20200806141610-86f49bd18e98
go: downloading golang.org/x/net v0.0.0-20190311183353-d8887717615a
go: downloading golang.org/x/text v0.3.0
(第一次會下載依賴)
2021/08/26 16:53:38 Received: world

客戶端

/project_path/grpc-go/examples/helloworld ((v1.35.0))
$ go run greeter_client/main.go
2021/08/26 16:53:38 Greeting: Hello world

服務端使用goland調試

Add Configuration ... => Add New Configuration => Go Build

Name: greeter_server/main.go
Run Kind: File
Files: C:\project_path\grpc-go\examples\helloworld\greeter_server\main.go  
這邊要複製出來在sublime手動編輯,去掉多餘路徑,直接UI上選出來的值會是 C:\project_path\grpc-go|C:\project_path\grpc-go\examples\helloworld\greeter_server\main.go

設定斷點,Debug

修改gRPC服務

在 examples/helloworld/ 目錄下,修改 helloworld/helloworld.proto
+++ b/examples/helloworld/helloworld/helloworld.proto
@@ -25,6 +25,8 @@ package helloworld;
 service Greeter {
   // Sends a greeting
   rpc SayHello (HelloRequest) returns (HelloReply) {}
+  // Sends another greeting
+  rpc SayHelloAgain (HelloRequest) returns (HelloReply) {}
 }

以下是完整的 helloworld.proto 


syntax = "proto3";

option go_package = "google.golang.org/grpc/examples/helloworld/helloworld";
option java_multiple_files = true;
option java_package = "io.grpc.examples.helloworld";
option java_outer_classname = "HelloWorldProto";

package helloworld;

// The greeting service definition.
service Greeter {
  // Sends a greeting
  rpc SayHello (HelloRequest) returns (HelloReply) {}
  // Sends another greeting
  rpc SayHelloAgain (HelloRequest) returns (HelloReply) {}
}

// The request message containing the user's name.
message HelloRequest {
  string name = 1;
}

// The response message containing the greetings
message HelloReply {
  string message = 1;
}
s

重新生成gRPC程式

$ protoc --go_out=. --go_opt=paths=source_relative \
     --go-grpc_out=. --go-grpc_opt=paths=source_relative \
     helloworld/helloworld.proto
(會重新生成 helloworld/helloworld.pb.go 和 helloworld/helloworld_grpc.pb.go )

更新服務端程式

在 greeter_server/main.go 新增以下程式
+++ b/examples/helloworld/greeter_server/main.go
@@ -43,6 +43,10 @@ func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloRe
     return &pb.HelloReply{Message: "Hello " + in.GetName()}, nil
 }

+func (s *server) SayHelloAgain(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) {
+        return &pb.HelloReply{Message: "Hello again " + in.GetName()}, nil
+}

更新客戶端程式

在 greeter_client/main.go 的main()裡面新增以下程式
+++ b/examples/helloworld/greeter_client/main.go
@@ -55,4 +55,10 @@ func main() {
         log.Fatalf("could not greet: %v", err)
     }
     log.Printf("Greeting: %s", r.GetMessage())
+
+    r, err = c.SayHelloAgain(ctx, &pb.HelloRequest{Name: name})
+    if err != nil {
+        log.Fatalf("could not greet: %v", err)
+    }
+    log.Printf("Greeting: %s", r.GetMessage())
 }

重新執行

服務端(Golang)

$ go run greeter_server/main.go
2021/08/26 17:29:35 Received: Alice

客戶端(Golang)

$ go run greeter_client/main.go Alice
2021/08/26 17:29:35 Greeting: Hello Alice
2021/08/26 17:29:35 Greeting: Hello again Alice



php

系統:windows 10
PHP 8.0

事前準備

PHP 7.0以上
pecl
composer

安裝grpc 擴展

windows 直接在 PECL 官網 
直接下載 DLL 裡面的 8.0 Non Thread Safe (NTS) x64 
解壓縮然後將 php_grpc.dll 放到  C:\BtSoft\php\80\ext 下
然後修改 C:\BtSoft\php\80\php.ini 加入
[gRPC]
extension=php_grpc.dll

然後在phpinfo()中檢查

安裝bazel

檢查系統

推薦:64 bit Windows 10, version 1703 以上
可使用 winver 檢查

安裝bazel的事前準備

Visual C++ Redistributable for Visual Studio 2015  (我沒裝,裝的時候報錯)

下載Bazel

下載 4.1.0 版本,我之前用4.2.0 後面會報錯


設定環境變數和檢查

$ /bazel_path/bazel-4.1.0-windows-x86_64.exe  version
Extracting Bazel installation...
Build label: 4.1.0
Build target: bazel-out/x64_windows-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri May 21 11:17:01 2021 (1621595821)
Build timestamp: 1621595821
Build timestamp as int: 1621595821

從github下載grpc

$ git clone --recurse-submodules -b v1.38.0 https://github.com/grpc/grpc
$ cd grpc


構建 protoc (使用bazel)

(在grpc目錄下)
$ /bazel_path/bazel-4.1.0-windows-x86_64.exe build @com_google_protobuf//:protoc
Starting local Bazel server and connecting to it...
...
error: invalid command 'bdist_wheel'

解法

https://stackoverflow.com/a/44862371  Why is python setup.py saying invalid command 'bdist_wheel' on Travis CI?
使用pip安裝wheel
$ pip install wheel

再試一次

$ /bazel_path/bazel-4.1.0-windows-x86_64.exe build @com_google_protobuf//:protoc
...
The target you are compiling requires Visual C++ build tools.
Bazel couldn't find a valid Visual C++ build tools installation on your machine.
Please check your installation following https://docs.bazel.build/versions/master/windows.html#using
...
FAILED: Build did NOT complete successfully

需安裝 Visual C++ build tools

https://stackoverflow.com/a/54136652  How to install Visual C++ Build tools?
到 https://visualstudio.microsoft.com/zh-hant/downloads/ 下載 Visual Studio 2019 的工具 => Build Tools for Visual Studio 2019 
然後安裝【使用C++的桌面開發】 

安裝完會要你重新開機

再試一次

$ /bazel_path/bazel-4.1.0-windows-x86_64.exe build @com_google_protobuf//:protoc
...
The target you are compiling requires Visual C++ build tools.
Bazel couldn't find a valid Visual C++ build tools installation on your machine.
Please check your installation following https://docs.bazel.build/versions/master/windows.html#using
...
FAILED: Build did NOT complete successfully
還是報相同錯誤

需設定 BAZEL_VC 環境變數

https://docs.bazel.build/versions/main/windows.html#using  Using Bazel on Windows
變數名稱:BAZEL_VC
變數值:C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC

再試一次

$ /bazel_path/bazel-4.1.0-windows-x86_64.exe build @com_google_protobuf//:protoc
Target @com_google_protobuf//:protoc up-to-date:
  bazel-bin/external/com_google_protobuf/protoc.exe
INFO: Elapsed time: 163.033s, Critical Path: 13.35s
INFO: 170 processes: 3 internal, 167 local.
INFO: Build completed successfully, 170 total actions

安裝構建在 bazel-bin/external/com_google_protobuf/protoc.exe 下

檢查

$ /path/grpc/bazel-bin/external/com_google_protobuf/protoc.exe  --version
libprotoc 3.15.8

構建 grpc_php_plugin (使用bazel)

$ /bazel_path/bazel-4.1.0-windows-x86_64.exe build src/compiler:grpc_php_plugin
Target //src/compiler:grpc_php_plugin up-to-date:
  bazel-bin/src/compiler/grpc_php_plugin.exe
INFO: Elapsed time: 15.245s, Critical Path: 5.66s
INFO: 15 processes: 5 internal, 10 local.
INFO: Build completed successfully, 15 total actions

安裝構建在 bazel-bin/src/compiler/grpc_php_plugin.exe 下

php gRPC客戶端從0開始

新增專案:grpc_php_client
複製 grpc-go的 helloworld.proto 到目錄下(使用同一個 proto)

使用.proto 生成PHP類

$ protoc -I=. helloworld.proto --php_out=. --grpc_out=. --plugin=protoc-gen-grpc=/path/grpc/bazel-bin/src/compiler/grpc_php_plugin.exe
就會生成 GPBMetadata/  Helloworld/ 兩個目錄

使用composer 安裝 grpc/grpc

$ composer require grpc/grpc

使用composer 安裝 grpc/grpc

因為比較容易安裝,或是使用pecl安裝 protobuf 擴展(效率比較好)
$ composer require google/protobuf

設定autoload

https://my.oschina.net/hongjiang/blog/3135474  PHP中使用gRPC客户端 - 5、使用 PHP 的 composer
修改 composer.json
--- a/composer.json
+++ b/composer.json
@@ -2,5 +2,13 @@
     "require": {
         "grpc/grpc": "^1.39",
         "google/protobuf": "^3.17"
+    },
+    "autoload": {
+        "psr-4": {
+            "GPBMetadata\\": [
+                "GPBMetadata/"
+            ],
+            "Helloworld\\": "Helloworld/"
+        }
     }
 }

然後
$ composer dump-autoload

複製 C:\path\grpc\examples\php\greeter_client.php 到  C:\path\grpc_php_client\ 
greeter_client.php
require dirname(__FILE__).'/vendor/autoload.php';

function greet($hostname, $name)
{
    $client = new Helloworld\GreeterClient($hostname, [
        'credentials' => Grpc\ChannelCredentials::createInsecure(),
    ]);
    $request = new Helloworld\HelloRequest();
    $request->setName($name);
    list($response, $status) = $client->SayHello($request)->wait();
    if ($status->code !== Grpc\STATUS_OK) {
        echo "ERROR: " . $status->code . ", " . $status->details . PHP_EOL;
        exit(1);
    }
    echo $response->getMessage() . PHP_EOL;
}

$name = !empty($argv[1]) ? $argv[1] : 'world';
$hostname = !empty($argv[2]) ? $argv[2] : 'localhost:50051';
greet($hostname, $name);

s

客戶端(PHP)

$ php greeter_client.php test
Hello test

服務端(Golang)

/path/grpc-go/examples/helloworld
$ go run greeter_server/main.go
2021/08/27 00:20:35 Received: test

php gRPC服務端

在 https://github.com/grpc/grpc 的 examples/php 中,greeter_server.php 有實作 gRPC服務端的代碼
  

require dirname(__FILE__) . '/../../src/php/lib/Grpc/MethodDescriptor.php';
require dirname(__FILE__) . '/../../src/php/lib/Grpc/Status.php';
require dirname(__FILE__) . '/../../src/php/lib/Grpc/ServerCallReader.php';
require dirname(__FILE__) . '/../../src/php/lib/Grpc/ServerCallWriter.php';
require dirname(__FILE__) . '/../../src/php/lib/Grpc/ServerContext.php';
require dirname(__FILE__) . '/../../src/php/lib/Grpc/RpcServer.php';
require dirname(__FILE__) . '/vendor/autoload.php';

class Greeter extends Helloworld\GreeterStub
{
    public function SayHello(
        \Helloworld\HelloRequest $request,
        \Grpc\ServerContext $serverContext
    ): ?\Helloworld\HelloReply {
        $name = $request->getName();
        $response = new \Helloworld\HelloReply();
        $response->setMessage("Hello " . $name);
        return $response;
    }
}

$server = new \Grpc\RpcServer();
$server->addHttp2Port('0.0.0.0:50051');
$server->handle(new Greeter());
$server->run();

  
但是代碼其中寫到:
/**
 * This is an experimental and incomplete implementation of gRPC server
 * for PHP. APIs are _definitely_ going to be changed.
 *
 * DO NOT USE in production.
 */
這個gRPC服務端的class還是未完成的實驗性質,不要在線上產品使用
而且, Helloworld\GreeterStub 找不到是用什麼方式生成的

使用 mix-php/grpc 實作gRPC服務端


手動安裝swoole

因為 mix-php/grpc 需要開啟 http2 ,寶塔後台安裝不會開啟,必須手動安裝swoole

下載swoole源碼

# wget https://github.com/swoole/swoole-src/archive/refs/tags/v4.7.1.tar.gz

解壓縮

# tar zxf v4.7.1.tar.gz

進入目錄

# cd swoole-src-4.7.1/

安裝

# phpize
# ./configure --enable-http2 --with-php-config=/www/server/php/80/bin/php-config
# make
# make install
Installing shared extensions:     /www/server/php/80/lib/php/extensions/no-debug-non-zts-20200930/
Installing header files:          /www/server/php/80/include/php/

配置php.ini

/www/server/php/80/etc/php.ini
[swoole]
extension=/www/server/php/80/lib/php/extensions/no-debug-non-zts-20200930/swoole.so

檢查

# php -i | less
...
swoole

Swoole => enabled
Author => Swoole Team <team@swoole.com>
Version => 4.7.1
Built => Sep  6 2021 08:57:19
...
http2 => enabled
...



下載protoc與相關plugin

centos 下載 protoc_mix_plugin  後,解壓縮把 protoc  protoc-gen-mix 放到 /usr/local/bin 目錄

實作

安裝mix/grpc
$ composer require mix/grpc
將前面的 helloworld.proto 複製到專案資料夾下(使用同一個proto)
然後使用 protoc 生成代碼:
$ protoc --php_out=. --mix_out=. helloworld.proto
執行命令後將在當前目錄生成以下文件
# tree -I 'vendor'
.
├── composer.json
├── composer.lock
├── GPBMetadata
│   └── Helloworld.php
├── Helloworld
│   ├── GreeterClient.php
│   ├── GreeterInterface.php
│   ├── HelloReply.php
│   └── HelloRequest.php
└── helloworld.proto

https://unix.stackexchange.com/a/47806  How do we specify multiple ignore patterns for `tree` command?
-I  忽略特定目錄

其中 HelloRequest.php、HelloReply.php 為 --php_out 生成,GreeterClient.php GreeterInterface.php 由 --mix_out 生成。
接下來我們將生成的文件加入到 composer autoload 中,我們修改 composer.json:
diff --git a/composer.json b/composer.json
index 335610d..3dd0f59 100644
--- a/composer.json
+++ b/composer.json
@@ -1,5 +1,11 @@
 {
     "require": {
         "mix/grpc": "^3.0"
+    },
+    "autoload-dev": {
+        "psr-4": {
+            "GPBMetadata\\": "GPBMetadata/",
+            "Helloworld\\": "Helloworld/"
+        }
     }
 }

修改後執行 composer dump-autoload 使其生效。

編寫一個 gRPC 服務


require __DIR__ . '/vendor/autoload.php';

// 編寫一個服務,實現 protoc-gen-mix 生成的接口
class SayService implements \Helloworld\GreeterInterface
{
    public function SayHello(\Mix\Grpc\Context $context, \Helloworld\HelloRequest $request): \Helloworld\HelloReply {
        // TODO: Implement SayHello() method.
        echo date('Y-m-d H:i:s')." Received:".$request->getName()."\n";
        $response = new \Helloworld\HelloReply();
        $response->setMessage(sprintf('hello, %s', $request->getName()));
        return $response;
    }

    public function SayHelloAgain(\Mix\Grpc\Context $context, \Helloworld\HelloRequest $request): \Helloworld\HelloReply {
        // TODO: Implement SayHelloAgain() method.
        $response = new \Helloworld\HelloReply();
        $response->setMessage(sprintf('hello again, %s', $request->getName()));
        return $response;
    }
}

$grpc = new Mix\Grpc\Server();
$grpc->register(SayService::class); // or $grpc->register(new SayService());

$http = new Swoole\Http\Server('0.0.0.0', 9595);
$http->on('Request', $grpc->handler());
$http->set([
    'worker_num' => 4,
    'open_http2_protocol' => true,
    'http_compression' => false,
]);
$http->start();
s

測試

客戶端(golang)

$ go run greeter_client/main.go test
2021/09/21 15:07:25 Greeting: hello, test
2021/09/21 15:07:25 Greeting: hello again, test

服務端(PHP)

# php hello_server.php
2021-09-21 15:07:26 Received:test

客戶端調用一個 gRPC 服務

require __DIR__ . '/vendor/autoload.php';

Swoole\Coroutine\run(function () {
    $client = new Mix\Grpc\Client('127.0.0.1', 9595); // 復用該客戶端
    $say  = new \Helloworld\GreeterClient($client);
    $request = new \Helloworld\HelloRequest();
    $request->setName('xiaoming');
    $ctx = new Mix\Grpc\Context();
    $response = $say->SayHello($ctx, $request);
    var_dump($response->getMessage());
    $response = $say->SayHelloAgain($ctx, $request);
    var_dump($response->getMessage());
    $client->close();
});
s

測試

客戶端(PHP)

# php hello_client.php
string(15) "hello, xiaoming"
string(21) "hello again, xiaoming"

服務端(PHP)

# php hello_server.php
2021-09-21 15:13:09 Received:xiaoming