# Kytensor 推理服务
kytensor推理服务是基于[NVIDIA Triton Inference Server](https://github.com/triton-inference-server)构建，将其各个模块重新组织在一个工程中便于管理和开发。

- server
- client
- core
- backend
- common
- third_party
- framework_backends

## 编译

### server

```
cmake -DCMAKE_INSTALL_PREFIX=`pwd`/install  -DTRITON_ENABLE_SERVER=ON -DTRITON_ENABLE_CLIENT=OFF  -DTRITON_ENABLE_ONNX_BACKEND=OFF ..
```

### client

```
cmake -DCMAKE_INSTALL_PREFIX=`pwd`/install -DTRITON_ENABLE_SERVER=OFF -DTRITON_ENABLE_CLIENT=ON -DTRITON_ENABLE_ONNX_BACKEND=OFF -DTRITON_ENABLE_CC_HTTP=ON -DTRITON_ENABLE_CC_GRPC=ON -DTRITON_ENABLE_EXAMPLES=ON -DTRITON_ENABLE_TESTS=ON -DTRITON_ENABLE_PYTHON_HTTP=ON -DTRITON_ENABLE_PYTHON_GRPC=ON  ..
```

## 测试
```
cmake -DCMAKE_INSTALL_PREFIX=`pwd`/install -DTRITON_ENABLE_SERVER=OFF -DTRITON_ENABLE_CLIENT=ON -DTRITON_ENABLE_ONNX_BACKEND=OFF -DTRITON_ENABLE_CC_HTTP=ON -DTRITON_ENABLE_CC_GRPC=ON -DTRITON_ENABLE_EXAMPLES=OFF -DTRITON_ENABLE_TESTS=ON -DTRITON_ENABLE_PYTHON_HTTP=ON -DTRITON_ENABLE_PYTHON_GRPC=ON -DTRITON_ENABLE_GPU=OFF -DTRITON_ENABLE_METRICS_GPU=OFF ..
```
