hailin
/
sglang.0.4.8.post1
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
sglang.0.4.8.post1/sglang/python/sglang
History
hailin 1a58b38c86 . 2025-07-24 13:09:00 +08:00
..
eval first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
lang first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
srt . 2025-07-24 13:09:00 +08:00
test first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
README.md first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
__init__.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
api.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
bench_offline_throughput.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
bench_one_batch.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
bench_one_batch_server.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
bench_serving.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
check_env.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
compile_deep_gemm.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
global_config.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
launch_server.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
math_utils.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
profiler.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
utils.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00
version.py first commit @ sglang v0.4.8.post1 2025-06-29 14:14:11 +00:00

README.md

Code Structures

  • eval: The evaluation utilities.
  • lang: The frontend language.
  • srt: The backend engine for running local models. (SRT = SGLang Runtime).
  • test: The test utilities.
  • api.py: The public APIs.
  • bench_offline_throughput.py: Benchmark the throughput in the offline mode.
  • bench_one_batch.py: Benchmark the latency of running a single static batch without a server.
  • bench_one_batch_server.py: Benchmark the latency of running a single batch with a server.
  • bench_serving.py: Benchmark online serving with dynamic requests.
  • check_env.py: Check the environment variables and dependencies.
  • global_config.py: The global configs and constants.
  • launch_server.py: The entry point for launching the local server.
  • utils.py: Common utilities.
  • version.py: Version info.