mingqiancha233's picture
Add model definition, architecture card and inference infrastructure
53aebab verified
|
Raw History Blame Contribute Delete
1.19 kB

Single-GPU inference container

Requires Docker with NVIDIA Container Toolkit, a compatible NVIDIA driver, and a CUDA GPU with sufficient BF16 memory. A 96 GB GPU is recommended. The container installs the pinned Python runtime but does not download model weights during its build.

From the root of the downloaded repository:

docker build -f infra/Dockerfile -t socrates-image:1.0 .
mkdir -p outputs
docker run --rm --gpus '"device=0"' --ipc=host \
  -v "$PWD:/model:ro" \
  -v "$PWD/outputs:/outputs" \
  socrates-image:1.0 \
  --model-path /model \
  --prompt "A small observatory on a mountain under a starry sky." \
  --resolution 1024 --steps 40 --seed 42 --output /outputs/sample.png

For a lower-resolution smoke run, use --resolution 256 --steps 4. The full component set is still loaded, so this does not eliminate the weight-memory requirement. The example runs one request and exits; it is not a hosted inference endpoint or concurrent serving implementation.

The inference container is provided as a reference configuration. The current release was smoke-validated on its original CUDA environment; this Docker build has not been GPU-validated separately.