ComfyUI Extension: ComfyUI-CloudArchive
Automatically archives generated files to Amazon S3 or S3-compatible storage services
Custom Nodes (0)
README
ComfyUI-CloudArchive
ComfyUI-CloudArchive is a plugin that monitors the ComfyUI output directory and automatically archives (saves) generated files to Amazon S3 or S3-compatible storage.
Features
- Monitors ComfyUI's output directory and detects new files (including subdirectories)
- Automatically uploads detected files to S3 or S3-compatible storage
- Intelligently waits until files are completely written (detects file size stabilization)
- Preserves the internal structure of the output directory when uploading
- Generates a unique session ID at startup for folder organization in S3
- RESTful API endpoints for status checking and control
- Easy configuration via environment variables
Installation
- Clone this repository into ComfyUI's custom nodes directory:
cd /path/to/ComfyUI/custom_nodes
git clone https://github.com/olduvai-jp/ComfyUI-CloudArchive.git
- Install the required Python packages:
pip install boto3 watchdog
Configuration
Set the following environment variables:
S3_ACCESS_KEY_ID
(required): Access key ID for the storage serviceS3_SECRET_ACCESS_KEY
(required): Secret access key for the storage serviceS3_REGION
(optional): Region for the storage service (default: us-east-1)S3_BUCKET
(required): Bucket name for uploadsS3_PREFIX
(optional): Prefix within the bucket (default: comfyui-outputs)S3_ENDPOINT_URL
(optional): URL for S3-compatible endpoint (MinIO, Wasabi, DigitalOcean Spaces, etc.)
Example environment variable setup (Linux/Mac):
export S3_ACCESS_KEY_ID=your_access_key
export S3_SECRET_ACCESS_KEY=your_secret_key
export S3_REGION=ap-northeast-1
export S3_BUCKET=your-bucket-name
export S3_PREFIX=comfyui/outputs
export S3_ENDPOINT_URL=https://minio.example.com
Windows:
set S3_ACCESS_KEY_ID=your_access_key
set S3_SECRET_ACCESS_KEY=your_secret_key
set S3_REGION=ap-northeast-1
set S3_BUCKET=your-bucket-name
set S3_PREFIX=comfyui/outputs
set S3_ENDPOINT_URL=https://minio.example.com
Using S3-Compatible Endpoints
This plugin supports not only Amazon S3 but also S3-compatible storage services. It can be used with services such as:
- MinIO
- Wasabi
- DigitalOcean Spaces
- Backblaze B2
- Scaleway Object Storage
- Other S3-compatible services
To use an S3-compatible service, set the S3_ENDPOINT_URL
environment variable to the endpoint URL of the service.
MinIO Example:
export S3_ACCESS_KEY_ID=your_minio_access_key
export S3_SECRET_ACCESS_KEY=your_minio_secret_key
export S3_BUCKET=your-minio-bucket
export S3_ENDPOINT_URL=http://minio.example.com:9000
Wasabi Example:
export S3_ACCESS_KEY_ID=your_wasabi_access_key
export S3_SECRET_ACCESS_KEY=your_wasabi_secret_key
export S3_REGION=us-east-1
export S3_BUCKET=your-wasabi-bucket
export S3_ENDPOINT_URL=https://s3.wasabisys.com
DigitalOcean Spaces Example:
export S3_ACCESS_KEY_ID=your_spaces_key
export S3_SECRET_ACCESS_KEY=your_spaces_secret
export S3_REGION=nyc3
export S3_BUCKET=your-space-name
export S3_ENDPOINT_URL=https://nyc3.digitaloceanspaces.com
Usage
Automatic Startup
The plugin starts automatically when ComfyUI is launched and monitors the default output directory (ComfyUI/output
).
API Endpoints
The following API endpoints are available:
Status Check
GET /cloud-sync/status
Example response:
{
"running": true,
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"total_files": 10,
"uploaded_files": 8,
"failed_files": 0,
"last_upload_time": "2023-04-01T12:34:56",
"errors": [],
"recent_uploads": [
{
"file_name": "image_01.png",
"s3_key": "comfyui-outputs/550e8400-e29b-41d4-a716-446655440000/image_01.png",
"upload_time": "2023-04-01T12:34:56",
"size_bytes": 1234567
}
]
}
Start Monitoring
POST /cloud-sync/start
Stop Monitoring
POST /cloud-sync/stop
Manual Upload
POST /cloud-sync/upload
Request body:
{
"file_path": "/path/to/file.png"
}
S3 Directory Structure
Uploaded files are stored in S3 with the following structure:
{S3_PREFIX}/{SESSION_ID}/{RELATIVE_PATH}
Example:
comfyui-outputs/550e8400-e29b-41d4-a716-446655440000/image_01.png
comfyui-outputs/550e8400-e29b-41d4-a716-446655440000/subfolder/image_02.png
The internal structure of the output directory (including subdirectories) is preserved.
Troubleshooting
Environment Variables Not Set
If the required environment variables are not set, error messages will be logged and displayed in the errors
field of the /cloud-sync/status
endpoint.
Upload Failures
If an upload fails, error messages will be logged and displayed in the errors
field of the /cloud-sync/status
endpoint. The failed_files
counter will also increase.
License
MIT