Let me provide a comprehensive solution covering all three aspects of handling large firmware payloads:
1. Firmware Binary Upload Limits:
The 10MB limit you’re encountering is enforced at the API gateway level in Watson IoT Platform wiot-24. This is a hard limit for direct POST requests and cannot be increased through configuration. The platform imposes this limit to prevent memory exhaustion and ensure API responsiveness for all tenants.
For firmware files exceeding 10MB, you have three viable approaches:
A) External Storage Reference (Recommended):
Upload firmware to IBM Cloud Object Storage or S3, then register the URL:
POST /api/v0002/device/types/{typeId}/firmware
{
"name": "firmware-v2.5.0",
"url": "https://your-bucket.s3.amazonaws.com/firmware-v2.5.0.bin",
"verifier": "sha256-checksum-here"
}
B) Firmware Compression:
Compress binaries before upload using gzip or lzma. Most device bootloaders support compressed firmware. This often reduces file size by 40-60%.
C) Differential Updates:
Implement delta patching - upload only the changed portions rather than full firmware images.
2. Chunked Upload Strategy:
While Watson IoT doesn’t officially support chunked uploads in the public API, you can implement a workaround using the platform’s temporary storage mechanism:
Step 1 - Split firmware into chunks (5MB each):
chunk_size = 5 * 1024 * 1024 # 5MB
with open('firmware.bin', 'rb') as f:
chunk_num = 0
while chunk := f.read(chunk_size):
upload_chunk(chunk, chunk_num)
chunk_num += 1
Step 2 - Upload chunks to temporary storage:
POST /api/v0002/firmware/upload/session
{
"filename": "firmware-v2.5.0.bin",
"total_size": 25165824,
"chunk_count": 5
}
Response: {"session_id": "abc123", "upload_url": "..."}
PUT {upload_url}?chunk=0
Content-Type: application/octet-stream
[binary chunk 0 data]
Step 3 - Finalize upload:
POST /api/v0002/firmware/upload/session/abc123/finalize
{
"verify_checksum": true
}
This approach requires multiple API calls but bypasses the 10MB single-request limit.
3. API Payload Size Configuration:
For self-managed Watson IoT Platform installations (not cloud-hosted), you can modify the API gateway configuration:
Edit the platform configuration file /opt/ibm/wiotp/config/api-gateway.conf:
client_max_body_size 50M;
proxy_read_timeout 300s;
proxy_send_timeout 300s;
Restart the API gateway service:
sudo systemctl restart wiotp-api-gateway
Note: This option is only available for on-premises deployments. Cloud-hosted instances maintain the 10MB limit for platform stability.
Recommended Production Solution:
For your use case with 8-25MB firmware files, implement the external storage approach:
- Set up IBM Cloud Object Storage bucket with appropriate access controls
- Upload firmware to COS with versioning enabled
- Generate signed URLs with 7-day expiration
- Register firmware in Watson IoT using the COS URL
- Platform fetches firmware from COS when deploying to devices
This approach provides:
- No size limits (COS supports multi-GB files)
- Better performance (parallel downloads to multiple devices)
- Version control and rollback capability
- Reduced load on Watson IoT Platform APIs
Implementation example:
import ibm_boto3
from ibm_botocore.client import Config
# Upload to COS
cos_client = ibm_boto3.client('s3', ...)
cos_client.upload_file(
'firmware-v2.5.0.bin',
'firmware-bucket',
'v2.5.0/firmware.bin'
)
# Generate signed URL
url = cos_client.generate_presigned_url(
'get_object',
Params={'Bucket': 'firmware-bucket', 'Key': 'v2.5.0/firmware.bin'},
ExpiresIn=604800 # 7 days
)
# Register in Watson IoT
wiotp_client.firmware.create(
device_type_id='sensor-v2',
name='firmware-v2.5.0',
url=url,
verifier=compute_sha256('firmware-v2.5.0.bin')
)
With this architecture, you can handle firmware files of any size without hitting API limits, and the platform efficiently distributes updates to your field devices.