File processing

Process uploaded files without making users wait

Store each upload immediately, queue the processing work, and run document, image, CSV, or AI tasks asynchronously.

Object StorageQueueFunction
  • Immediate upload response
  • Managed file storage
  • Background processing
  • Observable workflow

Why processing inside the upload request fails

Synchronous upload handler

  1. 1Upload
  2. 2Validate file
  3. 3Parse or transform
  4. 4Call external services
  5. 5Return response
  • Users wait while processing completes
  • Large files increase timeout risk
  • Temporary failures can lose work
  • Retries may upload or process files twice
  • Application requests consume processing capacity
  • Long-running AI or document tasks become unreliable

RocketStack workflow

UploadObject StorageUpload response
QueueFunctionProcessing complete
  • The upload completes independently of processing
  • Files remain available during temporary failures
  • Processing can be retried separately
  • Work happens outside the application request
  • Failures remain visible
  • Processing logic can scale independently

Three components, one file-processing pipeline

1

Object Storage

Store the original upload before processing begins.

  • Accept the file
  • Preserve the original object
  • Return or record the object path
  • Make the file available to the processing function
2

Queue

Hold processing jobs until a Function is ready to run them.

  • Decouple upload from processing
  • Carry the object path and job metadata
  • Retry supported temporary failures
  • Control processing delivery
  • Move exhausted messages to a dead-letter queue
3

Function

Download the object, perform the processing work, and store or return the result.

  • Parse CSV or document content
  • Transform images or files
  • Call an AI model or external API
  • Store processing results
  • Surface failures

Example workflow

See a file move through the workflow

Run an illustrative upload and processing sequence. Selected files stay in the browser — nothing is uploaded to RocketStack in this demo.

Example workflow

Simulated upload

Drag and drop a file here, or choose one with the file picker.

Activity timeline

Example timing

Select a sample file or choose a local file to begin.

  1. File accepted
    +0ms
  2. Object stored
    +40ms
  3. Processing job queued
    +70ms
  4. Upload response returned
    +85ms
  5. Function invoked
    +180ms
  6. File processed
    +320ms
  7. Result stored
    +410ms

Build the workflow

Use the RocketStack HTTP API or generated SDKs. Examples below use real request fields.

Step 1 of 4

Upload the file

Language

Store the original file and capture its RocketStack object path.

Upload the file · JavaScript
const form = new FormData();
form.append('file', file);
form.append('path', `uploads/${file.name}`);

const uploadRes = await fetch('https://api.rocketstack.sh/storage/objects', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.ROCKETSTACK_API_KEY}`
  },
  body: form
});

const { data } = await uploadRes.json();
// data.path, data.sizeBytes, data.contentType, data.etag
console.log(data.path);

Ready to process your first file?

Also see Object Storage, Queue and Functions product docs.

Add it to your existing application

Keep your application where it is. Upload files and enqueue processing from server-side code.

Keep your Next.js frontend and routes. Send files and processing jobs to RocketStack from server-side application code.

Next.js example
// app/api/uploads/route.ts — server-side only (keep API key off the client)
export async function POST(request: Request) {
  const formData = await request.formData()
  const file = formData.get('file') as File
  const path = `uploads/${file.name}`

  const uploadForm = new FormData()
  uploadForm.append('file', file)
  uploadForm.append('path', path)

  const uploadRes = await fetch('https://api.rocketstack.sh/storage/objects', {
    method: 'POST',
    headers: { Authorization: `Bearer ${process.env.ROCKETSTACK_API_KEY}` },
    body: uploadForm
  })
  const { data } = await uploadRes.json()

  await fetch(
    `https://api.rocketstack.sh/queues/${process.env.ROCKETSTACK_QUEUE_ID}/messages`,
    {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${process.env.ROCKETSTACK_API_KEY}`,
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({
        payload: {
          type: 'file.uploaded',
          objectPath: data.path,
          operation: 'parse-csv'
        },
        idempotencyKey: data.path
      })
    }
  )

  return Response.json({ accepted: true, path: data.path }, { status: 202 })
}

Related reading: Use RocketStack with Next.js

Next.js starter

Start from a working CSV-processing example

Clone a Next.js application with secure uploads, RocketStack Object Storage, Queue integration, and asynchronous Function processing.

File jobs should be safe to retry

Principle 1

Store before processing

Preserve the original file before starting transformations or external API calls.

Principle 2

Use a stable job identifier

Associate each processing job with the file path and a stable job or upload ID.

Principle 3

Make processing idempotent

A retried job should not create duplicate result files, notifications, or application records.

Principle 4

Separate permanent failures

Invalid files should be surfaced for review rather than retried indefinitely. After maxReceiveCount (default 10), messages move to a dead-letter queue.

Deploy the pipeline from your coding agent

Use RocketStack MCP to create the Storage, Queue, Function, and application integration behind the workflow. This workflow uses Object Storage, Queue, and Functions.

MCP deployment prompt
Add asynchronous file processing to this application using RocketStack.

Requirements:
- Accept file uploads through the existing server-side application
- Store each original file in RocketStack Object Storage
- Add a processing job to a RocketStack Queue
- Include the stored object path and job metadata in the queued payload
- Deploy a RocketStack Function to process each queued file
- Make processing safe to retry
- Store or return the processing result using supported RocketStack APIs
- Surface failures clearly
- Keep the RocketStack API key in server-side environment variables

Track every file from upload to result

Build this workflow on the Free plan

Every RocketStack primitive is available on Free with lower monthly usage allowances.

  • Object Storage0.5 GB
  • Queue messages500 / mo
  • Function invocations5,000 / mo

PRIVATE ALPHA

Build this workflow with the RocketStack team

Bring a real application, complete the setup during a guided session, and help shape the product.

Apply for the alpha

FAQ

Move file processing out of your application requests

Create a free account and deploy the Object Storage, Queue, and Function behind your first processing workflow.