In Beta, contributing an arcade photo meant attaching it to an email. An operator read the message, downloaded the attachments and registered them in the service. The gallery could display photographs, but users still had no upload route of their own. Delta introduced direct uploads on the site and moved the existing email material into the new storage arrangement.

Email submissions and storage capacity

A file picker makes uploading look straightforward. Photographs, however, remain after the request finishes. Each submission consumes storage, and each gallery visit generates traffic. Before opening uploads, we needed somewhere to keep an accumulating collection and a way to serve it. Preparing S3 meant learning both how to connect the storage service and how to operate a large file collection.

S3 stores files as objects, allowing the service to use external storage rather than continually filling its server disk. Moving the bytes is only part of the work. With email, an operator knew which arcade a photograph described and who had sent it. The server now needed to retain those relationships explicitly.

flowchart TD
    accTitle: Moving from email to site uploads
    accDescr: Operators previously registered email attachments manually. Direct uploads let the server check files and connect storage to document records.
    U[User photos] --> M[Email submission]
    M --> O[Operator checks and manual registration]
    U --> W[Direct site upload]
    W --> B[Server file checks and record creation]
    B --> S[S3 file storage]
    B --> R[Arcade and author relationship]

Photo uploads and S3 storage

S3 setup begins with a storage container called a bucket. Its name, region, endpoint and credentials go into PocketBase’s file-storage settings. Once enabled, PocketBase’s filesystem uses that configuration to store the files. Requests still pass through the same file handling used for local storage, so the interface does not need bucket addresses or secret keys.

When a user chooses photographs, the browser sends the arcade ID and files to the server in a multipart request. Multipart carries ordinary form values and file content together. The server checks the content and saves file fields on photo records; the configured PocketBase storage handles the bytes. We did not send S3 credentials to the browser to let it operate the bucket directly.

After configuring storage, verification needs to cover both uploading a file and opening it again on an arcade page. A filename in the database must match the actual object location, or a completed upload still will not appear in the gallery. Photo storage and database backups also need their own configuration: being able to store images does not establish a database backup.

Publishing uploaded photos

Consider uploading several photos together. Some may save while another fails. The upload API, /arcade/photo/upload, creates an arcade_photo_atoms record for each successful file, links it to the arcade and author, and initially keeps it private. The response returns successful record IDs separately from failed files so the user can see what actually happened.

After transferring the files, the user chooses the list for the document. /arcade/photo checks that the selected photos belong to that arcade and makes them public. It creates an arcade_photo record containing their IDs and updates the current photo reference and history together. A database transaction commits this selection without leaving the document with a partly updated list. These two stages also allow the user to choose which uploaded photos to publish.

sequenceDiagram
    accTitle: File storage and document publication in two steps
    accDescr: Uploads create private photo records. Saving the list checks venue membership and commits the public list and current reference.
    participant U as User
    participant B as Server
    participant S as File storage
    participant D as Database
    U->>B: Upload photo files
    B->>S: Store files
    B->>D: Create private photo records
    B-->>U: Successful IDs and failed items
    U->>B: Save selected photo IDs
    B->>D: Check venue and update public list reference and history
    B-->>U: Document save result

Migrating email photos and their provenance

Introducing uploads did not remove the need for earlier email contributions. A message contained the target arcade, contributor and receipt time as well as its files. Copying only the photographs would lose that context; copying only the message details would leave no image for the gallery.

Mail AppleScript extracted the relevant attachments and message details. ArcadeID linked the place, UID the contributor and Received_at the original receipt time. The migration created individual photo records and a group of their IDs, then prepared files under the collection-ID, photo-record-ID and filename layout used to connect S3 objects. Keeping the receipt time prevented old contributions from appearing newly submitted on migration day.

Reprocessing the same message could otherwise register duplicate photos. The importer compares arcade, receipt time, contributor and filename list with existing material to identify repeated submissions. That comparison finds previously imported data; it cannot establish that the image depicts the right arcade or has the right contributor. Those links still need checking against the original email.

flowchart TD
    accTitle: Migrate files together with their provenance
    accDescr: Extracted email files connect to stored objects while service records retain the place contributor and original receipt time.
    M[Email attachments and submission context] --> E[AppleScript extraction]
    E --> F[Prepare files and storage paths]
    E --> I[Place contributor and receipt time]
    F --> S[S3 objects]
    I --> R[Photo records and photo group]
    R --> S

Direct uploads reduced the work of reopening every message and registering its files. They also allow a state in which a file is stored but the public list has not been saved, so storage and document results need separate handling. Even an existing file can appear in the wrong gallery if its place relationship is wrong. Providing S3 therefore still leaves ongoing work to manage photo ownership and publication.

Reference: Musecat engineering notes.