(old) tX flow path

jag3773 · June 26, 2018, 7:49pm

Contributed by Rich, 2017-06-22

THREAD 1

Commit is made on git.door43.org (Gogs)
Gogs triggers webhook and sends commit payload to client at https://api.door43.org/client/webhook (Lambda)

Client downloads zip from Gogs, unpacks it, and loads the manifest from the repo
The manifest and details about this project is saved to the tx-manifest table (DynamoDB)
Using the preprocessors, it massages data according to the resource identifier, and zips modified files in to a zip file and uploads it to the tx-webhook-client bucket (S3)
Client creates a payload with the new job’s info including URL to the ZIP file and its own Gogs user_token and sends payload to tX manager @ https://api.door43.org/tx/job (Lambda)

tX Manager sets up the job for conversion by saving the job data to the database with a status of ‘requested’ (DynamoDB)
tX Manager immediately after inserting job to DB returns information about the job to the client in the API response

Client receives the job status JSON array from the Manager and returns it in the API response to Gogs

(END THREAD 1)

When job is saved to the tx-job table, triggers the Start Job function of tX Manager (Lambda)

If the job is of status ‘requested’, tX Manager determines if a converter can convert this job, sets the status to ‘started’ in the tx-job table (DynamoDB), and calls the converter’s function (Lambda)
Converter downloads the ZIP from ‘source’, converts the files in alphabetical order, zips up converted files, uploads to the ‘output’ location given by tX Manager, and returns status, log, warnings and errors in payload to tX Manager
tX Manager saves the result to tx-job table with a status of ‘finished’ (DynamoDB)
If the job request has a ‘callback’ URL, tX Manager calls that with the job info and results, The client above will give it https://api.door43.org/client/callback (Lambda)

Callback function, if conversion was a success, unpacks the ‘output’ ZIP into cdn.door43.org/u/<owner_of_repo>/<repo_name>/<commit_10_chars> and puts the info about the job in build_log.json in the same directory, as well as the manifest.json file of the repo if there is one (S3)

(END THREAD 2)

When a build_log.json is saved to the cdn.door43.org S3 bucket (both when the client gets a request and gets a callback, so twice per conversion request) it triggers the Door43 Deploy function (Lambda)

Door43 Deploy function templates the raw converted files in the cdn.door43.org S3 bucket based on resource_type and uploads those files, the build_log.json, the manifest.json and a project.json of all project commits to door43.org/u/<owner_of_repo>/<repo_name>/<commit_id> (S3)

(END THREAD 3)