LBRY on Kubernetes with Helm

[EnigmaCurry]

This is a working Helm chart and HOWTO for running lbrycrd, chainquery, lbrynet, and spee.ch on Kubernetes, using the containers from this repo.

README

[EnigmaCurry]

This requires changes to the lbrycrd image outlined in #70

[Leopere]

Would you be interested in showing a reasonable platform agnostic playthrough of this guide via a video similar to your Cloud-init?

[Leopere]

There's quite a lot to review here so I would like to sit down with you in slack today and look over its functionality and give it a run through test. Maybe record a video for embedding into the tutorial.

[EnigmaCurry]

I want to refine this process a bit with a few more goals:

• Don't install tiller on the cluster - https://rimusz.net/tillerless-helm
• Use namespaces by default
• Use a Makefile or frontend code to encapsulate helm and kubectl commands to make this simpler.
• Solve the data sharing problem between spee.ch and lbrynet. (share a single volume between containers? Stream results from lbrynet into spee.ch, for spee.ch to store and cache in it's own volume?)

[Leopere]

@EnigmaCurry I would say a shared volume is the solution for shared data between spee.ch and lbrynet.

The makefile could be a really good option however BASH is generally more available than Make is it possible to just use a bash script?

[EnigmaCurry]

I've updated my k8s-lbry branch with the new additions outlined above.

• There is a new run.sh wrapper script
• Tiller is run locally from run.sh and no longer installed on the cluster
• Everything installed to the k8s-lbry namespace
• Speech includes its own lbrynet sidecar container (and mounts a shared data dir between the two)

Right now I'm building my own spee.ch container from https://gist.github.com/EnigmaCurry/57e9773d9b3748b248a52fcebd447fba

Things to do before merge:

• Finish production spee.ch container
• Update image references in values.yaml
• Update HELM_REPO in run.sh to lbryio official repository.
• Publish production lbrynet v0.33.0 as this is the version that works with spee.ch right now
• Figure out why uploads don't work in spee.ch

[EnigmaCurry]

• Spee.ch upstream has been upgraded to SDK 0.37.2, speech-lbrynet container updated to new version.
• Made PR Multi-stage Dockerfile for spee.ch #74 for compatible spee.ch dockerfile
• Made PR lbrynet-multiarch-compiler steps require TRAVIS_COMMIT env var. #73 for building new lbrynet container. This needs to run on 0.37.2.
• Anonymous publishing is working, there is an error in the logs though:

2019-05-30T21:16:20.981Z - error: Error on /api/file/availability/479364455881011fc6306o2600/86e53ddc594fcc13e558dc0c1f94b31f4ee1fb99 
{ stack: 'TypeError: Cannot read property \'dataValues\' of null\n    at /spee.ch/server/bundle/server.js:1:638181\n    at <anonymous>',
  message: 'Cannot read property \'dataValues\' of null' }

• Once the claim is accepted onto the blockchain, the image resolves correctly.
• Test page (temporarily online only) : https://speech.rymcg.tech/8/479364455881011fc6306o2600
• Publishing within a channel is not working, with an error message of no channel password provided, although logging into the channel appears to work, and I do see the password in the log when the account was created.
• run.sh still points to my forked helm chart values, this still needs to be switched before merged.
• values.yaml contains docker images for my own repository that needs to be switched once we have spee.ch and lbrynet (v0.37.2) production images.
• I added the asciicast directly to the contrib/k8s-lbry/kick-ascii directory. Turn on github pages in the repository settings for the master branch.
• There is a helm repository created on S3 that is being used currently, this could be migrated to github pages as well. That would mean mainting a full history of tarballed helm chart releases in contrib/k8s-lbry/repository. Alternatively, we could just keep it as S3, and host it elsewhere. I think the latter option is a bit cleaner.

[Leopere]

Looks like the Readme is 97% of the way there let us get it the last 3% and I'll push this into the repository!

[Leopere]

Just waiting on @tiger5226 to enable the github pages for this repo so I can rebase and merge this at this point.

[tiger5226]

its enabled