Wroclove.rb 2019 highlights
Our team couldn’t miss Wroclove.rb conference in March. As usual, great community, talks, and the atmosphere. We took a lot of notes for the rest of the team but we think they are worth sharing in the blog. Look at some of our highlights.
Building UIs for Microservices
It was a talk in which Ethan Garofolo told that microservice architecture is not about deploying parts of our apps on different servers but about connections between those parts. Sometimes people just deploy parts of an app on different servers but those parts are still highly coupled together and know a lot about each other. That is not microservices but distributed monolith.
When we really have pretty independent microservices, we get new UI challenges. For example, we have an architecture with separated read and write models (or separate services). Sometimes we can’t amend read model immediately, and updating takes time when we create an entity in such a system
We should be aware of such things in our system and use special UI patterns for this. For example:
- A user enters invalid data. Our app accepts it immediately and returns an acknowledgment to the client. Then backend validates data in background and client polls for the result.
- Sometimes it’s ok to just show a loader.
We should think about UI at the very beginning or it will be too late to modify.
Handling file uploads for a modern developer
- modular design,
- plugin system,
- direct and resumable uploads,
- and probably more.
It is recommended to use it with Uppy 1, frontend lib for uploading file which is customizable and feature-rich.
Development with Axioms
Developers don’t like to argue about simple things. They rather want to work on complex problems and fix real issues. So the team of developers decided to create a list of axioms everybody in the team agrees on which should help them avoid unnecessary discussions.
Axiom is a statement everybody agrees on. For example:
- Downtime has no value.
- Redundancy provides no value.
- And others.
Then, based on their axioms they created assumptions and rules. And one of the rules was about priorities of operations they apply to code:
Remove > Fix > Refactor > Change > Add
Since “redundancy provides no value” when we have something we could delete we should delete it first, etc. And use those “remove”, “fix”, etc. verbs in commit messages and arrange their commits in a pull request in order of priority (remove things before refactoring, refactor before adding).
Having these axioms they may discuss fewer things in PRs.
I was inspired by how they communicated internally and deal with human problems. Thanks, Martin Gamsjaeger for the talk!
Autoloading is unpredictable, dangerous, and can’t be implemented right.
- You load module A.
- You load module B that depends on module A current state.
- You change module A and module B may become invalid.
Given that Rails reloads modules on change you can never tell which module is loaded first.
Also, since we have autoloading only in development we kind of develop our code in one environment but users execute it in a different one on a server in production. Which may be inconsistent.
One of the proposed solutions is to have a file where you list all your constants in particular order so Rails load it in this order.
Optimistic UI & Live updates with Logux
Even in the modern world, people get into a situation when they have either no internet connection or a very poor one. For example, we have limited connection in the subway.
Our usual approach is to either do nothing when we have no internet connection (page stuck) or to show a loader animation. Nobody likes loader animations and nobody likes to wait. Imagine you would see a spinner every time you click on like button in Twitter instead of a beautiful fireworks animation.
We could solve this by creating an optimistic UI: we apply changes on frontend first and wait until they sync with server. And there are a number of strategies on how to deal with conflicts (two users liked a tweet in offline at the same time) and inconsistency.
Logux gives us some tools to make optimistic UIs faster. It may send all the events we have in redux on the frontend to the server and vice-versa.
Towards the post-framework future
Frameworks make our life much easier. Unfortunately, often it is easier only for the first few months.
Victor Shepelev draws an analogy with industrial approaches:
- Industrial (framework): make dull and hard things cheap by making them similar.
- Post-industrial (post-framework): make custom things affordable by using modern tools and materials.
Frameworks are not the only choice. We could pick the right tools for the work and build our apps using different heterogeneous tools. It is totally ok to get Rails and use ROM instead of ActiveRecord, or cells instead of rails views.
Orchestrating video transcoding in Ruby: A story
A great story about building a service that transcodes a lot of videos. If you’re interested in video processing, I highly recommend watching the video. Here’s a list of tips I remember:
- Encode uploaded video to lower quality first because it would be quick and users will see it in UI quickly.
- AWS lambda is not the right place to do transcoding if you have a lot of videos uploaded to your site because it would cost a lot of money.
- Optimize video for streaming during encoding:
ffmpeg -movflags + faststart.
- If you allow processing remote files (user provided an URL of video) consider downloading a file first on your server and then transcode. It may be 50x faster because of reasons.
- FFmpeg has options presets, use them.
- Collect metadata of initial and converted video files. This will help to debug things.
ffprobegives you video file metadata.
The TruffleRuby Compilation Pipeline
They wrote compiler of ruby code to GraalVM and it works well. But our ruby programs consist not only with ruby code but also with C code from C extensions. That is the usual challenge for alternative ruby implementations. JRuby can’t work with C extensions so people usually create alternative gems (jdbc-postgres instead of pg).
In TruffleRuby they use LLVM to compile C extensions to GraalVM bytecode:
So TruffleRuby is fast they say.
I’ve just described this and realized that we probably have LLVM backend for JVM so we can use C extensions with JRuby the similar way.
Events. Events. Events!
It is a good and smooth introduction to Event Sourcing. The speaker Anton Davydov suggests reading “Designing Data-Intensive Applications” for more info on this topic.
How to hijack, proxy and smuggle sockets with Rack/Ruby
It is possible to make our browser to listen to a TCP socket using an extension (and a native binary but soon only extension will be required). It is possible to open a TCP connection from a browser to our server which will be able to send any arbitrary TCP traffic (not only http).
So we can make an incredible setup:
- Our browser listens for traffic on localhost:1234.
- ssh or vnc client connects to localhost:1234.
- Browser then redirects all this traffic to our server.
- Our server (in ruby) keeps connection with our browser and another server.
- Our server get traffic (ssh or vnc or any) from browser and proxies it to another server and vice versa.
It is easy to spawn a thread per connection and do the job but this doesn’t scale. The talk shows us how to make it performant enough for production by implementing some kind of an event loop and using specific operating system features such as EPOLL syscalls.
Dávid Halász is a developer at ManageIQ. They build a tool for developers and DevOps to manage their infrastructure. Sometimes it happens that a DevOps wants to access their server via vnc. ManageIQ server has access to vnc, but DevOps doesn’t. So we implement a proxy in ManageIQ which would proxy traffic between DevOps and their server. But ManageIQ is rails or rack-based app, and it accepts only HTTP queries, what can we do? Well, we make DevOps’ browser to connect to ManageIQ via HTTP and establish a long living connection. Then we make this connection to be able to send arbitrary TCP traffic so now DevOps’ browser can send any data to ManageIQ server. How to make it send vnc traffic? Well, we make DevOps’ browser to open a server at localhost:1234 so vnc client connects to this localhost:1234 and then the browser sends traffic to ManageIQ, and it sends it to vnc server.
As I understood nothing during the talk, I had to drink a couple of beers with David at the afterparty to finally make it clear.
There were a couple of SonicPi sessions during lightning talks. Check the video how it works (not from wroclove.rb but describes what Sonic Pi is):
The photos and speaker’s talks have taken from an official wroclove.rb twitter account.
Many thanks to organizers of the event and see you on Wroclove.rb 2020!