Write it simple

About side projects

2024-03-03T00:00:00+00:00

There is something about writing code for me and no one else.

It just feels different, it feels great.

Handcrafting a tool, that serves me exactly how I want.

It may be similar as composing a song.

For the past few weeks I’ve been working on a side project to generate slides from markdown. To be honest, I did nothing I just added a UI to existing tools I’ve been using. I’ve been teching Rails for a few years now. Two years ago I decided to revamp my slides and gave reveal.js a try. I fell in love.

Reveal.js is an amazing tool to create slides—not only from markdown. It has lots of configurations available, the speaker view is just fantastic, adding notes work just fine, vertical and horizontal slides served me very well to group topics in my lectures, and given it’s just HTML everything is easy to further customize.

These slides were written in Markdown and built with reveal.js—the script used can be found here.

In addition to that, after preparing a written document for a course I was taking in the university, the teachers asked us to present the work. Given we had already written the document in markdown, we only removed little bits and used reveal.js to create a presentation. The presentation was simple and nice, everything we needed to talk about the work done.

We used Pandoc to generate the PDF first. It used the markdown as source. And turns out Pandoc also supports reveal.js as an output format. Nice.

Eventually, this side project will not only support generating slides from markdown, but also a printable PDF files or ebooks or any other format I may need.

Who knows, maybe in time it becomes a ~~side~~ main project.

For now, I’ll keep spending my days writing code and tests, but at nights only writing code.

This is the start of Only Docs

How to deploy Apache Superset with Dokku?

2023-06-28T00:00:00+00:00

Why?

Because it is much easier than doing everything manually
Configuration changes are deployed via git (git push dokku main)
Reverse proxy (nginx) is configured by Dokku (no lines of nginx config files are touched, not even looked at)
SSL is configured by a Dokku plugin (I don’t even know how to use certbot)
Ideal to spin up a Superset instance quickly to test things out

Why not?

Well, if the plan is to serve thousands of dashboards and charts for thousands of users, maybe look into the kubernetes installation.

Before we start, ACME is used as an example of a company name. Feel free to replace with your own.

In your virtual machine

1. Install dokku

Go to Dokku’s installation page and follow the steps. There’s no need to have Docker pre-installed, Dokku installer will take care of that.

Make sure to follow all steps until the end including the ones about adding your SSH key and setting the global domain. A domain is required to run Superset with SSL.

The SSH key must belong to the machine from where you intend to deploy Superset, so that git push can authenticate.

2. Create the app in Dokku

dokku apps:create acme-superset

3. Install postgres/redis

Install postgres plugin and redis plugin.

Then create the services and link them to the app.

sudo dokku plugin:install https://github.com/dokku/dokku-postgres.git postgres
sudo dokku plugin:install https://github.com/dokku/dokku-redis.git redis

dokku postgres:create acme-superset
dokku postgres:link acme-superset acme-superset

dokku redis:create acme-superset
dokku redis:link acme-superset acme-superset

This is optional, but I recommend doing it.

By default Superset uses SQLite, which will not allow us to create multiple Datasets pointing to different tables and using the same name. This is solved by using Postgres as the metastore (metastore is how Superset documentation refers to the database where Supersets objects are stored—dashboards, charts, etc).

Redis is also optional but it’s nice to have in case you want to configure data caching.

Postgres and Redis connection strings will be present as environment variables in the acme-superset application:

dokku config:show acme-superset

After running the link command you may encounter a message like App image (dokku/acme-superset:latest) not found, just ignore them.

4. Configure the default port

dokku proxy:ports-add acme-superset http:80:8088

By default Superset uses the 8088 port, but in order to properly configure SSL it is required for us to proxy the port 80.

In your local machine

1. Create a new local repository

mkdir acme-superset
cd acme-superset
git init

Create a new folder and initialize git. We’ll use this folder as the Dokku application to deploy.

2. Create a config.py

curl https://raw.githubusercontent.com/apache/superset/2.1.0/superset/config.py -o config.py

The config.py from your repo will be placed instead of the default config.py.

Grab the default config.py from github.com/apache/superset/blob/VERSION/superset/config.py.

To use the database connection string from the DATABASE_URL env var, update the config.py file as follows:

SQLALCHEMY_DATABASE_URI = os.environ.get("DATABASE_URL")

To use the redis connection string from the REDIS_URL en var, update the config.py file as follows:

# Cache for datasource metadata and query results
DATA_CACHE_CONFIG: CacheConfig = {
    "CACHE_TYPE": "RedisCache",
    "CACHE_DEFAULT_TIMEOUT": int(timedelta(hours=1).total_seconds()),
    "CACHE_KEY_PREFIX": "superset_data_cache_",
    "CACHE_REDIS_URL": os.environ.get("REDIS_URL")
}

Configure anything else as needed.

3. Create a Dockerfile

touch Dockerfile

# ./Dockerfile
FROM apache/superset:2.1.0
USER root

# Set my secret key
ENV SUPERSET_SECRET_KEY=SUPER_SECRET_KEY_PLEASE_REPLACE_ME

# Use my config
COPY config.py superset/config.py

# Add database drivers
RUN pip install psycopg2
RUN pip install sqlalchemy-bigquery

# Adds vim to be able to enter the container and read files with vim
RUN apt-get update && apt-get -y install vim

USER superset

First line allows us to select what version to use. This example uses 2.1.0, latest version as of this writing.

Add any database driver needed. This example adds the driver for Bigquery. More drivers can be found here.

The base image can be found in Docker hub.

4. Add Dokku’s remote repository

git remote add dokku dokku@dokku.acme.com:acme-superset

Where dokku.acme.com is the global domain configured for Dokku and acme-superset is the name of the application in Dokku.

5. Deploy

Commit all files and push to Dokku’s remote.

git add .
git commit -m 'Superset initial configuration'

git push dokku main

The deploy will start and you should have output in the terminal about the Dockerfile steps we defined previously.

Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 8 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 23.48 KiB | 5.87 MiB/s, done.
Total 4 (delta 0), reused 0 (delta 0), pack-reused 0
-----> Cleaning up...
-----> Building jm-superset from Dockerfile
remote: build context to Docker daemon  66.56kB
Step 1/14 : FROM apache/superset:2.1.0
...

Finally configure SSL in the Dokku machine via the letsencrypt plugin.

# This is performed in the virtual machine, where Dokku is running
sudo dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git

# Set global email for letsencrypt
dokku letsencrypt:set --global email admin@acme.com

dokku letsencrypt:enable acme-superset

That’s it, open acme-superset.dokku.acme.com in your browser.

Superset is running and you are able to change its configuration, install new drivers, change version, and pretty much do anything in a very easy manner.

Final considerations

The first time you deploy superset, you may need to create an admin user, migrate the database and run superset init.

Enter the container by running:

dokku enter acme-superset

Then run:

# Create an admin
superset fab create-admin \
  --username admin \
  --firstname Superset \
  --lastname Admin \
  --email admin@superset.com \
  --password admin

# Migrate DB
superset db upgrade

# Init and setup roles
superset init

Refer to Superset documentation for more information about its configuration.

Happy dashboarding.

I have a Superset instance running smoothly in a Digital Ocean Droplet. Get $200 in credit over 60 days to try this out.

How to find the PID of a process using a specific port

2021-12-12T00:00:00+00:00

To find out what processes are using a specific port, use lsof.

lsof -i :PORT

Examples

List all processes using the 5432 port (commonly used by postgres)

➜ lsof -i :5432
COMMAND  PID   USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
postgres 777    1ma    7u  IPv6 0x0123456789abcdef 0t0  TCP localhost:postgresql (LISTEN)
postgres 777    1ma    8u  IPv4 0x1123456789abcdef 0t0  TCP localhost:postgresql (LISTEN)

List all processes using the 4000 port (this blog in development)

➜ lsof -i :4000
COMMAND   PID   USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
ruby    59182    1ma   10u  IPv4 0x2123456789abcdef      0t0  TCP localhost:terabase (LISTEN)

Alternatives

netstat can be used with grep to look for the port, but not possible in macOS as netstat won’t output the process ID.

ps -ef can be used with grep too, but we’ll need some bash sorcery to keep the first line of the ps output (the headers line).

How to cleanse your AWS bill

2021-05-23T00:00:00+00:00

Ever wondered what a black hole looks like? My AWS Billing—or yours—might be one of the closest things to a black hole on earth. So scary, and so hard to remove things from it, firstly because we don’t know what services to remove nor the regions for those unwanted services.

Let’s focus on how to remove things from the AWS bill, which is the main task that will definitively cleanse our bill. It’s so easy to spin up new services in AWS and forget about them, some might be cheap some others might not be that cheap; and given that AWS—at the time of this writing—provides over 175 services in several regions each, it becomes quite a task to manage all of your services in all of the regions you used.

Enter the Bills page. Go to your Billing Dashboard, and then on the left sidebar, under the Billing section, click on Bills. Below AWS Service Charges you can see all of the services you have signed up for, and inside each of them, you can see the incurred costs per region. This way you just need to figure out what services are not in use, then go to their regions and remove them.

This way I was able to spot an unused RDS instance and a very old Amplify instance, both burning money with no returns.

Why avoid too many abstractions in tests?

2021-02-13T00:00:00+00:00

Intro

There’s production code, and there’s test code. Production code refers to any portion of the codebase that is executed in production environments. Similarly, test code refers to any portion of the codebase that is solely executed in testing environments, where production code is exercised via the test code.

Test code needs to be explicit, easy to read and understand what’s going on. Test abstractions, like shared examples, are usually against what we need out of test code.

Why not to DRY test code?

Refactoring production code tested with several abstractions in test code becomes harder. This is because one single test abstraction is used across several places in test code, potentially causing your changes to break some tests but not all of them. Finally, triggering a refactoring task on the test abstraction as well or maybe even updating your tests that failed to stop using the abstraction cause it no longer fits the test.

Test reports with failures are harder to read. If a test using an abstraction failed inside the abstraction then the report will show where it failed first, and that is inside the abstraction. It requires the developer to read over the stack trace to check what test called the abstraction in order to find out what test is actually failing.

This is the output of RSpec’s Shared Examples

Finished in 3.17 seconds (files took 2.49 seconds to load)
47 examples, 5 failures

Failed examples:

rspec ./spec/requests/api/v1/users_spec.rb[1:2:2:2:1:1:1] # Api::V1::Users PATCH #update behaves like ...
rspec ./spec/requests/api/v1/users_spec.rb[1:2:4:1:1:1] # Api::V1::Users PATCH #reset_change_email behaves like ...
rspec ./spec/requests/api/v1/users_spec.rb[1:2:5:1:1:1:1] # Api::V1::Users PATCH #request_remove_user behaves like ...
rspec ./spec/requests/api/v1/users_spec.rb[1:2:1:1:1:1] # Api::V1::Users GET #show behaves like ...
rspec ./spec/requests/api/v1/users_spec.rb[1:2:3:1:1:1] # Api::V1::Users PATCH #reset_api_key behaves like ...

_{beer’s on me if you can tell where the failure occurred}

Don’t get me wrong

Too many abstractions for our test code is like highlighting every single word of the book you’re reading. You’d only want to highlight the most important parts right?

With test code is quite similar, we don’t want to abstract everything but we’d rather keep test code as explicit and readable as possible and at the same time provide abstractions for the parts that are not really meaningful for test cases. For instance, the factory pattern to create objects for test code is perfect – re: FactoryBot. Custom matchers are also a valid way to introduce abstractions while keeping our test code explicit and declarative.

To sum up, abstractions for non meaningful parts of test code are valid and really useful. Abstractions for test code that describes actual behavior of the system are never[*] good.

_{* I wrote a shared example last month, out of a code review suggestion, and I can live with it}

I don’t use nil

2020-10-12T00:00:00+00:00

I don’t use nil, or at least I try not to.

NoMethodError: undefined method 'some_method' for nil:NilClass

This is one of the most common errors seen on production systems. And it’s likely to happen everywhere where a variable can reference to a null value. Here are some case studies about the topic.

Migrations

When creating/updating a table I always consider adding a null: false constraint and a default value accordingly. For string values, the default would be an empty string. For number values, the default value would be zero.

This allows us to do operations with those columns without worrying about them being nil, because we took care of that at the database level. For instance, let’s consider a Post model in a blogging system:

# db/migrate/...
class CreatePosts < ActiveRecord::Migration[6.0]
  def change
    create_table :posts do |t|
      # [snip]

      # not bad
      t.string :excerpt

      # better
      t.string :excerpt, null: false, default: ''
    end
  end
end

# app/views/posts/show.html.erb
<p>
  <%= @post.excerpt.downcase %>
</p>

In this particular example, sending downcase to Post#excerpt will always work because we ensured that the value will always return a String object.

Enums

Let’s consider that any post can have a certain category. Since these categories won’t have any extra info nor behavior, we’ll use an enum attribute in the post model.

# db/migrate/...
class CreatePosts < ActiveRecord::Migration[6.0]
  def change
    create_table :posts do |t|
      # [snip]

      # not bad
      t.integer :category

      # better
      t.integer :category, null: false, default: 0
    end
  end
end

We can easily add a default enum option to represent a post without a category. And why is it better? For the simple reason that the Post#category method will always return the same object type. This will cause the code to avoid checking for nil when working with the category, and will also make the safe navigational operator useless when working with this attribute.

# app/models/post.rb
class Post < ApplicationRecord
  enum category: { general: 0, lifestlye: 1, art: 2, misc: 3 }
end

In this example we are using the category named general as the default one.

NullObject

This approach is an elegant way to represent a null value. It is somewhat similar to the enum’s case study. Here’s an excellent article from thoughtbot about this pattern.

undefined method ‘conclusion’ for nil:NilClass

To sum up, I don’t use nil because I don’t want the program to potentially raise this NoMethodError. To avoid it, it’s just as easy as not handling null values in the codebase.

And this is Tony Hoare, the inventor of nil:

I call it my billion-dollar mistake. It was the invention of the null reference in 1965. At that time, I was designing the first comprehensive type system for references in an object oriented language (ALGOL W). My goal was to ensure that all use of references should be absolutely safe, with checking performed automatically by the compiler. But I couldn’t resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years. src

When objects become super objects

2020-08-11T00:00:00+00:00

A pencil. We use a pencil to write. And that’s it. (Let’s forget about some pencils that come with an eraser in one end, in this world they don’t exist)

We are in the “digital transformation” era, right? So I came up with the idea of including a simple scanner in the body of the pencil, to be able to digitalize everything written with the pencil. It’s really handy!

I always struggled to share my technical drawings when exchanging thoughts in groups, so I think a mini projector on top of the pencil would be a great addition. It’s amazing, I can show my drawings directly in the wall, no need for any external projector!

Oh, and a laser pointer on it. You know, everyone loves playing with lasers.

We call it The Pencil 3000 ™

Let’s stop here. I’ve experienced this very same line of thinking when creating/updating classes.

Multiple responsibility classes: Real life examples

These are the places where I’ve encountered myself working with this approach:

CTMPS means “comment to my past self”.

1. Models

Several times I wrote methods in the model that were only being used in the views. CTMPS: It’d be better to use a presenter object for those.

Numerous times I found myself writing callback methods in the model, with the purpose of sending emails or adding default values in the model. CTMPS: Sending emails seems like a side effect of a user action, why don’t trigger them in the controller instead? And about setting default values, what about setting them using the ActiveRecord::Attributes API?

Don’t get me started with inline validation methods. This I one I abused. I’m sorry. CTMPS: There are no taxes for new classes, create a validator class and be done with it. Peace out.

2. Form objects

Select tags with options that are not DB backed. Guilty. I more than once wrote a method in the form object to provide the options to be used in the select tag, in the view. CTMPS: It’s ok no big deal, but what about using a presenter object for this? Given it’s a view-only matter.

At this point it feels like I’m seeking absolution from my sins. Hopefully you haven’t experienced any of these. But in any case, let me share you one more example.

3. Page objects

Testing usually requires 4 steps: setup, exercise, assertions and teardown. And when testing web applications, end users interact with HTML documents. Enter page objects. So we use page objects to wrap the HTML and provide a meaningful API to be used in test code. This includes the exercise and assertion steps of testing. But I gotta be honest, it is so easy and pleasant to just write a new method into the page object to do the setup and create instances, update stuff, set up environment variables and whatever needed for our tests to run. CTMPS: It’s as easy and pleasant to extract the creational part into a support module in your specs, and keep your page objects fulfilling their true purpose.

Remember The Pencil 3000™? I can’t imagine how would I sharp the pencil, with all that stuff around it.

Single-responsibility principle

“[…] each software module should have one and only one reason to change.” Better read it from here

Use constants to specify the version of your gem

2020-05-04T00:00:00+00:00

When developing gems or external libraries we frequently write the semantic versioning as a string.

module MyGem
  VERSION = '1.4.0'
end

And when releasing a new version, git diff will show us this line was modified.

Why not use a constant for each semver element instead?

Rewriting the previous version like this:

module MyGem
  MAJOR = 1
  MINOR = 4
  PATCH = 0
  VERSION = [MAJOR, MINOR, PATCH].join('.')
end

It will allow us to keep our git diff clean when releasing, and also make our version string less prone to typos.

Use error objects instead of strings

2020-05-03T00:00:00+00:00

Every once in a while we need to raise an error so that our program exits and displays the error in the standard error. And most of the time (seen it several times while code reviewing) we tend to write a lovely: raise "some error explanation"

Why not use an object instead of a string?

When we raise errors using objects we’ll be able to easily identify those errors, everywhere (logs, console, when rescuing). Take a look at this example:

[1] pry(main)> raise "some error"
RuntimeError: some error

It raised a RuntimeError which is an error that can occur not only here.

[2] pry(main)> class ThisErrorIKnow < StandardError; end;
[3] pry(main)> raise ThisErrorIKnow, "some error explanation"
ThisErrorIKnow: some error explanation

Here it raised our custom error class which in my opinion is better because we’ll be able to rescue this error specifically later if needed.

Do you use form objects in your Rails apps?

2020-04-10T21:00:00+00:00

Introducing yaaf, a gem to ease the usage of the form object pattern in rails apps.

My pain points (before yaaf)

The form object pattern is widely used across Rails apps, and yet we tend to write these form objects in a different manner from project to project.

Most of the time, they don’t even have the same interface across form objects in the same app.

We also tend to write validations twice, in the model and in the form object, just so that the form object has the errors and the view can display them.

Well, I can say that most of our in-house form objects make good use of database transactions. But I bet we rarely provide the controller the ability to use a banged method, expecting it to raise an error if things go south.

Business logic is present everywhere in your app, controllers, models, helpers (some hardcore scenarios might include Rails initializers), but is that right? Many times we find ourselves with the need to add business logic around the creation of an object, such as sending an email, updating other records or even calling an external service. We know that the controller is not a good place, and the model would be terrible as well because it violates SRP. So we end up creating several service objects and throwing the business logic there as if it were a trashcan.

Instead of all of that, we can…

use yaaf, it solves all of that in an easy way, let me explain:

Same interface? yaaf will encourage you to use save and just save to, well you know, SAVE the data from your form into models.

Contextual validations only? Please put them in your form objects, and keep data integrity validations where they belong, in your service objects, haha no, of course no, in your models! yaaf will take the liberty to promote all model errors collections into the form object errors collection, so you can show them in your views

So, one of the form objects need to raise an error when not saved? As you may have guessed already, use the dangerous save! method. yaaf already defines it so that it raises an error just as any Active Record model does. Of course, it will also trigger a database rollback when the models couldn’t be saved. Both save and save! methods are available.

Not sure where to put business logic related to the creation of an object? Well, the form object is the place. yaaf will allow you to use callbacks the same way as your model callbacks work. For instance, if you want to send an email after the form has been submitted and persisted, the after_commit callback is the one you’re looking for. More info can be found in the readme.

The Bob Ross of form objects

Using yaaf will allow you to standardize your form object definitions across your project’s production code.

Why not use this?

If you would rather have controllers deal with all these responsibilities, then it’s fine to keep doing it, you may not need yaaf if that’s the case.

Are you a service-objects-for-all person? Then you might feel better writing service objects rather than form objects.

If you work with Java, well you don’t want to use this.

A bit of history

We have been using this very same approach on a production app for almost a year now, more than 10 form objects written this way, and it has served us well. That’s why we decided to extract it to a gem. (yaaf is no more than 64 lines currently, and we need no more)

Big thanks to @santib for shaping up this gem from its very beginning.

The lovely branding from yaaf was done by Sofi Salazar

https://github.com/rootstrap/yaaf