Blog

All posts are my own opinion and do not represent any organization I am affiliated with.

The Next Generation of Web Is Here

WASM and WebGL are here, and they power the generation of the web. Web Assembly enables native performance applications with the distribution of a website. Web Assembly or WASM is a way to compile C++, C, Rust, and other programs into a format that can run in the web browser. WebGL is a graphics library that allows you to use the graphics card to have much more power and control over what is rendered on the screen than the standard web languages HTML, JS, and CSS. With WebGL, you can control how every frame and every pixel is drawn. Combining the UI and UX control of WebGL and the performance of WASM enables a new generation of web applications.

Few people are talking about this today, even though WASM + WebGL applications are already disrupting big business. For example, Adobe. A native performance application is required to edit photos and create graphics. Now with WASM, you can get native performance in the browser, and with WebGL, you can get the pixel-perfect controls that users need in a graphics editing program. If you've used Figma, you have used a WASM + WebGL application! Figma is disrupting Adobe because their application is effortless to start using. You go to their website, and it loads in less than 3 seconds. That's it. Adobe Creative Cloud takes over an hour to install. With Figma, you can share and collaborate on your work with the full power of the application. Previously people exported graphics from Adobe applications to show them to coworkers, now, they can share them in the application, and anyone can make changes. Being based on the web brings the whole team into an application previously only used by designers and illustrators. This team collaboration is enabled by the application being a website, and the performance is enabled by Web Assembly.

The magical part of WASM + WebGL is that there is nothing to install or configure, and the user doesn't even know it's being used. As of November 2017, WASM is supported in all major browsers, including first-class support on iPhone and Android. WASM runs everywhere. For the first time, you can build a native performance application that runs on every platform by default. This is a step forward from the Web2 world of HTML, JS, and CSS. Performance + distribution is why I argue that WASM and WebGL are the next generation of the web.

Let me prove to you that WASM completely changes the game. Here is a full Windows 98 emulator running in the browser. Yes, that is an entire operating system running in the web browser. It can even load already booted up because WASM is a consistent build target across all browsers and systems its behavior is also consistent across all platforms. This is a huge step forward from Web2 and enables native code to run consistently on all platforms. This changes the game for both Native Applications and websites by blurring the lines and allowing developers to combine the strength of both. Performance + distribution. 

WASM + WebGL has already significantly disrupted graphics and photo editing, so what's next? Applications where performance and user interface are tightly tied together that are critical to the usability of the application will be disrupted by WASM. It's my prediction this will have a massive impact on the gaming industry, as well as video editing. The native performance and instant wide distribution of WASM + WebGL will also disrupt data science. Projects such as duck-db are showing great traction in this space, they have built a performant in-memory database that runs seamlessly in WASM.

This is why I started a data science company rethinking spreadsheets. Our product is called Quadratic. We are building the first WASM + WebGL spreadsheet application. Quadratic runs entirely in the browser and can load millions of rows of data in seconds. You can use data science languages such as Python and SQL, all directly in cells. You can freely pan and zoom to navigate large data sets (like Figma). Our grids are as easy to use as Google Sheets and are as powerful as your computer running Excel, Python, and SQL natively. Performance + distribution. It would not have been possible to build Quadratic without WASM or WebGL.

PS We are hiring developers. Check out our careers page at https://careers.quadratic.to

End to End Testing for Expo Apps With CircleCI & Detox

It is becoming increasingly common for developers to begin to develop their apps with Expo with React Native. Expo is a great developer experience for getting started. However, some more advanced workflows for publishing production applications do not come out of the box. One of the missing workflows is end-to-end testing. So we set out to develop a method to run tests on our expo app using Continous Integration. Here is the configuration we came up with to successfully use CircleCI to test our Expo app using the testing framework Detox + Jest.

Below is our config.yml file. Our process works by performing the following steps:

  1. Install application dependencies.
  2. Log into the ExpoCLI using env vars (you have to set these in the CircleCI UI).
  3. Instruct Expo to do an iOS build with the target simulator.
  4. Download the build from Expo and extract it.
  5. Install testing packages.
  6. Run tests using detox test.

config.yml

Here is a copy of our .detoxrc.json file

And here is our testing/config.json file

Our test runner takes about 10 minutes to set up and run a set of ~10 tests. The test setup is probably partially redundant and could be sped up. However, this works for us today! We hope this is helpful for you in setting up Expo End to End Testing!

Django + Postgres Views

Over the years, the Django ORM has become my go-to way to interact with a database from Python. It is an incredibly robust binding between the database and Python. As Postgres added advanced database features, the Django ORM kept up. An early example of this is Django's implementation of JSONB fields. The JSONB fields allowed you to save JSON as a field and interact with the JSON object's contents from the SQL query level. Django quickly supported this powerful feature. Awesome.

Another powerful feature added Postgres added is Views. Views are a simple database feature that allows you to create an alias for a common query. Suppose you were routinely running a SELECT statement on a table of transactions for the category outdoor_products:

SELECT * FROM tranactions_table WHERE product_type='outdoor'

In Postgres, you can save this query as a view for future reference.

CREATE OR REPLACE VIEW 'outdoor_product_transactions_view' AS 
(SELECT * FROM tranactions_table WHERE product_type='outdoor')
Then you can just
SELECT * FROM outdoor_product_transactions_view
And easily apply additional filters
SELECT * FROM outdoor_product_transactions_view WHERE amount > 1000

This feature is great for complex queries you are regularly repeating.

In the application I am currently building, there are complex queries that I run regularly on our API. I would also like our BI tools to benefit from these complex queries without repeating the logic in SQL. So I set out to create an easy way for a Django queryset to manage a Postgres view.

Here is the result:

create_or_replace_view.py

from django.db import connection


def create_or_replace_view(view_name, view_qs):
    with connection.cursor() as cursor:
        queryset = view_qs
        compiler = queryset.query.get_compiler(using=queryset.db)
        sql, params = compiler.as_sql()
        sql = "CREATE OR REPLACE VIEW {view} AS {sql}".format(
            view=connection.ops.quote_name(view_name), sql=sql
        )
        cursor.execute(sql, params)

migrate_views.py

from postgresviews.create_or_replace_view import create_or_replace_view
from ledger.querysets.formattedTransactionsQS import FormattedTransactionsQS


## Define Views Here
## NOTE: Renamed or Deleted Views will not automatically clean up the old version.
create_or_replace_view(
    "view_formatted_transactions", FormattedTransactionsQS().get_qs()
)

migrate_views.py is run on every deployment of our Django API. This allows me to use complex querysets with our API via Django Rest Framework and build a Postgres view of the queryset available to our BI tools. Magic! I hope you find this useful in your own applications!

How To Level A 4Runner Trunk For Less Than $40

I love to sleep in my car. It's the most convenient way to spend a night or two away from home, often far away from civilization, often biking or skiing.

With my Subaru Outback, I could simply put the seats down and sleep on a foam pad. Easy! However, I just switched to a Toyota 4Runner and there was a problem. The bed isn't flat! 

4Runners (gen5) have this nasty angled part in the middle, with a 1.5" difference between the trunk section and where the seats fold down. So I set out to make it flat.

My Design Criteria

  • Cheep. This can be done for less than $40.
  • Easy to make, easy to remove.
  • No modifications to the vehicle itself.
  • Can be done without access to a workshop.

Here is the result!

It takes up only 1.5" of vertical trunk space and doesn't need to be adjusted when you put your seats up.

How To Build It

What's needed:

  • 4x8' Plywood Sheet 3/4 inch thick. $20
  • Outdoor Carpet (or rubber mat if you don't have a staple gun) $15
  • Staple Gun (optional)

Step 1: Cut The Plywood

This design takes a single sheet of plywood (4'x8'x3/4") and cuts it into 2 sheets which are stacked on top of each other to make a 1.5" tall surface. Because the shape we want to fill has an angle on one side, the top piece is cut 2" inches longer to fill the angled gap. See the images below.

How it sits in the bed:

This only takes 3 cuts to complete, and your local Home Depot or Lowes will do it for you if you ask nicely!

Step 2: Secure The Top and Bottom Sheets Together

I simply screwed them together with a few 1 1/4" screws. 

Step 3: Carpet The Top Layer

I laid the structure upside down on top of an outdoor carpet. Pulled the carpet around all sides to the back, and stapled it in place. This way all the seams and staples are on the bottom.

(If you don't have a staple gun, and don't care about seeing the plywood when your trunk is open. simply put a rubber entryway mat over your sheets and cut it to size, Lowes sells 4'x6' foot mats which are large enough for $19.)

Step 4: Slide It In Place

Done!

I wanted to share this design because it is the simplest and least expensive one I could find, which doesn't really require any tools. Let me know if you build one!

Cherish Every Moment

At this moment, I feel stressed.

I feel pulled in 1000 directions.

I feel like I am doing too much; I feel like I'm not nearly doing enough.

I love the people I get to spend my days working with. 

I love what we are working on.

I love the person (and dogs) I get to go home to tonight.

I love the direction of our work and our lives. 

But, I feel lost in the moment.

A few days ago, I asked some friends the question, "what is missing from your life?"

I was delighted to hear their answer was, "nothing."

And I realized my own answer was "nothing."

I know at almost every other time in my life, I would have had a different answer.

Today I don't.

So why the feeling of loss?

Being grateful is a choice. I was choosing to experience the problems.

Not to just experience the moment.

Good or bad, every moment with every person I interact with, I can simply cherish.

Cherish every moment.