Run a random walker in the browser using Pyodide

Since I got this site run­ning, I have been want­ing to be able to em­bed some kind of in­ter­ac­tive plot in my blog post. For in­stance, say I want the user to be able to per­form some ma­chine learn­ing com­pu­ta­tions and then vi­su­al­ize the re­sult. Currently, there are a few op­tions to achieve this,

• Pure Javascript so­lu­tion. Both com­pu­ta­tion and vi­su­al­iza­tion are per­formed us­ing javascript. This process can ei­ther hap­pens on the server or in the browser.

• Combination of python and javascript

  • ei­ther the com­pu­ta­tion is done with python but hap­pens on a server,
  • or the com­pu­ta­tion is done with python di­rectly in the browser.
  • Any com­mu­ni­ca­tion with DOM such as vi­su­al­iza­tion is through javascript or API in python.

I have al­ways been amazed by things peo­ple can do with javascript, such as deep learn­ing us­ing javascript in­side your browser. But I can’t imag­ine javascript tak­ing over python in sci­en­tific com­put­ing in near fu­ture. I, per­son­ally, am much more com­fort­able with python. Besides, the lan­guage has a much more ma­ture sci­en­tific li­brary ecosys­tem. To be able to use python to per­form the com­pu­ta­tion part is es­sen­tial, hence leav­ing us only with the sec­ond op­tion, which is that python code runs ei­ther on a server or di­rectly in­side the browser.

Using a server to per­form com­pu­ta­tions means com­mu­ni­ca­tion with the server. This can have some draw­backs,

• Depends on the us­age, one may need to pay for the server.
• Communication over­head can cause de­lays in user in­ter­ac­tion.

With my ex­pe­ri­ence with Binder, the sec­ond point can be a deal­breaker. The so­lu­tion would be sim­ple. Just elim­i­nate the server step! However, since for a long time javascript is the only pro­gram­ming lan­guage the browser can in­ter­pret di­rectly. No server means that we need to find some way to run python code in the browser di­rectly. There are quite a few op­tions, such as PyPy.js. However it is not pos­si­ble to use Numpy, Pandas and many other sci­en­tific/​data analy­sis li­braries in the browser un­til the Pyodide pro­ject came out re­cently. Pyodide al­lows python code to run in­side the browser through WebAssembly. The best thing is that it al­lows one to use a few most pop­u­lar sci­en­tific li­braries in­clud­ing Numpy, Matplotlib, Pandas, Scipy and even Scikit-learn in­side the browser! In fact, to my un­der­stand­ing, any python li­braries in prin­ci­ple can be used through Pyodide. I am by no means ex­pert on how Pyodide works. I sug­gest read­ing their blog post and check­ing out the pro­ject github repos­i­tory.

I have been ex­per­i­ment­ing with Pyodide for a few days. In this post, I would like to give a proof-of-con­cept demon­stra­tion. Since I deal with ran­dom walks a lot in my re­search, I would like to make a sim­ple ran­dom walk an­i­ma­tion demon­stra­tion which

• al­lows users to spec­ify the num­ber of the steps
• cal­cu­late the ran­dom walk tra­jec­tory in the browser on the fly
• an­i­mate the gen­er­ated tra­jec­tory of the ran­dom walk

In this ex­am­ple, I will use python code to gen­er­ate the tra­jec­tory of a sim­ple 2D ran­dom walker and use plotly.js to han­dle the vi­su­al­iza­tion.

Python code for 2D ran­dom walker #

For demon­stra­tion pur­pose, the ran­dom walk in this ex­am­ple is sim­ple,

• It is a two-di­men­sional walk
• At each step, the dis­place­ment along the $x$ and $y$ di­men­sions are in­de­pen­dent and drawn from a gauss­ian dis­tri­b­u­tion with mean zero and unit vari­ance.
• The num­ber of steps is spec­i­fied be­fore­hand

Here is the python code for gen­er­at­ing such ran­dom walk,

# load numpy library 
import numpy as np

# function for generating random walk
# it takes the number of steps as only parameter
def walk(n):
    # check if the number of steps is an integer
    if int(n) != n:
        print('number of steps should be an integer')
        return None
    # the initial position is (0,0)
    xy_0 = np.array([0.0, 0.0])
    # generate displacements of each step
    dxdy = np.random.randn(int(n), 2)
    # cumulative sum displacement to get positions at each step
    xy = xy_0 + np.cumsum(dxdy, axis=0)
    # insert the initial position at the head of the array
    xy = np.vstack((xy_0, xy))
    # since javascript has no 2D array, it is better to
    # return the x-position and y-position, separately
    return xy[:,0], xy[:,1]

Call our python func­tion in­side the browser #

Now we would like to be able to call this python code in the browser on de­mand. The browser then does the cal­cu­la­tion and get two ar­rays which con­tain the $x$ co­or­di­nates and $y$ co­or­di­nates. Then we can use plotly.js to an­i­mate the tra­jec­tory.

For bet­ter main­tain­abil­ity, I sug­gest to put python code in a github gist and fetch the con­tent on the fly. It also has an ex­tra ben­e­fit that it al­lows the mod­i­fi­ca­tion of python code with­out re­build­ing the site.

Before I con­tinue, I would like to point out that one of the biggest prob­lems of Pyodide is that it is very large. To use it, the browser needs to down­load about 24 Mb code and Numpy li­brary needs an­other 8 Mb which leads to a to­tal of 32 Mb down­load size. I want the user to down­load the Pyodide only when they want to. To achieve this, I dy­nam­i­cally load the Pyodide script only when the ini­tial­iza­tion but­ton is clicked (see demon­stra­tion be­low).

The python code is called through

gistFetchPromise.then(res => {pyodide.runPython(res)})

where gistFetchPromise is the promise ob­ject of fetch­ing the gist con­tent. Note that the python code needed to be parsed as a raw string. The pyodide.runPython() func­tion is called to ex­e­cute the python code. Once it is ex­e­cuted, all the python ob­jects are avail­able in the browser. The de­fined python func­tion walk can be ac­cessed through pyodide.globals.walk. Here is an ex­am­ple,

// Here is the javascript code
// we assign the python function [walk] to javascript [walk]
let walk = pyodide.globals.walk;
// we can call the function [walk] in javascript
let [x,y] = walk(1000);
// now x and y have values of positions of our random walker

Once we get the cal­cu­lated po­si­tions x and y, we can use plotly.js to plot the re­sult. Fortunately, plotly.js pro­vides a rel­a­tive sim­ple API for an­i­ma­tion. One can also use Bokeh, D3, or any other web vi­su­al­iza­tion tool out there. It is even pos­si­ble to do the vi­su­al­iza­tion in python di­rectly since Pyodide also work with Matplotlib. However, at this stage, I think it is more straight for­ward to use a javascript li­brary to han­dle the vi­su­al­iza­tion since it is de­signed to ma­nip­u­late DOM (HTML) af­ter all.

I don’t want to make this post su­per long, thus I won’t go into very de­tails of the vi­su­al­iza­tion part. The full javascript code we need to load in the page can be found in this gist. The file in­cludes the code for fetch­ing gist, vi­su­al­iza­tion us­ing plotly.js, Pyodide code and event han­dlers for but­tons.

Demonstration #

Here is the end prod­uct! Click the but­ton Initialize Pyodide to down­load the Pyodide and load Numpy. Once the ini­tial­iza­tion is fin­ished (it can takes about 20 sec­onds or even longer with slow net­work. Not good, I know …), the but­ton Reset, Start and Pause will be­come click­able and green. Then en­ter a step num­ber (or use the de­fault num­ber 100) and hit Start but­ton to watch the an­i­ma­tion of a 2D ran­dom walker. Click Pause to pause the an­i­ma­tion any­time and Start to re­sume. Click Reset but­ton to re­set the ran­dom walker.

Further read­ing #

• Running Python in the Browser
• Python-in-the-browser tech­nolo­gies
• WebAssembly
• Python in Web Assembly with Pyodide (This is a pod­cast)
• Pyodide doc­u­men­ta­tion
• Pyodide note­book demo
• Plotly.js an­i­ma­tion ex­am­ples