SkillsCapabilitiesExploreContact
Short Posts
Setting up kopano video conferencing in mattermost
Writing GatsbyJS configuration in Typescript
Recovering disk space in NixOS
Determining why docker command hands at startup in WSL2
Worse is Better: Setting up Gatsbyjs drafts
Replacing X window system with wayland and sway
Using Windows 10 to fix UEFI after BIOS update
Adding Jupyter Notebook (and Rust) to Gatsby
thumbnail

Adding Jupyter Notebook (and Rust) to Gatsby

2019-12-11
Blog
Skill
Software
Rust
Summary:
  • Integrated Jupyter Notebook execution into a Gatsby static site using gatsby-transformer-ipynb to render live code output at build time
  • Patched an upstream rendering bug in @nteract/notebook-render and applied the fix via Yarn selective dependency resolution
  • Enabled Rust code execution in blog posts by installing and configuring the evcxr_jupyter kernel
  • Built a GraphQL-driven page template that queries notebook metadata and safely renders pre-executed HTML in a static site

An insane plan to allow posts on this site to perform computations and display output

What? How?

Jupyter Notebook is an interactive environment where code can be typed into cells and the outputs visualized. evcxr_jupyter is a custom jupyter kernel that can run rust. gatsby-transformer-ipynb is a gatsby plugin that can render jupyter notebooks during site generation. Let’s get frankensteining!

Installing Jupyter

This computer is Windows and primary development is in WSL1 (Ubuntu-ish)

sudo apt install python3 python3-pip
pip install --user jupyterlab
which jupyter
> /home/me/.local/bin/jupyter
jupyter lab

stock jupyter

Installing in gatsby

yarn add @gatsby-contrib/gatsby-transformer-ipynb
// In your gatsby-config.js
plugins: [
  {
    resolve: `gatsby-source-filesystem`,
    options: {
      name: `content`,
      path: path.join(__dirname, "content"),
      ignore: [`**/.ipynb_checkpoints`],
    },
  },
  ...`@gatsby-contrib/gatsby-transformer-ipynb`,
]
cd content/blog/03-rusty-jupyter
jupyter lab

example python 3 jupyter post

notebook metadata

Correcting upstream bug

yarn develop
...
 ERROR #11321  PLUGIN

"@gatsby-contrib/gatsby-transformer-ipynb" threw an error while running the onCreateNode lifecycle:

Renderer for type `element` not defined or is not renderable

as noted on github 7 days ago (as of writing)

ooorah, let’s do this

Using yarn selective dependency resolution the notebook-render dependency is overridden to mine until the PR is accepted:

/// package.json
  "resolutions": {
    "@gatsby-contrib/gatsby-transformer-ipynb/@nteract/notebook-render": "https://github.com/sdobz/notebook-render"
  },

Adding pages

First we need to figure out how the notebook is added to graphql:

graphql explorer

One small distinction is that the front matter of the blog post is titled “metadata”

gatsby-node.js was customized to parse and validate the metadata, details are too complex for the scope of this post.

Creating a template

This is a rough representation of the template used:

const JupyterTemplate = ({
  data: {
    jupyterNotebook: {
      metadata: { title },
      html,
    },
  },
}) => (
  <PageContainer>
    <PageTitle title={title} />
    <Stack dangerouslySetInnerHTML={{ __html: html }} />
  </PageContainer>
)

export default JupyterTemplate

export const query = graphql`
  query JupyterPage($path: String!) {
    jupyterNotebook(metadata: { path: { eq: $path } }) {
      metadata {
        title
      }
      html
    }
  }
`

Viola, the example notebook source

Using Rust

cargo install evcxr_jupyter
evcxr_jupyter --install
jupyter lab

Creating a rust notebook…

jupyter notebook with rust kernel

all done — download the Rust notebook source

Next steps

Currently all previews on this blog have to be markdown, so notebooks will not appear there.

The formatting on the notebooks leaves a bit to be desired and must be improved

Theming between code snippets in markdown and in the notebooks could be unified

Sub posts:

Previous

Adding Jupyter Notebook (and Rust) to Gatsby

thumbnail

Correcting a bug in notebook-render

2019-12-11
1st article in Adding Jupyter Notebook (and Rust) to Gatsby

Debugged a server-side rendering crash in @nteract/notebook-render caused by remark-rehype producing unhandled "element" AST nodes

Oss
Skill
Software
Blog
Rusty Jupyter
thumbnail

Simplifying Jupyter integration by eliminating heterogenous lists

2019-12-23
2nd article in Adding Jupyter Notebook (and Rust) to Gatsby

Resolved heterogeneous content-type problem by embedding Jupyter notebooks as markdown custom blocks instead of abstracting into virtual post types

Philosophy
Skill
Software
Blog
Rusty Jupyter
Featured Work
CNC Surface Grinder RetrofitWelding PositionerStaübli Reborn: Industrial Robot with a Modern Control System
Company Info
About UsContact UsPrivacy Policy
Specific Solutions LLC
Portland, OR