- Rachit's Blog

Formally speaking, "Transpiler" is a useless word

2025-12-30T00:00:00+00:00

I find “transpiler” to be a useless word</a>: depending on the speaker, it can mean anything from a text replacement tool to a sophisticated optimizing compiler; I would much rather just use “compiler” to describe these systems. In reading this older post, someone pointed me to the dichotomy between lumpers and splitters: people who try to group similar things vs. people who try to separate out different things. I think this is an interesting idea and made me want to pen down, rigorously and formally, why I think compilers and transpilers are the same thing.

So, here is the game plan: I will introduce a couple of ideas that the programming languages (PL) community developed to carefully think about tools that manipulate programs. I will use these ideas to give three formal definitions of the word “transpiler” and work through why I do not like them. Even if you disagree, I hope that the formal framework provides a more rigorous ground for debate!

It Walks Like a Duck</h1>
Concrete syntax is the first layer of a programming language we interact with: the symbols and words that form a language’s vocabulary. The PL community formalizes the syntax of a language using the Backus-Naur form (BNF) grammar. For example, here is the syntax of simple imperative language with loops and conditionals:
$$ \begin{aligned} n &\in \mathcal{N} \quad b \in \{T, F\} \quad x \in variables \\ e &::= n | e + e | e - e | … \\ c &::= b | e > e | e = e | c\, \texttt{and}\, c | … \\ s &::= x := e \\ &|\, s;s \\ &|\, \texttt{if}(c)\,\{s\}\,\texttt{else}\,\{s\} \\ &|\, \texttt{while}\, c\, \{ s \} | … \end{aligned} $$ A BNF grammar is recursively-defined: the generator for expressions ($e$) defines terminals, such as numbers ($n$), as well as compound expressions, such as $ e - e $, which refer back to the generator. Other generators, such as statements ($s$) mention expressions as well as special tokens such as if</code> and while</code>. Formally, a programming language is the (infinite) set of all terms generated by this recursive definition:

Syntax of a Programming Language: A programming language ($L$) is the set of all terms enumerated by some BNF grammar ($G$). Therefore $L = \{ t | t \in G \} $. </blockquote> Here’s an intuitive way to think about this definition: there is a universe of all possible terms that can be constructed by combining the symbols in ASCII. Let’s call this the alphabet soup. A BNF grammar provides structure to the alphabet soup by saying which combinational of terms are a part of some language and which ones are not. For example, the alphabet soup contains the term >+++</code> the but language above does not. However, you can totally define it to be a meaningful term in another language. Using this idea of syntax, I give my first definition of the word “transpiler”:
Transpilers are Syntax Translators. A Transpiler transforms programs in a source language (S) into programs a target language (T) by converting syntax constructs in S to match the “most natural” ones in T. </blockquote> “Most natural” is my attempt to capture what people mean when they say “transpilers compile two languages at the same level of abstraction”. For example, when transpiling JavaScript programs to Python, for (...) { body }</code> is converted into for ...: body</code>. Unfortunately, when the target language does not have the same constructs as the source language, the “transpiler” needs to be able to do something else: it needs to preserve the intention of the programmer when translating between S and T. The formalism of syntax does not give us the ability to talk about the intention of the programmer. Let’s look at the next formalism.
It Quacks Like a Duck</h1> The execution model is the second layer of our interaction with programming languages: it defines how a concrete program is transformed into computation. PL researchers call this the semantics of the language and it is sacrosanct: by precisely describing what a program means, a semantics also defines what optimizations are valid, what guarantees a type system gives, and what intention of the programmer the compiler must preserve when translating a program between languages. There are many different flavors of formal semantics but I will talk about operational semantics. An operational semantics describes how to evaluate programs by defining a mathematical “machine” that rewrites programs into simpler ones (or reduces them). For example, let’s consider the expression: (1 + 2) > (3 + 4)</code>. Intuitively, we expect the following steps to occurs when this program runs:
(1 + 2) > (3 + 4)
3 > (3 + 4)
3 > 7
False
</code></pre>
Each one of these steps is a “reduction” within an operational semantics. We can formalize the possibly reductions using a big-step operational semantics which defines the relation $e_1 \Downarrow e_2$ (read as “$e_1$ reduces to $e_2$”):
$$
\begin{aligned}
&\frac{e_1 \Downarrow v_1 \quad e_2 \Downarrow v_2 \quad v = v_1 + v_2}{e_1 + e_2 \Downarrow v}
\\
&\frac{c_1 \Downarrow v_1 \quad c_2 \Downarrow v_2 \quad v_1 > v_2}{c_1 > c_2 \Downarrow \texttt{True}}
\\
&\frac{c_1 \Downarrow v_1 \quad c_2 \Downarrow v_2 \quad v_1 < v_2}{c_1 < c_2 \Downarrow \texttt{False}}
\end{aligned}
$$
Like the concrete syntax definition, the rules are recursive: in order to reduce large expressions, we first reduce their subcomponents and then perform some computation over the simplified values.


Tricks for encoding operational semantics.
</summary>

There are two interesting tricks we play when defining the operators $+$ and $>$ in the rules above.

The first rule reduces two expressions into values and then applies the $+$ symbol. But this seems circular: to define $+$, we use $+$. Absolute rigor would demand that we further describe the $+$ operator on numbers $v_1$ and $v_2$ using some sort of encoding such as Peano arithmetic or Church numerals.</li>
The $>$ operator has two rules: one for the True</code> case and another for the False</code> case. This is another common trick in defining operational semantics: we can encode choice by describing multiple rules. Because the two rules share almost all the conditions on the top, they will attempt to reduce expressions like $c_1 > c_2$ but only one of them will succeed for any given expression. For the acutely philosophical, we are using the built-in notion of “matching” within inference rules to define a conditional; this means that inference rules themselves have a semantics…</li>
</ol>

</details>
Let’s see why semantics are useful by studying a particular compilation example. First, we define our fragments of our source ($S$) and target ($T$) languages:
$$
\begin{aligned}
x &\in variables \\
e_s &::= b\, |\, e > e\, |\, e\, \texttt{and}\, e \\
e_t &::= b\, |\, x\, \texttt{and}\, x \\
&|\, \texttt{let}\, x = e\, \texttt{in}\, e \\
&|\, \texttt{if}\, x\, \texttt{then}\, e\, \texttt{else}\, e
\end{aligned}
$$
Let’s walk through the differences in the two languages by comparing the syntax for $\texttt{and}$:
the expression $(1 > 2) \,\texttt{and}\, (3 > 4)$ is a valid expression
in the source language ($e_s$) but not the target language ($e_t$). This is because the syntax for
$\texttt{and}$ for the target language requires the sub-expressions to be variables ($x$).
Given these definitions, let’s try to build a “transpiler” using our first definition. Staying true to our definition of translating to the most natural operator within $e_t$, we would like to generate another $\texttt{and}$ expression. To do this, we need to introduce some let</code>-bindings to name our expressions:
e1 && e2
(* Rewrite into *)
let x1 = e1 in
let x2 = e2 in
x1 && x2
</code></pre>
Transpilation complete! Except, this rewritten program has introduced errors that were not possible before. In Python, we can write:
if x is not None and x > 10: ...
</code></pre>
Our rewrite rules transforms this into:
p = x is not None
q = x > 10
if p and q: ...
</code></pre>
Executing this will generate and error saying that comparisons are not supported between NoneType</code> and int</code>. However, this error cannot occur in the first program. This is because and</code> implements short-circuiting behavior: if the left side evaluates to False</code>, the right side is not evaluated at all. By binding both the expressions before the and</code> operation, we’ve eliminated the short-circuiting. This highlights the folly of thinking about “transpilation” as a purely syntactic process; if you ignore the semantics, you change the meaning of programs.
An operational semantics can differentiate between the two possible semantics of the $\texttt{and}$ operator. This first rule, which looks very similar to the operational semantics above, captures a non-short-circuiting implementation of $\texttt{and}$:
$$
\begin{aligned}
&\frac{e_1 \Downarrow v_1 \quad e_2 \Downarrow v_2 \quad v = v_1 \land v_2}{e_1, \texttt{and}, e_2 \Downarrow v}
\end{aligned}
$$
The rule fully evaluates the left and right sub-expressions and then uses the logical and operator ($\land$) to compute the output. This second set of rules capture the short-circuiting behavior:
$$
\begin{aligned}
\frac{e_1 \Downarrow \texttt{False}}{e_1, \texttt{and}, e_2 \Downarrow \texttt{False}}
\\
\frac{e_1 \Downarrow \texttt{True} \quad e_2 \Downarrow \texttt{False}}{e_1, \texttt{and}, e_2 \Downarrow \texttt{False}}
\\
\frac{e_1 \Downarrow \texttt{True} \quad e_2 \Downarrow \texttt{True}}{e_1, \texttt{and}, e_2 \Downarrow \texttt{True}}
\end{aligned}
$$
The first rule skips the reduction of $e_2$ if $e_1$ evaluate to False</code>.
The correct way to encode this short-circuiting behavior is to use conditionals:
e1 && e2
(* Rewrite into *)
let x1 = e1 in
if x1 then e2 else False
</code></pre>
So, the syntax-based definition of a transpiler is wrong if we care about maintain correctness. This second definition of “transpiler” uses the idea of semantics:

Transpilers are semantics translators. Given two language $S$ and $T$ with semantics $\Downarrow_s$ and $\Downarrow_t$, a Transpiler is function $\mathcal{C}: S \rightarrow T$ such that $\forall s \in S : s \Downarrow_s v \implies C(s) \Downarrow_t v$.
</blockquote>
The math says that “if a source program $s$ evaluates to $v$, then the compiled program $C(s)$ should also evaluate to $v$.” In other words, a transpiler preserves the meaning of a program. This is a nice precise definition!
It also happens to be the exact definition of a compiler.
This puts us in a bind: The syntax translation definition of “transpilers” is too weak because it excludes transformations that so-called “transpilers” definitely support, while semantic translation definition is too strong because it includes normal compilers!
At this point, the wise critic would say that my academic top hat has blinded me and the obvious distinction between a transpiler and a compiler is that a transpiler is simple! Compilers can perform all sorts of complicated tricks to transform programs but the transpiler is elegant simplicity itself.
Let’s see what math has to say about simplicity.
Bills but no Webbed Feet</h1>
Simplicity is subjective but the history of the word “transpiler” might shed some light on what people mean. To my knowledge, “transpiler” originates within the JavaScript community and refers to tools like Babel</a> which enabled people to use new language constructs that lacked browser support. For example, JavaScript might implement a new for</code>-in</code> construct which allows a loop to walk over the keys of an object. However, browsers would take some time to implement the feature natively. In this case, Babel could compile for</code>-in</code> into an existing while</code>-loop with the right function calls allowing codebases to adopt it while maintaining compatibility. Such a rewrite seems simpler than the heavyweight responsibilities of a compiler.
So, how do we define “simplicity”? Some readers might call the for</code>-in</code> construct syntactic sugar: it makes it easier to write certain kinds of programs but does not fundamentally change the expressivity of the language. An important property of syntactic sugar is that rewriting it is compositional: I can rewrite any for</code>-in</code> loop into the equivalent while</code> loop without worrying about the surrounding context. This idea of “context independence” is precisely what the paper On the Expressive Power of Programming Languages</a> uses to formally distinguish between simple and complex rewrites:

Syntactic sugar. A feature $\mathcal{F}$ is syntactic sugar for language $\mathcal{L}$ if there is a rewrite rule $\phi$ such that $\phi$ distributes over all composition operators of $\mathcal{L}$. In other words, if $A(L_0, L_1, …) \in \mathcal{L}$, then $\phi(A(L_0, L_1, ..)) = A(\phi(L_0), \phi(L_1))$. Such a $\phi$ is called a local transformation.
</blockquote>
Deep breath. The math looks like scary mumbo-jumbo but it describes what it means for a rewrite rule to be simple. In short, if the rewrite rule distributes over language constructs like loops (while</code>), conditionals (if</code>-else</code>), and sequential composition (;</code>), it is simple syntactic sugar. Because the rewriting can distribute over constructs, it cannot, for example, look at one part of the program and use that information to transform a different part of the program. This rules out transformations that add expressive features to a language–exceptions in languages with simple control flow or mutability in pure functional languages–which require global and contextual transformation.
I find this idea to be pretty neat because it allows to talk about what features of a language add expressivity to it:

Expressive constructs. A feature $\mathcal{F}$ adds expressivity to $\mathcal{L}$ if there does not exist a local transformation to eliminate it.
</blockquote>
With these definitions, here is my third definition of a transpiler that uses the idea of expressivity and local transformations:

Transpilers are local rewriters. Transpilers convert programs in source program $S$ to target $T$ by only performing local transformations.
</blockquote>
This definition of “transpiler” is the only one I have any sympathy for since it captures the loose notion of simplicity that people often throw around. However, this definition is still far from being good enough because real programming languages have complex semantics that generally do require some amount of global transformation. For example, correctly implementing the semantics of JavaScript’s this</code> within Python would give anyone a headache.
Call It What You Want</h1>
To summarize, I have given three definitions of the word “transpiler” grounded in formal concepts:

Syntax Translator. Based on the formalism of concrete syntax. This probably most closely matches the intuition people have when they say “transpiler” but falls apart upon basic scrutiny.</li>
Semantic Translator. Based on the formalism of semantics. This is the definition of a compiler. If you mean this, then just use the word “compiler”.</li>
Local Rewriter. Based on expressivity of constructs. This definition is a little more nuanced and the one I am willing to cede some ground on. However, tools that are called “transpilers” often need to perform more than just local rewrites if they care about being correct.</li>
</ul>
I hope that, even if you disagree with my definitions, the formal tools shed some light on how my research community thinks about these problems. At the end of the day, I do not care what you call these tools; maybe there is some intuitive idea that is clear to your audience when you say “transpiler”. However, in a world full of vagaries and vibes, I look for precise thought and I find the definitions of the word “transpiler” lacking.



Reflecting on PLDI 2025
2025-07-02T00:00:00+00:00
PLDI is one of the marquee programming languages conference and one my favorite ones to attend. The number of domains that PL are working in and bringing ideas to is frankly overwhelming and inspiring. I had a bunch of interesting conversations that incited new ideas and perspectives for me.</p>
For the Masses or for the FLOPs?</h2>
Building and programming heterogenous systems has quickly become one of the key problems</em> being tackled by the systems community. For the PL community, this should present tons of interesting work on defining programming models for emerging accelerators (GPUs, TPUs) and programming large systems that use them. Given this, I was surprised by the lack of papers tackling these problems.</p>
After several long conversations with other conference attendees, I came up with the following framing. Programming languages research has two models of impact: building for the masses or building for the FLOPs (floating-point operations).
A vast majority</em> of the FLOPs</a> in the world today come from specialized hardware like GPUs and TPUs.
These accelerators generally use highly specialized programming models like CUDA or high-level languages like PyTorch.
On the other hand, a vast majority</em> of programmers use general-purpose languages</a> like Python or JavaScript.</p>
As a PL researcher, this poses a problem: you can work on building programming systems that enable more people to effectively interface with computers, or you could build better interfaces to a vast majority of the computation in the world.
I think the current iteration of the PLDI community focuses on the former problem–of building better programming systems for the most number of people–while conferences like ASPLOS publish a lot more of the work on the latter kinds of problems.

</label>
</span>


For example, the PyTorch 2</a> paper was published at ASPLOS ’24.
</span>

Like most distinctions, I don’t think this is a black-and-white divide but I did find this to be a useful lens to look through.</p>
Tools for Language Evolution</h2>
Turning to general-purpose languages, the new RPLS workshop</a> and Sukyong Ryu’s keynote</a> highlighted the need for tools for language evolution.
There has been a lot of work in the community on building tools that help specify formal semantics of languages and automatically generate tools like interpreters, symbolic verifiers, etc.
Instead of trying to use general-purpose semantics specification tools like the K framework</a>, Sukyong’s group has developed language specific tools: ESMeta for JavaScript, and SpecTec for WASM and worked hard to integrate into the community’s workflow.
For example, the JavaScript community already had an English-language specification for operations like addition</a> that are written with a precise pseudocode language.
It would be a drastic change of norms (and a huge waste) to throw out these specifications.
ESMeta</a> provides a formal specification language to describe JavaScript (or, as it is officially called, ECMAScript) and automatically generates the same English-language documentation.
This means that the community can move towards a formal, single source of truth model for defining the language and keeping it up-to-date.</p>
These kinds of single-source of truth language specifications seem to only be growing in importance.
The WASM community adopted the SpecTec project (also from Sukyong’s group), ARM defines the ISA semantics using the architecture specification language (ASL)</a> and the SAIL architecture specification language</a> has been successfully used to define the semantics of ISAs like ARM-A, x86, IBM Power, and CHERI.</p>
Formal Verification Looms Large</h2>
Formal verification of systems, languages, and semantics remains one of the primary focuses of PLDI.
Two out of the six distinguished papers</a> focused on formal verification of some form and Leo de Moura gave a keynote</a> on the Lean project</a>.
The talk went over the wide impact Lean has had over the mathematics and the software engineering community.
There were two projects that I particularly found interesting:</p>

The mathlib</code> project</a> which has slowly accumulated theorems and proofs from algebra, geometry, and analysis. The project provides a large number of building blocks for anyone interested in writing sophisticated mathematical proofs (like Terrance Tao’s proof of the PFR conjecture</a>).</li>
The Veil Framework</a> by folks at NUS which provides a domain-specific language verifying distributed algorithms. The cool thing about Veil is that because it is embedded within Lean, it can provide both automatic</em> and deductive</em> verification. The former uses SMT solver integration to perform proof search while the latter uses the Lean proof assistant to generate proofs through tactics. Furthermore, because Lean has extensive metaprogramming support, Veil programs actually look more like Alloy or TLA+ programs than like other Leans.</li>
</ul>
These two projects, to me, provide an existence proof for the potential of designing large-scale hardware-software verification stacks within Lean.
For example, think of a system that takes proofs about computations on fully homomorphically encrypted data and connects them all the way down to the hardware while allowing domain experts to use abstractions they’re familiar and comfortable with!</p>
Papers</h2>
The main conference has several parallel tracks so I couldn’t see all the talks I wanted to but here is a collection of talks I found interesting:</p>

Making Concurrent Hardware Verification Sequential</a>: Normally, when thinking about the behavior of a processor, we like to think of a sequential implementation</em>–an interpreter that run each instruction one at a time–as the ideal behavior of the processor. Pipelining, speculation, out-of-order execution all must be shown to be refinement</em> of (i.e., admitting fewer behaviors than) this sequential implementation. This feels like a pretty reasonable model and exactly how I thought of processors. However, this paper shows that it is not actually possible</em> to connect purely sequential implementations to more complex processors! The crux of the problem is that a real processor might, for example, have multiple memory requests in-flight before the first one returns, which is a behavior that the sequential implementation cannot</em> exhibit but a pipelined one might. I find this result pretty staggering and am very interested in diving into the details.</li>
Bean: A Language for Backward Error Analysis</a>: The talk presented a type system to quantify the backward error</em> present in a program due to floating-point errors (compared to a corresponding real-valued computation). I found this to be a really cool</em> combination of deep theory work (to make the notion of compositional error analysis work) and application focused ideas (error analysis is done very manually and seems to be very tedious). The paper presents a limited language with quite a few restrictions (which are justified and analyzed by the theory) but to me, the interesting part was that this is even possible at all</em>. The talk is well worth a watch when it shows up on the SIGPLAN channel.</li>
Task-Based Tensor Computations on Modern GPUs</a>: One of the only talks about GPU programming. Beyond the research contributions, I found the talk to be a pretty lucid description of the current programming model for GPUs (which is kind of insane and cool). The crux of the problem is that the newest NVIDIA GPUs have an asynchronous execution interface for the tensor cores (circuits that perform matrix-multiply) and the processor cores (streaming multi-processors</em>) need to find work to do while the tensor cores are doing their thing. The standard SIMT programming model makes it hard to find and do said work so the task-based approach makes a lot more sense.</li>
MISAAL: Synthesis-Based Automatic Generation of Efficient and Retargetable Semantics-Driven Optimizations</a>. MISAAL continues in the line of work from Vikram Adve’s group on automatically synthesizing compilers from ISA specifications. The work has a lot of moving pieces so I’ll leave the details to the paper but one thing I found to be interesting was that the authors parsed the pseudocode language used by different ISA vendors to generate specifications for ISA instructions! This is a pretty good example where specification languages like ASL can be useful in rapidly enabling this kind of work.</li>
</ul>
Conclusion</h2>
The golden age of computer architecture continues and with it, heterogeneity remains one of the most interesting and challenging programming problems. I continue to think that the toolset developed by the PL community is extremely well-suited to address these problems and I’m excited to see what folks build next!</p>


GitHub-centric Research Management
2023-09-15T00:00:00+00:00
I am a systems researcher</a> which means almost all of my research requires writing lots of a code.
I also believe in building in the open</a> and, being a person of hubris, like writing good code that other people can use.
Finally, as an academic, I also like teaching people how to write good code and build cool tools that other people can use.
Over the course of my PhD, my advisor and I have developed a set of guidelines that we often repeat to people working with us to help them write better code when building research projects.
I also have some experience</a> scaling up research projects, and I follow the same guidelines when building the management structure for new projects.</p>
Our guidelines revolve around using the GitHub code management platform to mechanize, track, and remember research tasks.
While I’m going to use GitHub-specific terms in this post, I expect alternatives like GitLab to also be perfectly capable of providing the same utility.
Before reading this post, I recommend getting familiar with git</a>.</p>
GitHub Basics</h2>
GitHub is a code management platform powered by git</a> that you to collaborate with people on code and manage its long-term health. Specifically:</p>

Issues</strong>: Tracks individual “tickets” that track outstanding work items. This can be as simple as fixing a bug or as complicated feature rewrites (making the “issues” title a bit of a misnomer). This is the core “planning section” for your projects. Any idea that requires more than 5 minutes of implementation work gets written up here. Once an issue has been addressed, it can be “closed” which hides it from the section.</li>
Pull Requests (PRs)</strong>: This is the “implementation section”. A pull request (confusingly named</a>) is a bundle of code changes that someone wrote up and wants to have merged</em> into the codebase. Often, but not always, a pull request will correspond to at least one issue created in the “issues” section.</li>
Continuous Integration/Deployment (CI/CD)</strong>: Automation that runs tests for you (integration testing) or deploys code artifacts for you (deployment) on every code commit. A good CI/CD system will ensure that new code doesn’t break existing features and automatically updates documentation.</li>
Linking</strong>: A key feature of GitHub that we use is linking issues and pull requests</a>. This allows us to build a breadcrumb trails that contextualize decisions. A key part of our philosophy is creating links between relevant issues and PRs.</li>
</ul>
Planning (Issues)</h2>
The Issues</em> section is where the bulk of planning and discussions about the code should happen.
The most common alternative to this is using messaging apps like Slack or Zulip which, I contend, is a bad idea.
Messaging apps, by design, keep focus on one thread of conversation while code development requires many different, interconnected thread of conversations over long periods of time.
Messaging apps don’t really provide effective mechanisms for continuing several conversations over multiple months and linking between them.</p>
Instead, the “Issues” section provides a permanent space for discussions to live and allows us to link together relevant things.
A good issue has the following two characteristics:</p>
Reproducible</strong>. The issue has enough information contained within it to allow someone who is not the author of the issue</em> to work on it.
Even if you’re the only person</em> working on the codebase, this is good practice because you today and you a year from now are different people.
Concretely, if there is a bug in the system, the issue should provide a minimal reproducible example</em> (MRE) along, a command to run reproduce the problem, and the expected behavior.
If the issue is a feature request, it should instead provide a sketch of the idea and outline the expected changes that need to be made to each component of the system.</p>
Contextualized</strong>. The issue should link to existing issues and pull requests that are related to it. This builds institutional knowledge because it allows us to trace why certain decisions were made about the code and the features.
A lot of projects get reproducibility</em> right but fail on this front because new contributors might not know enough about which things are related.
It is the job of senior contributors to aggressively track and link together things as the junior contributors develop the context of the codebase.</p>
These guidelines are missing one characteristic that is critical in large-scale projects: Actionable</strong>, which is the idea that the issue must be something that can be worked on in the short term. Research projects will necessarily have ideas and bugs that are not workable in the short term.
However, it is still useful to sketch out the idea (reproducible</em>) build a web of connections for those ideas (contextualized</em>).</p>
Labels</h3>
GitHub issues and pull requests can be tagged with “labels” to categorize them. My recommendation is to have two kinds of labels:</p>

Component</strong>. Which part of the codebase does this issue relate to? For example, it could be a specific tool, error message, UX, etc.</li>
Status</strong>. What is the status of this issue? Keep this to a few categories. Here’s my recommendation:

“Available”: This can be worked on.</li>
“Needs Triage”: We don’t know exactly how to work on this.</li>
“Needs Discussion”: We need to discuss whether this is something we should ever work on.</li>
“Blocked”: This issue is blocked by something else. When the last tag is used, make sure to link the issue blocking this one.</li>
</ol>
</li>
</ol>
Labels allow us to slice and dice the set of issues we care about and review them from time to time. For example, after a major feature is implemented, we can look at all the blocked issues and see which ones were unblocked. Similarly, if we’re putting more effort into a particular component, it could be useful to see which are the currently open issues.</p>
Guidance</h3>
There are two pieces of advice on issues:</p>

Feature proposal and discussions must be issues</em>: If any code-related discussion starts getting in the weeds on the messaging platform, move the discussion into an issue. This will avoid losing the thread of conversation and ensure other team members can chime in.</li>
Issue creation is cheap</em>: when in doubt, create an issue. If it is easily answered, a senior contributor will do so and close it.</li>
</ul>
Again, these guidelines don’t scale to large projects, but we’ve found them to be useful in instilling a sense of ownership with new contributors and building institutional memory.</p>
Doing (Pull Requests)</h2>
Pull requests</em> is the section where all the code must</em> travel through. This discipline is extremely powerful if practiced well: it allows people to review things and make sure changes don’t break other people’s code. Here are a couple of guidelines to enable this:</p>

Disable pushes to the main</code> branch. This means that no one is allowed to directly push to the main</code> branch.</li>
Require certain tests to pass before a pull request can be merged.</li>
Disable the “Merge” and “Rebase” options for pull requests and only allow for “Squashes”</a>. Also, require a linear history</a>. Along with (2), this means that every commit to main</code> is state where the tests pass.</li>
Build a culture of code review. This helps new contributors understand the culture of the project and enable transference of institutional memory.</li>
</ol>
Contributors should feel free to break tests and muck around with things as much as needed when working on a feature on a branch.
However, once the feature is ready to be merged, they should open a pull request and ensure that all tests pass on the final commit.
If using the “squash merge” strategy, this will merge all the branch commits into one commit that has all the tests passing.</p>
Testing</h3>
A good test suite is a mark of a real project.

</label>
</span>


If you’re a systems researcher and your projects don’t have tests, I’m putting you in the category of “people who build toys”.
</span>
 There is lots of good advice on how to write test suites, especially if you work on compilers</a>.
Once you have a test suite, make sure it is run on every commit.
GitHub makes this particularly easy through GitHub Actions</a>: you define a configuration to install all the tools needed to run your system, and define how to run commands.
GitHub will use this configuration to run tests on every commit, including pull requests and allow you to ensure that bad code doesn’t get committed.</p>
Reviewing</h3>
Code review is the practice of having a second person read your code before it gets merged into the codebase.
This is the place where a senior contributor helps the code author understand how their code affects other systems, if there are better ways to implement a feature, suggest refactorings, and remind them to write tests.

</label>
</span>


Yeah, I’m looking at you. I know you didn’t do it.
</span>

It is a particularly powerful tool for research mentorship: it allows you to teach junior contributors how to write good code, practice rigorous testing, and develop a sense of aesthetics about code architecture.</p>
Deployment</h3>
Deployment usually happens after a particular code change has been merged in to the main branch.
A common set of things to deploy can be:</p>

Release code artifacts such as packages or docker builds.</li>
Build and deploy a new version of a website.</li>
</ul>
Automating deployment is an upfront cost, but it can have tremendous benefits. Answers to questions like “how do I fix this typo on the website” or “how do I release a new docker build” simply become: “edit this file and open a pull request!”.
GitHub Actions</a> can again be used to build and deploy artifacts.</p>
Summary</h2>
Scaling up research projects requires balancing short-term goals like writing a paper or hacking together a feature for a deadline, and long term goals, like ensure code readability, reducing tech debt, etc.
It is also challenging to deal with the turnover on projects.
The above guidelines are designed to build enough institutional memory</em> and infrastructure</em> so that people can continue contributing and developing the system well beyond the “research prototype” phase.</p>


Your Eternal Spark
2023-09-01T00:00:00+00:00
Content Warning: Death</strong></p>
My dear friend and colleague Priya Srikumar passed away in a car accident.
I write this post to memorialize the impact they had on me and other people around them.
If you knew Priya in their capacity as an academic and would like to add something, please email me</a>.</p>
I started my PhD at Cornell in 2018 and Priya was an undergraduate student then.
I first met them when they gave a talk at the programming languages discussion group, which usually has research talks from PhD students.
I remember thinking that their talk would put most PhD students to shame: they deeply understood complex math that took years to grasp and gave an easy-to-follow talk which is no mean feat.</p>
Priya continued doing research with professors in the programming languages group and applied to various graduate programs.
They got into most of the top programming languages departments in the country.
After an arduous decision process, which I was a part of, Priya decided to stay at Cornell for their PhD and eventually started working with my advisor.</p>
Priya had an infectious spark for research. They would get excited about a topic and pursue it to obsessive depth.
Before our first conference together, ASPLOS 2023 in Vancouver, Canada, Priya sat with me and read a paper together with me on the train ride from Seattle.
This is an unconventional choice because the train ride has some beautiful views.
However, Priya always cherished the chance to learn something new.
They grasped the intricacies of a computer architecture paper (something that they only recently started learning about) and eventually asked some thoughtful questions from the paper’s authors.</p>
Most people I know struggle to network at conferences but watching Priya at the conference was remarkable.
They gave a widely-liked WACI talk in one of the most crowded rooms at the conference.
They made fast friends with tons of people, both junior and senior, and impressed them with their depth of knowledge in both computer architecture, formal methods, compilers, and programming languages.
Priya always made an impression on the people who interacted with them.</p>
Apart from research, Priya shared in my joy of food, music, and boba tea.
They showed me around their favorite food places in New York City and took me to Korea town to try my first ever bingsoo.</p>
My last interaction with them was at FCRC 2023 where they helped co-run a tutorial on Calyx.
Priya asked me about how to manage the mid-project blues, the part of research when you have a direction to pursue, but it gets hard to maintain the momentum.
This is something every researcher experiences, and so I encouraged them to continue pursuing their work.
I remember jokingly telling them, “I’m excited to read the paper that comes out of your work; don’t make me wait so long!”
They sparked up on this and promised me that I will be reading it soon.
We hugged and I left to catch my flight.</p>
Priya was full of potential. Potential to do great science, potential to be an amazing, kind, thoughtful mentor, potential to change the world.
I, and many others, imagined seeing them at every conference we went to.
They were supposed to be a permanent fixture of our lives.
At every conference I attend, I will miss them.
At every group discussion, I will miss them.
When I talk about my mentees, my friends, and my time in grad school, I will miss them.</p>
Goodbye Priya. Your spark is eternal.</p>
Joshua Turcotti said:</p>

It always appeared as if things came naturally to Priya, but the more you got to know them the more you realized that the thing that came most naturally was putting in more hours than I ever knew the day had methodically researching, preparing, and attacking the problems in their life. They always fought battles hard, but came out the other side with skills and passions that inspired others. Music was a huge part of their life that they worked so hard to cultivate, becoming a beautiful vocalist and an aspiring guitarist. They never wanted to be passionate about things alone, and so they took that love of music into their community, performing with our beloved Dexter and with a local band. They always brought out the best in the communities that were lucky enough to have them, and in short time they always become one the best and brightest elements of those communities themselves.</p>
</blockquote>
Ben Kushigian said:</p>

They were such a sweet person, and I was always so happy to see them in PLSE. They had a smile that would break me out of my grumpiest of moods. I can’t believe they’re gone, and I can only imagine what folks who were closer to them must be going through. They were a special person, and I’m really lucky to have known them, even just for a bit.</p>
</blockquote>
Omkar Bhalerao said:</p>

Priya has always struck me as a helpful educator in addition to a knowledgeable academic. They never hesitated to help out with the overall structure of the class and perform logistical duties even as a heavily burdened PhD student. During our time together as TAs, the work ethic that they displayed heavily inspired me to improve my own efforts as a TA for the class, and I also learned quite a lot from them.</p>
</blockquote>
Paulette Koronkevich said:</p>

I am so fortunate to have known Priya for too short of a time. I met them at POPL in 2020, and found an instant connection. They reminded me why I wanted to pursue research–because of the excitement of new discovery, but also because of the kindness of the community. Priya was an undergraduate at the time but was already participating in the community and learning so much. The next time we met in person was 3 years later, and it was as if no time had passed. We instantly wrapped each other in a huge hug. I expected that these wonderful meetings and hugs would happen for many, many more years. I am so proud of everything they’ve accomplished, and the impact they’ve had on all their communities, and I will miss them greatly. We’ve lost a piece of our heart, but Priya has shown us that our heart should remain open.</p>
</blockquote>
Zach Sisco said:</p>

I first met Priya at ASPLOS, and then FCRC, this year. I have fond memories of Priya at those conferences—whether it was talking shop, chatting about our shared passions and hobbies (music and film photography), or just goofing off. Conferences can be draining, but Priya had a level of energy and curiosity that was infectious (in a good way). If I was lucky to catch them in the hallways, they’d always pep me back up; I will always remember the kindness and compassion they showed me in those brief moments. Although I only knew Priya through academic conferences, I wish I could have known them more, and their bright presence will be missed at future ones.</p>
</blockquote>
Alexa VanHattum said:</p>

I first really got to know Priya when they were still an undergraduate—but already serving as a TA in Cornell’s challenging graduate programming languages class. The care and dedication Priya showed to our students was representative of who they were as a person and as a researcher. Priya was always, always helping someone with something—organizing discussion groups when they were still a first year Ph.D. student, rallying enthusiasm for department picnics, commiserating and offering me tips as we both were training our own puppies. Over the years, Priya and I would meet for Gimme coffee chats to talk about life in the department and what would come next. During our last chat earlier this year, Priya was so excited to hear about my job search, and I could already picture reading over Priya’s own faculty materials in a few years. Priya made such a mark on our community, and we will all miss them dearly.</p>
</blockquote>
Griffin Berlstein wrote a remembrance post</a>.</p>
The UW PLSE group wrote a remembrance post</a></p>


Transpiler, a meaningless word
2023-08-15T00:00:00+00:00
This tool is different from a compiler</em> which often has a complex frontend, an optimizing middle end, and code generators for various backends.
The big problem around most of the arguments to distinguish between compilers and “transpilers” focus on language syntax. However, anyone who wants one of these tools to actually work</em> has to contend with the fact that different languages will have different semantics</em> and translating between those is a complex task; a task that compilers</em> already do.</p>
Lie #1: Transpilers Don’t have Frontends</h2>
Let’s look at a simple Python to C transpiler.

</label>
</span>


Both Nuitka</a> and Mojo</a> both actually target this exact problem but sanely call themselves compilers.
</span>
 It takes python code that looks like this:</p>
def fact(n):
    x = 1
    for i in range(1, n):
        x *= i
    return x
</code></pre>
Into some C code like this:</p>
int fact(int n) {
    int x = 1;
    for (int i = 1; i < n; i++) {
        x *= i;
    }
    return x;
}
</code></pre>
Wow, pretty simple! But of course, that piece of python is not very idiomatic. We can make it a bit more terse using list comprehensions:</p>
import functools as ft
def fact(n):
    lst = range(1, n)
    return ft.reduce(lambda acc, x: acc*x, lst)
</code></pre>
Now our “transpiler” is in a little bit of trouble.
The implementation of reduce</code></a> is in pure Python so maybe we can still transpile it but range</code> is implemented purely in C</a>.</p>
Looking into the implementation, what’s even clearer is that matching the semantics of this program</em> is even harder:
range</code> is a Python generator which means that instead of actually computing the numbers from 1 to n</code>, it only produces them when asked.
This allows our method to save memory because we don’t actually have to allocate n</code> words and can work using just the memory for the lazy
implementation of the generator and the local variables.</p>
Another problem is that there are hundreds of built-in library functions that need to be compiled from Python from C.
Even a moderately useful subset would be unwieldy to implement by hand in our simple “transpiler”.
Maybe one strategy we can take is to build a some sort of tool that would simplify these hundreds of definitions into a more uniform representation
to work with.</p>
We’ll call it the transpiler-not-frontend</em> to make sure people understand we’re not building a compiler here.

</label>
</span>


It is not hard</a> to find examples of things mislabelled as transpilers. However, I won’t name any specific projects because this is just a dumb diatribe about words, I actually think the projects themselves are cool.
</span>
</p>
Lie #2: Transpilers are Simple</h2>
BabelJS is arguably one of the first “transpilers” that was developed so that people could experiment with JavaScript’s new language features that did not yet have browser implementations.

</label>
</span>


Technically, ECMAScript features.
</span>

For example, ES6 added support for generators (similar to those in Python) but a lot of browser frontends did not support them.
Generators are pretty nice:</p>
function *range(max) {
  for (var i = 0; i < max; i += 1) {
    yield i;
  }
}
// Force the evaluation of the generator
console.log([0, ...range(10)])
</code></pre>
Facebook’s regenerator</a> is a BabelJS-based “transpiler” to transform generators into language constructs that already existed
in JavaScript. Shouldn’t be too hard, right?</p>
var _marked = /*#__PURE__*/regeneratorRuntime.mark(range);
function range(max) {
  var i;
  return regeneratorRuntime.wrap(function range$(_context) {
    while (1) {
      switch (_context.prev = _context.next) {
        case 0:
          i = 0;
        case 1:
          if (!(i < max)) {
            _context.next = 7;
            break;
          }
          _context.next = 4;
          return i;
        case 4:
          i += 1;
          _context.next = 1;
          break;
        case 7:
        case "end":
          return _context.stop();
      }
    }
  }, _marked);
}
// Force the evaluation of the generator
console.log([0, ...range(10)]);
</code></pre>
Guess what, it is. Implementing generators is a whole-program transformation</em>: they fundamentally rely on the ability of the program to save its internal stack and pause its execution. In fact, making it fast requires enough tricks that we wrote a paper on it</a>.</p>
The point here is that people call arbitrarily complex tools “transpilers”. Again, the problem is the misguided focus on language syntax and a lack of understanding of the semantic difference.</p>
Lie #3: Transpilers Target the Same Level of Abstraction</h2>
This is pretty much the same as (2). The input and output languages have the syntax of JavaScript but the fact that compiling one feature</em> requires a whole program transformation gives away the fact that these are not the same language</em>.
If we’re to get beyond the vagaries of syntax and actually</em> talk about what the expressive power of languages is, we need to talk about semantics</a>.</p>
Lie #4: Transpilers Don’t have Backends</h2>
BabelJS has a list of “presets”</a> which target different versions of JavaScript.
This is not very different from LLVM having multiple different backends.

</label>
</span>


If you’re going to argue that the backends all compile to the same language, see (3).
</span>
 People might argue that when Babel is compiling its operations, it can do it piecemeal: that is, the compilation of nullish coaleascing operators</a> has nothing to how classes are compiled.</p>
This is exactly what compiler frontends do as well: they transform a large surface area of syntax into a smaller language and a lot of operations are simple syntactic sugar</em> which can be represented using other, more foundational primitives in the language.
For example, in the Rust compiler, the mid-level representation (MIR) does away with features like if</code>-let</code> by compiling them into match</code> statements. In fact, clippy</code>, a style suggestion tool for Rust, implements this as source-to-source transformation</em>: if you have simple match</code> statements in your program</a> in your program, Clippy will suggest a rewrite to you.</p>
Compilers already do things that “transpilers” are supposed to do. And they do it better because they are built on the foundation of language semantics instead of syntactic manipulation.</p>
Lie #5: Compilers only Target Machine Code</h2>
This one is interesting because instead of defining the characteristics of a “transpiler”, it focuses on restricting</em> the definition of a compiler.
Unfortunately, this one too is wrong. The term is widely used in many contexts where we are not generating assembly code and instead generating bytecode for some sort of virtual machine.
For example, the JVM has an ahead-of-time compiler from Java source code to the JVM bytecode and another just-in-time compiler to native instructions. These kinds of multi-tier compilation schemes are extremely common in dynamic languages like JavaScript as well.</p>
Lie #6: Transpilers are not Compilers</h2>
People seemed to scared of compilers and resort to claims like “I don’t want something as complex”, or “string interpolation is good enough”. This is silly. Anyone who has built one of these “transpilers” knows that inevitably, they get complex and poorly maintained precisely because of the delusion that they aren’t doing something complex.</p>
Programming languages are not just syntax; they have semantics too.
Pretending that you can get away with just manipulating the former is delusional and results in bad tools.</p>
Lindsey Kuper has a well-written article</a> on the same topic.</em></p>


The Stateless Manager
2023-07-15T00:00:00+00:00
I find myself repeating the following advice to my mentees: Assume your manager (advisor) remembers nothing about your previous meeting. Start from the top and build back the context</em> of your discussion before you dive into technical details.</p>
I have dubbed this the “stateless manager model” after functional programming paradigms where the program uses no state and instead acts only upon the inputs provided to it.
There’s a couple of reasons why this advice makes sense:</p>

Your manager/advisor probably has dozens of technical meetings in a week and a million other things they are working on. It is quite hard for them to remember all the details of your previous conversation.</li>
Even if</em> they remembered what the details were, you could’ve come up with a new way to think about the problem and building context from the ground up will reveal those to your manager.</li>
</ul>
The latter skill is quite important for junior developers and researchers.
The ability to effectively and concisely articulate what the problem is</em> is just as important, if not more important, than what the solution is.
In fact, a great many researchers are famous not for their ability to come up with a solution but their ability to articulate problems.
The Stateless Manager model is a particularly good way to practice this art.</p>

Disclaimer: This post is not a subtweet of my current or past advisors, all of whom are blessed far better memory than me. The point of the post is that even if</em> your manager remembers each conversation exactly, this is still</em> a good way to structure your approach to meetings.</p>
</blockquote>
Have comments? Email</a> or tweet</a> at me.</em></p>


Why Study Programming Languages
2022-09-19T00:00:00+00:00
This class is about the study of programming languages. Before we start, I want to perform two activities with folks here. First, I want us to answer two dumb questions:</p>

Why do we design new programming languages?</li>
What is a programming language?</li>
</ol>
While (2) seems to be the more fundamental question, we need to answer (1) to have any hope of even thinking about (2).</p>
So first, why do we design programming languages? Every program that can be written, can be written in C or assembly or Java or any of the dozens of languages we already have. So why do we design new languages?</p>
Common answers to this question will include words like abstraction, performance, convenience, usability etc. The problem with these answers is that apart from the measurable, they are all subjective, aesthetic choices. Convenience is a function of knowledge, familiarity, and community. Usability is similarly ill-defined and hard to measure. And of course, none of these metrics really predict which languages are widely used or popular.</p>
Consider the thought of inventing a whole new natural language just to express a new concept clearly. Explaining the rules of grammar and construction would certainly be simpler than any natural language provides. And yet, we’d have the small, troubling problem that this knowledge would be almost entirely useless; we need to learn a commonly known natural language to communicate with people. And yet, this is something that we can often find ourselves doing with programming languages with the hope that the concepts</em> learned in one language can be transferred into another; a world where being a polyglot is expected, not unusual.</p>
Perhaps this points to a striking similarity between programming languages. As they evolve, they take features from each other and converge into one language singular. They’re only differences being the syntax used to represent them.</p>
But of course, knowledge</em> of a language is different from mastery. An expert C programmer’s bit twiddling is akin of magic while a Haskell programmers tower of abstractions will make mere mortals cower away in fear.</p>
Here’s a hypothesis, the truth of which is unknown to me: we create programming languages to experience new ideas; ideas that would have remained inaccessible had we stayed with the old languages. Languages not just a form of expression but also a form of exploration. I do not create languages with the hope of expressing everything that was, but to express that which isn’t yet. It is the rare joy of a language designer to see their languages being used and abused to do something inconceivable to them.
I would point to dozens of historical examples of this, from ALGOL, to APL, every time a language has enabled expression and forward exploration, it has changed the course of computing.</p>
Now that we have some bearing of why we create programming languages, we can try answering what exactly is a programming language.</p>
Is a language just syntax? Surely not, since symbols don’t have any meaning to them. Perhaps it is the meaning of programs in the language, its semantics</em> that defines a language. But its meaning in terms of what? The results of programs? The internal states of this execution algorithm? Perhaps a purely mathematical description, detached from anything resembling a computer?</p>
Something resembling semantics of languages does seem to be a part of what defines a language but it is definitely not the full story. Ask a Python programmer why they like it and they’ll point to the amazing library ecosystem; ask a web developer why they like JavaScript, and they’ll wax poetic about Web 2.0; to a Haskell proponent, it’s type system, to a LISP programmer, macros, to a Go programmer, its concurrency model and so on. All of these characteristics define languages and yet have very little to do with semantics. So semantics alone do not define languages.</p>
Perhaps a tentative definition is that a programming language is defined by its syntax, semantics, and ecosystem. The former two are easy to study formally; we can teach you the mathematical tools needed to understand them. But for the latter, we must turn back to our first question: why do we design new languages. It is true that both Python and Go have ample libraries and a concurrency model. However, the exploratory power</em> of Python is enabled by the sheer quantity and quality of those libraries while Go’s power comes from its concurrency model.</p>
Therefore, I give my last definition of what a programming language is: syntax, semantics, and ecosystem in support of exploration; which parts of semantics and ecosystems to care about defined by what tools of exploration they provide. The study of programming languages encompasses all of these: syntax, semantics, type systems, runtime systems, garbage collectors, debuggers, IDEs, syntax highlighting, error messages, compilers, and design. Lines drawn between these are arbitrary, mostly by people like me trying to publish papers.</p>

I encourage everyone to create the most absurd, implausible, and impractical languages. Chasing the measurable is often useful, expressing the expressible is insightful, but never forget the true goal of language design: to explore and create what isn’t.</p>
</blockquote>


Lies Academics Believe
2022-08-02T00:00:00+00:00

I cannot be happy outside academia</li>
If it is novel, it is useful</li>
If it is useful, it is publishable</li>
Engineering does not matter</li>
Presentations do not matter</li>
Writing does not matter</li>
I will build and they will come</li>
Pedantry and insight are the same thing</li>
Critiquing and creating are the same skills</li>
This class will help my research</li>
This class won’t help my research</li>
Research area X is useless</li>
Research area Y is the ultimate truth</li>
Idea matters more than the execution</li>
Execution matters more than the idea</li>
Citation count indicates how smart someone is</li>
Industry does not do anything novel</li>
Industry does the hard 20% needed to make something real</li>
Everything was invented in the 80s</li>
</ul>
Addressed to my future self</em></p>

Other readings:</p>

Satirical ways to measure academics</a></li>
</ul>


Dear Sir, You Have Built a Compiler
2022-01-11T00:00:00+00:00
Dear Sir,</em></p>
I am afraid to inform you that you have built a compiler.
I know you wanted a “simple prototype” that would just add that
one feature to your programming model.
You said that “SSA is an overkill” and “it’s just way too much infrastructure
to maintain for a simple task” and yet, six months later, you have pile of
string mangling scripts that do not work—breaking every time a user input
slightly deviates from things you’ve seen before.</p>
Surely, switching to the unstable abstract syntax tree (AST) library provided by the compiler will be the
end of your woes; at least that way, someone else maintains a real parser and provides
at least a semblance of sane input for you to transform.
But wait, the AST is massive. “Do I really have to handle all 500 different AST nodes?” you ask yourself?
Surely not.
Your users aren’t crazy; they don’t use all the weird features of this language.
So you march on, and handle the 50 AST nodes that matter, certain, that this
will be the last of what you need to do to maintain this pile of hacks.</p>
Ah, but wait! Inevitably, someone wanted to nest a for loop inside a switch
statement, a struct definition within that loop, and a conditional in
expression position.
Tired, you patch in support for each feature, distracting from the crucial
features you should be working on and shipping.
One of your brilliant team member suggests a pre-processing stage: de-nest all definitions,
hoist all expressions, flatten out all control, and that way, you only have to handle
50 AST nodes because you will know</em>, yes know</em>, that the program cannot have any other
shape.</p>
Except once that engineer leave, who will know</em> what assumptions you encoded?
Those littered asserts? The inscrutable “unreachable code” errors? Who will know</em>,
how you simplified your AST?
So you rolled out your own AST library, so that you may compile, nay, transpile</em> your
code and expose your assumptions in your data structures.

</label>
</span>


Certain, of course, that because you’re transpiling</em> JavaScript to JavaScript, it is going to
way easier than what real compilers do.
</span>
 What glorious engineering, you say to yourself.</p>
In the last leg of your journey to avoid building a compiler, your manager tells you
that your code should run on older machines, which only support version 0.8.
Version 0.8, of course, does not support the brilliant type-level encodings your
transpiler generates to implement your feature.
Not my problem, your manager says.
So you write some code that simplifies your transpiled</em> code further, making
it use only features present in version 0.8.
Done at last, you say to yourself, without having to build a compiler.</p>
A parser, an intermediate representation, transformation passes, and a code generator.
Dear Sir, you have built a compiler.</p>
Addressed to,</em>
Those who did not want to build a compiler</em></p>

Other readings:</p>

Compilers for the Future</a></li>
If Architects had to work like Programmers</a></li>
</ul>


Personal Infrastructure for PhD Students
2022-01-09T00:00:00+00:00
Personal infrastructure loosely refers to all the tools and systems I have to
interact on a regular basis to do my job as a PhD researcher.
As a systems researcher, this includes obvious stuff like command-line tools
and programming workflow as well as as the set of tools to build websites,
write papers, manage TODO lists, and record videos</a>.</p>

Websites</a></li>
Software</a></li>
Technical Writing</a></li>
Programming Workflow</a></li>
</ul>
Websites</h2>
Let’s get this one out of the way–as a PhD student, you need to have a personal website</em>.
It doesn’t need to dazzle, it doesn’t need to use bleeding-edge web frameworks
and CSS animations but it needs to exist and it needs to be easy to find.</p>
My website is built using zola</a>, a static website generator written in Rust.
Almost all of the styling is written in plain CSS with rules to make the website responsive.

</label>
</span>


Please, for the love of god, make your website readable on a phone.
</span>
</p>
Tools.</strong> Using a static website generator, which takes all of your content written in a language of your
choice and makes it web ready is going to be your best bet to have a maintainable infrastructure.
Jekyll</a> is wildly popular but provides way more features that I’ve ever needed.
Previous versions of this website used:</p>

Hakyll</a>: Too slow and Haskell package management was a travesty.</li>
Frog</a>: Racket is amazingly expressive, but slow.</li>
Hugo</a>: Blazing fast but updates broke old code.</li>
</ul>
I’ve settled on zola</a> because it’s fast and provides just enough features to maintain my website.</p>
Styling and responsiveness.</strong> The style of your website is up to you. Many websites use
templates such academic</a> which are pretty good, but again, may provide far too many distracting
features.
If you’re hand rolling your own CSS, I recommend using CSS Flex</a> and Grid</a> to make your
content responsive. They provide responsive layout features that Bootstrap</a> provides without
all the cruft.</p>
Domain, deployment, and discoverability.</strong> I highly recommend buying your own personalized domain name–domains
provided by institutions are temporary often annoyingly hard to remember. For example, my Cornell web address
is https://cs.cornell.edu/~rnigam</code> which requires you to remember what weird internal ID Cornell gave me.
The second, potentially bigger reason is website deployment</em>.
My current setup uses Github Actions</a> to automatically update my website
when I push changes to my website
repository</a>.
Doing this with the Cornell web hosting platform requires maintaining my own
infrastructure of deployment hooks, servers, and scripts which is yet another
thing to debug.
Finally, discoverability makes it easier to find your website when someone looks up
your name on Google.
Doing this as an academic is pretty easy–make sure a bunch of .edu</code> websites point
to your website.

</label>
</span>


Unless you have the same name as someone famous in which case, tough luck.
Or take it as a challenge to be more famous than they are.
</span>
 This includes your advisor’s website, department website, research group website, etc.</p>
All of this advice generally applies to project websites</a>
as well, which again, I highly recommend to make your research more visible
and approachable.</p>
Software</h2>
I use a fairly minimal set of software tools to do my day-to-day work, make
presentations, and manage tasks.</p>
Slides.</strong> Most of my presentations are written in Apple Keynote.
People I know are divided on the use of animations in slides but I’m staunchly
in support–it gives your slides that much more polish and forces you to
memorize the transitions resulting in an overall better talk.
I have yet to see a good talk made in LaTeX beamer</a>; my unscientific
belief is that beamer encourages adding too much math on your slides which is
often the wrong thing to do for talks where the goal is to give an intuition
behind your work.

</label>
</span>


For teaching, however, I’ve found beamer to be a pretty good tool.
</span>
</p>
Recording.</strong> I’ve had to record talks for virtual conferences in the past and
have used Screenflow</a> for this. It works for what I do and I haven’t needed
anything more. There might be better alternatives out there.</p>
Task Management.</strong> Before and during my PhD, I’ve used a string of task management
systems, from Todoist</a>, Wunderlist</a>, and even Github projects</a>. I’ve settled
on Omnifocus</a> for the last two years and am quite happy with it.
It uses the Getting Things Done</a> school of task managements where you recording everything</em>
you need to do, file it under the right projects, and complete it when needed.
Omnifocus’s most powerful feature for me has been the defer</em> action which
hides unactionable tasks from my list and makes them visible on the right
day–providing an almost inhuman ability to recall commitments and tasks.</p>
Technical Writing</h2>
Technical writing is the bread and butter of researchers–I’m always writing
documents, either to discuss ideas with my team or to polish them up for a
paper.
I make copious use of Notability</a> to jot down notes on my iPad and
Markdown</a> to write down ideas about systems I’m building.</p>
Papers need to be far more polished and therefore I’ve only ever used LaTeX</a>
to write them.
When starting a new paper, I copy over the bibliography file from the most
recent paper and a pervasives.sty</code> file that contains all the accumulated
LaTeX hacks I’ve ever had to do.
I tend to write everything in one giant file which makes it easier to track
down a phrase in the paper and start editing it.

</label>
</span>


Many people prefer separating out each section into a new file which might suit
your team’s contribution style better.
</span>
</p>
Graphs and figures.</strong> Papers often need to contain visual elements like
system diagrams and measurement graphs. I use OmniGraffle</a> to make diagrams
and use python scripts that use seaborn</a> or Vega</a> to generate graphs.</p>
Programming Workflow</h2>
Onto the good stuff and things most likely to cause a flamewar. My configuration
for various tools is publicly available</a>.</p>
Editor.</strong> I use neovim</a> which is a modern rewrite of the vim</a> editor.
If you’re interested in emacs, I’ve heard good things about spacemacs</a>.
If you like the 21st century, you may use VS Code</a> instead.</p>
Shell.</strong> zsh</code> has been my favored shell for a long time. It provides a bunch
of quality of life improvements over bash</code> owing to its powerful plugin system.
Oh my Zsh</a> is a popular addon for zsh</code> and adds a bunch of nice features
to it but can often be slow and bloated. antigen</code></a> is a plugin manager for zsh</code>
that mostly circumvents these problems by only installing what you need and
aggressively caching slow things.
In a past life, I used the fish</code></a> shell but got frustrated with the POSIX
compatibility problems which would break a lot of build scripts.</p>
Tmux.</strong> Tmux</a> is a powerful terminal multiplexer that allows you manage
multiple shell sessions side-by-side.
When programming, I usually split my terminal into three sections: one for the
editor and two for interactive commands.
My tmux configuration changes the default keybindings to be easier to remember
as well as some visual elements to track the current window, name of session,
time, and date.</p>
Miscellaneous.</strong></p>

fzf</a> is a generic fuzzy-finding tool that supercharges your search history command among other things.</li>
entr</a> watches for file changes and executes a command.</li>
autojump</a> is database-backed cd</code> alternative.</li>
rg</a> is a modern grep</code> alternative which much, much faster.</li>
fd</a> is a find</code> alternative with saner defaults.</li>
</ul>
Have comments? Email</a> or tweet</a> at me.</em></p>


Commoditize the Complement of Your Research
2021-03-13T00:00:00+00:00
“Commoditize your complement” is an idea about how companies can build profitable
markets without complete vertical integration or monopolization.

</label>
</span>


I highly recommend “Laws of Tech: Commoditize Your Complement</a>” for a more
in-depth look into this idea.
</span>

Very briefly, the idea is this: every product has a substitute</em> and a complement</em>.
A substitute is a product that provides the same functionality and therefore
competes with your product.
For example, Zoom is a substitute for Skype.
A complement is product that people buy along with your product.
For example, operating systems complement personal computers.
So, what can researchers learn from creating markets?</p>
The Complement of Your Research</h2>
Some of my favorite research projects have an interesting characteristic:
instead of competing with people in a hot research area, these projects build
the infrastructure that everyone needs to use.
Over time, such projects win out in terms of research impact and ability to
do novel research because everyone</em> who does research in that area ends up
using these tools.</p>
Let me use a particular research tool as an example.
Rosette</a> is an embedded language in Racket that allows researchers to quickly
develop solver-aided tools</em>.
Solver-aided tools are a “hot topic” in programming languages research.
The high-level idea is encoding the semantics of a program into boolean (or richer
logics) and using SMT solvers like Z3</a> to either verify programs or
automatically synthesize them from specifications.

</label>
</span>


James Bornholt’s introduction to program synthesis</a> provides a good overview of
the area.
</span>
</p>
Roughly speaking, anyone who attempts to build a solver-aided tool has to do
three things: encode the semantics of programs as SMT, repeatedly query
the SMT solver, and transform the output of the SMT solver back to the input
language.
Before languages like Rosette, researchers would spend a lot of time building
compilers to painstakingly transform programs to SMT, debugging problems with
the encodings, and transforming the output from the SMT solver.
Every researcher</em> who wants to build a solver-aided tool would redo this work
or build upon someone else’s unmaintained research code.</p>
The idea with Rosette is simple—build a framework where you can write an
interpreter for your language and automatically</em> turn it into a solver-aided
tool.

</label>
</span>


This simple idea is, of course, built upon deep insights about how solvers
and symbolic execution work.
I recommend reading the [Rosette paper][rose-paper] for those
interested.
</span>

The original Rosette paper</a> was a novel and interesting contribution,
therefore justifying its publication.
However, the real research impact of Rosette has been from its continued use
long after the paper was published.</p>
As of writing this blog post, the Rosette project page</a> outlines
19 research projects that use it in some capacity.
An important reason for this is because the Rosette authors continued to
maintain Rosette, provide support, and build upon the original work.
However, these things only help if there a demand for a tool like Rosette—if
people didn’t care about building solver-aided tools, Rosette would not be as
successful.</p>
Rosette is a successful example of the “commoditize your complement”
principle—instead of competing with people working in a “hot area”, build
infrastructure that boosts the productivity of the people in that area.
This way, your work becomes foundational and people can more productively focus
on advancing the state-of-the-art using it.</p>
Building on Success</h2>
A long-term benefit of building tools that complement a research area is that
your tools and systems become legitimate grounds for follow-up research.
Generally speaking, it is hard to motivate a research project where you solve a
problem for your made-up system.
However, if even dozens of other groups use your system, you can both
reasonably claim that the follow-up work is important and</em> other people’s work
to evaluate your follow-up work.</p>
SymPro</a>, a tool built upon Rosette, is an example of this.
SymPro is a symbolic profiler for Rosette.
Put simply, it profiles programs written in Rosette and finds code that causes
the SMT encoding of a program to blow up.
SymPro was able to use the existing Rosette ecosystem to develop a robust tool
and evaluate it on code written by users of Rosette.
This is both a value-add for users of Rosette and a compelling case to justify
the research paper.

</label>
</span>


The research contributions of the paper are not tied to Rosette.
However, building upon Rosette makes the paper that much more compelling.
</span>
</p>
Finding a Complement</h2>
Rosette was not the first tool to address the needs of a particular research
community.
LLVM</a>, Valgrind</a>, KLEE</a>, the Click modular router</a>, GEM5</a> etc. all found research areas where people
were desperate to build tools but had no good infrastructure.
All of them capitalized on this need by building robust tools and providing
support.
So, the valuable takeaway from this is that research projects that seek to
support instead of compete might win out in the long-term.</p>
Have comments? Email</a> or tweet</a> at me.</em></p>


Languages, Tools, and Techniques for Accelerator Design
2021-02-17T00:00:00+00:00

If you’re excited by the ideas in this post, please consider registering for
and submitting to LATTE</a>: the first workshop on languages, tools, and
techniques for accelerator design co-located with ASPLOS.</p>
</blockquote>
FPGA-based accelerators have opened up a new frontier for accelerator design; instead of spending months building and fabricating silicon chips, programmers can buy a cloud instance to run custom hardware accelerators within hours. With the remarkable new hardware, there is a need for remarkable new software—existing tools and languages used to describe circuits provide assembly-like abstractions and cannot enable the kind of rapid iteration we’ve become used to in the software ecosystem. Innovation in languages, tools, and techniques for accelerator design is key in making accelerator design productive, accessible, and useful.</p>
The Perks and Perils of Custom Hardware</h2>
In contrast to conventional processors, hardware accelerators ruthlessly trade off the generality of input programs for simpler and faster hardware. For example, Google’s tensor processing units (TPUs) use systolic arrays which exploit the data-reuse patterns in linear algebra kernels by connecting processing units in an array-like configuration. While a TPU is not a general purpose processor, it can dramatically speed up linear algebra workloads while being power efficient. With the imminent death of Moore’s law, computational improvements will be driven by such hardware accelerators.</p>
While silicon-based accelerators provide the most performant implementation of an accelerator, designing them is a challenging and time-consuming task. Architects spend months outlining a high-level architecture, implementing it using low-level hardware description languages (HDLs), and fabricating it. Finally, the process of integrating these accelerators is tedious—each accelerator must be directly connected to a physical machine and programmed using low-level memory-mapped interfaces.</p>
Field programmable gate arrays (FPGAs) are key to a more radical approach to accelerator design—where programmers can rapidly reprogram and iterate on designs within hours instead of months. FPGAs represent a programmability-efficiency trade-off between ultra-specialized silicon accelerators and traditional processors. They can be programmed to simulate a particular hardware design and do so more efficiently than processors. On the other hand, while less efficient than silicon accelerators, FPGAs can be reprogrammed in a few seconds, cutting out the tedious fabrication process. This trade-off can be well worth it: FPGAs can provide an order of magnitude speedup and can accelerate diverse workloads.</p>
The limiting factor in the design and proliferation of FPGA-based accelerators is languages and tooling. Hardware description languages (HDLs), which operate at the abstraction of gates, wires, and clock cycles, are the dominant way of designing hardware. While useful for building high-end processors, these abstractions are inappropriate for designing accelerators. For example, a simple matrix-multiply accelerator can require hundreds of lines of carefully crafted HDL code to coordinate data and control flow. Accelerator designers are stuck specifying low-level circuitry instead of rapidly iterating on high-level architectures.</p>
New, higher-level programming models are the key to the ubiquitous use of FPGA-based accelerators. Beyond ridding developers from low-level abstractions, such programming models also enable novel solutions to classic problems such hardware verification and automatic optimization.</p>
High-Level Programming Models</h2>
The obvious benefit of higher-level programming models is the ability to specify hardware without dwelling on low-level details. For example, using high-level synthesis (HLS) compilers, programmers can compile C++ programs into hardware designs. The aforementioned matrix multiplier can be implemented in a few lines of code:</p>
for (int i = 0; i < N; i++)
  for (int j = 0; i < N; i++)
    for (int k = 0; i < N; i++)
      #pragma HLS UNROLL factor=5
      C[i][j] = A[i][k] * B[k][j];
</code></pre>
The challenge in such programming models, however, is exploiting the available hardware parallelism. For example, To express DOALL parallelism programmers can use the UNROLL</code> pragma to duplicate the loop body and perform five computations at the same time.</p>
However, loop unrolling demonstrates the key difference between a C++ program meant to run on processors and ones used to generate hardware. Processors take advantage of unrolled loops through their superscalar design and complex memory hierarchies that can service multiple requests every cycle. In contrast, unrolling a loop in an accelerator design instantiates five physical</em> multipliers in the final design which are connected to primitive memories that can only serve a single read or write request every cycle. This means that without careful manual organization of memories, the additional multipliers would stall most cycles waiting on read and write requests.</p>
The challenge here is the need to connect the high-level abstractions to the fundamentally physical</em> nature of hardware designs. This balance is precarious: expose too much information and we’re back to the abstraction level of hardware description languages; too little, and the programming model will unpredictably generate poor hardware designs without providing programmers any useful feedback.</p>
Recent work in this area demonstrates how programming languages techniques can help overcome these problems. For example, Dahlia</a> (my work) uses a substructural type system to enforce memory constraints in HLS programs while Aetherling</a> is a domain-specific language that automatically generates high-performance streaming accelerators.</p>
Automatically Optimizing Accelerators</h2>
The vast majority of optimization, analysis, and verification of hardware accelerators occurs at the level of hardware description languages (HDLs). High-level programming models, in addition to improving programmer productivity, can enable novel and scalable solutions to these problems.</p>
A key property of HDLs is that, by default, everything executes in parallel. In order to encode control flow, programs must implement their own state machines that activate the right set of actions to execute every cycle. Pseudocode for this pattern demonstrates how gnarly it can get:</p>
x = (state == 0) ? 1
  : (state == 1) ? x + 1
  : 0;

state = (state == 0) ? 1
      : (state == 1 && x < 10) ? 0
      : (state == 1 && !(x < 10)) ? 2;

y = (state == 2) ? 1 : 0;
</code></pre>
The input program, which uses control flow constructs, demonstrates the actual intent:</p>
x = 1;
while (x < 10) {
  x = x + 1;
}
y = 1;
</code></pre>
Not only is the latter program easier to write down, but it also reveals useful facts about the program. For example, the variable x</code> is never used after the loop i.e., it is no longer live</em>. Optimizing compilers, both for software and hardware, can use this fact to reuse the registers that store x</code>. However, yet again, software optimizations don’t precisely capture the nature of hardware designs. In a software program, sharing registers is almost always a good idea, limited only by the compiler’s knowledge of aliasing. On the other hand, sharing registers in accelerators requires instantiating multiplexers</em> which control the input and output signals from the register. The choice of trading off registers for multiplexers is target-dependent: registers are cheap on FPGAs but costly on silicon-based accelerators while multiplexers are the opposite. Attempting to port software optimizations without knowledge of hardware is futile.</p>
Language-based abstractions can capture such trade-offs.
For example, Calyx</a> (my work), proposes a new intermediate language for building
accelerator generating compilers.
Calyx uses a split representation of programs: a hardware-like language
captures structural facts while a software-like language is used to precisely
express the control flow.
Using both structural and control flow information, Calyx can build a set of
generic optimizations and analyses that benefit all compilers aiming to
generate hardware.
On the other hand, μIR</a> uses a task-parallel representation to optimize
accelerator designs while SPARK</a> automates speculative and parallelization
optimizations.</p>
Language-based abstractions similarly have the potential to enable scalable verification of accelerators. Instead of coping with the always-parallel semantics of HDLs, verification techniques can utilize higher-level control flow information to perform modular verification.</p>
State of the Art</h2>
A language-oriented view of classic problems in hardware design has resulted
in a slew of novel solutions:</p>

Verification</strong>: Formally verified hardware design ([Kami][]) aims to eliminate
the slow and tedious process of hardware verification.</li>
Virtualization</strong>: Language-based virtualization of FPGA designs
(Cascade</a>) has been shown to be a promising avenue for fast
state-snapshotting and transparent relocation.</li>
Programming Models</strong>: New programming models for designing
systolic arrays (SuSy</a>) and streaming accelerators (Aetherling</a>)
demonstrate the potential of a domain-specific approach to hardware design.</li>
Type Systems</strong>: Type systems that capture hardware constraints in a
high-level programming model (Dahlia</a>) can simplify manual optimization
ensure that well-typed programs go fast.</li>
</ul>
Languages, Tools, and Techniques for Accelerator Design</h2>
Recent work on language-oriented accelerator design is distributed across three research communities: (1) The electronic design automation (EDA) community, focused on HLS and tools for silicon-based architectures, (2) the compilers community, building infrastructure and optimizations for emerging architectures, and (3) the PL community, exploring new languages for designing and verifying hardware designs. There is a growing consensus that work on FPGA-based accelerators needs to be interdisciplinary—building a robust programming language requires precise semantics, construction of performant compilers, and characterization of the underlying architectures like FPGAs.</p>
In order to bring together people who are excited by the idea of a language-focused future for hardware design, we’re organizing the first workshop on Languages, Tools, and Techniques for Accelerator Design (LATTE)</a> which will be co-located with ASPLOS 2021. If you’re interested, consider submitting a 2-page position paper and/or come by!</p>
Have comments? Email</a> or tweet</a> at me.</em></p>


Compiling for the Reconfigurable Future
2020-04-16T11:59:11-04:00

FPGAs, a form of reconfigurable
architectures, already power a large number of datacenter applications. With
FPGA acceleration becoming mainstream, it is the perfect opportunity to think
about programming models for designing next-generation high-performance
hardware.</p>
</blockquote>
Moore’s law is in its death throes. With Global Foundries announcing</a>
that they are no longer pursuing 7nm production nodes, fabrication companies
focusing on incremental improvements</a>, and the end of the
arguably more important Dennard scaling</a>, we’re entering a new
era where general purpose architectures are no longer the solution.
Reconfigurable architectures are one of the hottest research topics and perhaps
hold the key to application-specific hardware acceleration. However, without
a sane programming model, reconfigurable architectures might not achieve the
success they deserve.</p>
Reconfigurable Architectures</h2>
Since the dawn of computer architecture, we’ve focused on building processors
that are good at executing every</em> conceivable program. The advances in
pipelined designs, speculative and out-of-order execution all try to
dynamically discover regularity and parallelism in arbitrary programs and
execute them as fast as possible. The performance benefits of these technologies
are inarguable. However, all good things come at a price. In their
single-minded zealotry to improve single threaded performance, processors introduce
an incredible amount of control overhead</em>. Figure 1 shows the energy
breakdown of executing an add instruction. The control dominates the cost of
executing an instruction.</p>



</img>

Fig 1.
Energy breakdown of executing an add instruction from
Computing’s Energy Problem [Horowitz, 2014]</a>
</figcaption>
</figure>
</center>
So while modern processors
can execute arbitrary programs quickly, they leave a lot of room for improvement
with an individual program.
Instead of paying for the cost of the general control structures in every program,
what if your processor could pay for exactly the amount of control required
to execute the current program.
What if you
could reconfigure</em> your architecture
based on the currently executing program?
Reconfigurable architectures refer to the general class of architectures
that allow some degree of application-specific reconfigurability. The term
“reconfigurable architectures” is incredibly broad and spans everything from
the reconfigurability of meshes in massive many-cores</a> to bit-level
reconfigurable architectures. In this post, we’ll be focusing on Field
Programmable Gate Arrays (FPGAs) as a reconfigurable accelerator.</p>
FPGAs as Computational Accelerators</h2>
FPGAs were initially developed as high-performance simulators for circuit
designs. Testing a hardware design requires simulating its behavior over
thousands of clock cycles. With larger and more complex, the computational
power required to simulate and track the state of a design becomes increasingly
hard. Unfortunately, simulating a hardware design on a traditional processor
does not scale—imagine trying to simulate an i3
processor on a Pentium 4. FPGAs were designed as simulation accelerators. They
provide bit-level</em> reconfigurability which allows them to simulate wires and
gates in a hardware design.</p>
The bit-level reconfigurability also made FPGAs
viable as a cheaper, low-volume alternate to application specific integrated
circuits (ASICs). Instead of taping-out custom chips, FPGAs could be used to
prototype and integrate such accelerators without paying for a full
silicon tape-out. In domains like
signal processing or networking, where real-time deadlines really matter and
CPUs struggle to meet high-throughput requirements, FPGAs were successfully
used as computational accelerators. The common thread in all of these use cases
is that we really want to design custom circuits but don’t want to pay the
costs of producing a whole new chip.</p>
FPGAs happily chugged along in these niche roles for a long time without taking
off in a big way. Researchers knew that FPGAs could play a big role as flexible
accelerators but didn’t have a “killer app”. Between 2010-2016, an exceptional
team of computer architects demonstrated
that FPGAs could be used as
computational accelerators inside datacenters</em> through the Catapult</a>
project. Catapult, and its successor BrainWave</a>, showed that not only can
FPGAs significantly improve the performance of modern large-scale applications,
they provide enough flexibility to be used in multiple domains, accelerating
everything from Bing search, Azure cloud network, and most recently, ML models.</p>
Other cloud services like AWS have jumped on this trend and now offer F1
instances</a> which provide access to high-end FPGA units through AWS’s
pay-what-you-use model.</p>
FPGA Programming 101</h2>
Owing to its root as a hardware simulator, FPGA programming toolchains repurpose
existing hardware design languages (HDLs). As a circuit simulator, this is
a really good idea. You can simply take your preexisting hardware design and
run it on an FPGA.

</label>
</span>


I apologize to my architect friends. Running designs on an FPGA in reality can be an incredible challenge. FPGAs have different kinds of memory and performance characteristics. Most hardware design codebases are carefully engineered to separate FPGA-specific design decisions from the core design.
</span>
</p>
Unfortunately, when trying to run high-level application code
the level of abstraction afforded by HDLs is far too low-level.
Imagine
trying to write a convolution kernel by specifying every wire connection
into every adder and the computation that occurs at every clock cycle. Proponents
of HDLs will point out that we can eek out every bit of performance from a
low-level hardware design. However, this also means that design iteration times
are much worse. It can take many weeks of engineering effort to implement
and optimize a design.</p>
I am by no means the first person to point this productivity-performance
trade-off. Practitioners and researchers have created a multitude of
HDLs to improve the level of abstraction: BlueSpec</a>, SystemVerilog</a>, PyMTL</a>,
Chisel</a>, etc. all aim to use host languages to improve the level of abstraction
in some manner. For example, Chisel is embedded in Scala and provides
modularity and parameterization mechanisms using Scala’s type system.
However, HDLs still fundamentally</em> operate at the gate-and-wire
level of abstraction. Chisel designs, after being typechecked by the Scala
compiler, are expanded into a structural specification of the hardware design.</p>
A more radical technique to lift the level of abstraction would be to specify
how</em> the computation occurs and use a compiler to generate the hardware for
that specification. The architecture community has been exploring the idea
of transforming behavioral (or functional) descriptions of computation
into hardware designs. This is commonly referred to High-Level Synthesis (HLS)
in the community.</p>
High-Level Synthesis</h2>
High-Level Synthesis

</label>
</span>


“Synthesis” is borrowed from hardware design workflows—circuits are synthesized into silicon. This is just a compiler.
</span>
 is the idea of compiling a computational description
in a high-level programming language, 

</label>
</span>


Architects operate at the level of gates, wires, and clocks. C++ is a huge
jump in abstractions.
</span>
 like C or C++, into an HDL like
Verilog. HLS has been quite successful in a multitude of domains—everything
from digital signal processing</a> to machine learning
accelerators</a> has been implemented in HLS.</p>
The semantic gap between a functional description and timed hardware structures
is quite large. Hardware designs are timed</em> because they explicitly describe
the
behavior of individual circuits at the granularity of clock cycles. An HLS
compiler needs to transform the functional description into a data path</em>,
which describes the hardware structures that perform computations, and
a control path</em>, which describes the computation performed by components every
cycle.</p>
The promise of transforming any</em> C++ program into hardware is absurd at its
face. C++ programs dynamically allocate memory, use complicated control
structures, and are notoriously hard to analyze. Compare this to physical
hardware where memory sizes and control structures need to statically generated
at compile time.</p>
I’ll leave the specifics of where HLS fails for a future blog post. If you’re
curious, dive into our paper</a> on Dahlia</a> which identifies
some of these problems and shows how little bit of programming languages magic
can help.</p>
If you’re curious about this area, jump onto these cool blog posts and papers:</p>

FPGAs Have the Wrong Abstraction</a> by Adrian Sampson.</li>
High-Level Synthesis for FPGAs: From Prototyping to Development</a>.</li>
A Cloud-Scale Acceleration Architecture</a>.</li>
</ul>
(If you’ve written a blog post on HLS-related stuff, email it to me so I can
add it here!)</p>
Thanks for Adrian Sampson</a> and Alexa VanHattum</a> for providing feedback on early
drafts of this blog post</em>.</p>
Have comments? Email</a> or tweet</a> at me.</em></p>


The First Two Years of My PhD
2020-04-08T00:40:17-04:00
With the end of the Spring ’20 semester a month away, I have spent almost
two academic years at Cornell. A quick rundown of everything that happened:</p>
Failures</h3>

Short paper on FuTIL</a> rejected from LCTES ’20.</li>
Rejected from Facebook fellowship ’20.</li>
Dahlia</a> rejected from ASPLOS ’20 with two weak rejects.</li>
Rejected from Microsoft research internship for summer ’19.</li>
Rejected from Qualcomm fellowship application ’19.</li>
Rejected from the Facebook fellowship ’19.</li>
Rejected from the Symantec fellowship ’19.</li>
</ul>
Successes</h3>

Short paper on Diospyros</a> accepted to LCTES ’20.</li>
Selected as a finalist for the Qualcomm fellowship ’20.</li>
Dahlia</a> accepted to PLDI ’20.</li>
Research internship at Facebook Reality Labs</a> for summer ’19.</li>
Gave an invited talk to the Princeton Architecture and PL groups.</li>
Organized the Great works in PL</a> seminar.</li>
Organized the programming languages retreat in Fall ’19.</li>
</ul>

Fall ’18</h3>
I started at Cornell and was terrified that I would not be able to find an
advisor. I set up meetings with the PL faculty and Cornell and decided to
do a rotation with Adrian Sampson</a> during the fall and switch to
working with Nate Foster</a>.</p>
Adrian pitched me three projects: Use program synthesis
to automatically partition programs for reconfigurable architectures,
build a type system for a high-level programming language
for FPGAs, and a type system for graphics
and shader programming languages.</p>
I decided to work on the type system for FPGA programming (called Dahlia). I
was unsure that I would be a good fit for this project because I had no
background in computer architecture research. I hoped that my programming
languages experience would be useful for the project and that I could learn
enough about architecture to contribute to the project.</p>
I started reading about FPGAs, implementing various features for the Dahlia
compiler, and writing down proofs for various type system properties.
I also got involved with the programming languages group and gave my
first pldg talk</a>.</p>
We designed several language features for Dahlia. A particularly thorny
design issue was supporting complex iteration patterns while providing
type safety. We came up with memory
views</a> to solve this.
The design of views felt inelegant.</p>
I volunteered at OOPSLA ’19 in Boston where I met a lot of new and old
friends. I applied to industrial fellowships and was rejected from them.
Adrian said that they prefer to accept more senior students and that applying
was more important than being accepted. I agreed.</p>
During the semester I also realized that I was enjoying working on
Dahlia and asked Adrian to formally be my advisor. He agreed.</p>
Winter ’18</h3>
I went back to India for the winter break where I read a bunch of papers
and reviewed applications for PhD applicants. I convinced Nate</a> to
help me organize the Great works in PL</a> seminar as an excuse to
read classic PL papers.</p>
Spring ’19</h3>
I came back to Cornell and started implementing memory views in Dahlia.
I kept feeling that the OCaml codebase was slowing me down so I rewrote
Dahlia</a> in Scala and implemented
memory views. The implementation demonstrated that views were inelegant
so we came up with a new implementation of memory views.</p>
Implementing views turned out to be a lot more challenging than I originally
expected and it took me four tries to get it right. Before the final attempt,
we realized there was a fundamental problem with checking views that we didn’t
know how to solve. I was feeling particularly down that day. During my
walk back home, I discovered an elegant solution for compositionally reasoning
about views.</p>
The biggest challenge with Dahlia was finding the right pitch for it.
We had some idea that it made hardware designs “more predictable” because
each language construct had a direct hardware mapping. However, we didn’t
know how to demonstrate this “predictability”. I was wary of qualitative
arguments. I spent the semester writing code and text. We started porting
an FPGA programming benchmark suite to Dahlia to see how it faired with larger
examples.</p>
In the background, I decided to do a summer internship that year and started
interviewed with MSR and Facebook Reality Labs (FRL). MSR rejected me and I
eventually accepted an offer from the silicon research team at FRL. I
also attended ASPLOS ’19 with Adrian and made a lot
of new architecture friends. Architects seemed to be livelier than PL
people because they’re living on the
EDGE</a>.</p>
Summer ’19</h3>
I spent my summer in Redmond at FRL using program synthesis to solve hardware
problems. Working on program synthesis is a roller coaster: the
solver gives you solutions and you’re happy. At some point it stops scaling
and you don’t know what to do anymore and everything is sad.</p>
I also wrote a few short sections for the Dahlia paper hoping to hit the
ASPLOS ’20 deadline.</p>
Fall ’19</h3>
My team at FRL sufficiently liked my project to ask me to continue working
on an offshoot. I realized that if I worked on a program synthesis project
alone, I would be sad all the time. I asked Alexa VanHattum</a> if
she wanted to collaborate on it with me and she said yes.</p>
I flew back to Ithaca a week before the ASPLOS ’20 deadline fully expecting
to miss the deadline since we didn’t have a lot of content in the paper.
Adrian said we should hit the deadline so I switched gears into paper
writing mode. We wrote a paper in a week and submitted it to ASPLOS. I didn’t
expect the paper to get in because of a weak evaluation.</p>
A central problem with the evaluation was that it simply reimplemented C++
benchmarks in Dahlia which resulted in the same area and latency numbers
as the baselines. The evaluation didn’t say anything interesting about how
Dahlia enabled “predictable hardware design”—which was the title of
our paper. I was starting to feel angsty about the project and felt like there
was no way evaluate it.</p>
I was burned out from the paper writing so I asked my friend Sam
Ginzburg</a> to host me at Princeton for a week. He recommended that I
give a talk to the architecture group which was a great idea but destroyed
my plans of not working during the Princeton visit. I visited Princeton, gave
a talk, and met a lot of cool people. Sam was working on a measurement project
and had a lot of pretty graphs. I decided that the only way to calm my angst
with Dahlia was to perform measurements and quantify predictability. I did
not yet know how.</p>
I continued spending my time implementing the compiler and getting the
benchmarks running. During an auspicious trip to the Applications Driving
Architecture (ADA) symposium, I came up with a plan to show that Dahlia enabled
predictable design.</p>
The plan was as follows:</p>

Take a hardware design and enumerate all the design points.</li>
Run all the points and extract statistics (area and latency).</li>
Show that the subset of design points Dahlia accepts smoothly trade off
area for latency and are therefore “predictable”.</li>
Profit.</li>
</ol>
The challenging part of this plan was getting all the data. A back-of-the-envelope
calculation showed that we’d need a few months of compute time to
get all the data. I had, unfortunately, reached a point where I needed to
build a distributed experimentation framework.</p>
I got to work building the framework on top of an existing in-house benchmarking
server. It took me three weeks of relentless Python hacking to get multiple
AWS machines to run FPGA designs. Once we had that, pretty graphs started
rolling out and I started confirming various claims about Dahlia quantitatively.
Around this time, Dahlia was rejected from ASPLOS.</p>
While this was expected, I was still sad for a few days. We decided to resubmit
to PLDI. With three weeks to go, I ran the capstone
experiment: enumerate 32,000 points and run them on the 80 workers. I
calculated that it would take 5 days to finish the jobs. I ran into numerous
issues like ls</code> being too slow, job uploads taking three days, and monitoring
scripts DDoS-ing the servers. I babysat the servers, painfully restarting
dead workers and failed jobs, and eventually got the
results. The graphs looked pretty and validated Dahlia’s claims. I was very
tired but happy.</p>
During the last week while writing and finishing up the final experiments, I
started staying late in the office. Three days before the deadline (Nov 19),
I finished all the experiments and got cookies at midnight to celebrate this.
After the cookies, I decided to bike back to home. I
started biking down at 2am. At 2.05am, I fell from my bike during a sharp turn
and broke my left wrist. My roommate took me the ER where I got a splint. I was heartbroken.</p>
I woke up the next day and went into the lab after getting a proper arm cast.
I could no longer type on a keyboard so I started handwriting the edits to
the paper which my co-authors then put into the paper. At 1am on November 23,
we submitted</a> the
Dahlia paper to PLDI ’20. I was unsure if the paper would get in but I was
proud of the work we had done.</p>
The semester rolled on and I started brainstorming ideas with Alexa and FRL
on a new project. We decided to use program synthesis to generate
high-performance kernels for DSPs.</p>
Winter ’19</h3>
I went back home to India to recuperate from the broken arm. I proposed
submitting a Qualcomm fellowship proposal for our DSP project. We quickly
hacked up a demo project (called Diospyros</a>) and submitted the proposal.</p>
Spring ’20</h3>
I came back to Cornell in the spring. Doctor told me that while my broken wrist
bone had healed, a cartilage tear in my wrist might never properly heal. I
wondered if a paper submission was worth a lifelong injury.</p>
The semester rolled on and we were accepted for stage 2 of the Qualcomm
proposal. We continued hacking on the project and wrote an even stronger stage 2
proposal with real graphs. Emboldened by the success, we also decided to write
a work-in-progress paper for LCTES ’20. In parallel, I joined another project
to build an intermediate language (called FuTIL</a>) for compiling high-level
languages to hardware circuits. I convinced my collaborators for that project
to submit an LCTES paper as well. We wrote two very good papers and submitted
them.</p>
In the meantime PLDI reviews came back and they were incredibly positive:
two strong accepts and two weak
accepts. Adrian said it was almost certainly enough to get into PLDI. We
wrote up a rebuttal and submitted it. Two weeks later, Dahlia was accepted
to PLDI ’20. Another week after the acceptance I submitted an artifact to
the PLDI artifact evaluation committee. I also volunteered for the committee
and reviewed some cool artifacts in the following weeks.</p>
I was generally happier about things, especially since I had published my first
grad school paper. However, at the start of March, everything turned upside
down. Due to the COVID-19 crisis, Cornell shut down its campus and PLDI
transformed into a virtual conference. I felt sad that I wouldn’t be able
to give a talk on Dahlia in paper. Sad enough to write a blog
post</a> about it.</p>
A few weeks into working from home and adjusting to our new reality, we heard
back from LCTES. The paper on Diospyros is accepted while the one on FuTIL is
rejected. We also hear back from Qualcomm saying that we made it to the final
stage.</p>
Epilogue</h3>
My first two years in grad school were a lot of expected and unexpected things.
The ups and downs of research were expected. The ups and downs of life were
not (injuries and global pandemics). This post leaves out a lot of my personal
accomplishments: I made a lot of friends, I took up biking and baking, I got
healthier, etc. Submitting my first paper was a big accomplishment for me but
I don’t I like the way I got to it. I sacrificed my personal health (due to
my own work ethic) and injured myself. Going forward, I want to set better
boundaries and think harder about the trade-offs between my life and my
research. I am grateful to the many people who made my first two years at
Cornell bearable.</p>
Have comments? Email</a> or tweet</a> at me.</em></p>


The Cost of Virtualizing CS Conferences
2020-03-18T23:15:05-04:00
Conferences in computer science are an odd occurrence. Unlike most other
research fields which primarily focus on publishing in journals, conferences
ended up being the primary publication and presentation venue in CS. They also
became the place where researchers network with each other,
learn about ongoing research, and drink beer with their grad school buddies.</p>
Because of this, conference presentations and networking play an incredibly
important part of a junior researcher’s career. Conferences allow us to show
our research to our community and have other people learn about us. In my field
of research, programming languages and systems, it takes anywhere between one
year to multiple years to complete a project. Add to that yearly deadlines and
specialized venues which results in a junior PhD student having anywhere from
two to four presentations before they go on the job market. Our recognition in
our community from our presentations and our papers is what gets us invitation
for interviews and job offers.</p>
Unfortunately, with the outbreak of COVID-19</a>, our world has been turned
upside down. Beyond the incredible amounts of fear, uncertainty, and human
suffering it has caused, it has also destroyed one of the core mechanisms of
conducting science—meeting people. Multiple major academic conferences
(ASPLOS</a>, ICLR</a>, PLDI</a>) have been canceled. Junior researchers, who had
decided to go on job markets, find internships, or visit another institutions
have had to cancel all of their plans. The impact of these things is
unquantifiable—how does one measure the effect of a missed serendipitous
research collaboration, or that one person on a hiring committee hearing about
your work?</p>
However, I am not here to complain about missed conferences. Canceling
conferences in the midst of a global pandemic is the right thing</em> to do.
I instead want to figure out how we as a community can recreate the
opportunities that conferences create for us every year. I am not an expert in
this so I will need help. I have attempted to summarize the crucial
opportunities conferences give us, what the challenges of running a virtual
conferences are, and what options we have given that physical meetings are out
of the question for a while.</p>
Goals of a conference</h3>
From my (second-year PhD student in a relatively small community) perspective,
conferences traditionally satisfy the following goals:</p>

Dissemination of research</strong>: The primary goal of any conference is to allow
researchers to present their work to their peers and discuss it.</li>
Welcoming new researchers</strong>: The bloodline of our communities are new
researchers. From undergrads who are attending conferences for the first
time to PhD student presenting their research.</li>
The “Hallway” track</strong>: Well understood to be the actual primary goal of
any conference, the hallway track is the colloquial name for researchers
hanging out with each other and discussing research and whatever else that
comes to their mind. It allows us to build long term connections within our
community.</li>
</ol>
Options for Virtual conferences</h3>
Given that most health organizations have recommended that non-essential
travel be suspended, our only choice is to have virtual conferences in some
format. Virtual formats present several challenges:</p>

Multiple time-zones</strong>: Since researchers are not directly traveling to
one physical location for the conference, it’s safe to assume they will
distributed across the world in different time-zones.</li>
Lack of commitment</strong>: As a friend of mine put it, it’s hard to set aside
the time to interact with presenters (who are possibly remote) when there
are other commitments like teaching a class or having research meetings.
Physical conferences act as a forcing function to set this time aside.</li>
</ol>
Both of these problems are challenging to solve. Following are some proposals
I’ve seen discussed/implemented at currently canceled conferences.</p>


Recorded presentations</strong>: The bare minimum any conference can do to
satisfy the first goal is to have authors record research talks and upload
them to YouTube. This will allow researchers to reach out to the people who
are most interested in their work and already know about it, but not a wider
audience that a physical conference gives access to. It might also be
possible to welcome new researchers through videos but they’d likely feel
impersonal.</p>
</li>

Chatrooms for discussing papers</strong>: In addition to uploading all the talks
to YouTube, ASPLOS 20</a> created a Slack channel to discuss
each paper and co-located workshops. This improves the possibility of direct
interactions by making community members available in the same place.
Unfortunately, there really is no way of creating the hallway track in such
a setup. As a junior student, it might be hard or impossible to get
introductions to/talk to other researchers when they are not present in
person. Furthermore, because of the asynchronous nature of chatrooms, it
might be hard to have detailed conversations with people in different
timezones.</p>
</li>

Livestreaming the conference</strong>: Livestreaming the conference in real time
brings the experience as close to a conference session as possible. People
are required to commit time beforehand and ask questions to right after
a talk. Setting this up is non-trivial owing to time zone issues. Again,
while this provides the opportunity for more direct conversations, there
doesn’t seem to be a good way to recreate a hallway track.</p>
</li>

Postpone the conference/Merge it with another</strong>: The nuclear option of
pushing back the conference and waiting out the pandemic. By definition,
this will recreate the experience of a conference. However, this would be
incredibly hard to do since conferences are carefully planned to not overlap
with other conferences in relevant areas. A different approach might to be
have a bigger conference the next year and have papers from both this year</em>
and next year</em> be presented there. Again, I imagine this would be a
nightmare to organize. It also fundamentally cannot recreate opportunities
for researchers who go on the job market this year.</p>
</li>
</ol>
None of the solutions here are perfect and I wouldn’t know which one to choose.
Each of these require hard trade-offs that we, as a community, have to make.
The lack of conferences and the opportunities they create is not measurable
which makes it easier to ignore their impact. I really hope that we can
come up with a solution that is cognizant of this and takes into consideration
the people most impacted by this.</p>
A personal note</strong>: I had a paper</a> accepted at PLDI 20. This
is my first first-author paper and I have been incredibly excited to present
this work for a really long time. I always imagined my first presentation
to an exciting and terrifying rite of passage that I would celebrate with
my friends, colleagues, and advisors. I feel a deep sense of loss, almost
as if all the hard work was “zeroed out” because I can’t present it anymore.
I assume other people in my situation feel similarly. I don’t know if senior
researchers put this much value in conference presentations (since they’ve
already given so many) but it seems important to acknowledge this feeling that
junior researchers have when we come up with solutions for virtual conferences.</p>
Have comments? Email</a> or tweet</a> at me.</em></p>


Project Management for PhD Students
2019-03-03T22:10:39-05:00
Collaborations in systems research is how I’ve built some of the best tools
in my research. A larger teams means an expanded vision and being able to pursue more
ambitious ideas but it also incurs an overhead – team management. Effectively
managing a team and keeping all team members up to date can be stressful and a
daunting task. I think one way to approaching management tasks is by asking
a few concrete questions:</p>

What’s the primary channel of communication?</li>
How often should we be meeting? What are the preparing expectations for a meeting?</li>
How are we managing our code base? What are the expectations about code knowledge?</li>
How are we managing our TODO items?</li>
How should we resolve conflicts?</li>
</ul>
The answers to these questions should evolve with a project. For example, a
project in its prototyping stage might have no restrictions on how or where the
code is kept but a more mature project associated with other projects or
deployments requires careful releases.</p>
The following sections answer these questions from my experience with teams.
The answers apply for a reasonably mature project with most core infrastructure
decisions already made (which language to use, which toolchains, etc.)</p>

Since I’m not the most experienced developer in the world, I would appreciate any suggestions (find my contact information at the end of the post).</p>
</blockquote>
Primary Communication</h2>
This is an easy one. Teams can either use email threads or one of the dozens
of chat applications to have conversations about the project.</p>
The benefit of using a emails is that the team can keep track of individual
threads of conversations easily. However, with multiple projects, this might
get unwieldy.</p>
Chat apps, on the other hand, make it really quick and easy to communicate with
the team but are usually bad at maintaining separate threads of conversations
cleanly.</p>
The choice of the primary communication is often already constrained by group
preferences so this is usually a straightforward decision.</p>
As a side note, team members should try to have long conversations in person.
Text based mediums make it really hard to accurately convey emotions and it
is easy to misread an offhand comment as being aggressive (I’ve certainly been
guilty of this!)</p>
Meetings</h2>
Meetings act as a synchronization point for the entire team and require some amount
of preparation. I suggest having at least two team meetings every week, one
with your advisor (main meeting) and one without them (student meeting).</p>
Main meeting</h3>
For the main meeting, every student should be prepared with the following:</p>

A short weekly update.</li>
Technical challenges faced during the assigned task.</li>
Questions or gotchas found during the assigned the task.</li>
</ul>
At the end of the main meeting, each student should leave with:</p>

At least one assigned task for the week. This can be a paper to read and explain, feature to implement, or a theorem to prove.</li>
A good sense of where to look for answers to their questions.</li>
</ul>
A lot of students (myself included) struggle with prioritizing tasks. Students
involved in research have tons of unstructured time which is not utilized
effectively without a good plan. Assigned tasks help me focus on a task that
I need to get done every week.</p>
Concretely, I try every week to either complete the tasks assigned to me or
have technical questions that are blocking me ready for the meeting.</p>
Student meeting</h3>
The student meetings are more informal and are meant for in depth discussions
about small issues that team members are facing in completing their tasks.</p>
Codebases</h2>
If you’re working on an applied systems project chances are you are building
a software artifact. Regardless of how many people are writing code, it is useful to
check in the code into source
control</a>. This makes the code
publicly viewable and commentable by the team members.</p>

The high-level principle behind these guidelines is to minimize the number of locations where critical information such as feature discussions are kept.</p>
</blockquote>
Since I primarily use git</a> and Github</a>, the following guidelines assume your
project is Github-based. When working on a artifact, I have the following
expectations with team members:</p>

The project leaders (graduate students or senior undergrads) should have a
good sense of what is going on with every aspect of the codebase. This means
having a high-level understanding of all pull requests and issue discussions.</li>
Use pull requests</a>
and branches</a>. Working
on big features on a separate branch allows other people to work in parallel
while leaving the code in a buildable state. Pull requests are a great way
to get the team’s attention on a big change and center discussions around it.</li>
Keep the git history clean by using git pull -r</code> and
rebasing</a> instead of
merging.</li>
</ul>
TODO List</h2>
Since most of my projects revolve around a software artifact, most of which
are on Github</a>, I use Github issues as
a tracking list. Other people I have worked with also use Trello</a> or one of the dozens of TODO apps.</p>

The todo list should make it easy to create tasks and have discussions around them and also allow team members to see who is working on what.</p>
</blockquote>
During the development phase of the project, I ask the team to use
issues liberally. The term ’‘global tracker’’ refers to the high level view of
all todo items. On Github, this is simply the issues page.</p>
Largely, I divide issues into three categories:</p>

Trackers</strong>. Trackers are a collection of smaller issues that logically belong
together but might pollute the global tracker. Use these for reading lists,
benchmark status, and low priority tasks. Example</a>.</li>
Proposal</strong>. Proposal are the heart of the global tracker. Use proposals to
discuss system features, implementation sketches, or big bugs.</li>
Miscellaneous</strong>: These include questions or small bugs. These should be
high frequency, i.e. created liberally, and answered quickly.</li>
</ul>
Conflict resolution</h2>
This is an often overlooked dimension of team dynamics. Research projects can
often be stressful, especially since students tend to be ambitious and prone
to overworking. Since this process so highly dependent on the team members
and project leads, my guideline can only be personalized for me.</p>
If a team member feels under too much pressure to do something or dislikes
someone’s personal behavior, they can either directly contact the person or
ask one of the team leads to mediate. While daunting, it is much better in the
long run to have frank discussions about team expectations and stresses instead
of letting things get worse.</p>
Conclusion</h2>
While there are several industrial strength methodologies for team managements,
I like having a much more lightweight team management style. A lot of research
is about exploring new ideas and pursuing crazy ideas. Regardless of which
guidelines you choose to follow, they should not take away the joy of programming
or research!</p>
Discussion on HackerNews</a>.</p>
Have comments? Email</a> or tweet</a> at me.</em></p>


Learning to Fail
2018-12-19T07:20:00+05:30
I often describe the basic philosophy of research using a metaphor: bash your
head in a wall over and over till you find a way to
break it and then repeat it ad nauseam. Sometimes you’ll know where the
cracks in the wall are, and sometime you’ll know what angle you need to hit the
wall with your head, but fundamentally, you’re hitting your head into a wall.</p>
This is perhaps an unnecessarily graphic description of what research is
like but the point I’m trying to get across is that research is hard</em> and
that failure is the expected outcome</em>. The primary skill of researcher is
not their ability to come up with good ideas or write code but to persevere
in the face of continuous failure.</p>
My undergraduate research experience is the primary reason that I skill. I
started research early but I failed. In fact, I failed almost every single
project I worked on. But this failure also removed any illusions of what
research is like and helped me redefine what “success” should mean.</p>
Here is a quick summary of my research experience as an undergrad:</p>
Spring 2016</h3>
I reached out a my undergraduate advisor in my first semester after being
fascinated with Scheme.

</label>
</span>


Yes, I am a walking PL cliché.
</span>

After some back and forth, I quickly started
a project. The project was to build a formal semantics for bash scripts. The
bash specification is large and complicated with a lot of subtle interactions.
The particular phase we were interested in formalizing were the bash shell
expansions</a>. We tried to build a Hoare logic style semantics for the expansion, because
we wanted to ultimately verify properties of these shell scripts. Unfortunately,
I showed that such a semantics becomes super complicated and we abandoned the
project.

</label>
</span>


Michael Greenberg, one of our collaborators, continued working on this and has
come up with some nice results</a>.
</span>

A few weeks into research and I had already failed a project.</p>
Summer 2016</h3>
I came back for the summer and started working on a new, and slightly related
project. The idea was to extend previous work on verifying Puppet
manifests</a> to
capture the semantics of snippets of shell programs people write into their
Puppet manifests. The previous work had modeled Puppet programs using a
small core calculus based on a Kleene Algebra with Tests (KAT</a>) and we wanted to create an active learning
mechanism to learn the underlying automaton by running the shell script in
a docker container.</p>
Unfortunately, I didn’t have a lot of background in either automata theory or
the low level details of system call tracing (which was the core mechanism to
figure out what system calls were being used). I spent half of the summer
jumping back and forth between learning about automata theory and systems and
implementing papers without much to show for it. While I didn’t know this
at the time, this project also fizzled out around this time.</p>
The reason the project fizzled out was because I joined another student’s
project where we were trying to automatically synthesize updates for Puppet
manifests by capturing system calls. I worked on this project for the rest
of the summer.</p>
Fall 2016</h3>
As the summer ended, my advisor proposed joining Fission, another project that
I had been interested in from the start of my summer. This project aimed to
build a single-tiered, secure programming model for writing web applications.
People on the project had built a frontend that could take JavaScript code and
compile it into something that could enforces security conditions.
Around the same time, the Puppet synthesis project slowed down because the
first author was applying to graduate schools and I was focusing more on
Fission. Eventually, I stopped working on Puppet synthesis entirely.

</label>
</span>


This eventually became a paper</a>.
</span>
</p>
To cap off the depressing string of half completed projects, it was around this
time I actually had minor clinical depression and my productivity collapsed. After
attending ICFP ’16</a> I decided to
start therapy to “fix” my clinical depression.

</label>
</span>


Researchers are people who sometimes work extraordinarily hard at the expense
of their own health.
It is important to realize that your work is significantly
less important that your health.
</span>
</p>
Meanwhile, we also published a workshop
paper</a>
on Fission. Unfortunately, after several rewrites of the compiler, people
leaving the project, and fundamental performance issues, it was becoming
painfully clear that Fission would not pan out.
If you’re keeping track, it’s 3/3 for failed projects.</p>
Spring 2017</h3>
While making slow progress on Fission, my advisor asked a new question, “What
would it take to build a client-side IDE?”. In order to build this IDE, we
started investigating different compiler frameworks for JavaScript. We built
multiple passes to simplify JavaScript constructs and around the same time, another
graduate student joined the project. This spring was perhaps the most productive
semester of my undergraduate research career. I had gained enough technical
and programming chops to push on the project without hands-on support. By the
end of this semester, we had managed to build an IDE and give a talk about it
at NEPLS ’17.</p>
Summer 2017</h3>
My advisor was going to be away for most the summer and he recommended that
I do an “academic internship”. I emailed a professor at Brown University who
took me in for the summer. After a meeting with him during spring break, I
convinced him to let me continue working on my spring research by promising
to integrate my work into the Pyret</a> programming
language.</p>
I spend a summer trying to improve the performance of our implementation
which didn’t work out. However, my collaborator back at UMass had figured
out a solution so continued pushing on.</p>
Towards the end of the summer, I started looking into integrating our work with
Pyret. The codebase of a production-ready compiler like Pyret that supports
thousands of users every day was daunting and hard to understand by myself.
I spent about two weeks trying to understand it, and frustrated at my lack of
progress, also wrote a Vim plugin</a> for Pyret. Once I understood the code base, it took me two days to
implement the first part of the integration.</p>
Fall 2017</h3>
Summer came to an end and my most stressful semester in undergraduate began.
I was graduating in three years so I was taking 6 classes, applying
to 10 graduate schools, applying for summer internships, and writing a paper
for our research. It was a lot of work but I did it all. We submitted a
polished paper to PLDI 2018 (which was later accepted</a>).
I was accepted to 8 graduate schools and a software engineering internship
at Google. I eventually decided to start my PhD at Cornell.</p>
Epilogue</h3>
Having spent a few years at Cornell, I have come to appreciate a lot of things
about my undergraduate experience:</p>

While I failed for more than a year, I learned a lot</em>. The
amount of implementation work I did made me good at rapid prototyping and
I came with a breadth of knowledge in configuration and web languages,
secure systems, and formal language theory.</li>
The infectious optimism of my advisor kept me going through all the failures.
The most important piece of advice he gave me was: “You’ll figure it out!”</li>
I learned that I work best when I collaborate with people. It is easier
to be excited about research when someone else is also excited about it
with you.</li>
It is really hard to execute research ideas. A lot of people can come up
with really good ideas but it takes a lot of work and dedication to
push through a project. I’ve come to respect the latter way more than the
former.</li>
</ul>
I feel privileged in having a undergraduate research career where I was
given the opportunity to fail. When I started my PhD, I had no illusions
about what research was: it requires a religious amount of faith and hard work
before you can see any progress.</p>
Have comments? Email</a> or tweet</a> at me.</em></p>


PhD at Cornell: The Free Agent System
2018-12-15T08:27:21+05:30

Update, 2022</strong>: After being at Cornell for couple more years, I’ve developed a more nuanced opinion of the “free agent system” at Cornell CIS.
I do not believe that this system scales well as the department has grown.
If you are an incoming student to Cornell CIS, I encourage you to set up rotation commitments with professors before you accept the offer.</em></p>
</blockquote>
Deciding which graduate school you’re going to spend the next n</em> years of your
life is one of the hardest decision of your life. One of the things
that made is hard for me was deciding between Cornell and my other
top choice was Cornell’s “Free agent system”. Here is a short post about what
the system is and why it worked for me.</p>
Graduate School Admissions</h3>
For most schools in the US, when you apply to a PhD program, students are usually
picked out by one or more professors who think you’d be a good fit. After
visiting the school, the student decides which professor they want to work with
and commit to the school. When the student starts at the school, they are
funded by the professor and they start doing great things together. However,
some schools don’t follow this system.</p>
Cornell’s Free Agent System</h3>
At Cornell, and a few other schools, the admission process looks a bit
different.  When a PhD student is admitted to Cornell, they are are admitted
to the department</em>, which highlights Cornell’s commitment towards the student’s
academic freedom. Concretely, this means that Cornell guarantees funding,
usually through teaching assistantship, for the student without tying them to
an advisor.  This is supposed to allow the students to explore and talk to
potential advisors without being worried about funding. This is the Free Agent
system at Cornell. Students are free agents till they decide who they want to
work with.</p>
The Problem</h3>
Cornell’s free agent system was devised when the department was young and the
incoming PhD students tended to have comparatively less research experience.
The free agent system allowed students to explore different areas without
being pressured into working on topics they might disliked. However, in the recent
years, the makeup of people applying to PhD programs has drastically changed —
students tend to come in with a lot more research experience and are usually
quite certain about the area they want to work in.</p>
Furthermore, the CS department is also structured in a way that assumes
students are free agents their first year. This means that they are expected to
take a lot of classes and</del> be teaching assistants (TAs) in their first
two semesters.

</label>
</span>


The CS department recently overhauled the course requirement to reduce the
number of classes and restrictions on which classes to take.
</span>
</p>
My Experience</h3>
The free agent system caused me a lot of angst during the decision process. For
some background, I had started doing programming languages (PL) research in the
first semester of my undergraduate degree and was certain about my future research
direction. Furthermore, I knew that Cornell was the best fit for my interest in
doing PL work at the intersection of other fields.</p>
Unfortunately, I was also afraid of not being able to find an advisor. After
about 6 hours of post visit day talks with professors and students in the PL
group, I decided to go to Cornell. Even after my acceptance, I wasn’t sure if
I’d be able to find an advisor.</p>
When I started in Fall, I emailed professors in the PL group to set up
meetings. This is where I found the true strength of the free agent system.
Since professors expect students to go talk to a lot of people, they expect and
often encourage students to do research rotations with professors they are
interested in working with.</p>
It also makes it easy to reach out to professors and learn about their work. I
cannot emphasize how important it is to me to learn about and have
conversations about research in different domains. One of my goals going into a
PhD is to have a broad sense of the different kinds of problems in different
domains and having access to professors in different areas makes it easy to do
so.</p>
I was also able to start working with my awesome advisor Adrian
Sampson</a> and we quickly found a project
I’m passionate about.</p>
Caveat Emptor</h3>
While the free agent system caused me some anguish in the decision process, it
was not the primary reason I decided join Cornell.</p>

My primary motivators were research that excited me, and people who are
just as excited about it as me.

</label>
</span>


Importantly, this includes other grad students.
Remember, you’re going to be spending a lot more time with other grad
students than you will with faculty.
</span>
</li>
The first year TAing requirement causes some amount of stress for
new students. However, the department is aware of the issues and is
trying to move away from this system.</li>
</ul>
Finally, here is a more detailed
post</a> from
Jean Yang</a> on what considerations matter when deciding
on schools. Good luck!</p>
Have comments? Email</a> or tweet</a> at me.</em></p>


Don’t be Programming Languages Researchers
2018-09-22T00:00:00+00:00

Instead of being a judgement of what PL research should be, this short post is
simply a reflection of my research interests and what role PL plays into it.</p>
</blockquote>
During a recent PLDG talk, the speaker said, “I think that, as a community, PL
people have a moral responsibility to step in and say, ‘No, you’re having fun
wrong!’.”
I have no qualms about the comment itself—jokes can be a useful
tool in presentations; however, it did led me to think about the
way PL research is applied in new domains.</p>
In a classic programming languages presentation, the speaker starts with an
overview of a domain, talks about the current state of the art of
programming languages and tools in it, and then go on to point out that most
tools and languages fail to make use of amazing and well-known PL
techniques. Then they describe their work which applies the aforementioned PL
technique and build cool and interesting language abstractions with the promise
of building better and improved tools for the domain.

</label>
</span>


It is rarely the case that the tool or language proposed actually solves the
problems in the motivation.
Yes, do tell me how your tiny language will stop Google from going down twice
a year.</em>
</span>
</p>
While I strongly endorse PL techniques and research being applied in new
domains, this story demonstrates a fundamental issue for me: Application of PL techniques is
done retrospectively. PL researchers are not there when a domain is shaping up
and people trying to build tools and programming languages for that domain.
Only once people have made build these tools, which in turn cement
themselves into domain, do PL researchers come into the scene and apply their
cool techniques—at a point where practitioners are unlikely to adopt something
new.</p>
So here is my solution: We should stop being PL researchers—we should take it
upon themselves to learn about new domains and apply ourselves well before
the standards are established.
Programming
languages are the fundamental way of communicating intent to computers. As
PL researchers, we should be actively helping people from other domains, not
waiting for them to realize the error of their ways and come to us.</p>

- Rachit's Blog

Formally speaking, "Transpiler" is a useless word

Reflecting on PLDI 2025

Papers</h2> The main conference has several parallel tracks so I couldn’t see all the talks I wanted to but here is a collection of talks I found interesting:</p>

GitHub-centric Research Management

Labels</h3> GitHub issues and pull requests can be tagged with “labels” to categorize them. My recommendation is to have two kinds of labels:</p> Component</strong>. Which part of the codebase does this issue relate to? For example, it could be a specific tool, error message, UX, etc.</li>

Guidance</h3> There are two pieces of advice on issues:</p>

Deployment</h3> Deployment usually happens after a particular code change has been merged in to the main branch. A common set of things to deploy can be:</p>

Your Eternal Spark

Transpiler, a meaningless word

The Stateless Manager

Why Study Programming Languages

Lies Academics Believe

Dear Sir, You Have Built a Compiler

Personal Infrastructure for PhD Students

Websites</h2> Let’s get this one out of the way–as a PhD student, you need to have a personal website</em>. It doesn’t need to dazzle, it doesn’t need to use bleeding-edge web frameworks and CSS animations but it needs to exist and it needs to be easy to find.</p>

Commoditize the Complement of Your Research

Languages, Tools, and Techniques for Accelerator Design

State of the Art</h2> A language-oriented view of classic problems in hardware design has resulted in a slew of novel solutions:</p> Verification</strong>: Formally verified hardware design ([Kami][]) aims to eliminate the slow and tedious process of hardware verification.</li>

Compiling for the Reconfigurable Future

The First Two Years of My PhD

The Cost of Virtualizing CS Conferences

Options for Virtual conferences</h3> Given that most health organizations have recommended that non-essential travel be suspended, our only choice is to have virtual conferences in some format. Virtual formats present several challenges:</p>

Project Management for PhD Students

Meetings</h2> Meetings act as a synchronization point for the entire team and require some amount of preparation. I suggest having at least two team meetings every week, one with your advisor (main meeting) and one without them (student meeting).</p>

Main meeting</h3> For the main meeting, every student should be prepared with the following:</p>

Student meeting</h3> The student meetings are more informal and are meant for in depth discussions about small issues that team members are facing in completing their tasks.</p>

Learning to Fail

PhD at Cornell: The Free Agent System

Don’t be Programming Languages Researchers

Meetings</h2>
Meetings act as a synchronization point for the entire team and require some amount of preparation. I suggest having at least two team meetings every week, one with your advisor (main meeting) and one without them (student meeting).</p>

Student meeting</h3>
The student meetings are more informal and are meant for in depth discussions about small issues that team members are facing in completing their tasks.</p>