RSS/Atom Feed Analysis
Analysis of https://steveklabnik.com/feed.xml
Feed fetched in 112 ms.
Warning Content type is application/xml, not text/xml.
Feed is 1,612,521 characters long.
Feed has an ETag of W/"b922f3bdd181b344e55e459232dc0b3c".
Feed has a last modified date of Tue, 01 Apr 2025 19:49:47 GMT.
Warning This feed does not have a stylesheet.
This appears to be an RSS feed.
Feed title: Steve Klabnik
Error Feed is missing a self link.
Feed has 215 items.
First item published on 2025-03-17T00:00:00.000Z
Last item published on 2009-12-02T00:00:00.000Z
Home page URL: https://steveklabnik.com/
Home page has feed discovery link in <head>.
Error Home page does not have a link to the feed in the <body>.
Formatted XML
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
<channel>
<title>Steve Klabnik</title>
<description>Steve Klabniks blog</description>
<link>https://steveklabnik.com/</link>
<item>
<title>Does unsafe undermine Rust's guarantees?</title>
<link>https://steveklabnik.com/writing/does-unsafe-undermine-rusts-guarantees/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/does-unsafe-undermine-rusts-guarantees/</guid>
<pubDate>Mon, 17 Mar 2025 00:00:00 GMT</pubDate>
<content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>When people first hear about <code>unsafe</code> in Rust, they often have questions. A very
normal thing to ask is, "wait a minute, doesn't this defeat the purpose?" And while
it's a perfectly reasonable question, the answer is both straightforward and has a
lot of nuance. So let's talk about it.</p>
<p>(The straightforward answer is "no", by the way.)</p>
<h2>Memory safe languages and their implementations</h2>
<p>The first way to think about this sort of thing is to remember that programming
languages and their implementations are two different things. This is sometimes
difficult to remember, since many programming languages only have one
implementation, but this general principle applies even in those cases. That is,
there are the desired semantics of the language you are implementing, and then
the codebase that makes programs in that language do those things.</p>
<p>Let's examine a "real" programming language to talk about this: <a href="https://en.wikipedia.org/wiki/Brainfuck">Brainfuck</a>.
Brainfuck programs are made up of eight different characters, each one performing
one operation. Brainfuck programs can only do those eight things, and nothing
more.</p>
<p>However, this simplicity leads to two interesting observations about the
differences between languages in the abstract and their implementations.</p>
<h3>Unsafety in implementation</h3>
<p>The first is that properties of the language itself are distinct from properties
of an implementation of the language. The semantics of the Brainfuck language
don't provide an opportunity to call some sort of re-usable function
abstraction, for example. Yet, when we write a Brainfuck interpreter or
compiler, we can use a programming language that supports functions. That
doesn't change that Brainfuck programs themselves can't define or call
functions.</p>
<p>And in fact, we often want to implement a language in another language that has
more abilities than the language we're implementing. For example, while it's
possible to implement Brainfuck in Brainfuck, it's much easier to do so in a
language that lets you define and call functions. Another classic example is
interacting with hardware. Brainfuck has the <code>.</code> instruction that allows you to
produce output. But in order to actually produce output to say, a terminal, you
interact with your platform's API to do so. But inside of that API, eventually
you'll hit a layer where there are no abstractions: you need to talk to hardware
directly. Doing so is not statically able to be guaranteed to be memory safe,
because the hardware/software interface often boils down to "put some
information at this arbitrary memory location and the hardware will take it from
there." Our Brainfuck program is operating in an environment just like any
other; at the end of the day it has to coordinate with the underlying operating
system or hardware at some point. But just because our implementation must do
so, doesn't mean that our Brainfuck programs do so directly. The semantics of
programs written in Brainfuck aren't changed by the fact that the implementation
can (and must) do things that are outside of those semantics.</p>
<p>Let's take Ruby as a more realistic example than Brainfuck. Ruby does not let us modify
arbitrary memory addresses from within Ruby itself. This means that pure Ruby
programs should not be able to produce a segmentation fault. But in the real
world, Ruby programs can segfault. This is possible if there's a bug in the Ruby
interpreter's code. Sure, it's our Ruby code that's making the call to a
function that results in a segfault, but the real fault lies with the code
outside of Ruby's purview.</p>
<p>That this can happen doesn't mean that Ruby's guarantees around memory
manipulation is somehow useless, or suffers segmentation faults at the same rate
as programs in a language that allow you to manipulate arbitrary memory as a
matter of course. But it does also mean that we don't need to look at our entire
program to figure out where the problem comes from: it instead comes from our
non-Ruby code. Ruby's guarantees have helped us eliminate a lot of suspects when
figuring out whodunit.</p>
<h3>Extending languages and unsafety</h3>
<p>The second property is that certain implementations may extend the language in
arbitrary ways. For example, I could write a Brainfuck interpreter and say that
I support a ninth instruction, <code>@</code>, that terminates the program when invoked.
This would be a non-portable extension, and Brainfuck programs written in my
variant wouldn't work in other interpreters, but sometimes, this technique is
useful.</p>
<p>Many languages find that these sorts of extensions are useful, and so offer a
feature called a "foreign function interface" that allows for your program to
invoke code in a different language. This provides the ability to do things that
are outside of the domain of the language itself, which can be very useful.
Brainfuck does not offer an FFI, but if it did, you could imagine that <code>@</code> could
be implemented in terms of it, making programs that use <code>@</code> portable again, as
long as the extended functionality was included somehow, often as a library of
some kind.</p>
<p>Just like our implementation has the ability to do things outside of our
languages' semantics, FFI and similar extension mechanisms also give us the
ability to do arbitrary things. I can write an extension for Ruby that writes to
arbitrary memory. And I can cause a segfault. But we're in the same place that
we were with our implementation issues; we know that if we get a segfault, the
blame lies not with our Ruby code, but instead, with either the implementation
or our FFI.</p>
<h3>So how is this memory safe?</h3>
<p>It may seem contradictory that we can call a language "memory safe" if
real-world programs have the ability to cause memory problems. But the thing is,
it isn't <em>really</em> programs in that language that caused the issue: it was FFI,
which is in a different language entirely, or it was implementation issues, and
implementations must do memory unsafe things, thanks to a need to interact with
the OS or hardware. And so the definition of "memory safe language" is commonly
understood to refer to languages and their implementations in the absence of
either implementation bugs or FFI bugs. In practice, these bugs occur at such a
low rate compared to languages that are clearly not memory safe that this
practical definition serves a good purpose, even if you may feel a bit
uncomfortable with these "exceptions."</p>
<p>But there's actually a deeper reason why these exceptions are acceptable, and
that's due to how we understand properties of programs and programming languages
in the first place. That is, this isn't just a practical "well these exceptions
seem fine" sort of thing, they're actually okay on a deeper level.</p>
<h2>Programs, properties, and proofs</h2>
<p>So how do we know anything about how programs and programming languages work at
all?</p>
<blockquote>
<p>Computer science is no more about computers than astronomy is about
telescopes.</p>
<ul>
<li>Edsger Wybe Dijkstra</li>
</ul>
</blockquote>
<p>A fun thing about computer science is that it's closely related to other
disciplines, including math! And so there's a long history of using math
techniques to understand computers and their programs.</p>
<p>One technique of building knowledge is the idea of proofs. If you get a
degree in computer science, you'll engage with proofs quite a lot. I even
took a class on logic in the philosophy department as part of my degree.</p>
<p>I don't intend to give you a full introduction to the idea of proofs in this
blog post, but there's some high-level concepts that are useful to make sure
we're in the same page about before we go forward.</p>
<h3>Aristotelian syllogisms</h3>
<p>Here is a very classic example of a form of reasoning called a "syllogism",
given by Aristotle in 350 BCE:</p>
<ol>
<li>All men are mortal.</li>
<li>Socrates is a man.</li>
<li>Therefore, Socrates is mortal.</li>
</ol>
<p>These first two lines are called "propositions," and the third is a conclusion.
We can base our conclusion on a logical relationship between the information given
to us by the propositions.</p>
<p>But how do we know propositions are true? Are all men mortal? What's important
here is that we, for the purposes of our proof, assume that propositions are
true. And we do this because, on some level, we cannot know everything. And so
to begin, we have to start somewhere, and make some assumptions about what we
know. It's true that later, we may discover a fact that disproves our
proposition, and now our proof no longer works. But that's just the way that
the world works. It doesn't prevent these sorts of proofs from being useful to
help us gain knowledge about the world and how it works, as best as we can tell
at the current time.</p>
<p>So on some level, this is also why Ruby is a memory safe language even though a
C extension can segfault: there's always some kind of things that we have to
assume are true. Memory safe languages are ones where the amount of code we have
to assume is memory safe, rather than is guaranteed to be by the semantics, is
small, and preferably indicated in the code itself. Put another way, the amount
of code we need to trust is memory safe is large in a memory unsafe language,
and small in a memory safe language, rather than zero.</p>
<h3>Logic and programs</h3>
<p>As time went on, so did our understanding of logic. And, of course, we even have
competing logics! And then this gets fun, because terms can mean something
slightly different. For example, in more recent logics, we'd call something like
"All men are mortal" to be an <em>axiom</em>, rather than a proposition. Same idea:
it's something that we accept without proof.</p>
<p>As computers appeared, people sought to apply the rules for mathematical logic
onto them. We even call circuits that carry out classic logical operations
"logic gates." Number theory and logic were foundational to making computers
work. And so, once high level languages appeared on the scene, there was
interest in applying these mathematical tools to understanding programs as well.
This discipline is called "formal verification," and the general idea is to
describe various properties we wish a system to have, and then use formal
methods from mathematics to demonstrate that this is true.</p>
<p>This area of study is very deep, and I don't plan to cover the vast majority of
it here. However, I do want to pursue one particular thread in this area.</p>
<h3>Hoare Logic</h3>
<p><a href="https://dl.acm.org/doi/10.1145/363235.363259">"Hoare Logic"</a> is, well:</p>
<blockquote>
<p>In this paper an attempt is made to explore the logical foundations of
computer programming by use of techniques which were first applied in the
study of geometry and have later been extended to other branches of
mathematics. This involves the elucidation of sets of axioms and rules of
inference which can be used in proofs of the properties of computer programs.
Examples are given of such axioms and rules, and a formal proof of a simple
theorem is displayed. Finally, it is argued that important advantage, both
theoretical and practical, may follow from a pursuance of these topics.</p>
</blockquote>
<p>Incidentally, C. A. R. Hoare, the author of this paper, and Graydon Hoare, the
creator of Rust, are unrelated.</p>
<p>How does Hoare logic work? Yet again, not going to cover all of it, but the
general idea is this: In order to figure out if a program does what it is
supposed to do, we need to be able to reason about the state of the program
after execution happens. And so we need to be able to describe the state before,
and its relationship to the state after. And so you get this notation:</p>
<pre><code>P { Q } R
</code></pre>
<p>P is a precondition, Q is a program, and R is the result of the program
executing with those preconditions.</p>
<p>But most programs have more than one statement. So how could we model this?
Hoare gives us the Rule of Composition:</p>
<p>If <code>P { Q1 } R1</code> and <code>R1 { Q2 } R</code> then <code>P { Q1; Q2 } R</code>.</p>
<p>This allows us to build up a program by proving each statement in turn.</p>
<p>Hoare logic is very neat, and I've only scratched the surface here. People did a
lot of work to extend Hoare logic to include more and more aspects of programs.
But then, something else happened.</p>
<h3>Separation Logic</h3>
<p>In 2002, <a href="https://www.cs.cmu.edu/~jcr/seplogic.pdf">Separation Logic: A Logic for Shared Mutable Data Structures</a>
was published.</p>
<blockquote>
<p>In joint work with Peter O'Hearn and others, based on early ideas of Burstall,
we have developed an extension of Hoare logic that permits reasoning about
low-level imperative programs that use shared mutable data structure.</p>
</blockquote>
<p>Hmm, shared mutable data structures? Where have I heard that before...</p>
<p>Let's see what they have to say:</p>
<blockquote>
<p>The problem faced by these approaches is that the correctness of a program
that mutates data structures usually depends upon complex restrictions on the
sharing in these structures.</p>
</blockquote>
<p>For sure. Well, what are we to do about this?</p>
<blockquote>
<p>The key to avoiding this difficulty is to introduce a novel
logical operation <code>P * Q</code>, called separating conjunction (or
sometimes independent or spatial conjunction), that asserts
that P and Q hold for disjoint portions of the addressable
storage.</p>
</blockquote>
<p>What disjoint portions of addressable storage might we care about?</p>
<blockquote>
<p>Our intent is to capture the low-level character of machine language. One can
think of the store as describing the contents of registers, and the heap as
describing the contents of an addressable memory.</p>
</blockquote>
<p>Pretty useful!</p>
<p>Before we talk a bit about how separation logic works, consider this paragraph
on why it's named as such:</p>
<blockquote>
<p>Since these logics are based on the idea that the structure of an assertion
can describe the separation of storage into disjoint components,we have come
to use the term separation logics, both for the extension of predicate
calculus with the separation operators and the resulting extension of Hoare
logic. A more precise name might be storage separation logics, since it is
becoming apparent that the underlying idea can be generalized to describe the
separation of other kinds of resources.</p>
</blockquote>
<p>The plot thickens.</p>
<p>Anyway, in Separation Logic, we use slightly different notation than Hoare Logic:</p>
<pre><code>{ P } C { Q }
</code></pre>
<p>This says that we start with the precondition <code>P</code>, and if the program <code>C</code>
executes, it will not have undefined behavior, and if it terminates, <code>Q</code> will
hold.</p>
<p>Furthermore, there is the "frame rule", which I am going to butcher the notation
for because I haven't bothered to install something to render math correctly
just for this post:</p>
<p>If <code>{ p } c { q }</code></p>
<p>then <code>{ p * r } c { q * r }</code></p>
<p>where no free variable in <code>r</code> is modified by <code>c</code>.</p>
<p>Why is it interesting to add something to both sides of an equation, in a sense?
Well what this gives us the ability to do is add any predicates about parts of
the program that <code>c</code> doesn't modify or mutate. You might think of <code>&amp;mut T</code> or
even just ownership in general: we can reason about just these individual parts
of a program, separately from the rest of it. In other words, we have some
foundational ideas for ownership and even bits of borrowing, and while this original
paper doesn't involve concurrency, eventually Concurrent Separation Logic would become
a thing as well.</p>
<p>I think the paper explains why the frame rule matters better than I can:</p>
<blockquote>
<p>Every valid specification <code>{p} c {q}</code> is "tight" in the sense that every cell
in its footprint must either be allocated by <code>c</code> or asserted to be active by
<code>p</code>; "locality" is the opposite property that everything asserted to be active
belongs to the footprint. The role of the frame rule is to infer from a local
specification of a command the more global specification appropriate to the
larger footprint of an enclosing command.</p>
</blockquote>
<p>What this gives us is something called "local reasoning," and local reasoning
is <em>awesome</em>. Before I talk about that, I want to leave you with one other
very interesting paragraph:</p>
<blockquote>
<p>Since our logic permits programs to use unrestricted address arithmetic,there
is little hope of constructing any general-purpose garbage collector. On the
other hand, the situation for the older logic, in which addresses are disjoint
from integers, is more hopeful. However, it is clear that this logic permits
one to make assertions, such as “The heap contains two elements” that might be
falsified by the execution of a garbage collector, even though, in any
realistic sense, such an execution is unobservable.</p>
</blockquote>
<p>Anyway. Let's talk about local vs global analysis.</p>
<h2>Global vs local analysis</h2>
<p>Proving things about programs isn't easy. But one thing that can make it even
harder is that, for many properties of many programs you'd want to analyze, you
need to do a <em>global analysis</em>. As an example, let's use Ruby. Ruby is an
incredibly dynamic programming language, which makes it fairly resistant to
static analysis.</p>
<h3>Global analysis</h3>
<p>Here is a Ruby program. Do you know if this program executes successfully?</p>
<pre><code>class Foo
def self.bar
"baz"
end
end
p Foo.bar
</code></pre>
<p>Yeah, it prints <code>baz</code>. But what about this Ruby program?</p>
<pre><code>class Foo
def self.bar
"baz"
end
end
require "foo_ext.rb"
p Foo.bar
</code></pre>
<p>We can't know. <code>foo_ext.rb</code> may contain no relevant code, but it also might
include something like this:</p>
<pre><code>Foo.class_eval { undef :bar }
</code></pre>
<p>In which case, when we try and call <code>Foo.bar</code>, it no longer exists:</p>
<pre><code>&lt;anonymous&gt;': eval:13:in `block in &lt;main&gt;': undefined method `bar' for class `Foo' (NameError)
eval:13:in `class_eval'
eval:13:in `&lt;main&gt;'
-e:in `eval'
</code></pre>
<p>Ouch. So in this case, we're going to need the whole code of our program in
order to be able to figure out what's going on here.</p>
<p>Incidentally, <a href="https://sorbet.org/">Sorbet</a> is a very cool project to add type checking to
Ruby. They do require access to the entire Ruby source code in order to do their
analysis. But they also made some decisions to help make it more tractable; if
you try Sorbet out on the web, the type checker is <em>fast</em>. What happens when you
try the above code with Sorbet?</p>
<pre><code>editor.rb:5: Unsupported method: undef https://srb.help/3008
5 |Foo.class_eval { undef :bar }
^^^^^^^^^^
</code></pre>
<p>This is a very fair tradeoff! It's very common with various forms of analysis to
choose certain restrictions in order to make what they want to do tractable. Ruby
has so much evil in it, just not supporting some of the more obscure things is
completely fair, in my opinion.</p>
<h3>Local analysis</h3>
<p>The opposite of global analysis is, well, local analysis. Let's consider a Rust
equivalent of our Ruby:</p>
<pre><code>struct Foo;
impl Foo {
fn bar() -&gt; String {
String::from("baz")
}
}
fn main() {
dbg!(Foo::bar());
}
</code></pre>
<p>Can we know if this program works? Sure, everything is here. Now, if we tried the
same trick as the Ruby code, we know that it would work, because Rust doesn't have
the ability to remove the definition of <code>bar</code> like Ruby does. So let's try something
else:</p>
<pre><code>struct Foo;
impl Foo {
fn bar() -&gt; String {
// body hidden
}
}
fn main() {
dbg!(Foo::bar());
}
</code></pre>
<p>Can we know if <code>main</code> is properly typed here? We can, even though we know
nothing about the body. That's because <code>dbg!</code> takes any value that implements
the <code>Debug</code> trait, and we know that <code>Foo::bar()</code> returns <code>String</code>, which
implements <code>Debug</code>. Typechecking <code>main</code> is local to main's implementation, we
don't need to look into the bodies of any function it calls to determine if it's
well-typed. We only need to know their signatures. If we didn't require <code>bar</code> to
have a type signature, we'd have to peek into its body to figure out what it
returns, and so on for any function that <code>bar</code> calls in its body. Instead of
this arbitrarily deep process, we can just look at the signature and be done
with it.</p>
<h2>Composability of proofs</h2>
<p>So why is local analysis so helpful? The first reason is either speed or
scalability, depending on how you want to look at it. If you are running global
analysis checks, they will get more expensive the larger your codebase, since
they will require checking the entire thing to work. Whereas local analysis only
requires a local context, it doesn't get any more expensive when you add more
code to your project, only when you change that local context. So when you're
trying to scale checks up to larger projects, local analysis is crucial.</p>
<p>But I also like to personally think about it as a sort of abstraction. That is,
global analysis is a leaky abstraction: changes in one part of the codebase can
cascade into other parts. Remember this line about the frame rule?</p>
<blockquote>
<p>The role of the frame rule is to infer from a local specification of a command
the more global specification appropriate to the larger footprint of an
enclosing command.</p>
</blockquote>
<p>If we have local reasoning, we can be sure that changes locally don't break out
of the boundaries of those changes. As long as the types of our function don't
change, we can mess with the body as much as we want, and we know that the rest
of the analysis of the program is still intact. This is really useful, in the
same way that abstractions are useful when building programs more generally.</p>
<h2>Rustbelt</h2>
<p>Okay, we have taken a real deep dive here. What's this all have to do with
unsafe Rust?</p>
<p>Well, seven years ago, <a href="https://plv.mpi-sws.org/rustbelt/popl18/paper.pdf">RustBelt: Securing the Foundations of the Rust
Programming Language</a> by Ralf Jung, Jaques-Henri Jourdan, Robbert
Krebbers, and Derek Dreyer, was published:</p>
<blockquote>
<p>In this paper, we give the first formal (and machine-checked) safety proof for
a language representing a realistic subset of Rust.</p>
</blockquote>
<p>But it's more exciting than that:</p>
<blockquote>
<p>Our proof is extensible in the sense that, for each new Rust library that uses
unsafe features, we can say what verification condition it must satisfy in
order for it to be deemed a safe extension to the language.</p>
</blockquote>
<p>Very good!</p>
<p>The paper begins by talking about why verifying Rust is such a challenge:</p>
<blockquote>
<p>Consequently, to overcome this restriction, the implementations of Rust's
standard libraries make widespread use of unsafe operations, such as "raw
pointer" manipulations for which aliasing is not tracked. The developers of
these libraries claim that their uses of unsafe code have been properly
"encapsulated", meaning that if programmers make use of the APIs exported by
these libraries but otherwise avoid the use of unsafe operations themselves,
then their programs should never exhibit any unsafe/undefined behaviors. In
effect, these libraries extend the expressive power of Rust's type system by
loosening its ownership discipline on aliased mutable state in a modular,
controlled fashion</p>
</blockquote>
<p>This is basically the question asked at the start of this post: can you honestly
say that a function is safe if it contains <code>unsafe</code> code as part of its
implementation?</p>
<p>They go on to describe the first challenge for the project: choosing the correct
logic to model Rust in (some information cut for clarity):</p>
<blockquote>
<p>Iris is a language-generic framework for higher-order concurrent separation
logic which in the past year has been equipped with tactical support for
conducting machine-checked proofs of programs in Coq. By virtue of being a
separation logic, Iris comes with built-in support for reasoning modularly
about ownership. Moreover, the main selling point of Iris is its support for
deriving custom program logics for different domains using only a small set of
primitive mechanisms. In the case of RustBelt, we used Iris to derive a novel
lifetime logic, whose primary feature is a notion of borrow propositions that
mirrors the “borrowing” mechanism for tracking aliasing in Rust.</p>
</blockquote>
<p>Separation logic!</p>
<p>So how does this work? There are three parts to the proof. I'm going to
summarize them because I'm already quoting extensively, please go read the paper
if you want the exact details, it is well written.</p>
<ol>
<li>Verify that the typing rules are sound when interpreted semantically.</li>
<li>Verify that if a program is well typed semantically, it does not exhibit
unsafe behavior.</li>
<li>For any library that uses unsafe, check that it satisfies the semantics that
its interface requires.</li>
</ol>
<p>If all three of these things are true, then the program is safe to execute. This
is also exciting because, thanks to 3, when someone writes new unsafe code, they
can get RustBelt to let them know which properties they need to satisfy to make
sure that their unsafe code doesn't cause problems.</p>
<h3>Weaknesses</h3>
<p>Of course, there is more to do following up on RustBelt:</p>
<blockquote>
<p>We do not model (1) more relaxed forms of atomic accesses, which Rust uses for
efficiency in libraries like Arc; (2) Rust's trait objects (comparable to
interfaces in Java), which can pose safety issues due to their interactions
with lifetimes; or (3) stack unwinding when a panic occurs, which causes
issues similar to exception safety in C++.</p>
</blockquote>
<p>However, the results were good enough to find some soundness bugs in the
standard library, and if I remember correctly, widen one of the APIs in a sound
manner as well. Another interesting thing about RustBelt is that it was
published before Rust gained Non-lexical lifetimes, however, it modeled them,
since folks knew they were coming down the pipeline.</p>
<h2>The theoretical vs the empirical</h2>
<p>So, we have some amount of actual proof that unsafe code in Rust doesn't
undermine Rust's guarantees: what it does is allow us to extend Rust's
semantics, just like FFI would allow us to extend a program written in
a language that supports it. As long as our unsafe code is semantically
correct, then we're all good.</p>
<p>... but what if it's not semantically correct?</p>
<h3>When things go wrong</h3>
<p>At some point maybe I'll write a post about the specifics here, but this post is
already incredibly long. I'll cut to the chase: you can get undefined behavior,
which means anything can happen. You don't have a real Rust program. It's bad.</p>
<p>But at least it's scoped to some degree. However, many people get this scope a
bit wrong. They'll say something like:</p>
<blockquote>
<p>You only need to verify the unsafe blocks.</p>
</blockquote>
<p>This is true, but also a bit misleading. Before we get into the unsafe stuff, I
want you to consider an example. Is this Rust code okay?</p>
<pre><code>struct Foo {
bar: usize,
}
impl Foo {
fn set_bar(&amp;mut self, bar: usize) {
self.bar = bar;
}
}
</code></pre>
<p>No unsafe shenanigans here. This code is perfectly safe, if a bit useless.</p>
<p>Let's talk about unsafe. The canonical example of unsafe code being affected
outside of unsafe itself is the implementation of <code>Vec&lt;T&gt;</code>. Vecs look something
like this (the real code is different for reasons that don't really matter in
this context):</p>
<pre><code>struct Vec&lt;T&gt; {
ptr: *mut T,
len: usize,
cap: usize,
}
</code></pre>
<p>The pointer is to a bunch of <code>T</code>s in a row, the length is the current number of <code>T</code>s
that are valid, and the capacity is the total number of <code>T</code>s. The length and the
capacity are different so that memory allocation is amortized; the capacity is
always greater than or equal to the length.</p>
<p>That property is very important! If the length is greater than the capacity,
when we try and index into the Vec, we'd be accessing random memory.</p>
<p>So now, this function, which is the same as <code>Foo::set_bar</code>, is no longer okay:</p>
<pre><code>impl&lt;T&gt; Vec&lt;T&gt; {
fn set_len(&amp;mut self, len: usize) {
self.len = len;
}
}
</code></pre>
<p>This is because the unsafe code inside of other methods of <code>Vec&lt;T&gt;</code> need to be
able to rely on the fact that <code>len &lt;= capacity</code>. And so you'll find that
<code>Vec&lt;T&gt;::set_len</code> in Rust is marked as unsafe, even though it doesn't contain
unsafe code.</p>
<p>And this is why the module being the privacy boundary matters: the only way to
set len directly in safe Rust code is code within the same privacy boundary as
the <code>Vec&lt;T&gt;</code> itself. And so, that's the same module, or its children.</p>
<p>This is still ultimately better than any line of code in the whole codebase, but
it's not quite as small as you may think at first.</p>
<h3>How can we check that things haven't gone wrong?</h3>
<p>Okay, so while RustBelt could give you some idea if your code is correct, I doubt
you're about to jump into Coq and write some proofs. What can you do? Well,
Rust provides a tool, <code>miri</code>, that can interpret your Rust code and let you know
if you've violated some unsafe rules. But it's not complete, that is, miri can
tell you if your code is wrong, but it cannot tell you if your code is right. It's
still quite useful.</p>
<p>To get miri, you can install it with Rust nightly:</p>
<pre><code>$ rustup +nightly component add miri
</code></pre>
<p>And then run your test suite under miri with</p>
<pre><code>$ cargo +nightly miri test
</code></pre>
<p>For more details, consult <a href="https://github.com/rust-lang/miri">miri's documentation</a>.</p>
<h3>Do things go wrong?</h3>
<p>Do people write good unsafe code, or bad?</p>
<p>Well, the obvious answer to the above question is "yes." Anyone who says all
unsafe code in Rust is sound is obviously lying. But the absolute isn't what's
interesting here, it's the shade of grey. Is unsafe pervasive? Does Rust have
memory related bugs at the same rate as other MSLs, or closer to memory-unsafe
languages?</p>
<p>It's still sort of early days here, but we do have some preliminary results that
are helpful. In 2022, <a href="https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html">Google reported</a> that:</p>
<blockquote>
<p>To date, there have been zero memory safety vulnerabilities discovered in
Android's Rust code.</p>
</blockquote>
<p>I am very interested to see a more up to date report about how that's changed,
but as they say:</p>
<blockquote>
<p>We don't expect that number to stay zero forever, but given the volume of new
Rust code across two Android releases, and the security-sensitive components
where it's being used, it's a significant result. It demonstrates that Rust is
fulfilling its intended purpose of preventing Android's most common source of
vulnerabilities. Historical vulnerability density is greater than 1/kLOC (1
vulnerability per thousand lines of code) in many of Android's C/C++ components
(e.g. media, Bluetooth, NFC, etc). Based on this historical vulnerability
density, it's likely that using Rust has already prevented hundreds of
vulnerabilities from reaching production.</p>
</blockquote>
<p>We'll see if these results replicate themselves, in the future, and also outside
of Google. But it at least looks like in practice, most unsafe code has not led
to problems. At least so far!</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3lkqjlfxypk2d" /&gt;</p></content:encoded>
</item>
<item>
<title>Choosing Languages</title>
<link>https://steveklabnik.com/writing/choosing-languages/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/choosing-languages/</guid>
<pubDate>Thu, 13 Mar 2025 00:00:00 GMT</pubDate>
<content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>I tried to write this post yesterday, but I didn't like my approach there. Last
night, I had a conversation with one of my best friends about it, and he
encouraged me to write this, but in a different way. I think this way is better,
so thanks, dude.</p>
<hr />
<p>The other day, Microsoft announced <a href="https://devblogs.microsoft.com/typescript/typescript-native-port/">that they are re-writing the TypeScript
compiler in Go</a>. A lot of people had a lot of feelings about this. I did
too, but in a different way than many other people.</p>
<p>The biggest questions people had were "Why not C#?" and "Why not Rust?". To
be clear, I do think that asking why someone chooses a programming language
can be valuable; as professionals, we need to make these sorts of decisions
on a regular basis, and seeing how others make those decisions can be useful,
to get an idea of how other people think about these things.</p>
<p>But it can also... I dunno. Sometimes, I think people have little imagination
when it comes to choosing a particular technology stack.</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3lk6p4clry22p" /&gt;</p>
<p>It is true that sometimes, there are hard requirements that means that choosing
a particular technology is inappropriate. But I've found that often, these
sorts of requirements are often more <em>contextually</em> requirements than ones that
would be the case in the abstract. For example, <a href="https://en.wikipedia.org/wiki/Lisp_machine">you can write an OS in
Lisp</a> if you want. But in practice, you're probably not working with
this hardware, or willing to port the environment to bare metal.</p>
<p>But it's also the case that you often don't know what others' contextual
situation actually is. I would have never guessed the main reason that Microsoft
chose Go for, which is roughly "we are porting the existing codebase, rather
than re-writing it, and our existing code looks kinda like Go, making it easy to
port." That's a very good reason! I hadn't really thought about the differences
between porting and re-writing in this light before, and it makes perfect sense
to me.</p>
<p>At the same time,</p>
<p>&lt;BlueskyPost uri="at://did:plc:4eukmtg5kmyjmp6qw3xkpite/app.bsky.feed.post/3lk6t4a4xa22w" /&gt;</p>
<p>people are going <em>way</em> overboard here.</p>
<p>&lt;BlueskyPost uri="at://did:plc:x2p6i7jmdhk4xj2ej3rflp7a/app.bsky.feed.post/3lk6was5n5227" /&gt;</p>
<p>There's some sort of balance here to be struck. But at the same time, I hate
"choose the right tool for the job" as a suggestion. Other than art projects,
has anyone ever deliberately chosen the wrong tool? Often, something that
seems like the wrong tool is chosen simply because the contextual requirements
weren't obvious to you, the interlocutor.</p>
<p>I think there's two other reasons why this situation is making me feel
complicated things. The first is RIIR, and the second is a situation from a long
time ago that changed my life.</p>
<p>Many of you have probably heard of the "re-write it in Rust" meme. The idea is
that there's this plague of programmers who are always suggesting that every
single thing needs to be re-written in Rust. While I am sure this happens from
time to time, I have yet to see real evidence that it is actually widespread. I
almost even wrote a blog post about this, trying to actually quantify the effect
here. But on some level, perception is reality, and if people believe it's true,
it's real on some level, even if that's not actually the case. Regardless of the
truth of the number of people who suggest this, on every thread about Rust,
people end up complaining about this effect, and so it has an effect on the
overall discourse regardless. And while I don't believe that this effect is real
overall, it absolutely has happened in this specific situation. A bunch of
people have been very aggressively suggesting that this should have been in
Rust, and not Go, and I get where they're coming from, but also: really? Go is
an absolutely fine choice here. It's fine. Why do you all have to reinforce a
stereotype? It's frustrating.</p>
<p>The second one is something that's been on my mind for over a decade, but I've
only started talking about it in places other than conversations with close
friends. Long-time readers may remember <a href="/writing/node/">that one time where I was an
asshole</a>. I mean, I've been an asshole more than once in my life, but
this one was probably the time that affected me the most. The overall shape of
the situation is this:</p>
<ol>
<li>Someone wrote <code>grep</code>, but in Node.</li>
<li>I made fun of it on the internet, because why would someone write a CLI tool in Node?</li>
<li>The author found out that I did and felt bad about it.</li>
</ol>
<p>Looking back at my apology, I don't think it's necessarily the best one. It was
better than the first draft or two of what I wrote, but I've gotten better at
apologizing since then. Regardless, this incident changed the trajectory of my
life, and if you happen to read this, Heather, I am truly, unequivocally, sorry.</p>
<p>There was a couple of things that happened here. The first is just simply, in
2013, I did not understand that the things I said had meaning. I hate talking
about this because it makes me seem more important than I am, but it's also
important to acknowledge. I saw myself at the time as just Steve, some random
guy. If I say something on the internet, it's like I'm talking to a friend in
real life, my words are just random words and I'm human and whatever. It is what
it is.</p>
<p>But at that time in my life, that wasn't actually the case. I was on the Rails
team, I was speaking at conferences, and people were reading my blog and tweets.
I was an "influencer," for better or worse. But I hadn't really internalized
that change in my life yet. And so I didn't really understand that if I
criticized something, it was something thousands of people would see. To me, I'm
just Steve. This situation happened before we talked about "cancel culture" and
all of that kind of stuff, but when the mob came for me, I realized: they were
right, actually. I was being an asshole, and I should not have been. And I
resolved myself to not being an asshole like that ever again.</p>
<p>And to do that, I had to think long and hard about why I was like that. I love
programming languages! I love people writing programs that they find
interesting! I don't think that it's stupid to write a tool in a language that
I wouldn't expect it to be written in! So why did I feel the reason to make fun
of this in that situation?</p>
<p>The answer? A pretty classic situation: the people I was hanging out with were
affecting me, and in ways that, when I examined it, I didn't like very much.</p>
<p>You see, in the Rails world at the time, there was a huge <a href="https://blog.aurynn.com/2015/12/16-contempt-culture">contempt
culture</a> at the time, especially around JavaScript and Node. And, when
you're around people who talk shit all the time, you find yourself talking shit
too. And once I realized that, well, I wanted to stop it. But there was a big
problem: my entire professional, and most of my social, life was built around
Ruby and Rails. So if I wanted to escape that culture... what should I do?</p>
<p>If you clicked on the link to my apology above, you probably didn't make note of
the date: Jan 23 2013. Something had just happened to me, in December of 2012: I
had decided to check out this little programming language I had heard of called
Rust.</p>
<p>One of the things that struck me about the "Rust community" at the time, which
was like forty people in an IRC room, was how nice they were. When I had trouble
getting Hello World to compile, and I typed <code>/join #rust</code>, I fully expected an
RTFM and flames. But instead, I was welcomed with open arms. People were chill.
And it was really nice.</p>
<p>And so, a month and a day later, I had this realization about the direction my
life was heading, and how I wasn't really happy about it. And I thought about
how much I enjoyed the Rust stuff so far... and combined with a few other
factors, maybe of which I'll write about someday, this is when I decided that I
wanted to make Rust a thing, and dedicated myself to achieving that.</p>
<p>And part of that was being conscious about the culture, and how it developed,
and the ways that that would evolve over time. I didn't want Rust to end up the
way that Rails had ended up. And that meant a few things, but first of all,
truly internalizing that I had to change. And understanding that I had to be
intentional with what I did and said. And what that led to, among other things,
was a "we talk about Rust on its own merits, not by trash talking other languages"
culture. (That's also partially because of Nietzsche, but I'm already pretty far
afield of the topic for this post, maybe someday.)</p>
<p>Anyway, the Eternal September comes for us all. The "Rust Community," if it ever
were a coherent concept, doesn't really make sense any more. And it is by no
means perfect. I have a lot of criticism about how things turned out. But I do
think that it's not a coincidence that "Rust makes you trans," for example, is
a meme. I am deeply proud of what we all built.</p>
<p>So yeah, anyway: people choose programming languages for projects based on a
variety of reasons. And sure, there may be better or worse choices. But you
probably have a different context than someone else, and so when someone
makes a choice you don't agree with, there's no reason to be a jerk about it.</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3lkblqoytls22" /&gt;</p></content:encoded>
</item>
<item>
<title>A Happy Day for Rust</title>
<link>https://steveklabnik.com/writing/a-happy-day-for-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/a-happy-day-for-rust/</guid>
<pubDate>Wed, 05 Mar 2025 00:00:00 GMT</pubDate>
<content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>A while back, I wrote <a href="/writing/a-sad-day-for-rust/">A sad day for Rust</a> about the Actix-web unsafe
controversy. The short of it was, Actix-web was using some unsound unsafe code.
When this was discovered, people responded pretty harshly, and the maintainer
quit. Others picked up the work, but it still wasn't great:</p>
<blockquote>
<p>I’m not sure where we go from here, and I’m not sure what we could have done
to prevent this from happening. But I feel like this is a failure, and it’s set
Rust back a bit, and I’m just plain sad.</p>
</blockquote>
<p>While the Rust community surely isn't perfect, today I'd like to share with you
a different story: one where there was some controvery, but it was handled in
the complete opposite way.</p>
<p><a href="https://rustup.rs/">Rustup</a> is the official installer and toolchain manager for Rust. It
has a much smaller team than Rust itself does, and releases on a much slower
schedule.</p>
<p>Back in August of last year, it was decided to <a href="https://github.com/rust-lang/rustup/pull/3985">change some behavior</a>.
The details aren't super important, what matters is that it's a breaking change
to a critical component of the ecosystem. I'm not here to say if this change is
good or bad; they had good motivations, there's also good reasons to keep the
previous behavior, it's one of those classic "you can't make everyone happy"
sort of deals.</p>
<p>The team knew that they'd need to let people know that this had happened:</p>
<blockquote>
<p>Yes, you've understood it correctly, and this breakage is intentional: #3635 (comment)</p>
<p>Before the actual release, we'll definitely clearly communicate this change with
the community, and we'll use the beta phase to collect user feedback.</p>
</blockquote>
<p>And they did <a href="https://internals.rust-lang.org/t/seeking-beta-testers-for-rustup-v1-28-0/22060">make a post on internals</a> about the change, seeking
feedback. This also landed in a This Week in Rust, in my understanding.</p>
<p>The problem is, not everyone reads internals, and not everyone reads This Week
in Rust. And so yesterday, some folks woke up to a broken CI, leading to <a href="https://github.com/rust-lang/rustup/issues/4211">this
bug report</a> among other upset posts elsewhere on the internet.</p>
<p>I have been on the other side of this before. It's frustrating when you try and
communicate something, and people don't hear about it, and then they find out
later and get mad at you about it. I also remember the Actix debacle, and many
other controversies in the broader open source ecosystem.</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3ljiqodr7q22f" /&gt;</p>
<p>However, this time, things went differently. As you can see, the issue is very
respectful, and the comments are also generally respectful. The team responded
<a href="https://github.com/rust-lang/rustup/issues/4211#issuecomment-2695620307">graciously</a>, and decided to put in the work to <a href="https://blog.rust-lang.org/2025/03/04/Rustup-1.28.1.html">release a new
version</a> to restore the previous behavior.</p>
<p>I was kind of worried yesterday that this would end up being another Actix
situation. The Rust community has changed a lot over time, and while it's still
pretty good overall, sometimes things can go poorly, just like with any random
group of thousands of people on the internet. But today, I'm grateful that
feedback was given constructively, it was taken with the spirit that it was
intended, and everyone should end up happy.</p>
<p>If only every controversy went this way, maybe I wouldn't be wondering about
coyotes.</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3ljnctebzec25" /&gt;</p></content:encoded>
</item>
<item>
<title>When should I use String vs &str?</title>
<link>https://steveklabnik.com/writing/when-should-i-use-string-vs-str/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/when-should-i-use-string-vs-str/</guid>
<pubDate>Wed, 16 Oct 2024 00:00:00 GMT</pubDate>
<content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>Rust has two main string types: <code>String</code> and <code>&amp;str</code>. Sometimes, people argue
that these two types make Rust code difficult to write, because you have to
think about which one you should be using in a given situation. My experience
of writing Rust is that I don't really think about this very much, and this
post is about some rules of thumb that you can use to be like me.</p>
<h2>Level 1: Don't think about it at all</h2>
<p>The very first thing you can do is follow the simplest rule:</p>
<blockquote>
<p>Always use <code>String</code>, never use <code>&amp;str</code>.</p>
</blockquote>
<p>That looks like this:</p>
<pre><code>struct Person {
name: String,
}
fn first_word(words: String) -&gt; String {
words
.split_whitespace()
.next()
.expect("words should not be empty")
.to_string()
}
</code></pre>
<p>This style means you may need to add <code>.to_string()</code> or <code>.clone()</code>
for things to work sometimes:</p>
<pre><code>fn main() {
let sentence = "Hello, world!";
println!("{}", first_word(sentence.to_string()));
let owned = String::from("A string");
// if we don't clone here, we can't use owned the second time
println!("{}", first_word(owned.clone()));
println!("{}", first_word(owned));
}
</code></pre>
<p>But that's okay, the compiler will let you know when you need to:</p>
<pre><code>error[E0382]: use of moved value: `owned`
--&gt; src/main.rs:21:31
|
18 | let owned = String::from("A string");
| ----- move occurs because `owned` has type `String`, which does not implement the `Copy` trait
19 |
20 | println!("{}", first_word(owned));
| ----- value moved here
21 | println!("{}", first_word(owned));
| ^^^^^ value used here after move
|
note: consider changing this parameter type in function `first_word` to borrow instead if owning the value isn't necessary
--&gt; src/main.rs:5:22
|
5 | fn first_word(words: String) -&gt; String {
| ---------- ^^^^^^ this parameter takes ownership of the value
| |
| in this function
help: consider cloning the value if the performance cost is acceptable
|
20 | println!("{}", first_word(owned.clone()));
| ++++++++
</code></pre>
<p>Hey, compiler, that's a great idea. Let's move on to level 2.</p>
<h2>Level 2: prefer <code>&amp;str</code> for function parameters</h2>
<p>A rule that's a little better is this one:</p>
<blockquote>
<p>Always use <code>String</code> in structs, and for functions, use <code>&amp;str</code> for parameters
and <code>String</code> types for return values.</p>
</blockquote>
<p>This is what the compiler error for level 1 was suggesting we do instead of <code>.clone</code>.</p>
<p>That results in code that looks like this:</p>
<pre><code>struct Person {
name: String,
}
fn first_word(words: &amp;str) -&gt; String {
words
.split_whitespace()
.next()
.expect("words should not be empty")
.to_string()
}
fn main() {
let sentence = "Hello, world!";
println!("{}", first_word(sentence));
let owned = String::from("A string");
println!("{}", first_word(&amp;owned));
println!("{}", first_word(&amp;owned));
}
</code></pre>
<p>We're now doing much less copying. We do need to add a <code>&amp;</code> on
to <code>String</code> values that we wish to pass into <code>first_word</code>, but
that's not too bad, the compiler will help us when we forget:</p>
<pre><code>error[E0308]: mismatched types
--&gt; src/main.rs:20:31
|
20 | println!("{}", first_word(owned));
| ---------- ^^^^^ expected `&amp;str`, found `String`
| |
| arguments to this function are incorrect
|
note: function defined here
--&gt; src/main.rs:5:4
|
5 | fn first_word(words: &amp;str) -&gt; String {
| ^^^^^^^^^^ -----------
help: consider borrowing here
|
20 | println!("{}", first_word(&amp;owned));
| +
</code></pre>
<p>Following this rule will get you through 95% of situations successfully. Yes,
that number was found via the very scientific process of "I made it up, but it
feels correct after writing Rust for the last twelve years."</p>
<p>For 4% of that last 5%, we can go to level 3:</p>
<h2>Level 3: return <code>&amp;str</code> sometimes</h2>
<p>Here's a slightly more advanced rule for certain circumstances:</p>
<blockquote>
<p>Always use <code>String</code> in structs, and for functions, use <code>&amp;str</code> for parameters.
If the return type of your function is derived from an argument and isn’t
mutated by the body, return <code>&amp;str</code>. If you run into any trouble here, return
<code>String</code> instead.</p>
</blockquote>
<p>That would look like this:</p>
<pre><code>struct Person {
name: String,
}
// we're returning a substring of words, so &amp;str is appropriate
fn first_word(words: &amp;str) -&gt; &amp;str {
words
.split_whitespace()
.next()
.expect("words should not be empty")
}
fn main() {
let sentence = "Hello, world!";
println!("{}", first_word(sentence));
let owned = String::from("A string");
println!("{}", first_word(&amp;owned));
println!("{}", first_word(&amp;owned));
}
</code></pre>
<p>This lets us remove a copy, we no longer have a <code>.to_string</code> in the body of
<code>first_word</code>.</p>
<p>Sometimes, we can't do that though:</p>
<pre><code>// because we are going to uppercase the first word, our return type can't
// be &amp;str anymore, because we aren't actually returning a substring: we are
// creating our own new string.
fn first_word_uppercase(words: &amp;str) -&gt; String {
words
.split_whitespace()
.next()
.expect("words should not be empty")
.to_uppercase()
}
fn main() {
let sentence = "Hello, world!";
println!("{}", first_word_uppercase(sentence));
let owned = String::from("A string");
println!("{}", first_word_uppercase(&amp;owned));
println!("{}", first_word_uppercase(&amp;owned));
}
</code></pre>
<p>How do you know that this is the case? Well, in this specific case,
<code>to_uppercase</code> already returns a <code>String</code>. So that's a great hint.
If we tried to return a <code>&amp;str</code>, we'd get an error:</p>
<pre><code>// this can't work
fn first_word_uppercase(words: &amp;str) -&gt; &amp;str {
&amp;words
.split_whitespace()
.next()
.expect("words should not be empty")
.to_uppercase()
}
</code></pre>
<p>would give us</p>
<pre><code>error[E0515]: cannot return reference to temporary value
--&gt; src/main.rs:7:5
|
7 | &amp;words
| ______^-
| | ______|
8 | || .split_whitespace()
9 | || .next()
10 | || .expect("words should not be empty")
11 | || .to_uppercase()
| || ^
| ||_______________________|
| |_______________________returns a reference to data owned by the current function
| temporary value created here
</code></pre>
<p>And that's really it. Following this rule will get you through virtually every
scenario where you need to wonder about <code>String</code> and <code>&amp;str</code>. With some practice,
you'll internalize these rules, and when you feel comfortable with a level, you
can go up to the next one.</p>
<p>What about that last 1% though? Well, there is a next level...</p>
<h2>Level 4: When to use <code>&amp;str</code> in a struct</h2>
<p>Here's the rule for level 4:</p>
<blockquote>
<p>Should you use a <code>&amp;str</code> in a struct? If you're asking that question, use
<code>String</code>. When you need to use <code>&amp;str</code> in a struct, you'll know.</p>
</blockquote>
<p>Storing references in structs is useful, for sure, and it's good that Rust
supports it. But you're only going to need it in fairly specific scenarios,
and if you feel like you're worring about <code>String</code> vs <code>&amp;str</code>, you're just
not in the position to be worrying about the complexity of storing a <code>&amp;str</code>
in a struct yet.</p>
<p>In fact, some people believe in this rule so strongly that they're working
on a language where storing references in structs isn't even possible, and
it's a language I've found very interesting lately:
<a href="https://www.hylo-lang.org/">Hylo</a>. They go a bit farther than that, even:
in Hylo, you think of everything as being values, rather than having references
at all. They think that you can write meaningful programs with this model.
But this isn't a post about Hylo. I'll write one of those eventually. My point
is just that you can really, truly, get away with not storing <code>&amp;str</code>s in structs
for a <em>lot</em> of useful Rust programs. So it's not really worth spending mental
energy on, until you determine that you must do so. That is the case when
you've profiled your program and determined that copying strings to and from
your struct is a big enough issue to bother with lifetimes.</p>
<p>I hope you've found this helpful, and if you have any other rules of thumb
like this, I'd love to hear about them!</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3l6nkx54sij2w" /&gt;</p></content:encoded>
</item>
<item>
<title>Against Names</title>
<link>https://steveklabnik.com/writing/against-names/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/against-names/</guid>
<pubDate>Mon, 12 Aug 2024 00:00:00 GMT</pubDate>
<content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>There's an old saying:</p>
<blockquote>
<p>There are only two hard things in Computer Science: cache invalidation and
naming things.</p>
<p>― Phil Karlton</p>
</blockquote>
<p>I also appreciate the joke version that adds "and off by one errors."</p>
<p>Lately, I've been thinking about this saying, combined with another old joke:</p>
<blockquote>
<p>"The patient says, "Doctor, it hurts when I do this." The doctor says, "Then
don't do that!"</p>
<p>― Henny Youngman</p>
</blockquote>
<p>Specifically, if naming things is so hard... why do we insist on doing it all
the time?</p>
<p>Now, I am not actually claiming we should stop giving things names. But I have
had at least two situations recently where I previously found names to be kinda
critical, and then I changed to systems which didn't use names, and I think it
improved the situation.</p>
<p>One of the most famous examples of not giving something a name,
lambdas/closures, took some time to catch on. But many folks already recognize
that naming every single function isn't always neccesary. I wonder if there are
more circumstances where I've been naming things where I didn't actually have
to.</p>
<p>Anyway, here's my two recent examples:</p>
<h2>Nameless Branches</h2>
<p>I haven't written much about it on my blog yet, but I'm fully converted away
from git to <a href="https://github.com/martinvonz/jj">jj</a>. I'll say more about this in the future, but one major
difference between the two is that jj has anonymous branches. If, like me,
you are a huge fan of git, this sounds like a contradiction. After all, the
whole thing about branches are that they're a name for some point in the DAG.
How do you have a nameless name?</p>
<p>Here's some output from <code>jj log</code>:</p>
<p>Here's some sample output from jj log:</p>
<pre><code> $ jj log --limit 5
@ pzoqtwuv [email protected] 2024-03-01 15:06:59.000 -06:00 9353442b
│ added some cool new feature
│ ◉ xrslwzvq [email protected] 2024-02-29 23:06:23.000 -06:00 a70d464c
├─╯ create hello and goodbye functions
│ ◉ yykpmnuq [email protected] 2024-02-29 23:03:22.000 -06:00 210283e8
├─╯ add better documentation
◉ ootnlvpt [email protected] 2024-02-28 23:26:44.000 -06:00 b5db7940
│ only print hello world
◉ nmptruqn [email protected] 2024-02-28 23:09:11.000 -06:00 90a2e97f
│ refactor printing
</code></pre>
<p>Here, we are working on change <code>pzoqtwuv</code>. (<code>@</code> means the working copy.) There
are colors in the real CLI to make the differences more obvious, and to show you
unique prefixes, so for example, you probably only need <code>p</code> or <code>pz</code> instead of
<code>pzoqtwuv</code> to uniquely identify the change. I'll use the full IDs since there's
no syntax highlighting here.</p>
<p>We have two anonymous branches here. They have the change IDs of <code>xrslwzvq</code> and
<code>yykpmnuq</code>. The log output shows the summary line of their messages, so we can
see "create hello and goodbye functions" on one branch, and "add better
documentation" on the other.</p>
<p>You don't need an additional branch name: the change ID is already there. If you
want to add even more better documentation, <code>jj new yykpmnuq</code> (or again, likely
<code>jj new yy</code> in practice) and you're off to the races. (jj new makes a new change
off of the parent you specify.)</p>
<p>(And if you're in a larger repo with more outstanding branches, you can ask <code>jj log</code> to show specific subsets of commits. It has a powerful DSL that lets you do
so. For example, say you only want to see your commits, <code>jj log -r 'mine()'</code> can
do that for you.)</p>
<p>That's all there is to it. We already have the commit messages and IDs, giving
an additional identifier doesn't help that much. In practice, I haven't missed
named branches at all. And in fact, I kind of really appreciate not bothering to
come up with a name, and then eventually remembering to delete that name once
the PR lands, stuff like that. Life is easier.</p>
<h2>Utility CSS</h2>
<p>Another technology I have learned recently is <a href="https://tailwindcss.com/">tailwind</a>. But Tailwind is just
one way of doing a technique that has a few names, I'm going to go with "utility
CSS". The idea is in opposition to "semantic CSS." To crib an example from a
<a href="https://adamwathan.me/css-utility-classes-and-separation-of-concerns/">blog post by the author of Tailwind</a> (which does a better job of thoroughly
explaining why doing utility CSS is a good thing, you should go read it),
semantic CSS is when you do this:</p>
<pre><code>&lt;style&gt;
.greeting {
text-align: center;
}
&lt;/style&gt;
&lt;p class="greeting"&gt;Hello there!&lt;/p&gt;
</code></pre>
<p>Whereas, with Tailwind, you end up instead having something like this:</p>
<pre><code>&lt;p class="text-center"&gt;Hello there!&lt;/p&gt;
</code></pre>
<p>We don't have a new semantic name <code>greeting</code>, but instead describe what
we want to be done to our element via a utility class.</p>
<p>So the thing is, as a previous semantic CSS enjoyer, this feels like using
inline styling. But there's a few significant differences. The first one is,
you're not writing plain CSS, you are re-using building blocks that are defined
for you. The abstraction is in building those utility classes. This means you're
not writing new CSS when you need to add functionality, which to me is a great
sign that the abstraction is working.</p>
<p>It's also that there is some sleight of hand going on here, as we do, on another
level. An objection that gets raised to doing things this way is "what happens
when you need to update a bunch of similar styles?" And the answer for that is
components. That is, it's not so much that utility CSS says that semantic names
are bad, it's that semantic names <em>at the tag level</em> are the wrong level of
abstraction to use names. To sort of mix metaphors, consider the lambda/closure
example. Here's a random function in Rust:</p>
<pre><code>fn calculate(list: &amp;[i32]) -&gt; i32 {
list.into_iter()
.filter(|i| **i % 2 == 0)
.sum()
}
</code></pre>
<p>The <code>**</code> is unfortunate, but this function takes a list of numbers, selects for
the even ones, and then sums them. Here, we have a closure, the argument to
<code>filter</code>, but it's inside a named function, <code>calculate</code>. This is what using
Tailwind feels like to me, we use names for higher level concepts (components),
and then keep things semantically anonymous for some of the tasks inside of them
(markup).</p>
<p>Heck, even the most pro-semantic-styles folks don't advocate that you must give
every single element a class. Everyone recognizes the value of anonymous things
sometimes, it's just a matter of what level of abstraction deserves to get
named.</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3kzkadmleqe2m" /&gt;</p></content:encoded>
</item>
<item>
<title>How Does BlueSky Work?</title>
<link>https://steveklabnik.com/writing/how-does-bluesky-work/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/how-does-bluesky-work/</guid>
<pubDate>Sat, 24 Feb 2024 00:00:00 GMT</pubDate>
<content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>One of the reasons I am enthusiastic about BlueSky is because of the way that
it works. So in this post, I am going to lay out some of the design and the
principles behind this design, as I understand them. I am not on the BlueSky
team, so these are my takes only.</p>
<p>Let's begin.</p>
<h2>Why does BlueSky exist?</h2>
<p>Here's what <a href="https://bsky.social">the BlueSky Website</a> says right now:</p>
<blockquote>
<p>Social media is too important to be controlled by a few corporations. We’re
building an open foundation for the social internet so that we can all shape
its future.</p>
</blockquote>
<p>This is the big picture.</p>
<p>Okay so that's a great idea, but like, what does that <em>mean</em>? Currently,
BlueSky is a microblogging application, similar to Twitter and Mastodon. How
does that fit into the big picture? Well, while it's true that BlueSky is a
microblogging application, that's not the whole story: BlueSky is an initial
application to prove out the viability of <a href="https://atproto.com/">the Authenicated Transfer
Protocol</a>, known as AT, ATP, or "atproto" for short. BlueSky is the
"building" and atproto is the "open foundation for the social internet."</p>
<p>An important thing to note: BlueSky is also a company. Some people look at a
company saying "hey we're building something that's too big to be controlled
by companies!" with skepticism. I think that's a healthy starting point, but
the answer for me is atproto.</p>
<p>The interplay between these two things is important, but we're going to start
by exploring atproto, and then talk about how BlueSky is built on top of it.</p>
<h2>Is this a cryptocurrency?</h2>
<p>The first thing we have to get out of the way: If you hear "oh it's a
distributed network called 'something protocol'" you may have a "is this
a cryptocurrency?" alarm bell going off in your head.</p>
<p>Don't worry, it's not a cryptocurrency. It does use some technologies that
originated in the cryptocurrency space, but this isn't a blockchain, or a DAO,
or NFTs, or any of that. Just some cryptography and merkle trees and the like.</p>
<h2>What is the big picture with atproto?</h2>
<p>Here's what <a href="https://atproto.com/guides/overview">the AT Protocol Overview</a> says:</p>
<blockquote>
<p>The Authenticated Transfer Protocol, aka atproto, is a federated protocol for
large-scale distributed social applications.</p>
</blockquote>
<p>Let's break that down:</p>
<blockquote>
<p>a federated protocol</p>
</blockquote>
<p>atproto is federated. This means that the various parts of the system can have
multiple people running them, and that they communicate with each other.</p>
<p>Choosing federation is a big part of how atproto delivers on the "can't be
controlled by one organization" promise. There are other parts too, but this
is an important aspect of solving this.</p>
<blockquote>
<p>for large-scale</p>
</blockquote>
<p>If you want to scale, you have to design with scale in mind. atproto makes
several interesting choices in order to distribute the load of running the
system more onto the actors that can handle load, and less on those that can't.
This way, applications running on top of atproto can scale up to large userbases
without issue.</p>
<p>That's the hope, at least. Earlier this week, BlueSky hit five million users,
and is far more stable than Twitter was in the early days. That's not as big
as many social applications, but it's not nothing either. We'll see how this
works out in practice.</p>
<blockquote>
<p>distributed social applications</p>
</blockquote>
<p>atproto is for connecting to others, so it's focused on social applications.
It also is currently 100% public, there are no private messages or similar. The
reasons for this is that achieving private things in a federated system is
very tricky, and they would rather get it right than ship something with serious
caveats. Best for now to only use this stuff for things you want to be public.</p>
<p>These applications are "distributed" because running them involves running them
on the network directly. There's no "BlueSky server," there's just servers
running atproto distributing messages to each other, both BlueSky messages and
whatever other messages from whatever other applications people create.</p>
<p>So that's the high level, but what does that mean concretely?</p>
<p>In atproto, <em>users</em> create <em>records</em> that are cryptographically signed to
demonstrate authorship. Records have a schema called a <em>Lexicon</em>.</p>
<p>Records are stored in <em>repositories</em>. Repositories run as a <em>service</em>, exposing
HTTP and WebSockets. They then can then talk to each other and federate the
records. These are often called PDSes, for "Personal Data Server." Users
either run their own PDS, or use one that someone else hosts for them.</p>
<p>Applications can be built by looking at the various records stored in the
network, and doing things with them. These services all called <em>App Views</em>,
because they are exposing a particular view of the information stored in the
network. This view is created via the Lexicon system: building an application
means that you define a Lexicon, structuring the data that you want to deal with,
and then look at records that use your lexicon, ignoring the rest.</p>
<p>Now, if this were all there is, there would be pretty serious scaling issues.
For example, if every time I post a new update on BlueSky, if I had to send
my post to every single one of my followers' repositories, that would be
extremely inefficent, and make running a popular repository very expensive to
run. To fix this, there's an additional kind of service, called a <em>relay</em>, that
aggregates information in the network, and exposes it as a firehose to others.
So in practice, App Views don't look at Repositories, but instead, look at
Relays. When I make a post, my respository won't notify my followers'
repositories individually. My repository will notify a Relay, and my followers
will use an App View that filters the ouput of the Relay to show only the posts
of people they're following. This does imply that Relays are often huge and
expensive to run, however you could imagine running a smaller relay that only
propogates posts from a smaller subset of users too. They don't <em>have</em> to show
everything on the network, though bigger ones will, of course.</p>
<p>Here this is in ASCII art:</p>
<pre><code> ┌─────┐ ┌──────────┐
│ PDS ├───────┐ │ App View │
└─────┘ │ └──────────┘
┌▼────────┐ ▲
┌─────┐ │ ├───────┘
│ PDS ├──────► Relay │
└─────┘ │ ├───────┐
└▲────────┘ ▼
┌─────┐ │ ┌──────────┐
│ PDS ├───────┘ │ App View │
└─────┘ └──────────┘
</code></pre>
<p>This is all you really need to know to understand the core of atproto: people
create data, it's shared in the network, and applications can interact with
that data.</p>
<p>However, there are additional service types being introduced, with the
possibility of more in the future. But before we talk about those, we have to
explain some ideological commitments to understand why things are shaped the way
they are.</p>
<h2>What is "speech vs reach"?</h2>
<p>Given that atproto is deliberately created to enable social applications, it
needs to consider not just connecting people, but also disconnecting people.
Moderation is a core component of any social application: "no moderation" is
still a moderation strategy. BlueSky handles these sorts of questions by
acknowledging that different people will have different preferences when it
comes to moderation, and also that moderation at scale is difficult.</p>
<p>As such, the protocol takes a "speech vs reach" approach to moderation. The
stuff we've described so far falls under the "speech" layer. It is purely
concerned with replicating your content across the network, without caring
what the semantic contents of that content is. Moderation tools fall under the
"reach" layer: you take all of that speech, but provide a way to limit the
reach of stuff you don't care to see yourself.</p>
<p>Sometimes, people say that BlueSky is "all about free speech" or "doesn't do
moderation." This is simply inaccurate. Moderation tooling is encoded into the
protocol itself, so that it can work with all content on the network, even
non-BlueSky applications. Moreover, it gives you the ability to choose your own
moderators, so that you aren't beholden to anyone else's choice of moderation or
lack thereof. But I'm getting ahead of myself: let's talk about feed generators
and labelers.</p>
<h2>What are feed generators?</h2>
<p>Most social applications have the concept of a "feed" of content. This is broken
out into its own kind of service in atproto, called a <em>feed generator</em>. A classic
example of a feed is "computer, show me the posts of the people I follow in
reverse chronological order." Lately, algorithmic feeds have become popular with
social networks, to the point of where some non-technical users refer to them
as "algorithms."</p>
<p>Feed generators take the firehose produced by a relay, and then show you a list
of content, filtered and ordered by whatever metric the feed generator desires.
You can then share these feeds with other users.</p>
<p>As a practical example, one of my favorite feeds is the <a href="https://bsky.app/profile/did:plc:vpkhqolt662uhesyj6nxm7ys/feed/infreq">Quiet
Posters</a> feed. This feed shows posts by people who don't post
very often. This makes it so much easier to keep up with people who get drowned
out of my main feed. There are feeds like <a href="https://bsky.app/profile/did:plc:vpkhqolt662uhesyj6nxm7ys/feed/followpics">the 'Gram</a>, which shows
only posts that have pictures attatched. Or <a href="https://bsky.app/profile/did:plc:q6gjnaw2blty4crticxkmujt/feed/bangers">My Bangers</a>, which shows
your most popular posts.</p>
<p>This to me is one of the killer features of BlueSky over other microblogging
tools: total user choice. If I want to make my own algorithm, I can do so.
And I can share them easily with others. If you use BlueSky, you can visit
any of those feeds and follow them too.</p>
<p>Feeds are a recent addition to atproto, and therefore, while they do exist,
they may not be feature complete just yet, and may undergo some change in the
future. We'll see. They're working just fine from my perspective, but I haven't
been following the lower level technical details.</p>
<h2>What are labelers?</h2>
<p>A <em>Labeler</em> is a service that applies <em>labels</em> to content or accounts. As a user,
you can subscribe to a particular labeler, and then have your experience change
based on the labels on posts.</p>
<p>A labeler can do this via whatever method it pleases: automatically by running
some sort of algorithm on posts, manually by having some human give a thumbs
up or thumbs down, whatever method the person running the labeling service
wants.</p>
<p>An example of a labeling service would be a blocklist: a label on the posts
authored by people whose content you don't want to see. Another example is
an NSFW filter, which may run some sort of algorithm over pictures in posts,
and labeling them if they believe they contain NSFW content.</p>
<p>Labeling exists, but I do not believe you can run your own labeler yet. BlueSky
runs their own, but there hasn't been an external release that I am aware of.
But once they do, you can imagine communities running their own services, adding
whatever kind of labels they'd like.</p>
<h2>How does moderation work in atproto?</h2>
<p>Putting this all together, we can see how moderation works: Feeds may choose to
transform the feed based on labels, or App Views may take feeds and apply
transformations based on asking a Labeler about it. These can
be mixed and matched based on preference.</p>
<p>This means you can choose your moderation experience, not just in applications,
but also within it. Want a SFW feed, but allow NSFW content in another? You
can do that. Want to produce a blocklist of people and share it with the
world? You can do that.</p>
<p>Because these moderation tools work at the network level, rather than at the
application level, they actually go <em>further</em> than in other systems. If someone
builds an Instagram clone on atproto, that could also use your blocklist
labeller, since your blocklist labeller works at the protocol level. Block
someone in one place, and they can be blocked on every place, if you so choose.
Maybe you subscribe to different moderation decisions in different applications.
It is 100% up to you.</p>
<p>This model is significantly different from other federated systems, because
you don't really have an "account" on an "instance," like in Mastodon. So a lot
of people ask questions like "what happens when my instance gets defederated"
which don't exactly make sense as stated. You can achieve the same goal, by
blocking a set of users based on some criteria, maybe you dislike a certain
PDS and want to ignore posts that come from a certain one, but that is <em>your</em>
choice and yours alone, it is not dictated by some "server owner" that your
account resides on.</p>
<p>So if you don't have a home server, how does identity work?</p>
<h2>How does identity and account portability work?</h2>
<p>There are a LOT of details to how identity works, so I'm going to focus on the
parts that I find important. I am also going to focus on the part that is
controversial, because that is important to talk about.</p>
<p>At its core, users have an identity number, called a "Decentralized Identifier,"
or <em><a href="https://www.w3.org/TR/did-core/">DID</a></em>. My DID looks like this: <code>did:plc:3danwc67lo7obz2fmdg6jxcr</code>.
Feel free to follow me! Lol, of course that's not the interface that you'll see
most of the time. Identity also involves a <em>handle</em>, which is a domain name.
My handle is <code>steveklabnik.com</code>, unsurprisingly. You'll see my posts on BlueSky
as coming from <code>@steveklabnik.com</code>. This system also works well for people who
don't own a domain; if you sign up for BlueSky, it'll give you the ability to
choose a name, and then your handle is <code>@username.bsky.social</code>. I started off
making posts as <code>@steveklabnik.bsky.social</code>, and then moved to
<code>@steveklabnik.com</code>. But because the DID is stable, there was no disruption to
my followers. They just saw the handle update in the UI.</p>
<p>You can use a domain as your handle by getting the DID your PDS generated for
you, and then adding a <code>TXT</code> record in the DNS you use for that domain. If
you're not the kind of person who uses or even knows what DNS is, I envy you,
but you can also use BlueSky's partnership with NameCheap to register a domain
and configure it to use as a handle without any technical knowledge necessary.
You can then log into applications with your domain as the handle, and
everything works nicely.</p>
<p>This is also how BlueSky delivers true "account portability," partially because,
well, there isn't really a concept of an account. The person who uses a given
DID uses cryptography to sign the content they create, and then that content
is replicated across the network. "Your account" can't really be terminated,
because that would mean someone forcibly stopping you from using keys that they
don't even have access to. If your PDS goes down, and you want to migrate to
a new one, there's a way to backfill the contents of the PDS from the network
itself, and inform the network that your PDS has moved. It is real, meaningful
account portability, and that is radically different from any similar service
running today.[^1]</p>
<p>But.</p>
<p>The devil is in the details, and I think this is one of the more meaningful
criticisms of BlueSky and atproto.</p>
<p>You see, there are different "methods" of creating a DID. BlueSky supports
two methods: <code>did:web</code>, which is based on domain names. There are some drawbacks
with this method that I don't personally fully understand well enough to describe,
I'm sure I'll write something in-depth about DIDs in the future.</p>
<p>So because of that weakness, BlueSky has implemented their own DID method,
called <code>did:plc</code>. The <code>plc</code> stands for "placeholder," because even though
they plan on supporting it indefinitely, it too has its weaknesses. And that
weakness is that it involves asking a service that BlueSky runs in order to
resolve the proper information. For example, <a href="https://plc.directory/did:plc:3danwc67lo7obz2fmdg6jxcr">here is my lookup</a>.
This means that BlueSky can ban you in a more serious way than is otherwise
possible thanks to the network design, which some people take to be a very
serious issue.</p>
<p>So, is the flaw fatal? I don't think so. The first reason is, if you really don't
want to engage with it, you can use <code>did:web</code>. Yes that isn't great for other
reasons; that's why <code>did:plc</code> was created. But you do get around this issue.</p>
<p>Another is that the BlueSky team has demonstrated, in my personal opinion,
enough understanding and uncomfortableness with being in control here, and it's
designed in such a way that if other, better systems develop, you can move
to them. They've also indicated that moving governance of <code>did:plc</code> to some sort
of consensus model in the future is possible. There are options. Also, others
could run a <code>did:plc</code> service and use that instead if they prefer, too.</p>
<p>I personally see this as an example of pragmatically shipping something, others
see it as a nefarious plot. You'll have to decide for yourself.</p>
<h2>How is BlueSky built on top of atproto?</h2>
<p>So, now that we understand atproto, we can understand BlueSky. BlueSky is
an application built on top of the atproto network. They run an App View, and
<a href="https://bsky.app/">a web application</a> that uses that App View to work. They also run a PDS
for users that sign up through the web app, as well as a relay that those PDSes
communicate with.</p>
<p>They publish two Lexicons, one as <code>com.atproto.*</code> and one as <code>app.bsky.*</code>. The
former are low level operations that any application on the network will need,
and the ones specific to BlueSky are in the latter.</p>
<p>But one nice thing about BlueSky in particular is that they've taken the product
goals that nobody should know any of this nerd shit to be able to use BlueSky.
The lack of instances means there's no "I need to pick an instance to create an
account" flow, and the portability means that if my host goes down, I can move,
and my followers are none the wiser.</p>
<h2>How will others build applications on top of atproto?</h2>
<p>You can create an atproto app by creating a Lexicon. You'll then want to run
an App View that does things with data on the network involving your lexicon,
and your application will want to give people the ability to write data to their
PDS using your lexicon.</p>
<p>I myself am considering doing so. We'll see.</p>
<h2>Concluding thoughts</h2>
<p>So yeah, on the technical side of things, that's an overview of how atproto and
BlueSky work. I think this design is very clever. Furthermore, I think the
separation of concerns between atproto and BlueSky are very meaningful, as having
a "killer app" for the network gives a reason to use it. It also is a form of
dogfooding, making sure that atproto is good enough to be able to build real
applications on.</p>
<p>I'm sure I'll have more to say about all of this in the future.</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3km6y2x6cxk2f" /&gt;</p>
<p>[^1]:
A commentor points out https://book.peergos.org/, which I had not heard of,
but apparently was known to the creators of BlueSky before they made it. Neat.</p></content:encoded>
</item>
<item>
<title>Using the Oxide Console</title>
<link>https://steveklabnik.com/writing/using-the-oxide-console/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/using-the-oxide-console/</guid>
<pubDate>Sat, 17 Feb 2024 00:00:00 GMT</pubDate>
<content:encoded><p>A very, very long time ago, I was introduced to <a href="https://en.wikipedia.org/wiki/Gerrit_(software)">Gerrit</a>. To be honest,
I hated it. However, lately I have become interested in divesting from
<code>git</code> and GitHub, and so have decided to re-visit various "forges" to see what's
out there. The "why" for this will come later, and I'm not leaving them just yet,
just, you know, doing some exploring.</p>
<p>Anyway, in order to play around with Gerrit, I need a server somewhere. This
website runs entirely on Vercel at the moment, so I don't happen to have a
more traditional server lying around to give Gerrit a shot. But as it just so
happens, I work at <a href="https://oxide.computer/">Oxide</a>, and so have access to a rack that I can play
around with. So I thought it would be fun to share what that looks like, given
that people are interested in what we do, but don't actually have access to
one of our machines.</p>
<p>If you'd like to poke at the console yourself, there's a <a href="https://oxide-console-preview.vercel.app/">live demo</a>!
Quoting <a href="https://github.com/oxidecomputer/console?tab=readme-ov-file#live-demo">the README</a>:</p>
<blockquote>
<p>At https://oxide-console-preview.vercel.app the console is deployed as a
static site with a mock API running in a Service Worker. You can create mock
resources and they will persist across client-side navigations, but they exist
only in the browser: nobody else can see them and the mock "DB" is reset on
pageload. Request and response bodies in the mock API match the Oxide API's
OpenAPI spec, but behavior is only mocked in as much detail as is required for
development and testing of the console and is not fully representative of the
real API.</p>
</blockquote>
<p>I happen to think this is extremely cool. If you want to, you could stop reading
this post and go play with that instead. It's also, like the rest of Oxide's software
stack, fully open source!</p>
<p>Before we begin, I would like to say that I am going to be doing this "manually"
via the console and a web browser rather than via the CLI. I am doing this for
two reasons:</p>
<ol>
<li>I think the console is pretty and I would like to show it off. The folks who
work on it do a fantastic job.</li>
<li>I don't have many images on this blog and so want to write a post with some
images and this will look nicer than screenshots of a terminal.</li>
</ol>
<p>To be clear, I imagine a lot of real production usage of the Oxide rack will be
driven by <a href="https://docs.oxide.computer/guides/introduction#_overview">the API</a>, either via our CLI tools or terraform or
whatever. Heck, the console is built on those APIs, so on some level, that's the
only way to do it. My point is, don't think of this as being the only workflow
to interact with Oxide. Maybe if people like this post I'll make some more
covering the CLI driven workflow. Let me know.</p>
<hr />
<p>After you SSO into the console, you're put on the Projects page:</p>
<p>&lt;img src="/img/2024-02-17/projects.png"&gt;</p>
<p>We're in a "silo" named Oxide. A silo is a grouping of users, projects, and
resources that are isolated from one another. But we don't care about that:
we want to make a new project. As you can see, I have censored my co-workers'
projects, as I didn't tell anyone I was making this post, and even though
the description here is hilarious, it wouldn't be right to share without asking,
and I'm already like ten layers deep in this yak shave.</p>
<p>Let's make a new project:</p>
<p>&lt;img src="/img/2024-02-17/new-project.png"&gt;</p>
<p>It's just that easy. After we push the button, we're put on the instances page:</p>
<p>&lt;img src="/img/2024-02-17/instances.png"&gt;</p>
<p>There's a nice little toast in the bottom left that I didn't manage to get a
screenshot of because I was re-sizing my window too. Anyway, now that we have
our project, we can add one or more instances to it. Instances are virtual
machines, and so setting one up looks like you might expect. You can give it
a name and description, but for this screenshot I'm only showing off one part
of this page because this is already a ton of screenshots:</p>
<p>&lt;img src="/img/2024-02-17/create-instance.png"&gt;</p>
<p>We can size instances however we'd like. Even though Gerrit is a Java
application these days, I'm going to keep the instance small, because it's just
for me and performance isn't a huge deal. Yes, I have made massive instances for
fun in the past.</p>
<p>We're using a Debian image that's been provided by this silo, because I am lazy
and that's fine. Is four gigabytes of disk enough? I don't know, let's find out.</p>
<p>A few seconds later, our instance is up and running:</p>
<p>&lt;img src="/img/2024-02-17/instance.png"&gt;</p>
<p>I am obscuring the external IP because, well, no reason to give that out.
Anyway, that's it! We can <code>ssh</code> in:</p>
<p>&lt;img src="/img/2024-02-17/login.png"&gt;</p>
<p>... how, you may ask? Well, I didn't show off the entire "Create instance" page:
I have set up my ssh keys in my account previously, and so by default, because
this image uses <code>cloud-init</code> to configure itself, my <code>ssh</code> keys are already
injected into the image. Furthermore, I left it on the default <a href="https://en.wikipedia.org/wiki/Virtual_private_cloud">VPC</a>, which
is currently configured with a firewall rule that allows for incoming <code>ssh</code>
connections. All of this is of course absolutely customizable, but given what
this rack is being used for, these defaults make sense for us, so I didn't have
to actually do much setup at all.</p>
<p>That being said, I do need to open port 8080 so that I can poke at Gerrit in
a web browser. If I go into the VPC configuration, I can add a new firewall rule:</p>
<p>&lt;img src="/img/2024-02-17/add-firewall-rule.png"&gt;</p>
<p>Further down, I can pick port 8080, and that I want to allow TCP. After setting
up Gerrit, we have success!</p>
<p>&lt;img src="/img/2024-02-17/success.png"&gt;</p>
<hr />
<p>And that's it! It's just that easy. There's of course a ton of stuff I didn't
talk about, but easy things should be easy, and for basic usage, it's just that
easy.</p></content:encoded>
</item>
<item>
<title>Memory Safety is a Red Herring</title>
<link>https://steveklabnik.com/writing/memory-safety-is-a-red-herring/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/memory-safety-is-a-red-herring/</guid>
<pubDate>Thu, 21 Dec 2023 00:00:00 GMT</pubDate>
<content:encoded><p>TL;DR:</p>
<p>I think that a focus on memory safe languages (MSLs) versus non memory-safe
languages is a bit of a red herring. The actual distinction is slightly bigger
than that: languages which have defined behavior by default, with a superset
where undefined behavior is possible, vs languages which allow for undefined
behavior anywhere in your program. Memory safety is an important aspect of this,
but it is necessary, not sufficient. Rust's marketing has historically focused
on memory safety, and I don't think that was a bad choice, but I do wonder
sometimes. Finally, I wonder about the future of the C++ successor languages in
the face of coming legislation around MSLs for government procurement.</p>
<p>Additionally, I would like to thank ixi for discussing some of this stuff with
me. I do not claim that they agree with me, these opinions are solely my own!
But talking about it helped me form those opinions.</p>
<hr />
<h2>ThreadId(1)</h2>
<p>Today I have been programming in Rust for 11 years. I usually write some sort
of blog post reflecting on the year. This year, I am reflecting on some old
questions:</p>
<ul>
<li>Should Rust have focused so much on memory safety in its advocacy?</li>
<li>Does Rust conflate a more general concept of "safety" with "memory safety?"</li>
<li>What do people even mean by "memory safe languages?"</li>
</ul>
<p>But before we get there, we have to talk about some other things.</p>
<h2>ThreadId(2)</h2>
<p>The other day, someone <a href="https://news.ycombinator.com/item?id=38616605">asked an interesting question on Hacker
News</a>:</p>
<blockquote>
<p>Can someone with a security background enlighten me, on why Python is on the
list of "memory safe" languages? Most of the python code I have worked with is
a thin wrapper on C. Wouldnt that make python vulnerable as well?</p>
</blockquote>
<p>This is a very reasonable question! If you click the link, you'd see my answer:</p>
<blockquote>
<p>You are correct that if you call C from Python, you can run into problems. But
the fault there lies with the C, not the Python. Pure Python itself is memory
safe.</p>
<p>Because memory safety is built on top of memory unsafety, it is generally
understood that when discussing things at this sort of level, that we are
speaking about the Python-only subset. (Same as any other memory safe language.)</p>
</blockquote>
<p>But this has been gnawing at me for a while. Like it's one of those things that
"everybody knows what I mean." But isn't that kind of... bullshit? Like something
is wrong with your definitions. But on the other hand, if we accept FFI as
being not safe, and that since you can FFI from anywhere, that means it's also
unsafe, then every single program is unsafe. Your operating system does not
offer memory-safe APIs. To do anything useful, you must call into the operating
system. This means every program would be infected via this conception of safety,
and therefore, as a definition, it would be useless.</p>
<p>If we instead make an exception for a language's runtime, which is allowed to
make unsafe calls, but users' code is not, that would draw an appropriate
boundary: only write code in the guest language, and you don't have to worry
about safety anymore. And I think that this sort of definition has been the
default for many years. You have "managed" languages which are safe, and you have
"unmanaged" languages which are unsafe. Unmanaged languages are used to implement
managed languages. What's that? A Java runtime written in Java? A Ruby runtime
written in Ruby? Yeah those are weird exceptions. Ignore those. And those
"native extensions" that segfault? Well they aren't written in the host language,
so of course that will happen, and yeah it's kinda annoying when it happens, but
it's fine. Sometimes you have to risk it for performance.</p>
<p>I just don't find this satisfying. It's too vague. You just kinda know it when
you see it. Ignore FFI, and we're good. And sometimes, you just can't have safety.
What if you want to write a runtime? You're gonna need to call into the system
directly, and now we're back to unsafe, so just use an unsafe language. It is
what it is.</p>
<h2>ThreadId(3)</h2>
<p>That comment is on an article titled <a href="https://media.defense.gov/2023/Dec/06/2003352724/-1/-1/0/THE-CASE-FOR-MEMORY-SAFE-ROADMAPS-TLP-CLEAR.PDF">The Case for Memory Safe
Roadmaps: Why Both C-Suite Executives and Technical Experts
Need to Take Memory Safe Coding Seriously
</a>.
This document, published by <a href="https://en.wikipedia.org/wiki/Five_Eyes">the Five Eyes</a>,
suggests... that executives and technical experts need to take memory safety
seriously. Call it boring if you want, but it is what it says on the tin.</p>
<p>More specifically, it says:</p>
<blockquote>
<p>Memory safe programming languages (MSLs) can eliminate memory safety
vulnerabilities.</p>
</blockquote>
<p>As well as an appendix, "Memory Safe Languages," which describes C#, Go, Java,
Python, Rust, and Swift as memory safe languages.</p>
<p>Finally, it says:</p>
<blockquote>
<p>Programming languages such as C and C++ are examples of memory unsafe programming
languages that can lead to memory unsafe code and are still among the most widely used
languages today.</p>
</blockquote>
<p>But it also acknowledges the reality that sometimes you can get around memory
safety:</p>
<blockquote>
<p>MSLs, whether they use a garbage collection model or not, will almost
certainly need to rely on libraries written in languages such as C and C++.
Although there are efforts to re-write widely used libraries in MSLs, no such
effort will be able to re-write them all anytime soon.</p>
<p>For the foreseeable future, most developers will need to work in a hybrid model
of safe and unsafe programming languages. Developers who start writing in an MSL
will need to call C and C++ libraries that are not memory safe. Likewise, there
are going to be situations where a memory unsafe application needs to call into
a memory safe library. When calling a memory unsafe component or application,
the calling application needs to be explicitly aware of—and limit any input
passed to—the defined memory bounds.</p>
<p>The memory safety guarantees offered by MSLs are going to be qualified when data
flows across these boundaries. Other potential challenges include differences
in data marshalling, error handling, concurrency, debugging, and versioning.</p>
</blockquote>
<p>Okay, that's enough quoting from the report. And I swear I'm not about to say
nice things about it because I am cited as #42, though to be honest it's real
fucking weird that searching "klabnik" on a page on defense.gov returns 1/1.
But I think this is an excellent formulation of the critical distinction between
the languages on one side of the line versus the other, though they make the
same conceptual mistake that I believe Rust may have: it's not just about
memory unsafety. "safety" itself is basically the idea that most of the time,
you're in a universe where everything is peachy keen, and occasionally, you'll
need to reach across that line into the scary other place, but you gotta do what
you gotta do. But there is a line there. But that line is more abstract than
just memory safety.</p>
<h2>ThreadId(4)</h2>
<p>Rust often describes itself as "data race free." But what the heck is a data
race anyway? Here's <a href="https://blog.regehr.org/archives/490">John Regehr</a>:</p>
<blockquote>
<p>A data race happens when there are two memory accesses in a program where both:</p>
<ul>
<li>target the same location</li>
<li>are performed concurrently by two threads</li>
<li>are not reads</li>
<li>are not synchronization operations</li>
</ul>
<p>There are other definitions but this one is fine; it’s from Sebastian Burckhardt at Microsoft Research.</p>
</blockquote>
<p>Oh. Well, I guess "Here's Sebastian Burckhardt:" instead.</p>
<p>Point is, data races are undefined behavior in Rust, and they're also undefined
behavior in C++. When looking for a succinct citation for that fact, I came
across <a href="https://www.hboehm.info/c++mm/why_undef.html">this 2006 post by Hans Boehm</a>.
Its first line:</p>
<blockquote>
<p>Our proposed C++ memory model gives completely undefined semantics to programs
with data races.</p>
</blockquote>
<p>I believe this was the model that was accepted, so that's the best citation you're
gonna get from me. Okay, as soon as I read that I said "come on Steve, that's stupid,
make sure it's not just trivial to actually find in the standard" and lo and
behold
<a href="https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/n4849.pdf">n4849</a>
says:</p>
<blockquote>
<p>The execution of a program contains a data race if it contains two potentially
concurrent conflicting actions, at least one of which is not atomic, and
neither happens before the other, except for the special case for
signal handlers described below. Any such data race results in undefined
behavior.</p>
</blockquote>
<p>Okay, the real citation was easier. The article by Bohm is still fascinating.</p>
<p>But data races aren't undefined behavior in every programming language. For
example, let's consider both Java and Go. Now I hate that I even have to
preemptively say this, but I am <em>not</em> saying Java and Go are bad. I think
these are good decisions for both of the two languages. And I think their
decisions here are very informative. Language wars are stupid. Stop it.</p>
<p>Java was the first time I remember hearing, "yeah you can have data races but
like, it's fine" about. I kinda lodged that fact in the back of my brain, and
didn't think about it much. But lately, I was wondering. So I went and read
the Java specification, and more specifically, <a href="https://docs.oracle.com/javase/specs/jls/se21/html/jls-17.html">Chapter
17</a>. It contains
this text:</p>
<blockquote>
<p>The semantics of operations other than inter-thread actions, such as reads of
array lengths (§10.7), executions of checked casts (§5.5, §15.16), and
invocations of virtual methods (§15.12), are not directly affected by data
races.</p>
<p>Therefore, a data race cannot cause incorrect behavior such as returning the
wrong length for an array.</p>
</blockquote>
<p>This is morally equivalent to "not undefined behavior" even though the
specification doesn't define "undefined behavior" in the way that the C, C++,
and Rust specifications do. Notably, the specification also defines "data race"
in a slightly different way:</p>
<blockquote>
<p>When a program contains two conflicting accesses (§17.4.1) that are not
ordered by a happens-before relationship, it is said to contain a data race.</p>
</blockquote>
<p>Not just any ordering or synchronization primitive, a happens-before relation
specifically. Fun! We have three technically-slightly different but basically
morally equivalent ways of defining a data race. Let's do a fourth. Go also
allows data races. Sort of. From <a href="https://go.dev/ref/mem">the Go memory model</a>:</p>
<blockquote>
<p>A data race is defined as a write to a memory location happening concurrently
with another read or write to that same location, unless all the accesses
involved are atomic data accesses as provided by the sync/atomic package.</p>
</blockquote>
<p>Given that Rust provides atomic ordering operations in a submodule of its
standard library, Rust and Go agree here, though Go's more specific about naming
it.</p>
<p>But they also have this to say:</p>
<blockquote>
<p>While programmers should write Go programs without data races, there are
limitations to what a Go implementation can do in response to a data race. An
implementation may always react to a data race by reporting the race and
terminating the program. Otherwise, each read of a single-word-sized or
sub-word-sized memory location must observe a value actually written to that
location (perhaps by a concurrent executing goroutine) and not yet overwritten.</p>
</blockquote>
<p>So it's not right, but it's also not exactly UB. It goes on:</p>
<blockquote>
<p>These implementation constraints make Go more like Java or JavaScript, in that
most races have a limited number of outcomes, and less like C and C++, where the
meaning of any program with a race is entirely undefined, and the compiler may
do anything at all. Go's approach aims to make errant programs more reliable and
easier to debug, while still insisting that races are errors and that tools can
diagnose and report them.</p>
</blockquote>
<p>While a Go program may exhibit what a Rust or C++ program would consider
undefined behavior, and it does also consider it an error, the consequences are
very different. You don't get time travel. You get 998 instead of 1,000.</p>
<p>(After publication, Edoardo Spadolini reached out to me and we had a productive
conversation about the above paragraph: while my overall point here is still
valid, there are slightly more serious consequences to UB in a Go program than
slightly incorrect sums. For example, torn writes to a slice can lead to serious
consequences, and there are other ways to write code that exploits UB to be
actively malicious, such as a function with the signature
<code>func Cast[T any](addr uintptr) *T</code> that works. However, it's also worth mentioning
that Go ships with a race detector, available by simply passing <code>-race</code>. For some
interesting usage of this at scale, check out <a href="https://www.uber.com/blog/dynamic-data-race-detection-in-go-code/">this post by Uber</a>. At the end of the day, this is still
very different than the "your whole program is invalid" style of issues you
can end up with in C or C++, I am slightly underselling the danger above.)</p>
<p>Go also provides <a href="https://go.dev/ref/spec#Package_unsafe">an unsafe package</a>:</p>
<blockquote>
<p>The built-in package unsafe, known to the compiler and accessible through the
import path "unsafe", provides facilities for low-level programming including
operations that violate the type system. A package using unsafe must be vetted
manually for type safety and may not be portable.</p>
</blockquote>
<p>Oh yeah, I forgot, Java also has <code>sun.misc.unsafe</code>, essentially the same thing.
They have actually wanted to remove it for a long time, and some progress has
been made, but it's not quite gone yet. <a href="https://blogs.oracle.com/javamagazine/post/the-unsafe-class-unsafe-at-any-speed">The Unsafe Class: Unsafe at Any
Speed</a>
is a good explanation of the topic.</p>
<h2>ThreadId(5)</h2>
<p>One weird thing about suggesting that the borrow checker is only about memory
safety, and that memory safety means the absence of data races, is that memory
safety is more than just the absence of data races. Consider a problem that
requires no threads to become an issue: iterator invalidation. And different
languages have different answers to this issue:</p>
<ul>
<li>In C++, it is undefined behavior.</li>
<li>In Java, some data structures detect this and throw a <code>ConcurrentModificationException</code>.</li>
<li>In JavaScript, Ruby, and other languages, you often get "weird" behavior:</li>
</ul>
<pre><code>numbers = [1, 2, 3, 4]
numbers.each do |number|
p number
numbers.shift(1)
end
p numbers
</code></pre>
<p>prints</p>
<pre><code>1
3
[3, 4]
</code></pre>
<p>But you don't get full-blown time-travel undefined behavior. Sort of like this
code in Rust:</p>
<pre><code>fn main() {
let i = i32::MAX + 1;
println!("{i}");
}
</code></pre>
<p>Okay in this case because it's all at compile time Rust can detect it:</p>
<pre><code>error: this arithmetic operation will overflow
--&gt; src/main.rs:2:13
|
2 | let i = i32::MAX + 1;
| ^^^^^^^^^^^^ attempt to compute `i32::MAX + 1_i32`, which would overflow
|
= note: `#[deny(arithmetic_overflow)]` on by default
</code></pre>
<p>But in the general case, you won't get a compile error. You may get a panic, you
may get <code>-2147483648</code>. It's weird. But it's not full-blown time-travel undefined
behavior.</p>
<p>Anyway, back to iterator invalidation:</p>
<ul>
<li>In Rust, it is prevented at compile time.</li>
</ul>
<p>This is neat! This is cool! This is good! And it's pretty unique to Rust.</p>
<p>It is nice when our programs do what we expect, instead of what we don't expect.
It is even nicer when, if our programs don't do what we expect, we have some sort
of waypost, some sort of sign where the problem might be. What "nice" means may
be a bit different in different languages, but the boundary is the important
part: we're gonna have to potentially violate the rules somewhere, so at least
give us some help when that inevitably goes wrong. Over here, logic bugs may
happen, some weird behavior may result, but over there? There be dragons.
Nobody can save you out there. Tread carefully.</p>
<h2>ThreadId(6)</h2>
<p>If we think about all of these designs, they all are very similar conceptually:
safe code is the default, but you can call into some sort of unsafe facility.
And everyone is very clear on the relationship between the two: while the unsafe
facility exists to be used, it must uphold the rules that the safe world relies
on. And that means that safety and unsafety have a super/sub-set relationship.
In the core is unsafety. But at some point, we draw a line, and on top of that
line, safety exists.</p>
<p>In Rust, we often talk about how unsafe is great, because it clearly draws a
line around code with which serious bugs may happen. More specifically, while
most people describe it as "where memory safety can originate from" or similar,
it's actually slightly broader than that: it's where undefined behavior can
originate from.</p>
<p><a href="https://doc.rust-lang.org/reference/behavior-considered-undefined.html">The nomicon</a>:</p>
<blockquote>
<p>Rust code is incorrect if it exhibits any of the behaviors in the following
list. This includes code within unsafe blocks and unsafe functions. unsafe
only means that avoiding undefined behavior is on the programmer; it does not
change anything about the fact that Rust programs must never cause undefined
behavior.</p>
<p>It is the programmer's responsibility when writing unsafe code to ensure that
any safe code interacting with the unsafe code cannot trigger these behaviors.
unsafe code that satisfies this property for any safe client is called sound;
if unsafe code can be misused by safe code to exhibit undefined behavior, it
is unsound.</p>
</blockquote>
<p>This is not a new insight, but basically, unsafe is like FFI in a managed
language. It's not something you do often. But it is something you sometimes
have to do. And when you do, you can at least contain the danger: the problem
has to lie somewhere behind that veil. You have clear points to begin your
search into where the issues lie, should the abyss claim another soul.</p>
<p>In some ways I'm also repeating another older slogan Rust had: "memory safety
without garbage collection." But it's more like "no UB and no garbage collection."
Gosh, why didn't we put <em>that</em> on the website? Practically rolls right off the
tongue. What makes Rust appealing, and I think especially to the "non-systems"
crowd, is that it shares a property of many managed languages, that of "no
undefined behavior by default," with a property of many systems languages, that
of "no runtime, as little overhead as possible." And sure, there's an escape
hatch, but it's rarely used, and clearly separates the two worlds. This is a
fundamentally different way of viewing the world than "unsafe by default."</p>
<p>Another way in which there's an interesting split here in language design
is between the scripting languages like Ruby, Python, and JavaScript, and
languages like Rust, Go, and Java, is that the boundary in the former is purely
FFI, while in the latter, there are also unsafe faculties in the host language
itself, as well as with FFI. In all three cases, they're behind either a specific
package or <code>unsafe {}</code>, but they give you some interesting tools that don't
require you to use another language to use. I think this is an under-appreciated
part of the overall design space.</p>
<h2>threads.into_iter().for_each(|t| t.join().unwrap());</h2>
<p>So: did Rust focus too much on memory safety? And what should newer languages
do in this area?</p>
<p>Well, I think that in some sense, the answer is obviously "no, Rust did not
focus too much on memory safety." Rust has been wildly successful, reaching
heights that I only could dream about eleven years ago. Yes, usually I have
tried to be fairly tempered when talking about Rust's successes, and some
folks still try to claim Rust is still super niche. But when I think about the
ways that Rust touched even this blog post, it's staggering: I wrote this post
in VS: Code, which uses Rust, on Windows, which has Rust in the kernel. I
published it to GitHub, which uses Rust for code search. I previewed it in
Chrome, which uses some Rust today and may start having more tomorrow. It's
hosted on Vercel, which uses Rust in projects like turbopack, though maybe not
in the critical path for this post. When you read it, the bytes probably passed
through CloudFlare, which uses a ton of Rust, and maybe you're reading this in
Firefox, which is about 12% Rust. And maybe you're on Asahi Linux, where the
GPU drivers are written in Rust. Rust has Made It, and is here to stay. And
the marketing we used got us here. So in some sense, it would be silly to
declare said messaging a failure.</p>
<p>However, I also wonder what else could have been. Could Rust have grown faster
with a different message? Maybe. That said, what would that message be?
"Undefined behavior is scary?" I'm not sure that's as compelling, even if I think
it's actually a bigger deal. While memory unsafety is only part of undefined
behavior, it is the part that is easy to understand: pointers are the bane of
every C newbie. It has consequences that are easy to understand: security
vulnerabilities are bad. Has Rust <em>really</em> conflated "memory safety" and "safety"
that much? I don't actually think so. People still talk about SQL injection,
about Remote Code Execution, about Cross Site Scripting. People don't <em>really</em>
believe that just because it's written in Rust, bugs are impossible. Sure you
can find a few comments on the internet as such, but the big organizations that
are using Rust for meaningful work do not take this position. Given <a href="https://en.wikipedia.org/wiki/Amdahl%27s_law">Amdahl's
Law</a>, if 70% of security
vulnerabilities are related to memory safety, focusing so strongly on it makes
sense, as fixing that will give you a much larger impact than other things.</p>
<p>So what should new languages do here? Well, pretty much the only language that
is gaining developer mindshare that's new and is memory unsafe by default is
Zig. I think Andrew is very sharp, and a tremendous engineer. I am very
interested to see how things shake out there. But as a language, it does not
appeal to me personally, because I think we've sort of moved past "unsafe by
default." That being said, it's also not that Zig is exactly like other
languages in this space either: if we are willing to go beyond "memory unsafe vs
memory safe at compile time" as a strict binary, and instead look at "memory
unsafe at compile time" itself, we see a pretty big gradient. Zig is doing a
bunch of things to mitigate issues where possible. Maybe "safe enough" truly is
safe enough. Time will tell. But beyond that, you also don't have to go full
Rust in the design space. Rust wants to be a language that you can (and at
Oxide, we do) use at the lowest levels of the system. But also memory safe.
And extremely fast. These constraints mean that Rust takes on a lot of
complexity in the type system, where languages that are willing to relax these
things wouldn't have to. Here are three languages that I think are sort of
"post-Rust" in a sense, that are learning from Rust but putting their own
twist on things:</p>
<ul>
<li><a href="https://www.hylo-lang.org/">Hylo</a> (formerly Val) tries to, in their own words,
"strengthens [value semantics], while de-emphasizing reference semantics".
This lets them avoid a lot of Rust's complexity. Hylo looks more explicitly
to Swift than Rust for positive inspiration in many cases, but I'm including
it here anyway, as I've sort of ended up not talking about Swift in this post.</li>
<li><a href="https://austral-lang.org/">Austral</a> is "if affine types are good for systems
programming, linear types are even better." Austral also believes that parsimony
is a virtue in language design, and that linear types can replace a lot of
different features in other programming languages, while also eschewing other
features that they deem as not carrying their weight.</li>
<li><a href="https://www.modular.com/max/mojo">Mojo</a>, "the usability of Python with the
performance of C." It also has this sort of "lower into a more complex language"
idea; you can Just Write Python for the most part and it works, but if you want
to get fancier, you can. <a href="https://github.com/modularml/mojo/blob/main/proposals/lifetimes-and-provenance.md">Lifetimes are a current
proposal.</a></li>
</ul>
<p>I think there are some other languages that are forming the next big cohort of
programming languages, and they're interesting too, but these three I think have
the most explicit connection to Rust in the design and goals of the language
itself. Shout out to Nu, Inko, Gleam, Roc, Oil Shell, Unison, and others. Lots
of cool stuff going on.</p>
<h2>int atexit(void (*func)(void))</h2>
<p>There is one way in which the question "is the focus on memory unsafety
misguided?" is much more serious than battles in the comment sections of
various fora: the government.</p>
<p>We talked about this earlier, but various government organizations have been
slowly putting out recommendations that organizations should be moving towards
MSLs. And the first reactions to these were always "well they're not laws so
they don't have teeth" as well as "laws are inevitably coming." To be honest,
I have mostly been in the former camp, and viewed the later as fearmongering.
Sure, maybe someday, but like, dipping your toes in does not mean that you're
about to cannonball moments later.</p>
<p>But then I got <a href="https://news.ycombinator.com/item?id=38700267">this interesting reply from David Tolnay on Hacker
News</a>. In it, he refers me to
<a href="https://adspthepodcast.com/about/">ADSP: The Podcast</a>, and more specifically,
<a href="https://adspthepodcast.com/2023/12/15/Episode-160.html">Episode 160: Rust &amp; Safety at Adobe with Sean
Parent</a>. In it, Sean
makes reference to two pieces of legislation, <a href="https://www.congress.gov/bill/118th-congress/house-bill/2670">one in the
US</a>
and <a href="https://digital-strategy.ec.europa.eu/en/library/cyber-resilience-act">one in
the EU</a>.
Now, I am not a politican, and these bills are huge, so I wasn't able to figure
out how these bills do this specifically, but to quote Sean:</p>
<blockquote>
<p>The one in the U.S. that's pending basically says that the Department of
Defense is going to within 270 days of the bill passing (and it's a funding bill
which means it will probably pass late this year - early next year) that the
Department of Defense will establish guidelines around safety and security
including memory safety for software products purchased by Department of
Defense. The E.U. has a similar wording in a bill that's slowly winding its way
through their channels. I don't have insight into when that will pass. The U.S.
one will almost certainly pass here within a month or two.</p>
</blockquote>
<p>This sounds much more realistic, of course. Now, this does not mean that
C++ is outlawed or something silly like that, but it does mean that using Rust
could become a more serious competitive advantage when it comes to selling to
government: if you don't need an exception for your product, that's an advantage
over a product which does.</p>
<p>This all also may go the way of another anecdote Sean talks about in the past:
when a similar bill tried to mandate POSIX compliance. What I'm saying is, go
listen to the podcast. And while this still isn't something as wild as "not
using MSLs is illegal," it is really shocking to me how quickly things seem to
be moving in that vague direction, though of course it will never actually
arrive there. There seems to be at least some sort of thinking that's not
immediately lauged out of the room that memory unsafety could be a consumer
safety issue in a similar sense that other unsafe materials are a consumer
safety issue. Is that right? I'm not sure, but it seems to be where the
conversation is heading.</p>
<p>So where does that leave existing widely used memory-unsafe languages? Well,
the C++ community has... had a wide variety of discussions on this topic. I
should also say I mean "community" at large, and the committee feels like it
has a big divide as well. Now I should re-iterate that while I pay attention to
the C++ standardization process, I am very much an outsider: I have never
attended a meeting, I have no direct involvement in these things, I just read
a lot. And while historically most of the community wrote Rust off as a novelty,
I think that at this point, those with their head in the sand about Rust's usage
are mostly a fringe minority. But the big question that remains is, what to do?</p>
<p>Two approaches have appeared:</p>
<ul>
<li>Improve the safety of C++ in some fashion.</li>
<li>Create a "successor language" that will fix C++'s problems.</li>
</ul>
<p>What is interesting to me about these approaches is that they are both good,
for different reasons. The former is a sort of harm reduction approach: don't
let the perfect be the enemy of the good. <a href="https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p3023r1.html">What can we do today to improve the
lives of a ton of
people?</a>
But if you want to move into the realm of an MSL, you have to break backwards
compatibility. This is tantamount to just doing #2 in the first place, and so
some are cutting straight to that approach. But if we look at it in a slightly
different way, an interesting thing happens:</p>
<ul>
<li>breaking backwards compatibility vs not</li>
<li>memory safety by default vs not</li>
</ul>
<p>This leads to four approaches:</p>
<ul>
<li>We do not break backwards compatibility, and we ignore the memory safety
issue (this is the 'do nothing' option).</li>
<li>We do not break backwards compatibility, but we try to make things as memory
safe as possible given the constraints we have.</li>
<li>We break backwards compatibility, and we try to make things as memory safe as
possible but still remain unsafe.</li>
<li>We break backwards compatibility, and we become memory safe by default.</li>
</ul>
<p>The first option is seemingly untenable, but is also where I see C heading.
There's seemingly not nearly as much desire for movement in the C space as the
C++ space, even though I am following what JeanHeyd Meneide is doing and
appreciate it.</p>
<p>The second option is how I view at least one of the possible C++ successor
languages, <a href="https://www.circle-lang.org/">Circle</a>. Circle is incredibly
interesting and is worth its own post. But in short, it is a C++ compiler that
also implements a lot of extensions, both ones that are proposals in the
committee, but also ones that its author is interested in. He has been doing
lifetime experiments.</p>
<p>The third option is where the most famous successor languages live:
<a href="https://github.com/carbon-language/carbon-lang">Carbon</a> and
<a href="https://github.com/hsutter/cppfront">cpp2</a>. I haven't gotten to spend a ton of
time with either of these yet, but they explicitly aim to do to C++ what C++ did
to C, or what TypeScript did to JavaScript: create a new language for new code,
while allowing you to use the older language for older code. This allows them
to break backwards compatibility more easily: new code can more easily be
written under the new constraints. This gives them a larger degree of freedom
to make changes, which may be necessary to move the needle significantly on
the safety issue. But to my knowledge, they do not attempt to be memory safe
by default. They're conceptually similar to Zig in this way.</p>
<p>But what about a fourth option? What if someone did a TypeScript for C++, but
one that was closer to what Rust does? You might argue that this is basically
just Rust? (Interestingly, it is also sort of what Zig is with C: you can use
the same compiler to compile a mixed codebase, which is close enough in my
opinion.) What makes me worry about option #3 is that it doesn't pass the
seeming test that is coming: memory safe by default. I still think moving
to a #3 style solution is better than staying with #1, but is that enough?
I don't know, time will tell. I wonder to what degree you can get "C++ but memory
safe" without just being Rust. As we talked about earlier, most languages that
are trying to improve on Rust in this space are willing to trade off some core
constraints on Rust's design to get them, but C++ and Rust share the same
constraints. That said, it would take a tremendous amount of hubris to declare
that Rust is the final, ultimate, best language in the "memory safe without GC"
space. Rust has made a lot of mistakes too. Maybe reflecting on those will make
for a good twelfth year post. Someone get started on Rust++, please!</p>
<p>The reason that today is my Rust-a-versary is that I usually play with a new
language over Christmas break, and in 2012, that language was Rust. Maybe this
Christmas, I'll give cpp2 a try.</p>
<p>In the final minutes before publishing this, I also found another <a href="https://www.cisa.gov/sites/default/files/2023-12/CSAC_TAC_Recommendations-Memory-Safety_Final_20231205_508.pdf">interesting
report</a>.
This is a report to the CISA director by the Technical Advisory Council on
how to engage with the memory safety question. Rust has 17 mentions. I haven't
read it yet, but I figured I'd include it here.</p></content:encoded>
</item>
<item>
<title>Updating Buck</title>
<link>https://steveklabnik.com/writing/updating-buck/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/updating-buck/</guid>
<pubDate>Mon, 08 May 2023 00:00:00 GMT</pubDate>
<content:encoded><p>Hey there! A shorter post today, but I wanted to continue my series on Buck
by going over some things that have changed since this series started.</p>
<h2>A series</h2>
<p>This post is part of a series:</p>
<ul>
<li><a href="using-buck-to-build-rust-projects">Using buck to build Rust projects</a></li>
<li><a href="using-cratesio-with-buck">Using Crates.io with Buck</a></li>
<li><a href="#">Updating Buck</a> (you are here)</li>
</ul>
<h2>Updating Buck</h2>
<p>Lots of great stuff has been happening since the initial release of buck2, and
we'd like to take advantage of that. If we try and update things, though,
we'll get (well, you might not, but I did) an error:</p>
<pre><code>~&gt; cargo +nightly-2023-03-07 install --git https://github.com/facebook/buck2.git buck2
&lt;snipped&gt;
Replacing C:\Users\steve\.cargo\bin\buck2.exe
error: failed to move `C:\Users\steve\.cargo\bin\cargo-installYqDCSV\buck2.exe` to `C:\Users\steve\.cargo\bin\buck2.exe`
Caused by:
Access is denied. (os error 5)
</code></pre>
<p>(Why yes, my console prompt has changed...)</p>
<p>See on Windows, we can't replace a program that's running, while it's running.
On other systems, this may work for you. However, it's probably a good idea to
not take advantage of this particular feature, because of the underlying
cause: buck runs a daemon in the background. This daemon is still going. Trying
to replace it while it's running means you'll still have the old one going
around in the back, and while that's fine (is it? I don't actually know), best
to cleanly shut down first. So do this:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; buck2 killall
Killed buck2.exe (4788). C:\Users\steve\.cargo\bin\buck2.exe --isolation-dir v2 daemon --dont-daemonize
Killed buck2.exe (40200). C:\Users\steve\.cargo\bin\buck2.exe --isolation-dir v2 daemon --dont-daemonize
</code></pre>
<p><code>buck2 killall</code> will kill every instance of the daemon. As you can see, it found
two of mine, and shut them down. As you can see, you want to run this command
from within one of your projects.</p>
<p>And now an upgrade works.</p>
<h2>Update Reindeer</h2>
<p>If you're building Rust code and using Reindeer, like we talked about in the
last post, go ahead and grab that too. Lots of good stuff in there:</p>
<pre><code>~&gt; cargo install --git https://github.com/facebookincubator/reindeer/ reindeer -f
</code></pre>
<p>Now that we're all updated, let's fix up our project to make use of the latest
changes that are relevant.</p>
<h2>Update the prelude</h2>
<p>I no longer need my weird toolchain hacks to get MSVC Rust working. Thank you
dtolnay! 🙏</p>
<p>Let's pull in changes to the prelude:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; cd prelude
~\Documents\GitHub\buck-rust-hello\prelude&gt; git fetch origin
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 17 (delta 12), reused 15 (delta 10), pack-reused 0
Unpacking objects: 100% (17/17), 3.60 KiB | 102.00 KiB/s, done.
From https://github.com/facebook/buck2-prelude
920d3f2..370cd4d main -&gt; origin/main
~\Documents\GitHub\buck-rust-hello\prelude&gt; git reset --hard origin/main
HEAD is now at 370cd4d Http_archive execution platform hack
~\Documents\GitHub\buck-rust-hello\prelude&gt; cd ..
~\Documents\GitHub\buck-rust-hello&gt;
</code></pre>
<p>And remove the hack (if you had to do it too) in <code>toolchains\BUCK</code>:</p>
<pre><code>system_cxx_toolchain(
name = "cxx",
visibility = ["PUBLIC"],
)
</code></pre>
<p>No more <code>link.exe</code> stuff!</p>
<h2>Fix some configuration</h2>
<p>Some good changes have landed in the <code>buck2 init</code> subcommand that we'll want
to add ourselves.</p>
<p>First up, we should <a href="https://github.com/facebook/buck2/commit/54e1eb5e29eca8a021579b859d0a34c73754eb2a">add an empty file named <code>.buckroot</code> to the root of our
repository</a>.
Because buck works with your filesystem hierarchy, it can and will traverse
upwards looking for things at times. Adding this file ensures that if it does
so, it will stop before it starts trying to traverse even higher. There's no
need for anything in the file, as the contents are ignored.</p>
<p>I think this kind of change is an interesting way to look at the usability of
various systems. Adding an empty file here is <em>sort of</em> "more complex" than
say, Cargo. But it's also more explicit. Which means it can be more... I want
to say "legible", if you also read insufferable books like I do sometimes. And
therefore, easier. Anyway, more thoughts on all of this in the future.</p>
<p>Next, we have a change that is demonstrated by this example. Can you guess
what it is?</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; buck2 run //src/bin:hello_world
File changed: root//.git/modules/prelude
Build ID: 1a24b418-acdb-4d23-a5e1-6e9b644c01e6
Jobs completed: 83. Time elapsed: 1.9s. Cache hits: 0%. Commands: 3 (cached: 0, remote: 0, local: 3)
</code></pre>
<p>There's no reason for buck to be watching our <code>.git</code> directory for changes. And
that's why <a href="https://github.com/facebook/buck2/commit/2083261102e867bce2b6385b78b598e758b424a8">it is now put in the default
configuration</a>
when you <code>buck2 init --git</code>. But we've gotta make that change ourselves. Open
up your <code>.buckconfig</code> and add this at the bottom:</p>
<pre><code>[project]
ignore = .git
</code></pre>
<p>We want it to ignore the <code>.git</code> directory. Seems good.</p>
<p>... I lied though, there's one more improvement we want to make: we also don't
want buck to bother listening to the <code>target</code> directory either, as those files
are for Cargo's output. So what we <em>actually</em> want is this:</p>
<pre><code>[project]
ignore = .git, target
</code></pre>
<p>After doing that we'll want to <code>buck2 kill</code> to shut the daemon down, so that
it can pick up our new configuration on the next boot.</p>
<h2>Regenerating reindeer</h2>
<p>Since we've got new bugfixes in Reindeer too, let's regenerate our config
for our dependencies:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; reindeer --third-party-dir third-party buckify
[WARN reindeer::fixups] semver-1.0.17 has a build script, but I don't know what to do with it: Unresolved build script at ..\..\..\..\.cargo\registry\src\github.com-1ecc6299db9ec823\semver-1.0.17\build.rs. Dependencies:
</code></pre>
<p>We still have to deal with the build script! We didn't talk about the
contents of <code>third-party\BUCK</code> last time, and we won't this time either. If you
want to see what's changed, you can take a peek though. One change that we
didn't explicitly talk about before, but you may end up noticing, is that it did
not generate a target to try and build our build script.</p>
<p>Let's try it out now:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; buck2 run //src/bin:hello_world
File changed: root//BUCK
File changed: root//third-party/BUCK
File changed: root//third-party
Build ID: 2eb0c121-d1fb-43d2-b8a4-f923d8dda657
Jobs completed: 20. Time elapsed: 1.1s. Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2)
</code></pre>
<p>Nice. Also note that we aren't seeing any more <code>.git</code> or <code>target</code> changes, not
that we've run anything that would inherently change those files, but go ahead,
invoke Cargo or git, and then build again. You shouldn't see notifications about
those directories anymore.</p>
<h2>Fixing some breakage</h2>
<p>Speaking of invoking Cargo, remember how I said this in the last post?</p>
<blockquote>
<p>We do have two different Cargo.tomls now. That is a bit of a bummer. But at
least it is easy to determine if there’s a problem: dependency failures are
loud, and if you’re building with both in CI, you’ll notice if stuff goes wrong.
There also may be a solution to this I’m just not aware of.</p>
</blockquote>
<p>Well...</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; cargo run
Compiling hello_world v0.1.0 (C:\Users\steve\Documents\GitHub\buck-rust-hello)
error[E0432]: unresolved import `semver`
--&gt; src\bin\main.rs:1:5
|
1 | use semver::{BuildMetadata, Prerelease, Version, VersionReq};
| ^^^^^^ use of undeclared crate or module `semver`
For more information about this error, try `rustc --explain E0432`.
error: could not compile `hello_world` due to previous error
</code></pre>
<p>Going back and re-reading my last post, I did have a <code>cargo add semver</code> in
there, so maybe I just forgot to commit that in my last post. Just in case,
we'll fix that:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; cargo add semver
Updating crates.io index
Adding semver v1.0.17 to dependencies.
Features:
+ std
- serde
</code></pre>
<p>With that, <code>cargo build</code> and <code>cargo run</code> are back in business.</p>
<p>We also have... well "breakage" isn't exactly right, but we have a buck
configuration issue. Let's try to build every target:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; buck2 build ...
Error running analysis for `root//:build (prelude//platforms:default#fb50fd37ce946800)`
Caused by:
0: Error looking up configured node root//:build (prelude//platforms:default#fb50fd37ce946800)
1: `root//src/bin:hello_world` is not visible to `root//:build` (run `buck2 uquery --output-attribute visibility root//src/bin:hello_world` to check the visibility)
Build ID: 51657a07-112c-46b4-a2eb-91d60a4b0aed
Jobs completed: 8. Time elapsed: 0.0s.
BUILD FAILED
</code></pre>
<p>We didn't declare that our binary was visible anywhere, and so when we try and
build it, it isn't happy. We do want this to be public, so change <code>src\bin\BUCK</code>
by adding this <code>visibility</code> line near the end. It should look like this:</p>
<pre><code>rust_binary(
name = "hello_world",
srcs = ["main.rs"],
crate_root = "main.rs",
deps = [
"//third-party:semver",
],
visibility = ["PUBLIC"],
)
</code></pre>
<p>And now that will work:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; buck2 build ...
File changed: root//src/bin/BUCK
Build ID: 9da23da4-3f99-4e78-8c58-ae5e2f1facaa
Jobs completed: 25. Time elapsed: 0.6s. Cache hits: 0%. Commands: 3 (cached: 0, remote: 0, local: 3)
BUILD SUCCEEDED
</code></pre>
<p>Okay, now that we've fixed both Cargo <em>and</em> buck... let's make sure that isn't
gonna happen again. We aren't testing any of this. So it broke. Just like I said
it was easy to not let break. Sigh.</p>
<p>We're going to use GitHub Actions because this is already on GitHub. I'm sure
you can adapt it to your setup of choice.</p>
<p>Put this in <code>.github\workflows\ci.yml</code>:</p>
<pre><code>name: CI
on:
pull_request:
push:
branches: [main]
jobs:
test:
name: ${{matrix.name || format('Rust {0}', matrix.rust)}}
runs-on: ${{matrix.os || 'ubuntu'}}-latest
strategy:
fail-fast: false
matrix:
include:
- rust: nightly
- rust: beta
- rust: stable
- rust: 1.69.0
- name: Cargo on macOS
rust: nightly
os: macos
- name: Cargo on Windows (msvc)
rust: nightly-x86_64-pc-windows-msvc
os: windows
steps:
- uses: actions/checkout@v3
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{matrix.rust}}
- run: cargo check
- run: cargo run
buck:
name: Buck2 on ${{matrix.os == 'ubuntu' &amp;&amp; 'Linux' || matrix.os == 'macos' &amp;&amp; 'macOS' || matrix.os == 'windows' &amp;&amp; 'Windows' || '???'}}
runs-on: ${{matrix.os}}-latest
if: github.event_name != 'pull_request'
strategy:
matrix:
os: [ubuntu, macos, windows]
steps:
- uses: actions/checkout@v3
with:
submodules: true
- uses: dtolnay/rust-toolchain@stable
with:
components: rust-src
- uses: dtolnay/install-buck2@latest
- run: buck2 build ...
- run: buck2 run //src/bin:hello_world
- uses: actions/cache/restore@v3
id: cache
with:
path: ~/.cargo/bin/reindeer${{matrix.os == 'windows' &amp;&amp; '.exe' || ''}}
key: ${{matrix.os}}-reindeer
- run: cargo install --git https://github.com/facebookincubator/reindeer reindeer
if: steps.cache.outputs.cache-hit != 'true'
- uses: actions/cache/save@v3
if: steps.cache.outputs.cache-hit != 'true'
with:
path: ~/.cargo/bin/reindeer${{matrix.os == 'windows' &amp;&amp; '.exe' || ''}}
key: ${{steps.cache.outputs.cache-primary-key}}
- run: reindeer buckify
working-directory: third-party
if: matrix.os == 'ubuntu'
- name: Check reindeer-generated BUCK file up to date
run: git diff --exit-code
if: matrix.os == 'ubuntu'
</code></pre>
<p>This is... you guessed it, based off of <a href="https://github.com/dtolnay/cxx/blob/master/.github/workflows/ci.yml">dtolnay's CI for
cxx</a>.
What can I say, he writes good code. This version is a bit stripped down, since
this is primarily a project for showing off a build system, rather than a
regular project. This has:</p>
<ul>
<li>Builds with Cargo on Linux against nightly, beta, stable, and 1.69.0 as a sort of MSRV check.</li>
<li>Builds with Cargo on MacOS and Windows against nightly Rust.</li>
<li>Buck on Linux, MacOS and Windows against stable Rust.</li>
<li>On Linux, checks that we have comitted any relevant changes to dependencies.</li>
</ul>
<p>This does not have:</p>
<ul>
<li>A "fail if there are any warnings" build</li>
<li>A clippy build</li>
<li>A full matrix of every os/language/build system combination</li>
</ul>
<p>or other things you may want out of a build. Again, you'll probably want to
customize this heavily, but this is what I'm going to do here.</p>
<h2>Wrapping up</h2>
<p>And with that, we are done! Next post we're going to deal with that build
script. And use some crates that have more intense dependencies, and get them
all working.</p>
<p>As always, you can <a href="https://github.com/steveklabnik/buck-rust-hello/commit/914e3aa7cdc269b2536f8118255eae12bf552f8d">check out the code</a> at this point if you'd like.</p></content:encoded>
</item>
<item>
<title>Using Crates.io with Buck</title>
<link>https://steveklabnik.com/writing/using-cratesio-with-buck/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/using-cratesio-with-buck/</guid>
<pubDate>Thu, 27 Apr 2023 00:00:00 GMT</pubDate>
<content:encoded><p>In <a href="using-buck-to-build-rust-projects">a previous post</a>, I laid out the basics
of building a Rust project with <a href="https://buck2.build/">buck2</a>. We compared and
contrasted it with Cargo. But what about one of the biggest and best features
that Cargo has to offer, the ability to use other Rust packages from crates.io?
They don't use buck, so how can we integrate them into our build?</p>
<h2>A series</h2>
<p>This post is part of a series:</p>
<ul>
<li><a href="using-buck-to-build-rust-projects">Using buck to build Rust projects</a></li>
<li><a href="#">Using Crates.io with Buck</a> (you are here)</li>
<li><a href="updating-buck">Updating Buck</a></li>
</ul>
<p>This post represents how to do this at the time that this was posted; future
posts may update or change something that happens here. Here's a hopefully
complete but possibly incomplete list of updates and the posts that talk
about it:</p>
<ul>
<li><code>build-script-build</code> target is no longer generated, see "Updating Buck"</li>
</ul>
<h2>Depending on semver</h2>
<p>Let's use the <code>semver</code> package example as our program. I am choosing this for
a few reasons:</p>
<ul>
<li>I used to be involved in maintaining it, and I'm sentimental</li>
<li>It's authored by dtolnay, who authors many great Rust crates.</li>
<li>More specifically than that, his excellent <a href="https://github.com/dtolnay/cxx/">cxx library</a> maintains build rules for Cargo, Buck, and Bazel, and examining how he uses Buck in cxx helped me write both the last post and this one. I wanted to make sure to shout that out.</li>
<li>It's pure Rust (this is easier than something that depends on C, since as I mentioned I'm still having an issue or two with getting my own C toolchain working so far).</li>
<li>It has one optional dependecy on serde, but no others. Again, this is just easier, not a fundamental limitation.</li>
</ul>
<p>Here's the example, from <a href="https://crates.io/crates/semver">the page on crates.io</a>:</p>
<pre><code>use semver::{BuildMetadata, Prerelease, Version, VersionReq};
fn main() {
let req = VersionReq::parse("&gt;=1.2.3, &lt;1.8.0").unwrap();
// Check whether this requirement matches version 1.2.3-alpha.1 (no)
let version = Version {
major: 1,
minor: 2,
patch: 3,
pre: Prerelease::new("alpha.1").unwrap(),
build: BuildMetadata::EMPTY,
};
assert!(!req.matches(&amp;version));
// Check whether it matches 1.3.0 (yes it does)
let version = Version::parse("1.3.0").unwrap();
assert!(req.matches(&amp;version));
}
</code></pre>
<p>Let's change <code>src/bin/main.rs</code> to contain this code. What do we need to do
to add this with Cargo?</p>
<pre><code>buck-rust-hello〉cargo add semver
Adding semver v1.0.17 to dependencies.
Features:
+ std
- serde
buck-rust-hello〉cargo run
Compiling semver v1.0.17
Compiling hello_world v0.1.0 (C:\Users\steve\Documents\GitHub\buck-rust-hello)
Finished dev [unoptimized + debuginfo] target(s) in 0.71s
Running `target\debug\main.exe`
buck-rust-hello〉
</code></pre>
<p>Easy enough! We expect no output, becuase the asserts should pass. (Note that
we didn't ask for the <code>serde</code> feature, so we aren't using it, and therefore
don't depend on anything other than <code>semver</code>.)</p>
<h2>How Cargo handles this</h2>
<p>Before we move on, let's talk for a moment about what Cargo actually does here.</p>
<p>First, <code>cargo add</code> has added this to our <code>Cargo.toml</code>:</p>
<pre><code>[dependencies]
semver = "1.0.17"
</code></pre>
<p>This is the latest release of semver at the time of this writing.</p>
<p>When we <code>cargo build</code>, Cargo will figure out all of the crates we need. It will
then check the cache for the source code for that crate, and if it's not there,
download it. On my system, this lives here:</p>
<pre><code> ~\.cargo\registry\src\github.com-1ecc6299db9ec823\semver-1.0.17\
</code></pre>
<p>This means we have a <em>global</em> cache of source code.</p>
<p>Cargo will then compile <code>semver</code>, as you can see in the output above. It places
that output in the <code>target</code> directory, more specifically</p>
<pre><code>.\target\debug\deps
</code></pre>
<p>This directory will contain <code>libsemver-ded1559592aad8f7.rlib</code>,
<code>libsemver-ded1559592aad8f7.rmeta</code>, and <code>semver-ded1559592aad8f7.d</code>. These have
hashes embedded in them so that if we had multiple versions of <code>semver</code> in our
project, they can be disambiguated. If you're not familiar with <code>rustc</code> output:</p>
<ul>
<li>rlib files are simlar to archive files, like Unix <code>ar</code>. The details are not standard, and may change at any time. Object code lives in here, as well as other things.</li>
<li>rmeta files contain crate metadata. This can be used for <code>cargo check</code>, for example.</li>
<li><code>.d</code> files are a dependency file. This format comes from gcc/make, and is sorta standardized. This is in theory useful for use with other build systems, but we won't be using this today.</li>
</ul>
<p>Cargo will then build our project, passing in <code>libsemver-*.rlib</code> as a
dependency.</p>
<p>If you're curious about the exact commands and flags, <code>cargo build -v</code> will
show that. Make sure to <code>cargo clean</code> first, or else you'll get no output, given
that we've already built this project.</p>
<p>For example, here's the <code>rustc</code> invocation that Cargo makes for building this step:</p>
<pre><code>rustc --crate-name main --edition=2021 src\bin\main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=c398fba113f38e08 --out-dir C:\Users\steve\Documents\GitHub\buck-rust-hello\target\debug\deps -C incremental=C:\Users\steve\Documents\GitHub\buck-rust-hello\target\debug\incremental -L dependency=C:\Users\steve\Documents\GitHub\buck-rust-hello\target\debug\deps --extern hello_world=C:\Users\steve\Documents\GitHub\buck-rust-hello\target\debug\deps\libhello_world-c5b898f1ca338cca.rlib --extern semver=C:\Users\steve\Documents\GitHub\buck-rust-hello\target\debug\deps\libsemver-ded1559592aad8f7.rlib
</code></pre>
<p>Since we are not using Cargo, we need to replace all of that stuff, and get
Buck to generate that <code>rustc</code> line, or at least, some equivalent of it.</p>
<p>Let's talk about the various things we need to do:</p>
<h3>The mismatch</h3>
<p>Let's start with one thing that is basically the same: <code>buck-out</code> and <code>target</code>
are both directories in our project that cache the output of our build. Yeah
the name and details are different, but we're not going to try and somehow
unify these, as they're both considered implementaiton details of the
respective systems, and trying to get them to share is a lot of work for not
much gain.</p>
<p>Buck does not have a central registry of packages that we can download code
from.</p>
<p>Buck is interested in reproducable builds, and therefore, a global cache of
source code doesn't make as much sense. You want the code stored locally, with
your project. The dreaded (or beloved) vendoring.</p>
<p>Buck does not understand the crate index, Cargo configuration for a given
package, and other implementation details. As a more general build system,
those are pretty much out of scope.</p>
<p>Lucikly, other people have done a lot of work here.</p>
<h3>Reindeer</h3>
<p>Enter <a href="https://github.com/facebookincubator/reindeer/">Reindeer</a>. Reindeer is a
project that will help us bridge this gap. Here's how this will work: Reindeer
will create and (mostly) manage a <code>third-party</code> directory for us. It will
generate <code>BUCK</code> files that we can then use to depend on these external crates.
We can even choose to vendor our sources or not.</p>
<p>You can install reindeer through Cargo:</p>
<pre><code>&gt; cargo install --git https://github.com/facebookincubator/reindeer/ reindeer
</code></pre>
<p>Let's set this up. First off, we need to create some files:</p>
<pre><code>&gt; mkdir third-party
&gt; code third-party\Cargo.toml # or vim, or whatever
</code></pre>
<p>In that <code>Cargo.toml</code>, we'll need to put this:</p>
<pre><code>[workspace]
[package]
name = "rust-third-party"
version = "0.0.0"
publish = false
edition = "2021"
# Dummy target to keep Cargo happy
[[bin]]
name = "fake"
path = "/dev/null"
[dependencies]
semver = "1.0.17"
</code></pre>
<p>We're creating a fake package here, so nothing truly matters except the
<code>[dependencies]</code> section. Here we depend on <code>semver</code> as usual.</p>
<p>We're also going to configure Reindeer to not vendor our source code, because
that's how I prefer to do things. Vendoring was the default behavior, with
non-vendoring being added very recently, so if you prefer to vendor, that
workflow works very well</p>
<p>Anyway put this in <code>third-party/reindeer.toml</code>:</p>
<pre><code>vendor = false
</code></pre>
<p>So run this inside of your <code>third-party</code> directory:</p>
<pre><code>〉cd third-party
〉reindeer buckify
[WARN reindeer::fixups] semver-1.0.17 has a build script, but I don't know what to do with it: Unresolved build script at vendor\semver\build.rs. Dependencies:
</code></pre>
<p>Oh no, build scripts! Yeah I said that I picked <code>semver</code> because it should be easy,
but it does have a build script, which is another Cargo-specific feature. Now,
the <code>semver</code> crate's build script is used as a feature to support older versions
of the compiler; all it does is detect old versions and then spit out some
configuration to make sure to not use the newer features of the language. This
is why this is a warning, not an error; in this case, we don't actually need
the build script since we are using a new compiler. So, we are going to skip this
<em>for now</em>, but we'll come back and fix it.</p>
<p>At this point, <code>reindeer</code> has generated a <code>BUCK</code> file for <code>semver</code>. Let's see
what targets we have now:</p>
<pre><code>〉buck2 targets //...
Jobs completed: 4. Time elapsed: 0.0s.
root//:build
root//src/bin:hello_world
root//src/lib:hello_world
root//third-party:semver
root//third-party:semver-1.0.17
root//third-party:semver-1.0.17-build-script-build
</code></pre>
<p>We have a few new ones! One for <code>semver</code>, one for the specific version of
<code>semver</code>, and one for the build script of <code>semver</code>. The general <code>semver</code> is
an alias for <code>semver-1.0.17</code>.</p>
<p>Do you know how to modify our build so that buck builds successfully?</p>
<p>Here's the answer: change <code>src\bin\BUCK</code>:</p>
<pre><code>rust_binary(
name = "hello_world",
srcs = ["main.rs"],
crate_root = "main.rs",
deps = [
"//third-party:semver",
],
)
</code></pre>
<p>And now we can build and run:</p>
<pre><code>〉buck2 run //src/bin:hello_world
File changed: root//src/bin/BUCK
Build ID: b18ba58d-8a77-439a-9d95-6051f3cf21d4
Jobs completed: 26. Time elapsed: 1.9s. Cache hits: 0%. Commands: 3 (cached: 0, remote: 0, local: 3)
</code></pre>
<p>Success! Our program has no output, if the assertions failed we would have
gotten something, but this is expected given the example code.</p>
<p>Now, whenever we need to add or remove a dependency, we can modify
<code>third-party\Cargo.toml</code>, re-run the <code>buckify</code> command, and we're good.</p>
<p>We <em>do</em> have two different <code>Cargo.toml</code>s now. That is a bit of a bummer. But
at least it is easy to determine if there's a problem: dependency failures are
loud, and if you're building with both in CI, you'll notice if stuff goes wrong.
There also may be a solution to this I'm just not aware of.</p>
<p>If you'd like to see the whole thing at this point, <a href="https://github.com/steveklabnik/buck-rust-hello/commit/2abd1ada7dbbc7f89cd8678eace1e07b3df2ae2f">this
commit</a>
should have you covered.</p>
<p>This should get you going with building more advanced projects in Rust using
buck2. In the next post, we'll talk about fixups, which you'll need for
dependencies that are more complex than <code>semver</code>.</p>
<p>By the way, if you're interested in this stuff, I've <a href="https://discord.gg/ZTEmwypZ6K">made a Discord for buck
fans</a>. Come hang out, chat about stuff, ask
questions, it's gonna be a good time.</p></content:encoded>
</item>
<item>
<title>Using buck to build Rust projects</title>
<link>https://steveklabnik.com/writing/using-buck-to-build-rust-projects/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/using-buck-to-build-rust-projects/</guid>
<pubDate>Thu, 13 Apr 2023 00:00:00 GMT</pubDate>
<content:encoded><p>A few days ago, Facebook/Meta/idk <a href="https://engineering.fb.com/2023/04/06/open-source/buck2-open-source-large-scale-build-system/">announced that buck2 is now open
source</a>.</p>
<blockquote>
<p>Buck2 is an extensible and performant build system written in Rust and
designed to make your build experience faster and more efficient.</p>
</blockquote>
<p>As it just so happens, I have taken an increasing interest in build systems
lately. I have mostly ignored the cool kids' build system things, because
I have never worked at a FAANG, or at a true monorepo shop. I also personally
try and avoid the JVM wherever possible, and the first generation of these tools
were all built on top of it. (Yes, that bias may be outdated, I'm not saying
<em>you</em> should avoid the JVM, just stating my own bias up front.)</p>
<p>So this timing was perfect! Let's explore what using buck looks like.</p>
<blockquote>
<p>A brief aside: listen, I have no love for Facebook. I <a href="im-deleting-my-facebook-tonight">deleted my
account</a> almost a dozen years ago. That
doesn't mean I'm not interested in using good tools they produce. If you
feel differently, fine, but that's not what I want to talk about today, so
I'm not going to.</p>
</blockquote>
<p>Oh, one last bit before I begin: I'm not going to do a lot of motivating on
"why would you want to use Buck?" in this post. There's a few reasons for that,
but for now, if that's what you're looking for, this post isn't it. We're doing
this purely for the novelty of trying out some new tech right now. I will
probably end up with a post giving better motivations at some point in the
future, but I think it makes more sense once you see how it works, rather than
starting there.</p>
<h2>A series</h2>
<p>This post is part of a series:</p>
<ul>
<li><a href="#">Using buck to build Rust projects</a> you are here</li>
<li><a href="using-cratesio-with-buck">Using Crates.io with Buck</a></li>
<li><a href="updating-buck">Updating Buck</a></li>
</ul>
<p>This post represents how to do this at the time that this was posted; future
posts may update or change something that happens here. Here's a hopefully
complete but possibly incomplete list of updates and the posts that talk
about it:</p>
<ul>
<li><code>buck2 init</code> also creates a file named <code>.buckroot</code>, see "Updating Buck"</li>
</ul>
<h2>Getting started with buck2</h2>
<p><a href="https://buck2.build/docs/getting_started/">The Getting Started page</a> will give
you instructions on installing buck. As of this moment, the instructions are:</p>
<pre><code>$ rustup install nightly-2023-01-24
$ cargo +nightly-2023-01-24 install --git https://github.com/facebook/buck2.git buck2
</code></pre>
<p>This is mega-convenient for me as a Rust user, but probably not if you don't
have Rust installed. That said, this is a first release, and so I don't expect
anything fancier. This is what <code>cargo install</code> is good for!</p>
<p>Let's make a new directory, <code>hello</code>:</p>
<pre><code>$ mkdir buck-rust-hello
$ cd buck-rust-hello
</code></pre>
<p>To initialize a project, we use this command:</p>
<pre><code>$ buck2 init --git
</code></pre>
<p>Before we move forward, let's examine what this generated for us.</p>
<h2>Initial project files</h2>
<p>We now have this stuff in our directory:</p>
<pre><code>$ git add .
$ git status
On branch main
No commits yet
Changes to be committed:
(use "git rm --cached &lt;file&gt;..." to unstage)
new file: .buckconfig
new file: .gitignore
new file: .gitmodules
new file: BUCK
new file: prelude
new file: toolchains/BUCK
</code></pre>
<p>Let's talk about each of these in turn.</p>
<h3><code>.buckconfig</code></h3>
<p>The <code>.buckconfig</code> file is... a configuration file for Buck, go figure. It looks
like this:</p>
<pre><code>[repositories]
root = .
prelude = prelude
toolchains = toolchains
none = none
[repository_aliases]
config = prelude
fbcode = none
fbsource = none
buck = none
[parser]
target_platform_detector_spec = target:root//...-&gt;prelude//platforms:default
</code></pre>
<p>That <code>none = none</code> is kind of amusing. Regardless of that, this file is
extremely important: it configures the entire thing. In a sense, it's like
<code>Cargo.toml</code>: just like a package is defined by the existence of a <code>Cargo.toml</code>,
a <code>.buckconfig</code> defines the existence of a... cell. Which defines a package.
We'll get there. Point is, this is the top level configuration. We say that
the repository root is in the current directory, we'd like to use the default
prelude and toolchains.</p>
<p>I uh... I don't know what the <code>none = none</code> is for. It might just be a bug.
I haven't seen it in some of the other config files I've poked at. Let's just
leave that alone for now. I do have a suspicion though... and it involves the
next section.</p>
<p>We also have a table for repository aliases. I couldn't find any documentation
on this, but I would imagine this means we could use the name <code>config</code> instead
of <code>prelude</code> later. Looks like we don't have any way to refer to <code>fbcode</code> and
<code>fbsource</code>, which makes sense, and same with <code>buck</code>.</p>
<p>(I wonder if this is what the <code>none = none</code> is about above, defining a sort of
"none repository" that we can then alias these to.)</p>
<p>Finally, we have a parser table, with one entry, pointing out where a thing
exists. I know this configures Buck's parser, but other than that... I'm sure
I'll figure it out eventually.</p>
<h3><code>.gitmodules</code> &amp; <code>prelude</code></h3>
<p>We have a git submodule, pointing to
<code>https://github.com/facebook/buck2-prelude.git</code>,
that lives at the <code>prelude</code> directory. If you poke around in there, you'll
find a bunch of <code>.bzl</code> files that implement useful features for us to use.
We'll get into those in a moment, but the point is that this is sort of like
a 'standard library' if you will. You could also not use it and define your own.
If you're that kind of person.</p>
<h3><code>.gitignore</code></h3>
<p>A very simple <code>.gitignore</code> will be created, that contains one line: <code>/buck-out</code>.
This is where buck stores artifacts produced by your builds, so we don't want
that checked into version control.</p>
<h3><code>BUCK</code></h3>
<p>Now we get to the good stuff. Here's the generated <code>BUCK</code> file:</p>
<pre><code># A list of available rules and their signatures can be found here: https://buck2.build/docs/generated/starlark/prelude/prelude.bzl
genrule(
name = "hello_world",
out = "out.txt",
cmd = "echo BUILT BY BUCK2&gt; $OUT",
)
</code></pre>
<p><code>genrule</code> is like a function, provided by our prelude. If you're curious, the
implementation is in <code>prelude/genrule.bzl</code>. This command, as you may imagine,
sets up a rule, named <code>hello_world</code>, that produces a file called <code>out.txt</code>.
It does this by running the <code>cmd</code>. Nice and straightforward. We'll give that
a try in a moment, but first, one last file:</p>
<h3><code>toolchains/BUCK</code></h3>
<p>This file describes a toolchain. Here's the contents:</p>
<pre><code>load("@prelude//toolchains:genrule.bzl", "system_genrule_toolchain")
system_genrule_toolchain(
name = "genrule",
visibility = ["PUBLIC"],
)
</code></pre>
<p>This loads a certain rule from the prelude, and then defines this as a public
toolchain. We can define as many toolchains as we want here, for example, if
we wanted to build both Rust and Python, we could define both toolchains here
for later use.</p>
<p>The "genrule" toolchain is used to generate files from a shell command, as we
saw before with our rule that produces <code>out.txt</code>. So, in my understanding, here
we are defining that we wish to actually use that. And then, in the <code>BUCK</code> file,
we're using this toolchain to implement our rule.</p>
<h2>Invoking our first build</h2>
<p>Okay, let's actually give this a shot. To instruct Buck to build something, we
invoke it with the "target pattern" as an argument. Let's ask Buck what targets
it knows how to build. To do this:</p>
<pre><code>C:\Users\steve\Documents\GitHub\buck-rust-hello〉buck2 targets //... 04/10/2023 02:01:50 PM
Build ID: cd778a29-2ba4-484b-8956-dc67f6fc0625
Jobs completed: 4. Time elapsed: 0.0s.
root//:hello_world
</code></pre>
<p>The <code>//...</code> is a "target pattern." The <code>/...</code> means "all build targets in build
files in subdirectories", and <code>/</code> means our root directory, so <code>//...</code> means
"all targets in all build files in all subdirectories." By passing this target
to <code>buck2 targets</code>, we can see every target in the project. This shows our
one target we've defined, <code>root://:hello_world</code>. This name was defined in our
<code>BUCK</code> file above. If you change that to</p>
<pre><code>genrule(
name = "lol",
</code></pre>
<p>then <code>buck2 targets //...</code> would show <code>root://:lol</code>.</p>
<p>Let's actually build our target:</p>
<pre><code>〉buck2 build //:hello_world
File changed: root//BUCK
Build ID: 73f4b797-2238-47bc-8e43-7ffcb2b7d9b7
Jobs completed: 36. Time elapsed: 0.0s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)
BUILD SUCCEEDED
</code></pre>
<p>Okay the build succeeded, but where is our <code>out.txt</code>? We can ask buck!</p>
<pre><code>〉buck2 build //:hello_world --show-output
Build ID: 7ce93845-ab1e-4b0a-9274-51fed9f9e295
Jobs completed: 3. Time elapsed: 0.0s.
BUILD SUCCEEDED
root//:hello_world buck-out\v2\gen\root\fb50fd37ce946800\__hello_world__\out\out.txt
</code></pre>
<p>It lives in a deeply nested subdirectory of <code>buck-out</code>, a new top-level directory
that was created for us. If you remember from before, this directory is ignored
in our <code>.gitignore</code>.</p>
<p>If we look at the file, you can see it contains the text we wanted it to
contain.</p>
<p>Let's build a second time!</p>
<pre><code>〉buck2 build //:hello_world
File changed: root//.git/index.lock
File changed: root//.git
File changed: root//.git/modules/prelude/index.lock
31 additional file change events
Build ID: c00e4bfa-a1f8-40c7-a61c-2a394dca5da5
Jobs completed: 5. Time elapsed: 0.0s.
BUILD SUCCEEDED
</code></pre>
<p>Buck has noticed that we've changed some files, but since our rule doesn't
depend on any of them, we're good to go.</p>
<h2>Building some Rust code</h2>
<p>Okay, <code>echo</code> to a file is fun, but let's actually build some Rust. Create a
file, <code>hello.rs</code>:</p>
<pre><code>fn main() {
println!("Hello, world!");
}
</code></pre>
<p>and then update the <code>BUCK</code> file to this:</p>
<pre><code>rust_binary(
name = "hello_world",
srcs = ["hello.rs"],
crate_root = "hello.rs",
)
</code></pre>
<p>This says "hey, we're building a Rust binary, it has this target name, these
source files, and the crate root lives here." Given we only have one file,
there's some reptition. It happens. Let's build:</p>
<pre><code>〉buck2 build //:hello_world
File changed: root//BUCK
Error running analysis for `root//:hello_world (prelude//platforms:default#fb50fd37ce946800)`
Caused by:
0: Error looking up configured node root//:hello_world (prelude//platforms:default#fb50fd37ce946800)
1: Error looking up configured node toolchains//:cxx (prelude//platforms:default#fb50fd37ce946800) (prelude//platforms:default#fb50fd37ce946800)
2: looking up unconfigured target node `toolchains//:cxx`
3: Unknown target `cxx` from package `toolchains//`.
Did you mean one of the 1 targets in toolchains//:BUCK?
Build ID: f126ce07-efe8-41d3-8aae-8b7d31a4dafc
Jobs completed: 4. Time elapsed: 0.0s.
BUILD FAILED
</code></pre>
<p>Oops! We didn't set up a rust toolchain! Let's do that now. Edit
<code>toolchains/BUCK</code>:</p>
<pre><code>load("@prelude//toolchains:rust.bzl", "system_rust_toolchain")
system_rust_toolchain(
name = "rust",
default_edition = "2021",
visibility = ["PUBLIC"],
)
</code></pre>
<p>And.... when we build again, the same error. Now. I am not 100% sure what's
going on here, but this is what we need to do:</p>
<pre><code>load("@prelude//toolchains:rust.bzl", "system_rust_toolchain")
load("@prelude//toolchains:genrule.bzl", "system_genrule_toolchain")
load("@prelude//toolchains:cxx.bzl", "system_cxx_toolchain")
load("@prelude//toolchains:python.bzl", "system_python_bootstrap_toolchain")
system_genrule_toolchain(
name = "genrule",
visibility = ["PUBLIC"],
)
system_rust_toolchain(
name = "rust",
default_edition = "2021",
visibility = ["PUBLIC"],
)
system_cxx_toolchain(
name = "cxx",
visibility = ["PUBLIC"],
)
system_python_bootstrap_toolchain(
name = "python_bootstrap",
visibility = ["PUBLIC"],
)
</code></pre>
<p>I <em>believe</em> that this is because, to compile the Rust compiler, we need
Python and a C++ compiler. Well, I <em>did</em> believe that, but after digging into
things some more, it's that the Rust toolchain from the prelude depends on
the CXX toolchain in the prelude, because the Rust toolchain invokes the C
compiler to invoke the linker. I'm still not 100% sure why Python needs to be
in there. Anyway.</p>
<p>Now, when I run, I got this:</p>
<pre><code>〉buck2 build //:hello_world -v 3
Action failed: prelude//python_bootstrap/tools:win_python_wrapper (symlinked_dir resources)
Internal error: symlink(original=../../../../../../../../../prelude/python_bootstrap/tools/win_python_wrapper.bat, link=C:\Users\steve\Documents\GitHub\buck-rust-hello\buck-out\v2\gen\prelude\fb50fd37ce946800\python_bootstrap\tools\__win_python_wrapper__\resources\win_python_wrapper.bat): A required privilege is not held by the client. (os error 1314)
Build ID: 57a66885-f7e7-474b-a782-b49fc4425be9
Jobs completed: 14. Time elapsed: 0.0s.
BUILD FAILED
Failed to build 'prelude//python_bootstrap/tools:win_python_wrapper (prelude//platforms:default#fb50fd37ce946800)'
</code></pre>
<p>I got this becuase I'm on Windows, and Windows restricts the ability to create
symlinks by default. Turning on "Developer Mode" (which I'm surprised that I
haven't had to turn on so far yet), I get further:</p>
<pre><code>&lt;whole bunch of output&gt;
= note: 'clang++' is not recognized as an internal or external command,
operable program or batch file.
</code></pre>
<p>Here's that "invokes the compiler to get the linker" thing I was referring to
above.</p>
<p>Now... by default, the Rust support is for the GNU version of the Windows
toolchain. I never use that. Upstream has said that they want everything
to be supported, so that change will come at some point, maybe by the time
you read this! But in the meantime, I could get my (pure Rust) projects
building with two small patches:</p>
<pre><code>diff --git a/prelude/toolchains/cxx.bzl b/prelude/toolchains/cxx.bzl
index c57b7b8..dc14ca7 100644
--- a/prelude/toolchains/cxx.bzl
+++ b/prelude/toolchains/cxx.bzl
@@ -39,7 +39,7 @@ def _system_cxx_toolchain_impl(ctx):
CxxToolchainInfo(
mk_comp_db = ctx.attrs.make_comp_db,
linker_info = LinkerInfo(
- linker = RunInfo(args = ["clang++"]),
+ linker = RunInfo(args = ["link"]),
linker_flags = ["-fuse-ld=lld"] + ctx.attrs.link_flags,
archiver = RunInfo(args = ["ar", "rcs"]),
archiver_type = archiver_type,
diff --git a/prelude/toolchains/rust.bzl b/prelude/toolchains/rust.bzl
index 8172090..4545d55 100644
--- a/prelude/toolchains/rust.bzl
+++ b/prelude/toolchains/rust.bzl
@@ -23,7 +23,7 @@ _DEFAULT_TRIPLE = select({
# default when we're able; but for now buck2 doesn't work with the MSVC
# toolchain yet.
"config//cpu:arm64": "aarch64-pc-windows-gnu",
- "config//cpu:x86_64": "x86_64-pc-windows-gnu",
+ "config//cpu:x86_64": "x86_64-pc-windows-msvc",
}),
})
</code></pre>
<p>Now a build works!</p>
<pre><code>〉buck2 build //:hello_world
File changed: root//BUCK
File changed: root//.git/index.lock
File changed: root//.git
6 additional file change events
Build ID: 65fc80aa-7bfa-433a-bfa7-57919147b550
Jobs completed: 65. Time elapsed: 1.0s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)
BUILD SUCCEEDED
</code></pre>
<p>We can run it to see the output:</p>
<pre><code>〉buck2 run //:hello_world
Build ID: 78b0ca23-2c7c-4c02-a161-bba15e3b38bd
Jobs completed: 3. Time elapsed: 0.0s.
hello world
</code></pre>
<p>Same idea as <code>cargo run</code>.</p>
<p>Speaking of <code>cargo run</code>, what might this look like with Cargo? Well, we can create
a <code>Cargo.toml</code>:</p>
<pre><code>[package]
name = "hello_world"
version = "0.1.0"
edition = "2021"
[[bin]]
name = "hello_world"
path = "hello.rs"
</code></pre>
<p>and try it out. Oh, and you'll probably want to put <code>target</code> into your
<code>.gitignore</code>.</p>
<p>Let's build. The "benchmark" command in nushell is sort of like <code>time</code> on a
UNIX system:</p>
<pre><code>〉benchmark { cargo build }
Compiling hello_world v0.1.0 (C:\Users\steve\Documents\GitHub\buck-rust-hello)
Finished dev [unoptimized + debuginfo] target(s) in 0.34s
416ms 490us 100ns
〉benchmark { cargo build }
Finished dev [unoptimized + debuginfo] target(s) in 0.00s
77ms 317us 200ns
</code></pre>
<p>Not too bad, a bit under half a second for the initial build, and near immediate
on a subsequent build. What about buck?</p>
<pre><code>〉benchmark { buck2 build //:hello_world -v 3 }
Running action: &lt;snip&gt;
Build ID: 47ebd9f1-3394-4f72-a0fb-02c936035d2b
Jobs completed: 58. Time elapsed: 0.8s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)
BUILD SUCCEEDED
945ms 557us
〉benchmark { buck2 build //:hello_world -v 3 }
Build ID: 5eed24e8-a540-454f-861a-855464aba3c9
Jobs completed: 3. Time elapsed: 0.0s.
BUILD SUCCEEDED
78ms 274us 100ns
</code></pre>
<p>Not too shabby; Buck is a <em>teeny</em> bit slower on the initial build, but when stuff
is cached, both are the same speed. All of this is fast enough to qualify as
"basically instant."</p>
<h2>Adding a library</h2>
<p>Let's up the complexity a bit, by adding a library that we want to depend on.
Here's a <code>lib.rs</code>:</p>
<pre><code>pub fn print_hello() {
println!("Hello, world!");
}
</code></pre>
<p>We want to change our top-level <code>BUCK</code> to add this:</p>
<pre><code>+rust_library(
+ name = "print_hello",
+ srcs = ["lib.rs"],
+ edition = "2021",
+ visibility = ["PUBLIC"],
+)
+
rust_binary(
name = "hello_world",
srcs = ["hello.rs"],
crate_root = "hello.rs",
+ deps = [
+ ":print_hello",
+ ],
)
</code></pre>
<p>Here, we make a new library, <code>print_hello</code>, and then make our binary depend
on it.</p>
<p>Let's change the code in <code>main.rs</code> to use the library:</p>
<pre><code>fn main() {
println!("hello world");
print_hello::print_hello();
}
</code></pre>
<p>And that's it! Let's examine our targets:</p>
<pre><code>〉buck2 targets //...
Build ID: 4646f2e7-0ea3-4d59-8590-3da0708ce96e
Jobs completed: 4. Time elapsed: 0.0s.
root//:hello_world
root//:print_hello
</code></pre>
<p>They're both there! We can now build one, the other, or everything:</p>
<pre><code># build everything
〉buck2 build //...
# build just the library
〉buck2 build //:print_hello
# build 'just' the binary, this will of course end up building the library too
〉buck2 build //:print_hello
</code></pre>
<p>Let's make sure it still prints our output:</p>
<pre><code>〉buck2 run //:hello_world
Build ID: d76c80fb-dd77-463a-86a1-b6443cea10f6
Jobs completed: 3. Time elapsed: 0.0s.
Hello, world!
</code></pre>
<p>Nice.</p>
<p>Let's compare that to Cargo. Modify <code>Cargo.toml</code>:</p>
<pre><code>[lib]
name = "print_hello"
path = "lib.rs"
</code></pre>
<p>and build:</p>
<pre><code>〉cargo run
Compiling hello_world v0.1.0 (C:\Users\steve\Documents\GitHub\buck-rust-hello)
Finished dev [unoptimized + debuginfo] target(s) in 0.42s
Running `target\debug\hello_world.exe`
Hello, world!
</code></pre>
<p>Nice.</p>
<h3>more cargo-ish</h3>
<p>Before we move forward, this isn't <em>exactly</em> an apples to apples comparison; we've been doing a
lot of configuration for Cargo that we normally wouldn't have to do, and also,
what if you already have a Cargo project, but you want to try out Buck with it?</p>
<p>Do this:</p>
<pre><code>&gt; mkdir src
&gt; mv hello.rs src/main.rs
&gt; mv lib.rs src/main.rs
</code></pre>
<p>And delete the configuration from <code>Cargo.toml</code>, leaving just the <code>package</code> table.
Finally, we need to change <code>src/main.rs</code>, given that we're using the default
crate name for the library crate, which is <code>hello_world</code> and not <code>print_hello</code>:</p>
<pre><code>fn main() {
hello_world::print_hello();
}
</code></pre>
<p>After this, <code>cargo build</code> works just fine. But what about Buck?</p>
<p>So. We have a bit of weirdness here, and I'm not sure if it's actually
work-around-able in Buck or not, since I'm still learning this myself. But
if we do the basic translation, we'll get an error. Let's try it. This is
how you modify the <code>BUCK</code> file:</p>
<pre><code>rust_library(
name = "hello_world",
srcs = glob(["src/**/*.rs"]),
edition = "2021",
visibility = ["PUBLIC"],
)
rust_binary(
name = "hello_world",
srcs = ["src/main.rs"],
crate_root = "src/main.rs",
deps = [
":print_hello",
],
)
</code></pre>
<p>Cargo produces a binary and a library, both called <code>hello_world</code>, but buck
doesn't like that:</p>
<pre><code>〉buck2 run //:hello_world
Error evaluating build file: `root//:BUCK`
Caused by:
Traceback (most recent call last):
* BUCK:8, in &lt;module&gt;
rust_binary(
error: Attempted to register target root//:hello_world twice, re-run the command with `--target-call-stacks` to obtain a call stack of the first registration
--&gt; BUCK:8:1
|
8 | / rust_binary(
9 | | name = "hello_world",
10 | | srcs = ["src/main.rs"],
11 | | crate_root = "src/main.rs",
12 | | deps = [
13 | | ":print_hello",
14 | | ],
15 | | )
| |_^
|
Build ID: d6a8925d-0180-4308-bcb9-fbc888e7eca1
Jobs completed: 4. Time elapsed: 0.0s.
BUILD FAILED
</code></pre>
<p>It's like hey! You have two targets named <code>hello_world</code>! That's confusing!
It also reveals a difference between Buck and Cargo. With Cargo, if you remember
our configuration, we had to point it to the crate root. From there, Cargo just
leans on <code>rustc</code> to load up all of the other files that may be required if you
have a bunch of modules. But with Buck, we need to tell it up front which files we use.
So as you can see above:</p>
<pre><code> srcs = glob(["src/**/*.rs"]),
</code></pre>
<p>We can use the <code>glob</code> command to glob up all of our files, which is nice, but
it's... it's actually wrong. We want to glob everything <em>except</em> <code>main.rs</code>.
If <code>main.rs</code> were to change, this would try and re-build both the binary and
the library, in my understanding. So that's annoying.</p>
<p>It's not just annoying for Buck, though. Having both a <code>src/main.rs</code> and a
<code>src/lib.rs</code> has led to so much confusion from beginners over the years. At some
point, someone puts <code>mod lib;</code> into <code>src/main.rs</code> and everything goes to hell.
The original intention of this layout, to make simple things simple, is a good
idea, but I think that sadly, we missed the mark here. Luckily, Cargo also
supports a <code>bin</code> directory:</p>
<pre><code>&gt; mkdir src/bin
&gt; mv src/main.rs src/bin
</code></pre>
<p>We can still <code>cargo run</code> and <code>cargo build</code> and this all works just fine.
This doesn't fix our glob issue, though, because <code>src/bin</code> is still inside
of <code>src</code>.</p>
<p>I <em>think</em>, in the abstract, I'd prefer a layout like <code>src/{lib,bin}</code>. You want
things to not really be nested. So let's do that. Both Cargo and Buck can
handle it! It's just not as nice as being purely default in Cargo, since that
convention is so strong.</p>
<pre><code># if you didn't do this above
&gt; mkdir src/bin
&gt; mv src/main.rs src/bin
&gt; mkdir src/lib
&gt; mv src/lib.rs src/lib/print_hello.rs
</code></pre>
<p>We have to change <code>src/bin/main.rs</code> to use <code>hello_world</code> again</p>
<pre><code>fn main() {
hello_world::print_hello();
}
</code></pre>
<p>And we have to re-add some configuration into <code>Cargo.toml</code>:</p>
<pre><code>[lib]
path = "src/lib/print_hello.rs"
</code></pre>
<p>Everything should build just fine. But what about Buck?</p>
<p>So, once you start getting into subdirectories, you can also start using
multiple <code>BUCK</code> files. So we can empty out our root <code>BUCK</code> file (I'm leaving it
existing but empty, if you want to delete it you can but you'll recreate it in
the next part anyway), and create two new ones. Here's <code>src/lib/BUCK</code>:</p>
<pre><code>rust_library(
name = "hello_world",
srcs = glob(["**/*.rs"]),
crate_root = "print_hello.rs",
edition = "2021",
visibility = ["PUBLIC"],
)
</code></pre>
<p>and <code>src/bin/BUCK</code>:</p>
<pre><code>rust_binary(
name = "hello_world",
srcs = ["main.rs"],
crate_root = "main.rs",
deps = [
":hello_world",
],
)
</code></pre>
<p>We added in a <code>crate_root</code> to the library as well. Okay, let's try this:</p>
<pre><code>〉buck2 run //:hello_world
File changed: root//src/lib/BUCK
Unknown target `hello_world` from package `root//`.
Did you mean one of the 0 targets in root//:BUCK?
Build ID: d5059fc9-8001-47c4-ba5a-6ba605a4182c
Jobs completed: 2. Time elapsed: 0.0s.
BUILD FAILED
</code></pre>
<p>Oops! Since we moved files around, the names of our targets have changed. Let's
examine them:</p>
<pre><code>〉buck2 targets //...
Build ID: c4165964-cb87-49b4-8afe-4a3fc2c526bc
Jobs completed: 4. Time elapsed: 0.0s.
root//src/bin:hello_world
root//src/lib:hello_world
</code></pre>
<p>We had only seen very basic target patterns, but this is enough to show off:</p>
<p><code>root/src/bin:hello_world</code></p>
<p>can be read as</p>
<blockquote>
<p>The "hello_world" target defined in <code>/src/bin/BUCK</code>.</p>
</blockquote>
<p>Our target names changing also means we made one mistake in our new <code>BUCK</code> files.
Let's give it a try:</p>
<pre><code>〉buck2 run //src/bin:hello_world
Error running analysis for `root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)`
Caused by:
0: Error looking up configured node root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)
1: Cyclic computation detected when computing key `root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)`, which forms a cycle in computation chain: `root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800),root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)`
Build ID: 930ab541-c2dd-44f5-aef1-f6658a2b7c53
Jobs completed: 2. Time elapsed: 0.0s.
BUILD FAILED
</code></pre>
<p>Right. Our binary depends on <code>:hello_world</code>, which it is itself named <code>hello_world</code>,
so there's a problem. But that's just it, we don't want to depend on any old
<code>hello_world</code>, we want to depend on our libary. Can you write out the target
pattern that should go in <code>src/bin/BUCK</code>?</p>
<p>It looks like this:</p>
<pre><code> deps = [
"//src/lib:hello_world",
],
</code></pre>
<p>"The <code>hello_world</code> target in <code>/src/lib/BUCK</code>. Perfect. And now it works!</p>
<pre><code>〉buck2 run //src/bin:hello_world
File changed: root//src/bin/BUCK
Build ID: c6d2fdaa-298a-425a-9091-d3f6b38c4336
Jobs completed: 12. Time elapsed: 0.5s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)
Hello, world!
</code></pre>
<p>It kinda stinks to have to type all of that out. Luckily, Buck supports aliases
for target patterns. Take our top-level <code>BUCK</code> file, and add this:</p>
<pre><code>alias(
name = "build",
actual = "//src/bin:hello_world",
visibility = ["PUBLIC"],
)
</code></pre>
<p>And now we can use it:</p>
<pre><code>〉buck2 build
Build ID: a87ed1e2-cfab-47b0-830e-407217997fd7
Jobs completed: 2. Time elapsed: 0.0s.
BUILD SUCCEEDED
</code></pre>
<p>Fun.</p>
<p>Okay! This is getting a bit long, so let's end there. We have more to learn
before buck can actually replace Cargo in our normal development workflow, but
I hope this helped you see how you could get started with Buck if you wanted to.</p>
<p>If you want to check out this on your own, I've published this on GitHub:
<a href="https://github.com/steveklabnik/buck-rust-hello/tree/024ef54ba45627e87a65aaf2f69c6661198c336c">https://github.com/steveklabnik/buck-rust-hello/tree/024ef54ba45627e87a65aaf2f69c6661198c336c</a></p>
<p>Next up, we'll be tackling other features, like "using crates from crates.io."
No promises on when that'll get published though!</p></content:encoded>
</item>
<item>
<title>Rust's Golden Rule</title>
<link>https://steveklabnik.com/writing/rusts-golden-rule/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rusts-golden-rule/</guid>
<pubDate>Mon, 27 Mar 2023 00:00:00 GMT</pubDate>
<content:encoded><p>I find myself thinking about a particular design principle of Rust today. I'm
not sure I've ever seen it named specifically before, but it gets referred to
from time to time, and I think it's an under-rated but very important aspect
of why Rust works so well. I was going to refer to it as "the signature is the
contract" today, but then I ended up changing it. Regardless of that, if someone
else has already written this out somewhere, and used a different name, please
let me know!</p>
<p>Magic: the Gathering is a really interesting project. I say "project" rather
than "card game" because while it is a card game, it also pioneered a whole
bunch of incedental other things that had big effects on related hobbies.</p>
<p>I learned MtG in the late 90s. The rules were a bit different then, but many
of them are the same. The very first rule I was taught was sort of the "Magic
Golden Rule," though in today's Comprehensive Rulebook, there are four of them.
This one is still the first, though:</p>
<blockquote>
<ol>
<li>The Magic Golden Rules</li>
</ol>
<p>101.1. Whenever a card’s text directly contradicts these rules, the card takes
precedence. The card overrides only the rule that applies to that specific
situation. The only exception is that a player can concede the game at any
time (see rule 104.3a).</p>
</blockquote>
<p>This rule is the most important rule because it kind of creates the spaces of
possibilities for cards: many cards exist to tweak, modify, or break some sort
of fundamental rule.</p>
<p>That being said, all these years later, this idea is so pervasive in games like
this that it's barely even considered an actual rule. It's just part of the
physics of the genre, it's how these games work. Yet it's critical to the
entire enterprise.</p>
<p>Rust also has a rule. It's kinda funny, because in some senses, this rule
is almost the opposite of Magic's, if you can even stretch the comparison this
far. Here it is:</p>
<blockquote>
<p>Whenever the body of a function contradicts the function's signature, the
signature takes precedence; the signature is right and the body is wrong.</p>
</blockquote>
<p>This rule is also so pervasive in Rust that we take it for granted, but it
is really, truly important. I think it is also important for Rust users to
internalize the implications of this rule, so that they know why certain things
work the way that they do.</p>
<p>Here is the most famous implication of this rule: Rust does not infer function
signatures. If it did, changing the body of the function would change its
signature. While this is convenient in the small, it has massive ramifications.</p>
<p>Consider this example program:</p>
<pre><code>fn foo(x: i32) -&gt; i32 {
dbg!(x);
x
}
</code></pre>
<p>This function prints out <code>x</code>, and then returns it. Nothing fancy going on here,
this is just random stuff to make an example. This compiles just fine. But let's
imagine that we have a version of Rust that infers our signatures. So we could
type this instead:</p>
<pre><code>fn foo(x) {
</code></pre>
<p>This is what a Ruby-ish Rust might look like; we declare the name of our
argument but not its type, and we don't declare a return type. Now, let's
do a small refactoring, we're gonna comment out the final value there:</p>
<pre><code>fn foo(x) {
dbg!(x);
//x
}
</code></pre>
<p>the final expression has changed; it's no longer <code>x</code>, but instead is <code>()</code>,
which is what <code>dbg!(x);</code> evaluates to. Because of type inference, the inferrred
type of <code>foo</code> is now <code>fn(i32) -&gt; ()</code>. Our function typechecks! It's all good,
right?</p>
<p>Well, no:</p>
<pre><code>error[E0369]: cannot add `{integer}` to `()`
--&gt; src/lib.rs:5:11
|
5 | y + 1
| - ^ - {integer}
| |
| ()
</code></pre>
<p>Wait, what? We don't have a <code>y + 1</code> anywhere in our code?! Where's that
error coming from... <code>src/lib.rs:5</code>. What's at the top of <code>lib.rs</code>?</p>
<pre><code>mod bar {
fn baz() -&gt; i32 {
let y = crate::foo(5);
y + 1
}
}
</code></pre>
<p>Oh. Some other code was using <code>foo</code>. When its signature changed, we broke
this invocation. It's nice that alt-Rust caught this for us, but this error is
(would be, anyway!) really bad: no longer is it telling us that our code
is wrong, but instead points to some other code somewhere else we weren't even
messing with. Sure, we can go "hey why is <code>y</code> <code>()</code>?" and then figure it out,
but compare that to today's error:</p>
<pre><code>error[E0308]: mismatched types
--&gt; src/lib.rs:9:19
|
9 | fn foo(x: i32) -&gt; i32 {
| --- ^^^ expected `i32`, found `()`
| |
| implicitly returns `()` as its body has no tail or `return` expression
10 | dbg!(x);
| - help: remove this semicolon to return this value
</code></pre>
<p>This error is far better: it points out that the body of our function
contradicts the signature. It points out what in our body is generating the
value that contradicts the signature. And it doesn't complain about callers.</p>
<p>So sure, this gets us nicer error messages, but is that really a big deal?
I think it is, but it's not the only implication here. I'd like to talk about
two more: one that's clearly an advantage, and one that has led to some pain
that people would like to resolve. Balance :)</p>
<p>First, the one that's an advantage. The advantage is modularity. That makes
some forms of analysis much more reasonable, or sometimes even possible, as
opposed to super difficult. Because everything you need for memory safety is
described in the signature of the function, Rust doesn't need to examine your
entire program to determine if there's some shenanigans going on elsewhere.
This is far, far less work than just checking the signatures of the functions
you call. Each function can be checked in isolation, and then assembled
together. This is a very nice property.</p>
<p>Second, where this leads to some pain. Users have nicknamed this one "borrow
splitting," or "partial borrowing." It looks like this:</p>
<pre><code>struct Point {
x: i32,
y: i32,
}
impl Point {
pub fn x_mut(&amp;mut self) -&gt; &amp;mut i32 {
&amp;mut self.x
}
pub fn y_mut(&amp;mut self) -&gt; &amp;mut i32 {
&amp;mut self.y
}
}
</code></pre>
<p>I find accessors, and especially mutators, to be where this sort of thing pops
up most often. This is the classic example. The above code is fine, but if we
try to do this:</p>
<pre><code>// this doesn't work
impl Point {
pub fn calculate(&amp;mut self) -&gt; i32 {
let x = self.x_mut();
let y = self.y_mut();
// yes I picked multiplication because this looks kinda funny
*x * *y
}
}
// we would call it like this:
let answer = p.calculate();
</code></pre>
<p>We get this:</p>
<pre><code>error[E0499]: cannot borrow `*self` as mutable more than once at a time
--&gt; src/lib.rs:19:17
|
18 | let x = self.x_mut();
| ------------ first mutable borrow occurs here
19 | let y = self.y_mut();
| ^^^^^^^^^^^^ second mutable borrow occurs here
20 |
21 | *x * *y
| -- first borrow later used here
</code></pre>
<p>However, if we didn't have these accessors, <code>x</code> and <code>y</code> were instead just
public, this very similar free function:</p>
<pre><code>fn calculate(x: &amp;mut i32, y: &amp;mut i32) -&gt; i32 {
*x * *y
}
// called like this:
let answer = calculate(&amp;mut point.x, &amp;mut point.y);
</code></pre>
<p>works just fine. Why? Because of the signatures. This signature:</p>
<pre><code>pub fn calculate(&amp;mut self) -&gt; i32 {
</code></pre>
<p>and these signatures:</p>
<pre><code>pub fn x_mut(&amp;mut self) -&gt; &amp;mut i32 {
pub fn y_mut(&amp;mut self) -&gt; &amp;mut i32 {
</code></pre>
<p>says "hey, I am going to borrow all of <code>self</code> mutably," which implies an
exclusive reference to <code>self</code>. That the body of <code>calculate</code> borrows two
different parts of <code>self</code> independently is 100% irrelevant, that's what the
signature says! And so rustc looks at this and says "hey wait a minute,
calling <code>x_mut</code> borrows <code>self</code> mutably, and calling <code>y_mut</code> borrows <code>self</code>
mutably. That's aliasing! Bad programmer!</p>
<p>Whereas in the second example, this signature:</p>
<pre><code>fn calculate(x: &amp;mut i32, y: &amp;mut i32) -&gt; i32 {
</code></pre>
<p>says "hey, I have two mutable references, to two different integers." And
at the call site, Rust sees that we're creating two different borrows to two
different integers, even though they're both part of our <code>point</code>, and so it
okays things.</p>
<p>This is kind of a pain in the butt! But what it saves us from is that scary
action at a distance in our typecheck example. Imagine that Rust somehow
inferred that the first version was okay, due to the body only being disjoint.
What happens in the future, when we refactor our function, and the borrows need
to change? That would have the same problem as before; we'd get weird errors
elsewhere in our code. There have been some proposals over the years to possibly
fix this pain point, but it's a tough one. Putting "here's the fields I'm
borrowing in the body" into the signature, in order to conform with the Golden
Rule, looks very odd. Maybe something will happen here, maybe not. I have
complicated feelings.</p>
<p>So beyond error messages and making accessors/mutators awkward, how does this
affect you day to day? Well, one way is that I find I end up paying more
attention to signatures and less attention to bodies, when I'm trying to sort
something out. In many cases, what you're doing in the body is irrelevant, I
care only about what your signature affords me. This isn't <em>quite</em> as true
as in some pure functional languages, but it has that vague feel to it. Another
way, and a slightly bigger one, has to do with the relationships between types
and TDD, but I think I'm going to save that for its own post.</p></content:encoded>
</item>
<item>
<title>You're probably learning a technology in its seventh season, not its pilot</title>
<link>https://steveklabnik.com/writing/you-re-probably-learning-a-technology-in-its-seventh-season-not-its-pilot/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/you-re-probably-learning-a-technology-in-its-seventh-season-not-its-pilot/</guid>
<pubDate>Fri, 27 Jan 2023 00:00:00 GMT</pubDate>
<content:encoded><p>I first heard this adage in the Rails space. I don't know who to attribute this
to, but I also never wrote it down. So, time to write it down, and if you know
of the origin of this aphorism, let me know and I'd love to put it here!</p>
<p>I <em>think</em> that the first formulation of this sentiment goes something like this:</p>
<blockquote>
<p>Rails is like a soap opera. You're getting started with it in a later season.
The cast of characters has already been established, there's a lot of history.
You kinda just gotta dive in and go with the flow and catch up on older stuff
at your leasure.</p>
</blockquote>
<p>That's obviously a bit long, so I think I'll be using something like this
in the future:</p>
<blockquote>
<p>You're probably learning a technology in its seventh season, not its pilot.</p>
</blockquote>
<p>I was thinking about this today because I happened to come across <a href="https://www.reddit.com/r/rust/comments/xj2a23/comment/ipd6tcv/">this reddit post from a few months ago</a>.</p>
<blockquote>
<blockquote>
<p>In fact, there is hardly any analysis of the downsides of Rust’s error model in the first place, which is quite disheartening.</p>
</blockquote>
<p>Not really planning to add much to the convo, but back in mid 2019 when I first
started learning and writing Rust, about half the posts I saw here in the
subreddit were related to errors, how to do them right, how to minimize the
boilerplate, and so on and so forth...</p>
<p>Over the last couple years, the convo has died down because we've all more or
less agreed that the major pain points around error handling have been solved
with a simple rule: thiserror for library code, anyhow for application code.</p>
<p>There's obvs still more things that can be done around error handling. try is
one such example, along with standardizing thiserror and anyhow so its part of
the stdlib.</p>
<p>But to say there is no discussion about Rust and it's error handling story is
very dishonest imo as that discussion was raged for years before I saw it and
only recently came to a proper conclusion for the majority of cases, and thus
theres now much less talk about it. I even learned 3 different "defacto standard
error libs" (that are all now defunct and no one cares about) out of a much
longer list of them that came and went before them due to better and better
patterns being being found over the years!</p>
<p>There was plenty of healthy and constructive discussion on this topic for an
insanely long time and just because you didn't see it doesn't mean everyone was
satisfied with the error handling story as it is in std from the 1.0 of Rust.
Rust users actually engage with the system constantly, so ofc they'd discuss the
problems with it and work towards fixing it! Hell, there's still people that
prefer things other than thiserror and anyhow and thats part of why those havent
been standardized yet too, and why there still might seem to be "no progress" on
fixing things.</p>
<p>TL;DR: Rust developers use Rust's error handling literally all the time. Your
assertion that they do not understand its shortcomings and pain points and
refuse to even discuss them is weird given that there's been a multi-years long
battle between all kinds of error handling crates that only recently resulted in
a proper and clear winner.</p>
</blockquote>
<p>Now, I don't want to take a position on the original thread, or what they have
to say, or if they actually <em>are</em> falling afoul of this aphorism. But I think
that this particular text shows the meaning of what I mean by "you're in the
seventh season" beautifully. Sometimes, when we learn a new technology, we go
"wtf, why is this like this?" And sometimes, that outside perspective is good!
But sometimes, it just means that you weren't there for the discussion, you either
don't know or aren't in the same contexts, or a variety of other reasons why
something is the way that it is, even if that way is surprising to you at first.</p>
<p>In that sense, I think this idea is pretty closely related to Chesterton's Fence.
It may even just be the same thing, formulated in a slightly different way.</p></content:encoded>
</item>
<item>
<title>Too many words about Rust's function syntax</title>
<link>https://steveklabnik.com/writing/too-many-words-about-rusts-function-syntax/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/too-many-words-about-rusts-function-syntax/</guid>
<pubDate>Tue, 03 Jan 2023 00:00:00 GMT</pubDate>
<content:encoded><p>There's a famous quote by Eleanor Roosevelt:</p>
<blockquote>
<p>great minds discuss semantics, average minds discuss syntax and small minds discuss syntax of comments</p>
</blockquote>
<p>(Apologies, Phil Wadler.)</p>
<p>I'd like to preface this post that I do <em>not</em> consider this to be an actual
proposal to change Rust, even a pre-RFC. I don't think that making changes like
this has enough benefits once a language is as established as Rust is. Consider
this post more of some musings about some things that there wasn't really time
to truly consider when Rust was younger, so that maybe someone who's making a
new language can learn from those things early enough in its life to do so.</p>
<p>Also, I am not the first person to think about this, but I have lost the
previous post to time. There also may be a newfangled language that has this or
similar syntax already I'm not aware of. If you know of similar previous
proposals or languages, please let me know so that I can link to them here.</p>
<p>Anyway, consider this post a "Steve decided to write up a thing instead of
just tweeting, it is not particularly well thought out but at least it's down
on paper" kind of thing. A serious proposal here, even if I thought the idea was
good in general, which to repeat again, I do not, would require examining a lot
more corner cases than I do in this post.</p>
<p>With all of that out of the way, let's begin.</p>
<hr />
<h2>The fn syntax</h2>
<p>Rust does a really good job of being orthogonal in many ways. While it isn't a
tiny language, that things fit together nicely is one of the things that can
make it feel a lot smaller, in my opinion. There's one thing, though, that
is really a bit inconsistent today, and that's the syntax and semantics of
functions.</p>
<p>Let me explain: you declare a function item (read: define a function) like this:</p>
<pre><code>fn foo(x: i32) -&gt; i32 {
todo!()
}
</code></pre>
<p>Simple enough. However, there's a lot more going on for more complex
declarations. <a href="https://doc.rust-lang.org/1.66.0/reference/items/functions.html">Here</a> is the beginning of the current grammar
of functions:</p>
<pre><code>Function :
FunctionQualifiers fn IDENTIFIER GenericParams?
( FunctionParameters? )
FunctionReturnType? WhereClause?
( BlockExpression | ; )
</code></pre>
<p>I read this like this: "A function consists of some function qualifiers, fn,
an identifier, and an optional list of generic parameters. We then have some
optional function parameters inside of parenthesis, an optional return type,
an optional where clause, and then a block expression or a semicolon."</p>
<p>This is in sharp contrast to a C-style function definition, which would
look like this:</p>
<pre><code>int foo(int x) {
// todo
}
</code></pre>
<p>In general, most new languages have tended to settle closer to Rust's syntax
here than C's. There are a few reasons, and it's not like Rust invented this
style in the first place. One of the core nice things about Rust style syntax
is that <code>fn IDENTIFIER</code>, that is, if you want to find out where <code>foo()</code> is
defined, grep for <code>fn foo</code> and you can find it pretty quickly. You can't
do this as well with the <code>int</code> syntax. Some code styles have you write</p>
<pre><code>int
foo(int x) {
// todo
}
</code></pre>
<p>Notably, the GNU project uses this style:</p>
<blockquote>
<p>It is also important for function definitions to start the name of the
function in column one. This helps people to search for function definitions,
and may also help certain tools recognize them.</p>
</blockquote>
<p>You instead grep for <code>^foo</code>, and you can get a similar benefit.</p>
<p>So, what's bad about this syntax? Well, there are useful 'modifiers' to
functions, and Rust has added some and may add more in the future. For example,
if we want <code>foo</code> to be callable in a const context, we'd make it a <code>const fn</code>:</p>
<pre><code>const fn foo(x: i32) -&gt; i32 {
todo!()
}
</code></pre>
<p>That's all well and good, we still even retain the "grep for <code>fn</code>" idea here,
but these are <em>also</em> useful on arbitrary blocks of code:</p>
<pre><code>let x = const {
1 + 2
};
</code></pre>
<p>This is of course a trivial example that's not very useful, but it shows off
the syntax here. My point is... this starts to feel a bit... nonuniform. In
the function delcaration, we put const at the front, but when it's any other
block expression, we put it right before the block. Rust's function bodies
are similar to blocks in other ways, such as the whole "evaluates to the final
expression" thing. But they're different when they're being used to describe
the body of a function. That's a bit less than ideal.</p>
<p>We could try the naive thing:</p>
<pre><code>fn foo(x: i32) -&gt; i32 const {
todo!()
}
</code></pre>
<p>This is... weird, but it does sort of unify some syntaxes. We now can turn
any block of code, including a function body, <code>const</code> by adding the <code>const</code>
right before the opening brace. However... I kinda shudder at <code>i32 const</code>.
There's not enough separation between the return type and the modifier to make
quick sense of things, in my opinion. However, there's another small tweak we
could make here. But first, a small divergence into another corner of Rust's
syntax: variables.</p>
<h2>Variables</h2>
<p>Rust also has a different syntax for variable declaration. Like C, it's similar
to its function declaration syntax:</p>
<pre><code>// Rust
let x: i32 = 5;
</code></pre>
<pre><code>// C
int x = 5;
</code></pre>
<p>We have <code>let</code> instead of <code>fn</code>, we have the <code>name: type</code> instead of <code>type name</code>.
We can also declare a variable but not initialize it:</p>
<pre><code>let x: i32;
</code></pre>
<p>Rust will make us initialize it before we use it, but we can declare the
variable on its own, and then give it an initial value later.</p>
<p>But what's this has to do with functions?</p>
<h2>A marginally better syntax</h2>
<p>Functions also happen to have a "declare the signature but not the body" syntax
in Rust too, though it's almost exclusively used in an FFI context. Remember
the very end of our grammar?</p>
<blockquote>
<p>and then a block expression or a semicolon.</p>
</blockquote>
<p>If we have a function <code>foo</code> that we're calling through FFI, we can define it
like this:</p>
<pre><code>#[link(name = "something")]
extern {
fn foo(x: i32) -&gt; i32;
}
</code></pre>
<p>We don't provide a body, but we do provide a semicolon. This is strikingly
similar to the variable syntax. So why not have the regular function definition
also follow the variable syntax?</p>
<pre><code>fn foo(x: i32) -&gt; i32 = {
todo!()
}
</code></pre>
<p>We've now added one little extra bit to the grammar: a <code>=</code> after the return
type, if any. This one little change allows us to unify the rest of the syntax
around blocks more easily:</p>
<pre><code>// full signature
fn foo(x: i32) -&gt; i32 = {
todo!()
};
// empty return type
fn foo(x: i32) = {
todo!()
};
// const
fn foo(x: i32) -&gt; i32 = const {
todo!()
};
</code></pre>
<p>I happen to <em>really</em> like this. It's a pretty small tweak but I think it
cleans stuff up nicely.</p>
<pre><code>let name = value;
fn name() = value;
</code></pre>
<p>Where both could be values or blocks, with <code>let</code> it's most often a value but
with <code>fn</code> it's most often a block. The symmetry pleases me.</p>
<p>(Sharp readers will also notice the introduction of a semicolon. Leaving it
off would be closer to the old syntax, requring it would be similar to the
variable syntax, but then that would diverge from the other item declaration
syntax... I don't feel strongly about it but just in case anyone noticed I
figured I'd mention that as well, yet another rabbit hole to chase down.)</p>
<h2>Going too far</h2>
<p>This of course raises another divergence. But I'm not sure that fixing this one
is worth it; it starts to get <em>too</em> weird, I think. But maybe I'm wrong. This
one is about the types.</p>
<p>One of the things that makes <code>name: type</code> syntax nicer than <code>type name</code> syntax
is that it degrades nicely in the presence of type inference:</p>
<pre><code>// Rust
let x: i32 = 5;
let x = 5;
</code></pre>
<pre><code>// C23 (https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3007.htm)
int x = 5;
auto x = 5;
</code></pre>
<p>Rust's function syntax isn't like this though, partially because there's no
type inference on function signatures, so there's not a lot of pressure for
it to degrade nicely. The choice not to offer inference is the right one for
Rust, but for a future language where that's not the case, we could unify
them with something like this:</p>
<pre><code>// explicit type
fn foo(x): fn(i32) -&gt; i32 = {
todo!()
}
// inferred
fn foo(x) = {
todo!()
}
</code></pre>
<p>This one... this may be too far afield. And, while Rust doesn't have named
parameters, and possibly never will, I have no idea how this style of syntax
would work with that. There's also some repetition of <code>fn</code> here that might be
worth trying to remove, but I'm trying to make the smallest possible deltas
from existing syntax here, and <code>fn(i32) -&gt; i32</code> <em>is</em> the type of that function
in today's syntax.</p>
<p>What I mostly take away from this part of the exercise is that consistency is
a good goal, but it's not the <em>only</em> goal. Even if Rust didn't allow for
inference, making its function declaration syntax be like this may have simply
been a bridge too far for wide adoption. Rust already takes a lot of flak for
being weird, and spending some <a href="the-language-strangeness-budget">Strangeness Budget</a> to unify these
syntaxes probably wouldn't be the right choice. But maybe I'm too conservative!
Given that Rust already takes that flak, maybe a bigger jump would have been
okay here. Who knows.</p>
<p>... actually, we have one way to find out, I guess. This is <em>very</em> similar
to the syntax that Herb Sutter came up with for <a href="https://github.com/hsutter/cppfront">cppfront</a>:</p>
<pre><code>foo: (x: i32) -&gt; i32 = {
// todo
}
</code></pre>
<p>There's no <code>fn</code> preceding. The type syntax works a bit differently. But it's
close. And Herb is targetting the people who I'd be most worried about hating
this syntax: C++ folks. Maybe this idea isn't so farfetched at all. He's
also got an example of how making it any expression after the <code>=</code> can be
nice for one-liner functions. If we applied this to our fake Rust syntax:</p>
<pre><code>fn main() = println!("Hello, world!");
fn hello(name): fn(&amp;str) = println!("Hello, {name}!");
</code></pre>
<p>That's kinda nice too.</p></content:encoded>
</item>
<item>
<title>Ten Years of Ru...ewriting my website</title>
<link>https://steveklabnik.com/writing/ten-years-of-ru---ewriting-my-website/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/ten-years-of-ru---ewriting-my-website/</guid>
<pubDate>Wed, 21 Dec 2022 00:00:00 GMT</pubDate>
<content:encoded><p>December 21 is the anniversary of when I first heard about Rust way back in
2012. I used to write yearly posts about it; the last time I did was <a href="../six-years-with-rust">in
2018</a>. That makes today ten years. I thought I'd have
something big to say here, but... I just don't.</p>
<p>Part of that is because I haven't written a blog post since late 2020.
Part of <em>that</em> is well, <em>gestures wildly at everything</em>, but another part of it
is that the way I had set up my website was cool, but fragile, and I didn't have
the time or energy to keep up with it.</p>
<p>You see, last time I had basically taken a starter project for a new framework
I had never used before, and just kinda wrote code until it works. That's one
of the joys of having your own website; you can do whatever you want with it,
including things you wouldn't do in a professional context. The downside is that
you can do whatever you want with it, including things you wouldn't do in a
professional context. My RSS feed has been broken for who knows how long, and
I just couldn't muster up the effort to fix it.</p>
<p>So last night, I did something new: I did the tutorial for a new framework I
had never used before, and wrote code until it works.</p>
<p>Yeah yeah that's funny and all, but I actually feel much better about things
this time. We'll see how I feel in six months. The framework this time is
<a href="https://astro.build/">Astro</a>, and I think it's really neat. At some point after
I do some more with it and digest some more of the way that it works, I'll give
a more thorough writeup, but I find the <a href="https://docs.astro.build/en/concepts/why-astro/">Why Astro?</a> page on their
site does a good job of explaining.</p>
<p>Why not a Rust based framework? I want to do more in that space too, but the goal
here is to stretch my legs, and using a language (TypeScript, in this case) that
I don't use often is more interesting than the thing I use daily.</p>
<p>However, one little thing did come up during the development of this project
that I thought was kind of amusing, in a 'compare and contrast' kind of way.</p>
<p>I like for my blog to have entries grouped by year. There's no database in this
project, so I have to sort things myself. Here's the relevant code as it stands
today, with some small omissions for clarity:</p>
<pre><code>import type { MarkdownInstance } from "astro";
const allPosts = await Astro.glob('../pages/writing/*.md');
let allPostsByDate = (() =&gt; {
let posts: { [key: string]: MarkdownInstance&lt;Record&lt;string, any&gt;&gt;[] } = {};
allPosts.map((post) =&gt; {
let year = new Date(post.frontmatter.pubDate).getFullYear();
let array = posts[year.toString()];
// if this is the first post of this year, we have to create the inner array
if (!array || !array.length) {
posts[year.toString()] = [];
}
posts[year.toString()].push(post);
})
return posts;
})();
const years = Object.keys(allPostsByDate).reverse();
---
&lt;BaseLayout&gt;
&lt;p&gt;This is my blog.&lt;/p&gt;
{years.map((year) =&gt; (
&lt;h3&gt;{year}&lt;/h3&gt;
&lt;ul&gt;
{allPostsByDate[year].map((post) =&gt; &lt;BlogPost url={post.url} title={post.frontmatter.title} /&gt;)}
&lt;/ul&gt;
))}
&lt;/BaseLayout&gt;
</code></pre>
<p>I'm sure there's like, ten thousand things wrong with this code; like I said,
TypeScript beginner over here. I figured this stuff out myself, there's
probably a better way to do these things, but this works. (Okay, actually while
writing this I realized I didn't sort <em>within</em> the years, so the real code now
handles that too, but I left it out because it's not super relevant to the issue
at hand here.)</p>
<p>This is a pretty classic "I'm doing something in a scripting language" style
setup: you make some sort of data structure out of maps (or objects in this
case, of course) and arrays, and then spit it out in some form. But TypeScript
prefers types, it's right there in the name! And I had configured TypeScript
to be in its strictest mode, becuase I like types too. Here's the one big type
annotation I had to use:</p>
<pre><code>let posts: { [key: string]: MarkdownInstance&lt;Record&lt;string, any&gt;&gt;[] } = {};
</code></pre>
<p>If I didn't put a type on <code>posts</code>, it complains later in the program:</p>
<pre><code>`Element implicitly has an 'any' type because expression of type 'string' can't be used to index type '{}'.
No index signature with a parameter of type 'string' was found on type '{}'.
</code></pre>
<p>It needs a bit of help to figure out the type here. Figuring out how to say
"hey this is on object with string keys and arrays of <code>MarkdownInstance</code> as
values" took me a second, but it's not too bad. But I did struggle with it a bit,
first to figure out why I was getting the error in the first place, and second
so that I could fix it. I kinda laughed at struggling with types in this dymanic
feeling context.</p>
<p>So anyway, I also had to do a bit of translating markdown files; I had exported
all of my old posts from Notion, but I couldn't just plop them in and expect
them to work; Notion doesn't give you the metadata in a frontmatter block, for
starters. I have something like 200 posts, so doing this by hand is <em>just</em>
annoying enough that I said "I bet I can code this up faster than if I did it
all by hand, even with a vim macro."</p>
<p>But by now, I was in "get stuff done" mode, so I decided to switch back to Rust.
The program I wrote needed to:</p>
<ol>
<li>Open each markdown file in the directory.</li>
<li>Try to parse out the metadata.</li>
<li>use that metadata to write out a new file, with a new filename, and the
metadata in a more friendly format.</li>
<li>Delete the old file.</li>
</ol>
<p>This is a classic scripting task! But at this point, I know how to do it in Rust
easier than in a scripting language. I started like this:</p>
<pre><code>&gt; cargo new migrate
&gt; cd migrate
&gt; cargo add anyhow
&gt; cargo add walkdir
&gt; code .
</code></pre>
<p>And ended up writing this:</p>
<pre><code>use anyhow::Result;
use std::io::Write;
use std::{
fs::{self, File},
path::PathBuf,
};
use walkdir::WalkDir;
fn main() -&gt; Result&lt;()&gt; {
let from = std::env::args().skip(1).next().unwrap();
println!("from: {}", from);
for entry in WalkDir::new(from) {
let entry = entry.unwrap();
println!("Processing: '{}'", entry.path().display());
if entry.file_type().is_dir() {
continue;
}
let contents = fs::read_to_string(entry.path())?;
if &amp;contents[0..3] == "---" {
println!("already migrated, skipping");
continue;
}
let header: Vec&lt;_&gt; = contents.lines().take(7).collect();
if &amp;header[0][0..2] != "# " {
eprintln!("couldn't read title, skipping");
continue;
}
let title = &amp;header[0][2..];
if &amp;header[3][0..6] != "Blog: " {
eprintln!("couldn't read blog, skipping");
continue;
}
let blog = &amp;header[3][6..].to_ascii_lowercase().replace(' ', "-");
if &amp;header[4][0..6] != "Date: " {
eprintln!("couldn't read date, skipping");
continue;
}
let date = &amp;header[4][6..];
if &amp;header[6][0..6] != "Slug: " {
eprintln!("couldn't read slug, skipping");
continue;
}
let slug = &amp;header[6][6..];
dbg!(slug);
let mut slug = PathBuf::from(slug);
slug.set_extension("md");
let output = entry.path().parent().unwrap();
let output_path = output.join(slug);
println!("writing to: {}", output_path.display());
let mut output = File::create(&amp;output_path)?;
write!(
output,
"---
layout: ../../layouts/MarkdownPostLayout.astro
title: \"{}\"
pubDate: {}
blog: {}
---\n\n",
title, date, blog
)?;
for line in contents.lines().skip(7) {
writeln!(output, "{}", line)?;
}
fs::remove_file(entry.path())?;
}
Ok(())
}
</code></pre>
<p>76 lines. Not too bad. This is <em>quick and dirty</em>, it is not perfect. For
example, like ten of my posts had no 'blog' metadata (that tells me which of
my old blogs the post was originally written for), and so it didn't work on
them. I still needed to do some small cleanup, but I wanted to double check
every post anyway, so that was fine.</p>
<p>But there's a few other things that are kind of amusing to me about this code:</p>
<ol>
<li>I struggled with types in the scripty scenario above, but I did not here. I
am more experienced with Rust so that's probably it, but I needed to annotate
one type, just like in the TypeScript.</li>
<li>I didn't deal with lifetimes at all. Owned values, baby! And passing references
to some functions.</li>
<li>I didn't deal with memory at all. No borrow checker complaints, none of that.
It didn't take me any extra time to "think about ownership and borrowing."</li>
<li>It didn't take me that long to write this. You can actually see a vestigial
thing of how it was implemented, can you spot the <code>dbg!</code> statement? I wrote
each part in sequence, printing out the information to manually validate that
it looked correct, and then eventually adding the "write new file" and
"delete old file" stuff when I was sure the process would complete correctly.
Lots of copy/paste there too, as you can see. I don't care. I'm never using
this code again now that the migration is done. That's also why I didn't try
to do some sort of struct and "deserialize" the contents, instead just
parsing it out like I'd never heard of the term "parsing" before. It's fine.</li>
</ol>
<p>So yeah, I had too many type problems in my 'real' JavaScript program, and no
type problems in my script-y Rust. Kinda funny.</p>
<p>I guess I have ended up making this a post about Rust anyway, even though I
didn't think I had much to say when I started. Now that Rust has grown so much,
people who love it are using it for all kinds of things, and sometimes that
seems to piss some people off. "You can't use a systems language for scripting
tasks, nooo!!" they cry. If you had asked me ten years ago if I should write web
backends in Rust or replace shell scripts with Rust, I'd have said the same
thing. But now that I've used Rust enough, a lot of those sort of generic
complaints about Rust don't speak to my actual experience: I had a problem, I
wrote ~70 lines of code in not very much time, problem solved. Seems like Rust
worked just as well as anything else would have. Yes, it's true that maybe if
you've just picked up the language, you'd have written that code much more
slowly than in another language you're more familiar with, or maybe you'd have
hit some sort of error that was confusing and took up a bunch of time, just like
I did with my TypeScript. But just because you may have those problems, doesn't
mean that everyone does. And, it also means that in the future, if you stick with
it, you won't have those problems either! Or at least, it's possible not to. Not
every tool clicks with every person.</p>
<p>Here's to another ten years.</p></content:encoded>
</item>
<item>
<title>Are out parameters idiomatic in Rust?</title>
<link>https://steveklabnik.com/writing/are-out-parameters-idiomatic-in-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/are-out-parameters-idiomatic-in-rust/</guid>
<pubDate>Fri, 16 Oct 2020 00:00:00 GMT</pubDate>
<content:encoded><p>There is a pretty straightforward design pattern called "out parameters" that you may have heard of before. In short, the question is, does your function wear pants... err, I mean, does your function modify data like this:</p>
<pre><code>fn foo() -&gt; i32 {
// body elided
}
let x = foo();
</code></pre>
<p>or like this:</p>
<pre><code>fn foo(out: &amp;mut i32) {
// body elided
}
let mut x = 0;
foo(&amp;mut x);
</code></pre>
<p>That is, does your function return a value, or modify a parameter?</p>
<p>There are a number of different reasons you may or may not use one or the other of these techniques in different languages, and they often depend on the semantics of the language itself.</p>
<p>In Rust, you almost always want to return a value from a function, that is, prefer option one over option two. There are a few reasons for this.</p>
<p>The first is that it's generally semantically more accurate. Your function is producing a value of some kind, so it should, well, produce a value. Additionally, with Rust's relatively rich data type support, it's easy to do things like return a tuple if you need to return more than one thing.</p>
<p>The second is that, in some languages and with some compilers, the out parameter is a performance optimization that you as a programmer do by hand. Imagine if I wasn't returning an <code>i32</code>, which is just a few bytes. The most straightforward way of implementing the function in assembly is to copy the return value from the function to its parent's stack. This also may be required by your ABI. Now, I am not an expert here, so I am not 100% sure of the exact current state of things, but at least in theory, optimizations can be done by the compiler. This is a pretty well-known optimization, see <a href="https://en.wikipedia.org/wiki/Copy_elision">Wikipedia for more</a>. But, I think that we do already do this today in some cases, if I compile both of these functions:</p>
<pre><code>fn returns_huge(out: &amp;mut [u64; 1024]) {
*out = [0; 1024];
}
fn returns_huge() -&gt; [u64; 1024] {
[0; 1024]
}
</code></pre>
<p>I get identical assembly:</p>
<pre><code>playground::returns_huge: # @playground::returns_huge
# %bb.0:
movl $8192, %edx # imm = 0x2000
xorl %esi, %esi
jmpq *memset@GOTPCREL(%rip) # TAILCALL
# -- End function
</code></pre>
<p>However, it is possible that this breaks for other examples, and so you may, may, may want to move to an out parameter if you see extraneous copies appear in your performance profiling.</p>
<p>C++ guarantees this optimization in certain circumstances; it would be really nice for Rust to do so as well. My understanding is that's the intention.</p>
<h2>When you should use out parameters</h2>
<p>There is one case where it is idiomatic to use an out parameter in Rust though. That's when you are writing into some kind of buffer, and you want to be able to allow your caller to provide the buffer to you, rather than making a new one. This would allow the caller to re-use the same buffer for multiple operations.</p>
<p>(I say "buffer" here but really, this generalizes to any sort of grow-able container, to be clear. I am using "buffer" in a loose sense here.)</p>
<p>You can see this play out in the Rust standard library. <code>std::io::Read</code> looks like this:</p>
<pre><code>pub trait Read {
fn read(&amp;mut self, buf: &amp;mut [u8]) -&gt; Result&lt;usize&gt;;
}
</code></pre>
<p>This accepts a <code>&amp;mut [u8]</code> rather than returning a new buffer. This allows for significant savings by re-using a buffer. For example, in <a href="https://doc.rust-lang.org/stable/book/ch02-00-guessing-game-tutorial.html">Chapter 2 of the book</a>, we have this example:</p>
<pre><code>use rand::Rng;
use std::cmp::Ordering;
use std::io;
fn main() {
println!("Guess the number!");
let secret_number = rand::thread_rng().gen_range(1, 101);
loop {
println!("Please input your guess.");
let mut guess = String::new();
io::stdin()
.read_line(&amp;mut guess)
.expect("Failed to read line");
let guess: u32 = match guess.trim().parse() {
Ok(num) =&gt; num,
Err(_) =&gt; continue,
};
println!("You guessed: {}", guess);
match guess.cmp(&amp;secret_number) {
Ordering::Less =&gt; println!("Too small!"),
Ordering::Greater =&gt; println!("Too big!"),
Ordering::Equal =&gt; {
println!("You win!");
break;
}
}
}
}
</code></pre>
<p>We aren't <em>quite</em> properly re-using the buffer here, but we could modify this to look like this:</p>
<pre><code>use rand::Rng;
use std::cmp::Ordering;
use std::io;
fn main() {
println!("Guess the number!");
let secret_number = rand::thread_rng().gen_range(1, 101);
let mut guess = String::new();
loop {
println!("Please input your guess.");
guess.clear();
io::stdin()
.read_line(&amp;mut guess)
.expect("Failed to read line");
let guess: u32 = match guess.trim().parse() {
Ok(num) =&gt; num,
Err(_) =&gt; continue,
};
println!("You guessed: {}", guess);
match guess.cmp(&amp;secret_number) {
Ordering::Less =&gt; println!("Too small!"),
Ordering::Greater =&gt; println!("Too big!"),
Ordering::Equal =&gt; {
println!("You win!");
break;
}
}
}
}
</code></pre>
<p>Here, we create our string outside of the loop, and then on each iteration of the loop, clear it. Clearing it does not deallocate the backing memory; it only changes the length to zero. When the next line is read in, if it's smaller than the existing size of the string, it won't allocate any new memory. This would not be possible if <code>stdin()</code> returned a new <code>String</code> every time, so even though the out parameter is a bit less nice to use, it can be much faster.</p>
<p>It took 26 releases of Rust for <a href="https://doc.rust-lang.org/stable/std/fs/fn.read.html"><code>std::fs::read</code></a> and <a href="https://doc.rust-lang.org/stable/std/fs/fn.read_to_string.html"><code>std::fs::read_to_string</code></a> to be added for this reason; we eventually did because in many cases, you don't care about re-using the buffer, and these functions are nicer to use, but they weren't included originally because they're pretty simple, and are built on top of the re-usable solutions:</p>
<pre><code>pub fn read_to_string&lt;P: AsRef&lt;Path&gt;&gt;(path: P) -&gt; io::Result&lt;String&gt; {
fn inner(path: &amp;Path) -&gt; io::Result&lt;String&gt; {
let mut file = File::open(path)?;
let mut string = String::with_capacity(initial_buffer_size(&amp;file));
file.read_to_string(&amp;mut string)?;
Ok(string)
}
inner(path.as_ref())
}
</code></pre>
<p>That said, I am exceedingly glad we did add them. It is a much nicer user experience.</p>
<p>(You may be wondering "what's up with that inner function there?"... that maybe deserves another post! It's not neccesary, strictly speaking, but is another case of "the compiler does not do an optimization yet so write some code in a strange way by hand.")</p></content:encoded>
</item>
<item>
<title>The CXX Debate</title>
<link>https://steveklabnik.com/writing/the-cxx-debate/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-cxx-debate/</guid>
<pubDate>Sat, 22 Aug 2020 00:00:00 GMT</pubDate>
<content:encoded><p>Let's talk about safe and unsafe in Rust.</p>
<p>To review, Rust has a concept of "unsafe code." Unsafe Rust is a superset of safe Rust, letting you do some extra things that are dangerous. It's your job as a programmer to verify that your code is playing by the rules, because the compiler cannot check them for you.</p>
<p>Imagine we have a C program, <code>add.c</code> that looks like this:</p>
<pre><code>#include &lt;stdint.h&gt;
uint32_t add_one(uint32_t x) {
return x + 1;
}
</code></pre>
<p>We can call this function from Rust like this:</p>
<pre><code>extern {
fn add_one(x: u32) -&gt; u32;
}
fn main() {
let x = 5;
let y = unsafe { add_one(x) };
println!("{}", y);
}
</code></pre>
<p>We need to invoke <code>unsafe</code> here because any <code>extern</code> function doesn't come from Rust; it comes from another language, and it's possible that that function does something that would cause undefined behavior. We look at <code>add_one</code>, see that it's not violating any rules, and we're good.</p>
<p>However, we know that we can pass any <code>u32</code> to this function, and it's safe. We don't need to example the value of the argument every single time, they're all fine. So we do this:</p>
<pre><code>mod internal {
extern {
pub fn add_one(x: u32) -&gt; u32;
}
}
fn add_one(x: u32) -&gt; u32 {
unsafe {
internal::add_one(x)
}
}
fn main() {
let x = 5;
let y = add_one(x);
println!("{}", y);
}
</code></pre>
<p>This is a classic way that unsafe code gets used in Rust; we have our binding, <code>internal::add_one</code>, and our wrapper, <code>add_one</code>. The wrapper is safe, but uses unsafe internally. This is totally fine! The idea is that this gives us a sort of "seam" in our code. Imagine that <code>add_one</code> would do bad things if we passed it a number less than five. We could use our wrapper to check this:</p>
<pre><code>// we would use a real error type in real code but we're using () here because we don't
// actually care about the error handling here.
fn add_one(x: u32) -&gt; Result&lt;u32, ()&gt; {
if x &lt; 5 {
return Err(());
}
unsafe {
Ok(internal::add_one(x))
}
}
</code></pre>
<p>But, <code>add_one</code> doesn't need to do this. Every <code>u32</code> we pass is totally fine.</p>
<p>Nothing so far is controversial here, this is just background and setup. The purpose of these examples is for you to read and ask yourself, "Is this okay? How do I feel about this?"</p>
<p>Somewhere after this point, opinions start to diverge.</p>
<hr />
<p>Our C library is growing new functionality! Here it is:</p>
<pre><code>#include &lt;stdint.h&gt;
uint32_t add_one(uint32_t x) {
return x + 1;
}
uint32_t add_two(uint32_t x) {
return x + 2;
}
</code></pre>
<p>We can now add two, not only one. Amazing, I know. Let's write another wrapper:</p>
<pre><code>mod internal {
extern {
pub fn add_one(x: u32) -&gt; u32;
pub fn add_two(x: u32) -&gt; u32;
}
}
fn add_one(x: u32) -&gt; u32 {
unsafe {
internal::add_one(x)
}
}
fn add_two(x: u32) -&gt; u32 {
unsafe {
internal::add_two(x)
}
}
fn main() {
let x = 5;
let y = add_one(x);
let z = add_two(x);
println!("{}, {}", y, z);
}
</code></pre>
<p>Two functions, both have an unsafe binding, and a safe wrapper.</p>
<p>Is this okay? How do you feel about this?</p>
<p>Now, imagine we had 100 of these functions. Writing this all out ends up being a massive amount of error-prone boilerplate. Heck, I almost forgot to switch to <code>internal::add_two</code> after I copy/pasted the new function there. Rust has a tool to abstract over code like this: macros! So let's refactor our code with a macro:</p>
<pre><code>macro_rules! call_c {
// implementation elided
}
call_c! {
fn add_one(x: u32) -&gt; u32;
fn add_two(x: u32) -&gt; u32;
}
fn main() {
let x = 5;
let y = add_one(x);
let z = add_two(x);
println!("{}, {}", y, z);
}
</code></pre>
<p>I am not showing you the macro because the exact contents are gnarly and immaterial to the point. I also think that maybe this needs to be a procedural macro not a <code>macro_rules</code> macro but again, those details don't really matter. The point is, we have a macro, it generates the exact same structure that we had before.</p>
<p>Is this okay? How do you feel about this?</p>
<hr />
<p>This wrapper is also known as <a href="https://github.com/dtolnay/cxx">CXX</a>. The syntax is slightly different, but conceptually, this is what it is doing. The reason it is controversial is that some people do not in fact feel that the above macro is okay. The reason that they feel it is not okay is that you do not write <code>unsafe</code> by hand, yet, if you're not careful, things can go wrong. I think this is a totally understandable and reasonable reaction, but I also don't think that it makes a ton of sense once you actually dig into the details.</p>
<p>We actually don't need C to think about this problem, we can do this with unsafe Rust purely. And I think that using unsafe Rust makes it a bit more clear, so let's talk about this in some more depth, but purely with Rust.</p>
<p>The <code>unsafe</code> keyword has two, complimentary uses. It can be used to require your caller to check invariants, and it can be used to tell the compiler "I have checked the invariants." In other words:</p>
<pre><code>// I am leaving off parameters and return types because it doesn't matter, but real code
// would of course have them.
// When calling foo, please check some invariants.
unsafe fn foo() {
// ...
}
// I have checked the invaraints, and this is okay to do.
unsafe {
foo();
}
</code></pre>
<p>The compiler forces these combinations of things; using a feature that generates a requirement to check something will require you to also declare that you've checked it. These requirements bubble up, in other words, we can also pass that obligation to our callers:</p>
<pre><code>unsafe fn foo() {
// ...
}
fn bar1() {
// I have checked the invariants, and this is okay to do
unsafe {
foo();
}
}
// I have not checked the invariants, so since I'm calling foo, you must check
// the invariants
unsafe fn bar2() {
foo();
}
</code></pre>
<p>Rust's built-in FFI works like <code>bar2</code>; if the contents of <code>foo</code> are written in another language, it has to be unsafe. This means that, even if it is perfectly safe to call <code>foo</code> with any possible arguments, or in any circumstances, we still need to write a wrapper.</p>
<p>CXX's take on FFI is that it works like <code>bar1</code>; by using the library we are saying "hey, I know that <code>foo</code> is safe in all circumstances, so please generate the wrapper for me." In other words, in some sense, it pushes the unsafe block across the other side of the boundary, because that's where it actually belongs in this case.</p>
<hr />
<p>This has generated a lot of feelings on both sides. The people who don't like it are upset that the letters <code>['u', 'n', 's', 'a', 'f', 'e']</code> are not present, and feel that this makes it quite easy to make a mistake and accidentally use the macro on some code that is not actually safe to call in all circumstances. The people who do like it argue that when you have a large number of calls that are actually safe, using a macro to abstract them away is fine, like you'd abstract away any other boilerplate. Sometimes, repeating the same thing over and over again makes mistakes more likely, not less.</p>
<p>In the meantime, <a href="https://www.reddit.com/r/rust/comments/ielvxu/the_cxx_debate/g2i1r7a/?context=3">dtolnay has decided</a> to change the macro and re-add the <code>unsafe</code> token, which helps assuage the fears of the people who are upset. But it also centralizes these sorts of uses of <code>unsafe</code>, which helps achieve the goals of the library in the first place. I am continually impressed by dtolnay's ability to design libraries, and hope he keeps writing Rust code forever.</p>
<p>Anyway, if you were wondering what this all was about, that's my take on it. I personally feel that CXX is fine the way that it is, but I'm also happy with this compromise, and hope that others are too.</p></content:encoded>
</item>
<item>
<title>Today is my first day at Oxide Computer Company</title>
<link>https://steveklabnik.com/writing/today-is-my-first-day-at-oxide-computer-company/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/today-is-my-first-day-at-oxide-computer-company/</guid>
<pubDate>Mon, 15 Jun 2020 00:00:00 GMT</pubDate>
<content:encoded><p><img src="/img/2020-06-15/oxide.png" alt="Oxide Logo" /></p>
<p>Today is my first day at <a href="https://oxide.computer/">Oxide Computer Company</a>, where I will be doing engineering work.</p>
<hr />
<p>When Oxide introduced themselves to the world, I saw a lot of folks who were pretty confused. At its core, what Oxide is doing is very straightforward: you give Oxide money, and Oxide gives you a computer. This company sells goods, for money. That raises the issue of what kind of computer, and why would you want one?</p>
<p>In the late 90s and early 2000s, if you were running a web service of some kind, you'd have a server sitting in a rack in a data center somewhere. Got a lot of traffic? You'd buy a bigger, beefier server. This was the era of "vertical scaling." Servers were big, expensive, and hard to manage, so you didn't buy more servers, you bought bigger servers.</p>
<p>That started changing. In 2006, you saw a bunch of articles like <a href="https://www.networkworld.com/article/2304459/google-builds-own-servers-for-efficiency.html">this one</a>:</p>
<blockquote>
<p>Google, typically tight-lipped about the technology behind its data centers, builds its own servers to save costs and because standard products don't exactly meet its needs, Google's senior vice president of operations said on Thursday.</p>
</blockquote>
<p>This era saw companies like Google start to reject the notion of "Server grade" hardware, and rely purely on cheaper, consumer-grade hardware. As the quote said, this required building a bunch of software that would make the cluster of servers more reliable.</p>
<p>But eventually, there were downsides to this approach, as well. As these companies grew, so did a new generation of companies building "the cloud" as we know it today. And once you start managing that many computers, possibilities open up to gain certain efficiencies and cut costs. Years later, we see stuff <a href="https://www.geekwire.com/2017/amazon-web-services-secret-weapon-custom-made-hardware-network/">like this</a>:</p>
<blockquote>
<p>It’s not unusual for internet and software giants to design and even make their own hardware, to increase efficiency and build a competitive advantage. Google custom-designs its own servers, filling them with millions of chips from Intel, and announced in May that it has designed its own application-specific integrated circuit (ASIC) for use on neural networks. Facebook uses its own switches in its data centers. But market-leading public-cloud company Amazon Web Services may have gone farthest down this path — designing not only its own routers, chips, storage servers and compute servers but also its own high-speed network.</p>
</blockquote>
<p>The cloud was printing money, and folks were running into scaling issues. The solution was pretty straightforward: hire some people to build amazing infrastructure. Why not just buy it? Well, it simply didn't exist for purchase in the first place. The current crop of computer companies really grew up in the vertical scaling era. But beyond that, you also have to remember that this stuff was pretty <em>new</em> at that time. The Cloud was still cutting-edge, and so the requirements weren't really known. There was a lot of R&amp;D work to figure out what things would actually work in these environments. It turns out that a lot of it was not just hardware, but the software too. There is a ton of junk and waste in off-the-shelf server systems today, in all parts of the system.</p>
<p>Because these developments were proprietary, it also meant that if you weren't willing to build your own hardware, the only way you could get access to this new tech was by renting it. The public cloud's usage-based billing model is pretty standard, and pretty lucrative. Custom hardware helped dig an even deeper moat around cloud providers. If you want to actually own computers, your options are to use previous-generation tech, or hire a team and try to catch up to the big players. So the world kind of split in two: the hyperscalers and everyone else.</p>
<p>There is a <strong>ton</strong> more to say here, but this is the short of it: what Oxide is doing is building computers that are suitable for hyperscalers, but selling them, rather than building another public cloud. These computers will have hardware and software designed together to create excellent systems. Or I should say "will be selling them," it is still early days.</p>
<hr />
<p>I do not come from a hardware background. But there's a lot more to a system than hardware. And as you might guess from the name of the company, Oxide will be building a lot of software in Rust. I also have known two of the three founders of the company for a while, and they are folks I respect and whose work I admire.</p>
<p>I am also looking forward to doing engineering work again. I still love doing product work, but I think I prefer it in a small company setting. In a new company, engineers can still have a lot of ownership over product, so I won't be giving that up completely. But I'm excited to finally have a job where I get to write Rust code as a direct responsibility. It's not like I've not written any Rust over the past few years, but I'm looking forward to writing more of it, regardless.</p>
<p>Finally, I'm excited to continue to help others as well; Oxide cannot only hire folks who are already Rust experts. There just aren't enough of them yet, and there's folks who do fantastic work who don't yet write Rust. I'll be helping to bridge that gap.</p>
<p>There is so much work to do. It's almost 9am. Time to get started.</p></content:encoded>
</item>
<item>
<title>The next iteration of my blogs</title>
<link>https://steveklabnik.com/writing/the-next-iteration-of-my-blogs/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-next-iteration-of-my-blogs/</guid>
<pubDate>Mon, 08 Jun 2020 00:00:00 GMT</pubDate>
<content:encoded><p>I've had a few different blogs over the years. Every so often, I change things. One of the nice things about having your own site is that you can do whatever you want! So, I'm folding my old blogs' content into my main website. I'm sick of adding new subdomains, and it all feels better in one place these days. The old URLs will redirect to posts on steveklabnik.com, but I wanted to record what they looked like at the time.</p>
<hr />
<p>My oldest blog that's survived is <a href="https://web.archive.org/web/20200519230707/http://blog.steveklabnik.com/">Literate Programming</a>. It looked like this:</p>
<p><img src="/img/2020-06-08/Untitled.png" alt="untitled" />
<img src="/img/2020-06-08/Untitled%201.png" alt="untitled 01" />
<img src="/img/2020-06-08/Untitled%202.png" alt="untitled 02" /></p>
<p>This was my own static site generator! I loved making that table of contents stuff over on the left.</p>
<hr />
<p>Then, there's <a href="https://web.archive.org/web/20200604152340/https://words.steveklabnik.com/">words</a>. This used <a href="https://svbtle.com/">Svbtle</a>, which is a great platform. I liked the Meeple, and the red, and the Kudos button that made random internet people <em>so mad</em> for some reason. Anyway, it looked like this:</p>
<p><img src="/img/2020-06-08/Untitled%203.png" alt="untitled 03" />
<img src="/img/2020-06-08/Untitled%204.png" alt="untitled 04" />
<img src="/img/2020-06-08/Untitled%205.png" alt="untitled 05" /></p>
<p>The kudos count does nothing, other than record that you did it. The style was really nice, it was easy to write Markdown. Then Medium got popular. Oh well.</p>
<hr />
<p>Now, I'm writing this in <a href="https://www.notion.so/">Notion</a>. I really, really like writing in Notion. Someone put together <a href="https://github.com/ijjk/notion-blog">some code</a> to use Notion like a headless CMS. Exactly what I wanted. I've modified it a bunch, I'm using I'm using <a href="https://concrete.style/">concrete.css</a> as the base of my look, for example. They're using <a href="https://nextjs.org/">Next.js</a> and <a href="https://vercel.com/">Vercel</a>. I've been a fan of those folks (and pretty much everyone in the JAMStack space, to be honest) for a while, so I'm giving that a go.</p>
<p>I think it's going pretty good so far.</p></content:encoded>
</item>
<item>
<title>How often does Rust change?</title>
<link>https://steveklabnik.com/writing/how-often-does-rust-change/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/how-often-does-rust-change/</guid>
<pubDate>Sat, 11 Apr 2020 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve been thinking a lot lately about how often Rust changes. There are some people that assert that Rust stays fairly static these days, and there are some people who say Rust is still changing far too much. In this blog post, I want to make a <em>data driven</em> analysis of this question. First, I will present my hypothesis. Next, my methodology. Then, we’ll talk about the results. Afterward, we’ll discuss possible methodological issues, and possible directions for further research.</p>
<p>As <a href="https://twitter.com/adam_chal/status/1248705340163620866">said on twitter</a>:</p>
<p><img src="https://svbtleusercontent.com/66pLpg8EDEH5dvt5BGQ7LH0xspap_small.jpg" alt="https://svbtleusercontent.com/66pLpg8EDEH5dvt5BGQ7LH0xspap_small.jpg" /></p>
<h2>The hypothesis</h2>
<p>My personal take on Rust changing: to me, it feels like we had more change in the past than we have today. The changes are also getting more minor. This will be my hypothesis.</p>
<h2>The methodology</h2>
<p>There are tons of different ways that we could measure the question “how often does Rust change?” So before I started collecting data, I had to decide some details.</p>
<p>I realized that my premise here is about how people <em>feel</em> about Rust. So, how do they tend to understand how Rust changes? Well, the primary way that we communicate change to the Rust community is through release blog posts. So I decided to start there.</p>
<p>What I did was, I opened up every release blog post for Rust, from 1.0.0 to 1.42.0, in a new tab. For fun, I also opened the release notes for 1.43, which will come out soon.</p>
<p>I then looked at every single change announced, and categorized them as follows:</p>
<ul>
<li>adds syntax?</li>
<li>language</li>
<li>standard library</li>
<li>toolchain</li>
<li>major</li>
<li>medium</li>
<li>minor</li>
</ul>
<p>I initially included “deprecation” and “soundness” as categories too, but I ended up not including them. More on that later.</p>
<p>From here, I had to decide what these categories <em>meant</em>. What standard would I be using here? Here’s the easier ones:</p>
<p><em>language</em> changes mean some sort of modification to the language definition. These are additive changes, thanks to our stability policy, but they’re new additions.</p>
<p><em>standard library</em> means a change to the standard library, primarily, new functions, new types, new methods, stuff like that. This one is pretty simple as a criteria but there’s some small interesting methodological issues we’ll get into later.</p>
<p><em>toolchain</em> changes related to <code>cargo</code>, <code>rustup</code>, new compiler target support, stuff like that. Not part of the language proper, but part of the Rust distribution and things Rust programmers would use.</p>
<p>Now, for the harder ones:</p>
<p><em>major</em>/<em>medium</em>/<em>minor</em> changes are how big of a change I think that the change is. There’s a few interesting parts to this. First of all, this is, of course, very subjective. I also decided to try and evaluate these as of today, that is, there are some changes we maybe thought were major that aren’t used all that often in practice today, so I categorized those as <em>medium</em> rather than <em>major</em>. This felt more consistent to me than trying to remember how I felt at the time.</p>
<p><em>adds syntax</em> changes <em>sound</em> easy, but are actually <em>really</em> tricky. For example, consider a feature like “<a href="https://github.com/rust-lang/rfcs/blob/master/text/1492-dotdot-in-patterns.md">dotdot in patterns</a>”. This does change the grammar, for example, so you could argue that it adds syntax. But as a programmer, I don’t <em>really</em> care about the grammar. The summary of this feature is “Permit the .. pattern fragment in more contexts.” It goes on to say:</p>
<blockquote>
<p>This RFC is intended to “complete” the feature and make it work in all possible list contexts, making the language a bit more convenient and consistent.</p>
</blockquote>
<p>I think these kinds of changes are actually very core to this idea, and so I decided to categorize them according to my opinion: this is <em>not</em> a new syntax change. You already had to know about <code>..</code>.</p>
<p>I believe this may be the most controversial part of my analysis. More on this later, of course.</p>
<p>Okay, so that’s the stuff I covered. But there’s also stuff I didn’t cover. I was doing enough work and had to draw the line somewhere. I left this stuff out:</p>
<ul>
<li>compiler speedups. This is interesting with numbers, but that means actually compiling stuff, and I don’t have time for that. This is a whole study on its own.</li>
<li>Documentation work. This tends to be not tracked as new features, but sometimes it appears in release notes, for bigger things. Better to just leave it out. I also don’t think that it affects my hypothesis at all.</li>
<li>Library types implementing new traits. These can be hard to number, especially with generics involved. I decided that it would be better to leave them off.</li>
<li>compiler internal news. We sometimes talk about things like “MIR exists now!” or “we ported the build system from make to cargo!” but similar to documentation, we talk about it very little. It’s also not a change that affects end users, or rather, it affects them more directly by stuff that is counted. MIR enabled NLL, and NLL was tracked as a language feature.</li>
</ul>
<h2>The results &amp; analysis</h2>
<p>I am not as good with Google Sheets as I thought, so I asked for help. Big thanks to <a href="https://twitter.com/ManishEarth">Manish</a> for helping me get this together. I should note that he set me up, but then I tweaked a ton of stuff, so any mistakes are mine, not his. For example, I got a little lazy and didn’t realize that the colors aren’t the same across the various charts. That’s my bad.</p>
<p>Here’s some quick totals of how many changes and of each kind I found, with some sparklines. You can click on any of the images in this post to enlarge them:</p>
<p><img src="https://svbtleusercontent.com/i84Jyi8Ku2vJ2VRGfYmyBV0xspap_small.png" alt="https://svbtleusercontent.com/i84Jyi8Ku2vJ2VRGfYmyBV0xspap_small.png" /></p>
<p>And here’s some charts, per release:</p>
<p><img src="https://svbtleusercontent.com/oZTrBYjy26f5AKURN7z3db0xspap_small.png" alt="https://svbtleusercontent.com/oZTrBYjy26f5AKURN7z3db0xspap_small.png" /></p>
<p>First of all, it is fairly clear that, at least numerically speaking, the standard library is the part of Rust that changes most often. It’s a clear outlier by volume. I find this result kind of hilarious, because Rust is well-known for having a small standard library. We’ll talk more about why I think this is in the “Issues and further research” section below.</p>
<p>Let’s look at the chart without the standard library:</p>
<p><img src="https://svbtleusercontent.com/qXu9v6Li5qzg8Ag6HV1j8p0xspap_small.png" alt="https://svbtleusercontent.com/qXu9v6Li5qzg8Ag6HV1j8p0xspap_small.png" /></p>
<p>You can see there’s a <em>ton</em> of toolchain changes early on. We had a lot of work to do early on, and so made a ton of changes! It’s calmed down in recent times, but toolchain changes happen almost twice as often as language changes. I should also note that this reduction in toolchain changes may be due to methodology; this is talked about later in the post.</p>
<p>So, the core question here: it looks like the language is changing <em>more</em>, recently, rather than less. I think it’s pretty clear that this graph is <em>not</em> exactly what I would imagine, that is, I expected a nice gentle curve down, but that’s not what has happened. I want to dig into this a bit more, but first, let’s look at some other graphs:</p>
<p><img src="https://svbtleusercontent.com/3Xa9rr1SM3atm69gAohMkM0xspap_small.png" alt="https://svbtleusercontent.com/3Xa9rr1SM3atm69gAohMkM0xspap_small.png" /></p>
<p>Changes that add syntax are a subset of language changes. There hasn’t been a whole ton of this, overall. Rust 2018 was the large bump. Half of our releases do not add syntax, though 10 out of 43 did introduce language changes. Our first 29 releases, skipping 1.26, had around one or two changes on average, but every since, it’s been between three and four. I believe this has to do with my methodology, but at the same time, this pretty resoundingly refutes my hypothesis. Very interesting!</p>
<p>Here’s major/medium/minor:</p>
<p><img src="https://svbtleusercontent.com/cm4iWVjB9XZw4qp3c5b94o0xspap_small.png" alt="https://svbtleusercontent.com/cm4iWVjB9XZw4qp3c5b94o0xspap_small.png" /></p>
<p>There’s a peak around Rust 2018, and one from 1.12 to 1.19, but otherwise, it’s been pretty steady lately, in terms of overall changes. If we look at just major changes:</p>
<p><img src="https://svbtleusercontent.com/N3Dud8Du3fUEC8LL8a9Fn0xspap_small.png" alt="https://svbtleusercontent.com/N3Dud8Du3fUEC8LL8a9Fn0xspap_small.png" /></p>
<p>Rust 2018 had a huge bump. After 1.0, and after Rust 2018, it calmed. I think this chart is <em>very</em> interesting, and demonstrates the 3 year edition cycle well. We ship an edition, stuff calms down, things slowly build back up, we ship an edition, they calm down again.</p>
<p>So, why was my hypothesis wrong? I think this conclusion is <em>fascinating</em>. I think part of it has to do with my methodology, and that maybe, that difference explains some of the ways people feel about Rust. You see, even if we release extremely often, and a lot of releases don’t have a <em>ton</em> of changes in them (as you can see, it’s vaguely around 8 or 10, when you exclude the standard library, on average), the very fact that we do release often and have a decent, but fairly small amount of change means that things get into the release notes that may not if we had the exact same amount of change, but did yearly releases.</p>
<p>So for example, in 2019, we released Rust 1.32 to 1.40. That involved 35 language changes. Would I have included, say, “you can now use <code>#[repr(align(N))]</code> on enums” if I had been writing a post about all of these changes for the whole year? Probably not. But because there were only eight total changes in that release, with half of those being language changes, it made sense to include in the 1.37 release post.</p>
<p>Does this mean that folks who read the release posts and think Rust changes a lot are wrong? No, it does not. For those of us in the thick of it, a lot of the smaller changes kind of blur into the background. I actually initially typed “<code>#[repr(N)]</code> on enums” when writing the above sentence, because this change isn’t relevant to me or my work, and is pretty small, and makes things more orthogonal, and so it is just kind of vague background noise to me. But to someone who isn’t as in-tune with the language, it’s harder to know what’s big and what’s small. The endless release posts with tons of stuff in them makes it <em>feel</em> like a lot is happening. But the same amount of stuff may be happening in other languages too, you just don’t see them in release posts, because they only put out one once a year. You both see them less often, as well as see less in them.</p>
<p>I don’t think this means that the Rust project should change its release schedule, and I’m not entirely sure if this means we should change how we write release posts. However, maybe there’s a better way to surface “these are big changes” vs “this is a small thing,” or something like that. I’m not sure. But I do think this helps me understand the disconnect a lot better.</p>
<h2>Issues and further research</h2>
<p>There are some big issues with this methodology. No analysis is perfect, but I want to be fair, so I’m calling out all of the stuff I can think of, at least.</p>
<p>The first of which is, this is inherently subjective. There’s different degrees of subjectivity; for example, “total changes in the notes” is fairly objective. But not completely; folks have to decide what goes into the release notes themselves. So there’s already a layer of filtering going on, before I even started looking at the data. Doing this analysis took me two days of real time, but only a few hours of actual work. Something that analyses the git repo in an automated fashion may see other results. I argue that this level of subjectivity is okay, because we’re also testing something subjective. And I think I designed my methodology in a way that captures this appropriately. On some level, it doesn’t matter if the “real” answer is that Rust rarely changes: folks still get this info by reading the posts, and so I don’t think saying that “oh, yeah you feel that way, but this graph says your feelings are wrong” is really going to help this debate.</p>
<p>The second problem expands on this. We sort of had three eras of release note writing: the first few posts were written by Aaron and Niko. As of Rust 1.4, I took over, and wrote <em>almost</em> every post until 1.33. I then stepped back for a while, though I did help write the 1.42 post. The release team wrote 1.34 onward, and it was more of a collaborative effort than previous posts. This means that the person doing the filter from release notes -&gt; blog post changed over time. This can mess with the numbers. For example, there are some blog posts where I would have maybe omitted a small family of features, but the folks writing that post included them all, individually. This may explain the uptick in language changes lately, and why many of them were not rated as “major.” Additionally, the people who did the filter from “list of PRs” to “release notes” <em>also</em> changed over time, so there’s some degree of filtering change there, as well.</p>
<p>To complicate things further, starting with Rust 1.34, Cargo lost its section in the release blog posts. Big changes in Cargo still made it into the text, but the previous posts had more robust Cargo sections, and so I feel like post-1.34, Cargo was under-counted a bit. Cargo changes are toolchain changes, and so the recent calm-ness of the toolchain numbers may be due to this. This still is okay, because again, this analysis is about folks reading the release notes, but it’s worth calling out. Likewise, I think the standards for writing the library stabilizations changed a bit over time too; it’s gone from more to less comprehensive and back again a few times.</p>
<p>This was mentioned above a bit, but for completeness, the structure of Rust releases means there’s a variable criteria for what makes it into a release post. If it’s a smaller release, smaller changes may get featured, whereas if those same feature were in a bigger release, they may not have been. I think this aspect is a significant factor in the rejection of my hypothesis, as I mentioned above.</p>
<p>I think there’s interesting future research to be done here. I initially tried to track deprecations and soundness fixes, but soundness fixes happened so little it wasn’t worth talking about. There were 7 in those 42 releases. This is another weakness / area for future research, because I did not include point releases, only 1.x.0 releases. This would have added a few more soundness fixes, but it also would have added a lot of random noise where not much happened in the graphs, so that’s why I left those out. They also don’t happen on the same regular schedule, so time would get a bit funny… anyway. Deprecations was also just hard to track because there weren’t a lot of them, and sometimes it was hard to say if something was deprecated because of a soundness issue vs other reasons.</p>
<p>Is my critera for adding syntax correct? I think a reasonable person could say “no.” If you have something that’s special cased, and you’re making it more general, there’s a good argument that this is <em>removing</em> a restriction, rather than adding a feature. But reasonable folks could also say you’ve added something. I don’t think that this part is core to my analysis, and so I think it’s fine that I’ve made this decision, but you may not care for this section of results if you disagree.</p>
<p>This analysis doesn’t get into something that I think is a huge deal: the ecosystem vs the language itself. I only track the Rust distribution itself here, but most Rust programmers use many, many ecosystem libraries. When people talk about churn in Rust, do they really mean the ecosystem, not the language? In some sense, doing so is correct: Rust having a small standard library means that you have to use external packages a lot. If those churn a lot, is it any better than churn in the language itself?</p>
<p><em>Does</em> Rust have a small standard library? There were 962 changes in 42 releases, that’s almost 23 changes per release. A lot of them are small, but still. Maybe Rust has a small, but deep standard library? Due to coherence, I would maybe think this is true. Just how big are standard libraries anyway? I’ve only seen discussion of this concept in terms of design philosophy. I’ve never seen a numerical analysis of such. Is it out there? Do you know of one? I’d love to hear about it!</p>
<p>I think there’s a number of reasons that the standard library changes dominate the total change rate. Primarily, this section of the blog post <em>tends</em> to be the most complete, that is, out of the total amount of changes, more standard library changes make it into blog posts than other kinds of changes. Why? Well, it’s really easy to measure: count up stuff stabilized and put it in a big list in the post. It’s also about how these changes are measured; when a method is added to every number type, that’s <code>u8</code> + <code>u16</code> + <code>u32</code> + <code>u64</code> + <code>u128</code> = <code>5</code> changes, even if conceptually it’s one change, in a different sense.</p>
<p>This leads to another question about those standard library changes: what is up with that weird peak around Rust 1.33? Well, we added the <code>const fn</code> feature in Rust 1.31. After that landed, we could <code>const</code>-ify a number of functions in the standard library. The initial huge bulk of changes here landed in 1.33. It had 138 changes, but only 13 were not “this function is now <code>const</code>.” And the feature continues to drive change outside of 1.33; the initial landing of <code>const fn</code> in 1.31 was very limited, and as it expands, more functions that already existed can be made <code>const</code>.</p>
<p>In conclusion, even though I still believe that Rust has slowed down its rate of change a lot, I think that it makes total sense that not everyone agrees with me. By some metrics, I am just flat-out wrong.</p></content:encoded>
</item>
<item>
<title>"C is how the computer works" is a dangerous mindset for C programmers</title>
<link>https://steveklabnik.com/writing/c-is-how-the-computer-works-is-a-dangerous-mindset-for-c-programmers/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/c-is-how-the-computer-works-is-a-dangerous-mindset-for-c-programmers/</guid>
<pubDate>Tue, 31 Mar 2020 00:00:00 GMT</pubDate>
<content:encoded><p>A long time ago, I wrote <a href="https://words.steveklabnik.com/should-you-learn-c-to-learn-how-the-computer-works">“Should you learn C to “learn how the computer works”?"</a>. This was part 1 in a three part series. A year later, I got around to part 2, <a href="https://words.steveklabnik.com/c-is-not-how-the-computer-works-can-lead-to-inefficient-code">““C is not how the computer works” can lead to inefficient code"</a>.</p>
<p>It’s been five more months. I don’t think I really want to write out the third part; I expected this to be a quick series of blog posts, not something that was on my plate for a year and a half. Here’s the thesis of part 3:</p>
<blockquote>
<p>Part three is going to show what happens if you make a mistake with the ideas from part two. If you incorrectly assume that C’s abstract model maps directly to hardware, you can make mistakes. This is where UB gets dangerous. While you can take advantages of some properties of the machine you’re relying on, you have to know which ones fit within C’s model and which ones don’t.</p>
</blockquote>
<p>I honestly don’t feel like writing more than that, so I’m just gonna leave it there. Basically, the overall thrust of this series has been this: C is not the hardware, it’s an abstract machine. But that machine runs on real hardware, and abstractions are leaky. If you go too far into “purely only the abstract machine,” you may not be able to accomplish your tasks. If you go too far into “C runs directly on the hardware,” you may be surprised when a compiler does something that you didn’t expect.</p></content:encoded>
</item>
<item>
<title>A sad day for Rust</title>
<link>https://steveklabnik.com/writing/a-sad-day-for-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/a-sad-day-for-rust/</guid>
<pubDate>Fri, 17 Jan 2020 00:00:00 GMT</pubDate>
<content:encoded><p><code>actix-web</code> is dead.</p>
<p>This situation is bad, from all sides. When Rust was a tiny, tiny community, I thought to myself, “wow, I wonder how long this will last? Probably 1.0 will kill it.” Sort of playing off of <a href="https://en.wikipedia.org/wiki/Eternal_September">Eternal September</a>, I assumed that over time, the community would grow, and we’d encounter problems.</p>
<p>Today is the first day where I say to myself, okay, has that happened?</p>
<p>This story is not super clear-cut. I’m not going to link to a dozen citations, or try to prove that I’m some sort of neutral party here. I’m going to give you account of this story as I remember it and as I felt it. Because this isn’t really about playing judge. This is about thinking about the future.</p>
<p>It’s been very clear from the beginning that the Rust project saw Rust as more than just the language. The community and the people mattered. From the earliest days, leadership explicitly took the position that it wasn’t just the code, but the people around the project were important. Of course, people are also people, and so this wasn’t perfect; we’ve made several fairly large mis-steps here over the years. But Rust has been an experiment in community building as much as an experiment in language building. Can we reject the idea of a BDFL? Can we include as many people as possible? Can we be welcoming to folks who historically have not had great representation in open source? Can we reject <a href="https://blog.aurynn.com/2015/12/16-contempt-culture">contempt culture</a>? Can we be inclusive of beginners?</p>
<p>Rust has a necessary feature, <code>unsafe</code>, that allows you to escape Rust’s guarantees. This is a really important part of Rust, but also a very dangerous one, hence the name. But one of the things that’s important about it is that, since <code>unsafe</code> means “I have checked this code, and it upholds all of the guarantees the compiler cannot check,” you can build a safe interface with unsafe guts.</p>
<p>Interacting with anything outside of the language, for example, using the operating system to print to the screen, or write a file, has to be <code>unsafe</code>, and so without this property, all programs would be <code>unsafe</code>. This premise is treated skeptically by many, but has generally been demonstrated to have worked out. Several folks are in the process of providing tooling to prove that your <code>unsafe</code> code is correct, and provided proofs for a bunch of the standard library’s <code>unsafe</code> code. It not only found a few places where there were bugs, but also a few places where the restrictions were too tight!</p>
<p>But “generally” is doing a lot of work in that previous sentence. This setup creates a gnawing fear in the back of people’s minds: what if some foundational package uses <code>unsafe</code>, but uses it incorrectly? What happens when this causes problems for every package that uses that package? This relationship between safe and unsafe is also a bit hard to understand, and so, when bugs are found around unsafe, people <em>outside</em> of Rust often use it to suggest that all of Rust is a house of cards. While Rust has “made it” in many senses, it has not quite yet in many others, and so I think this produces a fundamental anxiety in the community.</p>
<p>Speaking of making it, actix-web is a good web framework. It came onto the scene, and people generally liked it. It also performed extremely well. Its score on <a href="https://www.techempower.com/benchmarks/">Techempower</a> in particular was impressive, and got more impressive over time. Its author also works at Microsoft, and suggested that Microsoft is using actix-web in production. This was really exciting to people. This was the second project using Rust at Microsoft, and so this seemed really fantastic.</p>
<p>But then, the anxiety.</p>
<p>Before we get into that, though, there’s also the matter of Reddit. The team has basically rejected Reddit for said social reasons. We provide alternate forums for folks, but Reddit is a huge place. The Rust subreddit has ~87,000 subscribers, partially thanks to this. And, for a while, the Rust reddit was a decent place. I still think it’s better than most reddits, but it’s degraded much more quickly than the community at large. “Why Reddit is like this” is a whole other essay; I think it’s built into reddit’s structure itself. But the point is, while Reddit is not official, and so not linked to by any official resources, it’s still a very large group of people, and so to suggest it’s “not the Rust community” in some way is both true and very not true. For the purposes of this story, I think it has to be included in the broader community, and I think that this situation has brought new questions about the relationship of the project and Reddit, though this is already far too long.</p>
<p>So, someone takes a peek under the covers, and it turns out actix-web is using a <em>lot</em> of unsafe code. That’s not inherently a problem. The problem is, a lot of that unsafe code is not actually needed. This is where things start to go wrong.</p>
<p>The author of actix-web basically says “nah it’s fine.” This is basically the perfect storm of unsafe anxiety: a big, important, visible library, a bunch of <code>unsafe</code>, and an unresponsive author.</p>
<p>Now, it’s important to mention that people were not just complaining: they were writing code. It is easy to frame this as a case of open source entitlement, and it still very may well be, but this isn’t just empty complaints. Patches were included, they just weren’t welcome. And that’s also not inherently wrong either; a project maintainer shouldn’t be required to accept a patch just because it exists…</p>
<p>Reddit whips itself into a frenzy. Lots of ugly things are said. More issues are opened, and closed, and locked. Eventually, the author accepts some code that drops the total unsafe count significantly and has more sound usage of what’s left.</p>
<p>People were not happy about how all this played out, for a variety of reasons and in a variety of ways. I haven’t spoken to actix-web’s author, but I can’t imagine that he was very happy about it either.</p>
<p>And then it happens all over again. More unsafe, more patches rejected, more Reddit, more bad blood. I thought that at this point, the author said he was quitting, but that may be faulty memory, or maybe he changed his mind. Regardless, how responsive would you be, if you were the maintainer, after the shit-show of last time?</p>
<p>People were not happy about how all this played out, for a variety of reasons and in a variety of ways.</p>
<p>All was quiet for a while, and actix-web kept climbing the Techempower benchmarks, reaching the top of most of them by a <em>lot</em>. It is wicked fast.</p>
<p>And then, yesterday, a post titled <a href="https://medium.com/@shnatsel/smoke-testing-rust-http-clients-b8f2ee5db4e6">Smoke-testing Rust HTTP clients</a> gets published. It focuses on a, well, smoke test of Rust HTTP clients, but it also comments a lot on the complexity of each project, and the amount of <code>unsafe</code>. The author found and filed a lot of bugs.</p>
<p>But:</p>
<blockquote>
<p>A quick glance at the dependencies reveals that it relies on actix-service, which underpins all of Actix and has a bespoke and unsound Cell implementation. For example, this method violates memory safety by handing out multiple mutable references to the same data, which can lead to e.g. a use-after-free vulnerability. I have reported the issue to the maintainers, but they have refused to investigate it.</p>
</blockquote>
<p>This causes the now-usual Reddit uproar. It’s extra nasty this time. Some people go far, far, far over the line.</p>
<p>And now the maintainer has quit.</p>
<p>This means, on some level, this situation is over: there will not be a fourth huge actix-web drama. But it’s really left a bad taste in my mouth. It’s unfortunate that a well-known project had soundness issues. It’s unfortunate that the maintainer wasn’t receptive to fixing them. At the same time, I’ve been there: being a maintainer is tough. It’s also unfortunate that there’s this <em>style</em> of response, and kind, and volume. It <em>is</em> true that the situation was dangerous. But did it have to be handled this way? You have to understand a lot of nuance here to even know the main points of the story. One version of this story that will certainly be told is “The Rust community says they’re nice but they will harass you if you use unsafe wrong.” Is that what we want? I, for one, do not. If you’re a part of Rust, you gotta ask yourself: are you happy with this outcome? How did your actions (or lack thereof) contribute to it? Could you have done something better?</p>
<p>I’m not sure where we go from here, and I’m not sure what we could have done to prevent this from happening. But I feel like this is a failure, and it’s set Rust back a bit, and I’m just plain sad.</p>
<p>EPILOGUE</p>
<p>actix-web has new maintainers, and will now live on: https://github.com/actix/actix-web/issues/1289</p></content:encoded>
</item>
<item>
<title>"C is not how the computer works" can lead to inefficient code</title>
<link>https://steveklabnik.com/writing/c-is-not-how-the-computer-works-can-lead-to-inefficient-code/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/c-is-not-how-the-computer-works-can-lead-to-inefficient-code/</guid>
<pubDate>Fri, 25 Oct 2019 00:00:00 GMT</pubDate>
<content:encoded><p>A little over a year ago, I wrote <a href="https://words.steveklabnik.com/should-you-learn-c-to-learn-how-the-computer-works">“Should you learn C to ‘learn how the computer works’”</a>. It was a bit controversial. I had promised two follow-up posts. It’s taken me a year, but here’s the first one.</p>
<p>In that post, I argued that C doesn’t “work like the computer”, but rather, by a concept called the “C abstract machine.” It happens to be close to how computers operate in some sense, but that when you write C, you’re writing for the abstract machine, not the computer itself. Here’s the trick, though: the C abstract machine doesn’t define every single aspect of how computation happens. And so sometimes, you can write code in two different ways, that the C abstract machine says are equivalent, yet are <em>not</em> equivalent in performance. This means that, even if you buy into my original point, you still need to know the boundaries of this definition, and can exploit those boundaries in important ways.</p>
<p>Let’s start with a made-up example, and then we’ll show how this happens in real code. Imagine that we have a very simple programming language, built on an abstract machine. It defines only the semantics for the multiplication of two 32-bit integers. It also ignores things like overflow. You could write a program like “3 * 4” and this machine would say “hey the result is 12.” Let’s also imagine we have a very simple computer: it has a single instruction, “add two numbers together.” This takes one clock cycle to execute. If we wrote a compiler for this machine, and didn’t apply any optimizations, it would execute “4 + 4”, and then take that result and add four to it. We have two add operations, and so our program takes two clock cycles.</p>
<p>Let’s imagine a new version of our computer comes out. It adds a new instruction: multiplication. This also takes one cycle, but can multiply two numbers directly. Our compiler adds a new back-end, and when compiling our “3 * 4” program for this fancy new machine, it emits a single multiplication instruction. Our program now takes one clock cycle.</p>
<p>Now imagine a new version of our compiler. It is able to do “constant folding”, an optimization. Our program does no computation at all, only produces a “12” directly. This now means our program takes zero cycles on both bits of hardware.</p>
<p>Is all of this okay?</p>
<p>Yes! The machine only defines how the <code>*</code> operator works on numbers in a “what number comes out the other side” sense. It doesn’t specify how to actually accomplish this task. Compilers for our little language are free to implement the details in any way they choose, as long as they respect the rules of the abstract machine. Other details are fair game. As long as the program “3 * 4” produces “12”, the rest of the computational process doesn’t actually matter.</p>
<p>This is how you can write “portable” code that is portable in one sense, but not at all in another sense. As long as you respect the rules of the abstract machine, you’ll be guaranteed (we live in a world with no compiler bugs, of course) to get the correct result. If it takes too long, or takes no time at all, that’s not the language’s concern, strictly speaking.</p>
<p>If this idea seems silly to you, well, I can assure you it’s real. For example, let’s consider the actual C language, and some actual hardware, specifically, x86.</p>
<p>Imagine we have an array of arrays, four by four:</p>
<pre><code>0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
</code></pre>
<p>There are two naive ways to process it: row by row:</p>
<pre><code>1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
</code></pre>
<p>or column by column:</p>
<pre><code>1 5 9 13
2 6 10 14
3 7 11 15
4 8 12 16
</code></pre>
<p>Here’s a fun question: is one of these faster than the other?</p>
<p>At this small size, probably not. But there’s a way to do this that’s faster than either of these ways, and that’s called “blocking” or “tiling”. It means that you process a sub-set, or “block”, of the array each time. Something like this:</p>
<pre><code>1 2 5 6
3 4 7 8
9 10 13 14
11 12 15 16
</code></pre>
<p>On x86, this is <em>meaningfully</em> faster in many circumstances. This has to do with the way that the CPU caches information. For more, see <a href="https://software.intel.com/en-us/articles/how-to-use-loop-blocking-to-optimize-memory-use-on-32-bit-intel-architecture">this page from Intel</a>, including a visualization that’s probably better than mine.</p>
<p>This raises an interesting point: is this code portable? Imagine a machine that didn’t have the cache behavior of x86 chips here. This code would run on them, but it may be 14 times slower. Strictly speaking, this program is “portable” in the sense that you’ll get the right computation in the end, but maybe this speed is part of the user requirements for this software to be successful, and so being super slow means that it “doesn’t work” according to those requirements, even though from a programming language point of view, it’s all good.</p>
<p>Because C’s abstract machine is so thin, these kinds of details can really, really matter, as we’ve seen. And this is where the good part of the “C teaches you how the computer works” meme comes in. Because the machine is thinner, you can learn more about the details, and as long as they’re not details that the C abstract machine cares about, exploit them. And there’s a good reason I mentioned an optimizing compiler above; it’s also the job of compiler authors to realize the difference between a given computer and the C abstract machine, and then exploit the difference to make your code go as fast as possible. But this really blurs the line between abstract machine and physical machine. And that’s why we all argue online about this all the time.</p>
<hr />
<p>I have a third post to wrap up this series. I hope it won’t take me another year, but no guarantees…</p></content:encoded>
</item>
<item>
<title>Porting steveklabnik.com to Workers Sites and Zola</title>
<link>https://steveklabnik.com/writing/porting-steveklabnik-com-to-workers-sites-and-zola/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/porting-steveklabnik-com-to-workers-sites-and-zola/</guid>
<pubDate>Thu, 26 Sep 2019 00:00:00 GMT</pubDate>
<content:encoded><p>One fun thing about having a personal domain is that you can keep the contents the same forever, but take a few days every so often to port it to some new stack you want to try out.</p>
<p>A while back, I had my site on <a href="https://pages.github.com/">GitHub Pages</a> using <a href="https://jekyllrb.com/">Jekyll</a>. This has long been the sort of default, go-to stack for me. At some point, I decided to simplify even further, and wrote a small script to take the few Markdown files I have and just make HTML out of them. I then used <a href="https://www.netlify.com/">Netlify</a> to do the hosting. I wanted to try out their legendary ease of use, and it was indeed super simple.</p>
<p>Earlier this year, I <a href="https://words.steveklabnik.com/i-m-joining-cloudflare">took a job at Cloudflare</a> to work on part of <a href="https://workers.cloudflare.com/">Cloudflare Workers</a>. To get more familiar with the project I’d be working on, <a href="https://www.cloudflare.com/products/workers-kv/">Workers KV</a>, I came up with a plan: I’d put the Markdown files into KV, and then write a worker in Rust and WebAssembly that would generate the HTML on the fly, and run that in a Worker. This worked really well, even though it was <em>incredibly</em> overkill for a simple static site.</p>
<p>Today we’re launching a new product, <a href="https://blog.cloudflare.com/workers-sites/">Workers Sites</a>. You see, I wasn’t the only one who was interested in using Workers for static site hosting; I’d see tweets every so often where others were doing the same. But doing so was a completely manual process, you had to write the code on your own, you had to do all the setup yourself, and there was no assistance in doing so. Workers Sites changes this, and makes it easy to host your own site, if you’d like.</p>
<p>The rest of this post is going to walk you through my porting process for <a href="https://www.steveklabnik.com/">steveklabnik.com</a>.</p>
<hr />
<p>I decided to go with <a href="https://www.getzola.org/">Zola</a> for my static site generator, as it’s written in Rust. I’d been wanting to give Zola a try for a while, and this was a good excuse to do so.</p>
<p>I needed these pre-requisites:</p>
<ul>
<li>Zola, instructions <a href="https://www.getzola.org/documentation/getting-started/installation/">here</a>.</li>
<li>Wrangler, instructions <a href="https://github.com/cloudflare/wrangler">here</a></li>
<li>A Cloudflare Workers account, <a href="https://workers.cloudflare.com/">here</a> if you don’t have a Cloudflare account already, and if you do, you can click the Workers tab in your dashboard to set it up.</li>
</ul>
<p>Additionally, you need to have a $5/month subscription to Workers, to get access to Workers KV. I want us to offer a free tier of KV in the future, but we’re not there yet. That $5/month gets you a lot more than one site, to be clear, but this isn’t a completely free solution today.</p>
<p>From there, it was time to make a new site with Zola:</p>
<pre><code>&gt; zola init steveklabnik.com
&gt; cd steveklabnik.com
</code></pre>
<p>Zola places the contents of your site in the <code>contents</code> directory. I only have one level for my site, so I dumped my two pages in there, as well as re-named <code>index.md</code> to <code>_index.md</code> to follow Zola convention.</p>
<p>I had a few YAML headers on pages, I had to convert those to Zola’s TOML format:</p>
<pre><code>+++
title = "Deleuzional"
+++
I really like the writing of [Gilles Deleuze]. I need
</code></pre>
<p>The <code>+++</code>s are like <code>---</code> in Jekyll, and the metadata is in TOML instead of YAML. I only needed <code>title</code> because I am keeping things very simple.</p>
<p>I then needed to set up templates. Like most static site generators, the template forms the outer contents of the HTML on a page, and renders the Markdown into the middle somewhere. I have a slightly different template for my root than the other two pages, and Zola handles this pretty well.</p>
<pre><code>&gt; ls .\templates\
Directory: C:\Users\Steve Klabnik\src\steveklabnik.com\templates
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 9/25/2019 5:28 PM 1288 index.html
-a---- 9/25/2019 5:28 PM 1234 page.html
</code></pre>
<p><code>index.html</code> is for the root page, and <code>page.html</code> is for the other pages. This is a Zola convention.</p>
<p>Zola templates are written in <a href="https://tera.netlify.com/docs/templates/">Tera</a>, which is sorta like <a href="https://shopify.github.io/liquid/">Liquid</a> or <a href="https://jinja.palletsprojects.com/en/2.10.x/">Jinja2</a>. Here’s a snippit of my template:</p>
<pre><code> &lt;body&gt;
&lt;h2&gt;{{ page.title }}&lt;/h2&gt;
{{page.content | safe}}
&lt;/body&gt;
</code></pre>
<p>The <code>| safe</code> bit says that it shouldn’t escape the HTML produced by <code>page.content</code>, which is the rendered contents of the Markdown pages.</p>
<p>Finally, I have some static assets for the site: a CSS file, some JS, and the favicon, etc. These get dumped in the <code>static</code> directory, and Zola will copy them over when I build the site.</p>
<p>Previewing during this process is very useful:</p>
<pre><code>&gt; zola serve
</code></pre>
<p>It took me a bit to figure out the conventions and get everything in place, and being able to change the contents and watch my browser refresh was invaluable.</p>
<p>Once I had my site ported to Zola, it’s time to deploy it to Workers Sites!</p>
<p>Wrangler is the command-line tool for Cloudflare Workers. If you haven’t used it before, you’ll have to log in:</p>
<pre><code>&gt; wrangler config
</code></pre>
<p>To check that your auth is set up correctly, you should run this command:</p>
<pre><code>&gt; wrangler whoami
You are logged in with the email '[email protected]'.
</code></pre>
<p>Of course, this should show your own email, not mine.</p>
<p>Let’s add Workers Sites to our project:</p>
<pre><code>&gt; wrangler init --site website
</code></pre>
<p>This creates two main things: <code>wrangler.toml</code>, used to configure Wrangler, and <code>workers-sites</code>, a directory with your worker in it. I used the extra <code>website</code> parameter to wrangler becuase it defaults to the name of the subdirectory, and the period in <code>steveklabnik.com</code> is invalid in a worker name, so I chose <code>website</code>. You can put whatever you’d like.</p>
<p>We need to set up <code>wrangler.toml</code>:</p>
<pre><code>&gt; code wrangler.toml
</code></pre>
<p>Mine looks like this:</p>
<pre><code>account_id = "f5518bfbf88c230551a64797c221c7da"
name = "website"
type = "webpack"
route = "www.steveklabnik.com/*"
zone_id = "9359b254c46ff9f5c546203686794862"
workers_dev = false
[site]
bucket = "./public"
entry-point = "workers-site"
</code></pre>
<p>All of these IDs are fine to be public; <code>wrangler</code> stores your sensitive stuff in <code>~/.wrangler</code>.</p>
<p>Important bits:</p>
<p>The <code>route</code> needs to be set up properly; this one means that the worker will serve everything on <code>www.steveklabnik.com</code>. My account and zone IDs are needed; you can find these on the Cloudflare dashboard on the bottom right of the main page for it. <code>workers_dev</code> is false; I’m deploying to my site, not to a subdomain of <code>workers.dev</code>. If we did, we wouldn’t need the <code>zone_id</code>, or <code>route</code>. Finally, the <code>bucket</code> setting needs to be set to whatever your site generator’s output directory is, which for Zola is <code>public</code>.</p>
<p>And that’s it! You can test out what this looks like:</p>
<pre><code>&gt; wrangler preview
</code></pre>
<p>If that all looks good, we can deploy:</p>
<pre><code>&gt; zola build
&gt; wrangler publish
</code></pre>
<p>And that’s it! And wow is it fast:</p>
<p><img src="https://svbtleusercontent.com/hS1xpZgYuQ9sRhUxP1iqq20xspap_small.png" alt="https://svbtleusercontent.com/hS1xpZgYuQ9sRhUxP1iqq20xspap_small.png" /></p>
<hr />
<p>… well, we’re <em>almost</em> done. Right now, this means I can only publish from my laptop. I’d prefer to have GitHub deploy this. We don’t have that as a built-in feature today, so I’d have to write an Action to do so. I haven’t yet, so I can’t blog about that. Maybe next time.</p>
<p>Additionally, there’s one more twist. Zola generates different URLs than my previous ones. I had previously had <code>.html</code> on the ends of my pages, but Zola does not do this.</p>
<ul>
<li>old: https://www.steveklabnik.com/deleuzional.html</li>
<li>new: https://www.steveklabnik.com/deleuzional/</li>
</ul>
<p>I think I’d prefer the new style anyway, so rather than figure out how to get Zola to do this, let’s set up some redirects. This is one cool thing about using Workers for this kind of task; while it’s slightly more manual, we can also make customizations very easily. The contents of the Worker that serves up the site lives in <code>workers-sites/index.js</code>. It’s very short.</p>
<pre><code>import { getAssetFromKV } from "@cloudflare/kv-asset-handler";
addEventListener("fetch", event =&gt; {
event.respondWith(handleEvent(event));
});
async function handleEvent(event) {
try {
return await getAssetFromKV(event);
} catch (e) {
let pathname = new URL(event.request.url).pathname;
return new Response(`"${pathname}" not found`, {
status: 404,
statusText: "not found"
});
}
}
</code></pre>
<p>Given that I only have two URLs, rather than doing something big and fancy, I did something dumb and straightforward:</p>
<pre><code>import { getAssetFromKV } from "@cloudflare/kv-asset-handler";
addEventListener("fetch", event =&gt; {
event.respondWith(handleEvent(event));
});
async function handleEvent(event) {
// redirect some old URLs
let pathname = new URL(event.request.url).pathname;
if (pathname === "/security.html") {
return Response.redirect("https://www.steveklabnik.com/security/", 301);
} else if (pathname === "/deleuzional.html") {
return Response.redirect("https://www.steveklabnik.com/deleuzional/", 301);
}
try {
return await getAssetFromKV(event);
} catch (e) {
let pathname = new URL(event.request.url).pathname;
return new Response(`"${pathname}" not found`, {
status: 404,
statusText: "not found"
});
}
}
</code></pre>
<p>I’m constructing <code>pathname</code> twice here, but I just don’t care enough to bother to refactor this out. Additionally, it’s nice to purely add, rather than customize what was generated too much.</p>
<p>But with that, and another <code>zola build &amp;&amp; wrangler publish</code>, I have the redirects set up for my site.</p></content:encoded>
</item>
<item>
<title>The culture war at the heart of open source</title>
<link>https://steveklabnik.com/writing/the-culture-war-at-the-heart-of-open-source/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-culture-war-at-the-heart-of-open-source/</guid>
<pubDate>Sun, 26 May 2019 00:00:00 GMT</pubDate>
<content:encoded><p>There’s a war going on. When isn’t there a war going on? But I’m not talking about a physical war here: I’m talking about a war over meaning. This particular war is a fight over what “open source” means.</p>
<p>Let’s take a few steps back.</p>
<h2>The Free Software Foundation</h2>
<p>People organize into groups for many reasons. This story starts with the story of an organization called the “GNU Project.” It was started in 1983, and <a href="https://groups.google.com/forum/#!msg/net.unix-wizards/8twfRPM79u0/1xlglzrWrU0J">here’s the initial announcement on USENET</a>. I’ve pulled out four important paragraphs:</p>
<blockquote>
<p>Starting this Thanksgiving I am going to write a complete Unix-compatible software system called GNU (for Gnu’s Not Unix), and give it away free to everyone who can use it. Contributions of time, money, programs and equipment are greatly needed.I consider that the golden rule requires that if I like a program I must share it with other people who like it. I cannot in good conscience sign a nondisclosure agreement or a software license agreement.So that I can continue to use computers without violating my principles, I have decided to put together a sufficient body of free software so that I will be able to get along without any software that is not free.If I get donations of money, I may be able to hire a few people full or part time. The salary won’t be high, but I’m looking for people for whom knowing they are helping humanity is as important as money. I view this as a way of enabling dedicated people to devote their full energies to working on GNU by sparing them the need to make a living in another way.</p>
</blockquote>
<p>There’s a lot of different ways to talk about this post: it contains all sorts of interesting things. But for right now, I want to talk about the motivations here. There were two main ones: first, to produce a software artifact. The second one is the big one: the artifact needs to be produced because existing ones were not compatible with a particular set of values.</p>
<p>The word “ideology” is a tricky one, but let’s go with this definition, which is <a href="https://en.wikipedia.org/wiki/Ideology">from Wikipedia</a>:</p>
<blockquote>
<p>An ideology is a collection of normative beliefs and values that an individual or group holds for other than purely epistemic reasons.</p>
</blockquote>
<p>The GNU project was formed to produce software according to a particular ideology: one of sharing software. I’m not here to argue if the project has accomplished this goal, or if this goal is good or not. My point is that the origins of the GNU project were specifically motivated by a collection of normative beliefs and values.</p>
<p>Two years later, the Free Software Foundation would be formed to support the GNU project, and promote the concept of Free Software. Free Software was software that aligns with the ideology of the GNU project, and a definition of Free Software was created and published in Feburary of 1986. Here is that definition:</p>
<blockquote>
<p>The word “free” in our name does not refer to price; it refers to freedom. First, the freedom to copy a program and redistribute it to your neighbors, so that they can use it as well as you. Second, the freedom to change a program, so that you can control it instead of it controlling you; for this, the source code must be made available to you.</p>
</blockquote>
<p>Since then, the Free Software Definition has expanded to four points. You can read the current definition <a href="https://www.gnu.org/philosophy/free-sw.en.html">here</a>.</p>
<h2>Open Source appears</h2>
<p>A decade passes, and trouble is brewing. I’ll <a href="https://en.wikipedia.org/wiki/Open_source#Origins">quote wikipedia again</a>:</p>
<blockquote>
<p>[T]he modern meaning of the term “open source” was first proposed by a group of people in the free software movement who were critical of the political agenda and moral philosophy implied in the term “free software” and sought to reframe the discourse to reflect a more commercially minded position. In addition, the ambiguity of the term “free software” was seen as discouraging business adoption. The group included Christine Peterson, Todd Anderson, Larry Augustin, Jon Hall, Sam Ockman, Michael Tiemann and Eric S. Raymond. Peterson suggested “open source” at a meeting held at Palo Alto, California, in reaction to Netscape’s announcement in January 1998 of a source code release for Navigator. Linus Torvalds gave his support the following day, and Phil Hughes backed the term in Linux Journal.</p>
</blockquote>
<p>Here we see the creation of the open source movement. If you’d prefer a primary source, here’s <a href="http://www.catb.org/~esr/open-source.html">Eric S. Raymond</a>:</p>
<blockquote>
<p>Specifically, we have a problem with the term “free software”, itself, not the concept. I’ve become convinced that the term has to go.The problem with it is twofold. First, it’s confusing; the term “free” is very ambiguous (something the Free Software Foundation’s propaganda has to wrestle with constantly). Does “free” mean “no money charged?” or does it mean “free to be modified by anyone”, or something else?Second, the term makes a lot of corporate types nervous. While this does not intrinsically bother me in the least, we now have a pragmatic interest in converting these people rather than thumbing our noses at them. There’s now a chance we can make serious gains in the mainstream business world without compromising our ideals and commitment to technical excellence – so it’s time to reposition. We need a new and better label.</p>
</blockquote>
<p>Shortly after, the <a href="http://opensource.org/">Open Source Initiative</a> was founded. A sort of mirror of the FSF, the OSI would support the promotion of the term “open source,” and the ideology behind it. Like the Free Software Definition, the <a href="https://opensource.org/osd">Open Source Definition</a> was created, derived from Debian’s guidelines for producing Free Software.</p>
<p>Again, we have an organization that’s created along ideological lines. But in this case, slightly different ones. <a href="https://web.archive.org/web/20090413035630/https://opensource.org/history">An older version of the OSI website says</a>:</p>
<blockquote>
<p>The conferees decided it was time to dump the moralizing and confrontational attitude that had been associated with “free software” in the past and sell the idea strictly on the same pragmatic, business-case grounds that had motivated Netscape.</p>
</blockquote>
<p><a href="https://opensource.org/history">Today’s version says</a>:</p>
<blockquote>
<p>The conferees believed the pragmatic, business-case grounds that had motivated Netscape to release their code illustrated a valuable way to engage with potential software users and developers, and convince them to create and improve source code by participating in an engaged community. The conferees also believed that it would be useful to have a single label that identified this approach and distinguished it from the philosophically- and politically-focused label “free software.”</p>
</blockquote>
<p>The idea here is plain: Free Software, but for business.</p>
<p>For more on this topic, I recommend <a href="https://thebaffler.com/salvos/the-meme-hustler">The Meme Hustler</a>.</p>
<h2>Moving forward</h2>
<p>Twenty years have passed, and a lot has changed.</p>
<p>For ideological movements to persist, they need to re-produce their values in new membership. And for a while, the FSF and OSI did a good job of that. These two movements, Free Software and Open Source, produced a lot of software, and gained a lot of new converts. But then… something happened.</p>
<p>I’m not sure exactly how it happened. I think the lazy answer is “GitHub!!!!”. I do think GitHub played a role, but I think the answer is more complex than that. I personally think that gender plays a huge role. But that’s a different essay. Regardless of <em>why</em> it happened, something did happen.</p>
<p>Somewhere along the way, Open Source ran into a problem that many movements face: the members of the movement no longer understood the ideology that created the movement in the first place.</p>
<p>If you ask a random developer what “open source” means to them, you won’t often hear “software that follows the Open Source Definition.” If you ask them “what’s the difference between free software and open source software,” you’ll often hear “aren’t those the same thing?” or “you can charge money for open source software, it’s not always free.” You may even hear “it’s on GitHub.”</p>
<p>In talking to developers about open source, you’ll also hear something else. Something that’s been itching in the back of my brain for a while, and the thing that led me to write this. You’ll often hear developers talk about how the relationship between businesses and open source developers is messy. Complicated. Businesses don’t “give back enough,” won’t pay people to work on open source. That it’s all take and no give. I’ve said a lot of that stuff myself.</p>
<p>But here’s the thing: that’s why the concept of open source was created in the first place. It’s there, plain as day, if you ask the folks who actually created it. Open source was a way to make free software better for businesses. This doesn’t mean that it’s beyond critique, of course. The relationship can improve. I don’t think these developers are stupid, or hypocrites.</p>
<p>This is where we’re at today. The Free Software movement was created, and then Open Source was created as a reaction to that. Today’s developers have never learned about this history, or don’t care about it, or actively think it’s irrelevant. And so we have a war. A war for the meaning of open source. A war fought with tweets, and blog posts, and discussions. A war between the old guard, those who created the idea originally, and the new generation, the developers who have taken the base idea and run with it.</p>
<p>I think history will repeat itself. In the same way that the open source movement said “we’re like free software, but with these changes,” I think we’ll end up with a new movement. For the same reasons that “open source” came up with a new name, I think the movement that will arise from today’s developers will also need a new name. I’m not sure what that movement will look like, and I’ll explore why in another post. To give you a teaser: the problem is in the way that both Free Software and Open Source are formulated. The rot is in the roots, and I’m not yet sure what will replace it.</p></content:encoded>
</item>
<item>
<title>I'm joining Cloudflare</title>
<link>https://steveklabnik.com/writing/i-m-joining-cloudflare/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/i-m-joining-cloudflare/</guid>
<pubDate>Fri, 05 Apr 2019 00:00:00 GMT</pubDate>
<content:encoded><p><img src="https://svbtleusercontent.com/jpH1brqXUvSAz1tNQ8uVKb0xspap_small.png" alt="https://svbtleusercontent.com/jpH1brqXUvSAz1tNQ8uVKb0xspap_small.png" /></p>
<p>Yesterday, I signed an offer letter from Cloudflare. I’ll be the product manager of Storage, which is the part of the company that’s working on data storage products for the <a href="https://www.cloudflare.com/products/cloudflare-workers/">Cloudflare Workers</a> platform. I’ll be starting at the end of the month.</p>
<p>I think a lot of you will say “that makes perfect sense,” but many more will say “wait, what?”. I hope this post will make things a bit more clear.</p>
<p>Before we start, I also want to say that I’ll still be continuing my work on Rust, though obviously in a more limited capacity. I’ll still be working on revisions to <a href="https://doc.rust-lang.org/stable/book/">The Rust Programming Language</a>, and doing core team work, for example.</p>
<hr />
<p>Before I worked on Rust, I worked on and with Ruby on Rails. When I got out of the web application game, the hot topic was this new “platform as a service” stuff. I’ve been watching all of the latest changes from afar, and they’ve been quite interesting. I’ve always loved the web, and the people building it. In the last 18 months or so, I’ve also been extremely interested in WebAssembly. Working at Cloudflare is the intersection of four things for me:</p>
<ul>
<li>Edge compute</li>
<li>WebAssembly</li>
<li>Rust</li>
<li>Personal and career growth</li>
</ul>
<h2>Edge Compute</h2>
<p>There’s sort of been four eras of “how do I web server”:</p>
<ul>
<li>Physical servers (“on premise”)</li>
<li>Infrastructure as a service</li>
<li>Platform as a service</li>
<li>Function as a service</li>
</ul>
<p>A lot of people see the relationship between these things as “what you control” vs “what does the provider control”, but I don’t think that’s quite right. But I’m getting ahead of myself.</p>
<p>Back in the day, if you wanted a web site, you bought a physical server, and you put it in a data center somewhere. I wasn’t around for those days. The point is, you managed everything: the hardware, the software. All of it.</p>
<p>In <a href="https://www.shapeways.com/blog/archives/305-CloudFab-Distributed-Fabrication-3D-printing,-Nick-Pinkston-interview.html">my days as a startup CTO</a>, my services were deployed to a Virtual Private Server. Instead of owning servers ourselves, we rented a virtual machine, running on someone else’s servers. This is “infrastructure as a service”; I didn’t have to worry about machines, I could just ask for one, and it’d be given to me. I still had to manage the operating system and everything above it.</p>
<blockquote>
<p>Fun side note: CloudFab was a Rails application with a big C extension to do data processing that was too much for Ruby. This later informed a lot of my Rust work.</p>
</blockquote>
<p>Next up came “platform as a service.” To me, Heroku was the platonic ideal of this era. Now, you didn’t even need to manage servers. You got to worry about your application, and that’s it. Maybe you’d need to set a slider to scale during times of demand.</p>
<p>I’ve been reading a lot about “functions as a service” over the past few years. Instead of deploying an entire application, each endpoint becomes an application, basically. This is where I think the previous thinking about the relationships between these categories of service is a little wrong. The actual underlying trend here is something else: what affordances do I have to scale?</p>
<p>In the on-premise days, if I wanted to add capacity, I’d need to buy new servers, install them, set everything up. There’s a <em>lot</em> of latency there. And if I need to remove capacity, well, I have to sell those servers? The move to infrastructure as a service was significant, because it more clearly separated concerns. Each company didn’t need a data center; a data center provides homogeneous compute. As a user, if I need to scale, I can spin up some more virtual servers much more quickly than I can physical ones. Managing that required a lot of tooling, however.</p>
<p>This was the insight that led to platform as a service: by managing this tooling, you could make it even easier for users to scale. There’s a reason why the slider became the ubiquitous synonym with PaaS. There’s nothing that you could do with PaaS that was impossible with IaaS, but it was much, much easier.</p>
<p>This brings us to Functions as a Service, also known as “serverless.” The reason that this architecture matters is one very similar to previous insights. By breaking your application into multiple functions, they can scale independently. This is one of the dreams of microservice architecture, but there’s a bit of a difference. Microservice architecture focuses on how you build your system, and how you split up components internally. FaaS focuses on how people interact with your system, and splits up the components according to that. This framing is a bit more significant to your ability to scale, because it means that during times of high load, you can scale only the parts under that load. In the same way that IaaS said “let’s not force people to buy servers to scale” and PaaS said “let’s not force people to build out infrastructure to scale”, FaaS says “let’s not force people to scale their help page to scale their shopping cart.”</p>
<p>This brings us to edge compute. Edge compute is a particular kind of FaaS, but an interesting take on it. Here’s the concept: your users are all over the world. Why do their packets need to go to <code>us-east-1</code> and back? To which you might say, “sure, that’s why we’ve developed our application to work in multiple regions.” That’s the same relationship between IaaS and PaaS. That is, why should you have to worry about making your application be available across multiple regions? You may not be worrying about individual physical servers anymore, but you’re still worrying about data centers. And some FaaS products, like AWS Lambda, can already run easily across multiple regions, and with some setup, go between them. However, Amazon has about 20 total regions.</p>
<p>So how is edge compute different? Well, at some point, various CDN companies realized “wait a minute. We have severs around the world. Why can’t we run compute on them as well?”</p>
<blockquote>
<p>Side note: Cloudflare is not really a CDN company, though that’s how I always thought of them. They do a lot more stuff than CDNs.</p>
</blockquote>
<p>This is interesting because, well, <a href="https://aws.amazon.com/about-aws/global-infrastructure/">here’s those 20 Amazon regions</a>:</p>
<p><img src="https://svbtleusercontent.com/qxGa9h6TshtRrXZmh6tsRU0xspap_small.png" alt="https://svbtleusercontent.com/qxGa9h6TshtRrXZmh6tsRU0xspap_small.png" /></p>
<p>Compare this with <a href="https://www.fastly.com/network-map">Fastly’s network</a>:</p>
<p><img src="https://svbtleusercontent.com/ePzMAv7Q4NVLARUKambWH80xspap_small.jpg" alt="https://svbtleusercontent.com/ePzMAv7Q4NVLARUKambWH80xspap_small.jpg" /></p>
<p>And finally, <a href="https://www.cloudflare.com/network/">Cloudflare’s network</a>:</p>
<p><img src="https://svbtleusercontent.com/bQzn9XqBY98mw269Q1tsdH0xspap_small.png" alt="https://svbtleusercontent.com/bQzn9XqBY98mw269Q1tsdH0xspap_small.png" /></p>
<p>(the page is animated, so those highlighted ones don’t mean anything)</p>
<p>You’re much likely to be physically closer to a Fastly or Cloudflare server than you are an AWS one. And that’s what we mean by “edge compute”, that is, you’re running your code on the edge, rather than in the data center.</p>
<p>Now, I should also mention that there is a form of Lambda called Lambda@Edge that runs off of CloudFront locations, rather than the above map. So that looks like this:</p>
<p><img src="https://svbtleusercontent.com/vwJvtVvGK7DpGZYEPcrjzW0xspap_small.png" alt="https://svbtleusercontent.com/vwJvtVvGK7DpGZYEPcrjzW0xspap_small.png" /></p>
<p>Much better! My point about regions is more applicable to “edge compute” vs more general compute, than saying something about Lambda as a particular product.</p>
<p>CloudFlare Workers don’t require you to think about where your service is deployed: when you upload it, it’s automatically running on all 175 locations, within minutes.</p>
<p>I think this space is interesting and significant. Cloudflare has <a href="https://www.cloudflare.com/products/cloudflare-workers/">Workers</a>, and Fastly has <a href="https://wasm.fastlylabs.com/">Terrarium</a>. Amazon has <a href="https://docs.aws.amazon.com/lambda/latest/dg/lambda-edge.html">Lambda@Edge</a>. These are all very interesting platforms for building high performance applications.</p>
<p>My role as part of Storage will be to consider “what does data access and storage look like in this world?” If your code is moved to the edge, but your data is still in a central server, you don’t gain the full benefit of having the code close to the client. There’s a lot of interesting stuff in this space!</p>
<h2>WebAssembly</h2>
<p>Both ClouldFlare Workers and Fastly’s Terrarium use <a href="https://webassembly.org/">WebAssembly</a> as a core part of the platform. And that’s significant:</p>
<blockquote>
<p>If WASM+WASI existed in 2008, we wouldn't have needed to created Docker. That's how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let's hope WASI is up to the task! https://t.co/wnXQg4kwa4
— Solomon Hykes (@solomonstre) March 27, 2019</p>
</blockquote>
<p>In order for this stuff to work, you need a runtime for WebAssembly. One interesting thing we’re seeing in this space is sort of two parallel paths emerge: do you want to support JavaScript, or not? Recently, Fastly open sourced <a href="https://www.fastly.com/blog/announcing-lucet-fastly-native-webassembly-compiler-runtime">Lucet</a>, the WebAssembly runtime that powers Terrarium. It’s an impressive bit of tech. There are others too, like <a href="https://wasmer.io/">Wasmer</a> and <a href="https://github.com/CraneStation/wasmtime">Wasmtime</a>. By focusing purely on WebAssembly, you can build specialized tech, and it can be really, really fast.</p>
<p>However, you can’t compile JavaScript to WebAssembly. You can sort of do it with AssemblyScript, a TypeScript subset the compiles to wasm. But JavaScript is arguably the most popular programming language in the world. I personally believe in JavaScript, even with the rise of WebAssembly. And so Cloudflare is taking a different approach than those projects. Instead of building a wasm-only runtime, they’re building on top of Google’s V8. This means that they can support both WebAssembly and JavaScript in workers. Additionally, by leaning on V8, you can take advantage of all of the excellent engineering resources that Google pours into it, similar to how the Rust compiler gets to take advantage of people working on improving LLVM.</p>
<p>Anyway, the point is, WebAssembly is core to this new edge compute world. And so I’m excited to be in the space. I know people that work at Fastly, wasmer, and Mozilla, and they’re all doing great work. I think there’s space for both approaches, but I’m mostly excited to be there and see how it all turns out.</p>
<p>Oh, one other thing I wanted to say: right now, everything around Workers is closed source. Kudos to Fastly for open-sourcing Lucet. I asked about this in my interview, and Cloudflare is very interested in doing more open source work, and so I’m hoping it doesn’t take us long to catch up in this regard. We’ll see, of course, I haven’t actually started yet. But I think that this stuff needs to be open source, personally.</p>
<h2>Rust</h2>
<p>You may remember a little story called <a href="https://en.wikipedia.org/wiki/Cloudbleed">CloudBleed</a>. In short, Cloudflare had a pretty bad security bug in 2017. This bug happened because of a parser, written in Ragel. Here, <a href="https://blog.cloudflare.com/incident-report-on-memory-leak-caused-by-cloudflare-parser-bug/">I’ll let Cloudflare explain it</a>:</p>
<blockquote>
<p>The Ragel code is converted into generated C code which is then compiled. The C code uses, in the classic C manner, pointers to the HTML document being parsed, and Ragel itself gives the user a lot of control of the movement of those pointers. The underlying bug occurs because of a pointer error.</p>
</blockquote>
<p>CloudBleed happened due to memory unsafety. When talking to Cloudflare today, it’s pretty clear that this bug was taken extremely seriously, to the point where it created a culture change inside the company itself. These days, it seems to me (I haven’t started yet, mind you) that you have to use a memory safe language by default, and only use a memory unsafe one if you have an extremely good justification. But Cloudflare needs to be able to have a lot of performance, and control latency, in much of their stack.</p>
<p>Enter Rust. Cloudflare started exploring Rust after CloudBleed, and now uses a significant amount of Rust in production. They host the local Rust meetup here in Austin, and sometimes the one in San Francisco. I’m excited to help Cloudflare be successful with Rust, and to use that experience to help improve Rust too.</p>
<p>My understanding is that I’ll be dealing with a bunch of TypeScript and possibly Go as well, which is also pretty cool.</p>
<h2>Career growth</h2>
<p>Finally, the move to product management. I really enjoy product work, and have had several jobs that were sorta-kinda product jobs. I don’t think that management jobs are inherently better than engineer jobs, of course, but it’s a way for me to grow. My role at Balanced was sorta-kinda-PM-y, and I loved that job.</p>
<p>I don’t have too much more to say about this, just that it’s a bit different, but something I’m really excited to do.</p>
<h2>I’m excited for the future</h2>
<p>So yeah, that’s my extremely long-winded explanation of what’s going on. I’m not starting until the end of the month, so I’ve got a few weeks to purely enjoy. I’ll still be playing a lot of <a href="http://www.celestegame.com/">Celeste</a> and Fortnite, writing blog posts, and working on some Rust open source. But then I get to get back to work. I’m pumped about all of this! New role, building some new tech with tech I already love.</p></content:encoded>
</item>
<item>
<title>What comes after open source</title>
<link>https://steveklabnik.com/writing/what-comes-after-open-source/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/what-comes-after-open-source/</guid>
<pubDate>Tue, 02 Apr 2019 00:00:00 GMT</pubDate>
<content:encoded><p>In <a href="https://words.steveklabnik.com/the-culture-war-at-the-heart-of-open-source">a previous post</a>, I discussed the history of open source, and ended with this claim:</p>
<blockquote>
<p>Today’s developers have never learned about this history, or don’t care about it, or actively think it’s irrelevant. … For the same reasons that “open source” came up with a new name, I think the movement that will arise from today’s developers will also need a new name.</p>
</blockquote>
<p>We talked about the ideological history of open source, but that’s not what developers object to, really. I don’t think developers are moving back towards a world of making source code private. Instead, it’s something related to a very old discussion in free software. To quote <a href="https://www.gnu.org/philosophy/free-sw.en.html">the FSF</a>:</p>
<blockquote>
<p>“Free software” means software that respects users’ freedom and community. Roughly, it means that the users have the freedom to run, copy, distribute, study, change and improve the software. Thus, “free software” is a matter of liberty, not price. To understand the concept, you should think of “free” as in “free speech,” not as in “free beer”. We sometimes call it “libre software,” borrowing the French or Spanish word for “free” as in freedom, to show we do not mean the software is gratis.</p>
</blockquote>
<p>In a similar fashion, I don’t think that developers are turning against the concept of “free as in free speech”. I think that they don’t believe that the current definitions of free software and open source actually produce software that is “free as in speech.”</p>
<h2>What does “freedom” mean anyway?</h2>
<p>What “free as in speech” means is itself, yet another schism between different camps in the Free Software/Open Source camp. At the root of this schism is a difference of <em>strategy</em>. The specific tactic is straightforward: there are two kinds of licenses, and which do you choose? The two kinds are:</p>
<ul>
<li><em>permissive</em> licenses</li>
<li><em>viral</em> licenses if you don’t like them, <em>copyleft</em> licenses if you do</li>
</ul>
<p>Open Source advocates prefer permissive licenses, and Free Software advocates prefer viral/copyleft licenses. What’s the difference? Well, that’s the topic for a different essay. Because here’s the thing: the jig is up. We’ve already hit the root problem. And you probably didn’t even notice. It took me a long, long time to notice.</p>
<p>Before I expose the trick, a personal interlude.</p>
<h2>Losing my religion</h2>
<p>I used to be a card-carrying, passionate advocate of Free Software. I used to see Stallman lecture. I only installed Free Software on my personal computer to the fullest extent possible. I was considering buying a coreboot-compatible laptop to get rid of the rest.</p>
<p>But then, slowly, I stopped caring. It felt really similar to when I decided to leave Catholicism. I started noticing that these beliefs weren’t really actually helpful, and were frankly, mostly harmful. Now, I’m not saying Free Software is harmful. What I am saying is that refusing to use a wifi connection on your laptop because there aren’t free drivers helps basically nobody, and only harms yourself. Even if you are the kind of person who thinks boycotts work, there just aren’t enough free software diehards to form a large enough boycott. And even then, you’d want to do the boycott <em>together</em>, <em>simultaneously</em>, to send the message.</p>
<p>I realized that a lot of software I used was under a permissive license, and even though I’d prefer if it was GPL’d, the only way to actually realize the dream of improving the software thanks to the source I’d been given was to contribute back, under those terms. So I started to.</p>
<p>And at that point, the permissive licenses were becoming more and more popular. So I found myself in a strange position: even though I was a passionate Free Software advocate, I found myself near-exclusively building Open Source software. All around me, I saw the assault on copyleft, from all sides. And copyleft was losing.</p>
<p>At some point, I just stopped caring.</p>
<h2>Free Software and Open Source are about licenses</h2>
<p>With that out of the way, here’s the problem: note that I seamlessly switched above from talking about what Free Software and Open Source <em>are</em>, to immediately talking about <em>licenses.</em> This is because these two things are effectively synonymous. Quoting <a href="https://www.gnu.org/licenses/licenses.html">the FSF again</a>:</p>
<blockquote>
<p>Published software should be free software. To make it free software, you need to release it under a free software license.</p>
</blockquote>
<p>The <a href="https://opensource.org/osd">OSI</a>:</p>
<blockquote>
<p>Open source doesn’t just mean access to the source code. The distribution terms of open-source software must comply with the following criteria:</p>
</blockquote>
<p>Both Free Software and Open Source are, almost by definition, specific design constraints on software licenses. While this started with the FSF, the OSI inherited it.</p>
<p>I think a quote from <a href="https://www.gnu.org/licenses/licenses.html#WhatIsCopyleft">further down the FSF’s page</a> really drives this home:</p>
<blockquote>
<p>To copyleft a program, we first state that it is copyrighted; then we add distribution terms, which are a legal instrument that gives everyone the rights to use, modify, and redistribute the program’s code or any program derived from it but only if the distribution terms are unchanged. Thus, the code and the freedoms become legally inseparable.</p>
</blockquote>
<p>The FSF sees copyleft, and therefore copyright, as a legal tool to enforce software freedom. This sometimes puts Free Software advocates in a strange position. For example, they will <a href="https://www.fsf.org/campaigns/drm.html">campaign passionately against Digital Rights Management</a>, a software tool to protect copyrighted works, yet also <a href="https://fsfe.org/news/2019/news-20190326-01.en.html">support expanding copyright in some cases</a> when it can further the aims of the GPL. (Due to controversy, they’ve stepped back a bit since, as you can see.)</p>
<h2>Licenses are not sufficient</h2>
<p>So why is it a problem that the concepts of free software and open source are intrinsically tied to licenses? It’s that the aims and goals of both of these movements are about <em>distribution</em> and therefore <em>consumption</em>, but what people care about most today is about the <em>production</em> of software. Software licences regulate <em>distribution</em>, but cannot regulate <em>production</em>. (technically they can, but practically, they can’t. I get into this below.) This is also the main challenge of whatever comes after open source; they cannot rely on the legal tactics of the last generation. I don’t have solutions here.</p>
<p>Let’s talk about what developers want first, and then we’ll get into why licenses can’t accomplish this.</p>
<h2>Developers care about production</h2>
<p>In <a href="https://words.steveklabnik.com/the-culture-war-at-the-heart-of-open-source">my previous post</a>, I claimed that developers are mostly confused about open source. Here’s two concrete examples of what I mean.</p>
<ul>
<li>Imagine a company wants to take some open source software using a permissive license and use it in their product. In the course of using it in their product, they improve it. Are they required to give anything back to the parent project, at all?</li>
<li>Imagine a company wants to take some free software using a copyleft license and use it in their product. In the course of using it in their product, they improve it. In order to comply with the license, they include a line in their 250 page “software license agreement” that says “Certain components of the software, and third party open source programs included with the software, have been or may be made available by $COMPANY on its Open Source web site (http://www.opensource.mycompany.com/”. That web site contains zip files with the contents of the (heavily modified) source. Are they required to do anything more than that?</li>
</ul>
<p>When presenting these questions to most developers today, I suspect you’d get these answers to these two questions, in order:</p>
<ul>
<li>Yes, and that’s why open source is such a mess; companies take and don’t give back.</li>
<li>… that sounds extremely shady.</li>
</ul>
<p>Those of you who have been around the block a few times may recognize situation number two: it’s what Apple did with WebKit. There was a project called KHTML, which was licensed under the GPL. Apple forked it, and in order to comply with the GPL, did exactly the above. They were completely within their rights to do so. Yet, many recognized, even at this time, that this “wasn’t in the spirit of open source.” These tactics are sometimes called “source dumps” or “code bombs.”</p>
<p>But that attitude, that this may be following the letter, <em>but not the spirit</em>, is the crux of it here. Most developers don’t understand open source to be a particular license that certain software artifacts are in compliance with, but an attitude, an ideology. And that ideology isn’t just about the consumption of the software, but also its production. An open source project should have a public bug tracker. There should be a mailing list, for discussion. You should be able to observe, and ideally participate in, the development of the software. Focusing on the code being open is putting the cart before the horse. In fact, this is one of the reasons why there’s such derision for “source available” licenses; it’s not about the source being open. It’s that the source being open is a necessary, but not sufficient, component of being open source. Now, this is normally framed as a problem of distribution, but I think that many also understand it as a problem of production.</p>
<p>I believe that this focus on process is why the GNU project has fallen out of favor as well. The tools that the GNU project uses to develop its Free Software are arcane, ugly, and unique to them. It’s the same with many of the old-school Open Source projects as well. If I never have to look at a Bugzilla instance again, I will die happy. This is why GitHub took off; it provided a significantly nicer developer experience for building software. You may not personally agree, but the numbers speak for themselves. The FSF didn’t move to GitHub because GitHub is proprietary, and they see that as inconsistent with their values. Most developers see that you can use it for no money, and that the software produced with it is open source. They see this as consistent with their values.</p>
<p>When developers talk about problems they see in open source, it’s often that there are production problems. Companies don’t “give back” money or developer hours. Programmers today don’t seem to be upset that, if they’ve developed any proprietary extensions to their open source software, that those extensions are not shared back with the community. They care that the production process is impeded by additional pressure, without providing resources. If a company were to add a proprietary feature to an open source project, yet pays five employees to develop the open source part further, the FSF sees this as a tragedy. The commons has not been enriched. The new generation of open source developers sees this as a responsible company that thankfully is contributing to the development of something they use and care about.</p>
<p>Software licenses can only restrict what people can do when they distribute the source code, and that’s it. It cannot force someone to have a bug tracker, or a code of conduct, or accept your patch. Copyleft can force an absolute minimal “contribution” back to your project, but it can’t force a good-faith one. This makes it an inadequate tool towards building something with the kinds of values that many developers care about.</p>
<h2>The challenges</h2>
<p>How would one go about building a software movement around the open <em>production</em> of software? There are a number of challenges.</p>
<p>First of all, do you even want the legal system to enforce such a thing? There are pros and cons. Without legal teeth, companies are unlikely to comply. That’s part of why we’re in this mess to begin with!</p>
<p>Okay, so you want the legal system to somehow enforce this kind of control over the production of software. But how? If we look closer at the strategy used by Free Software and Open Source, they use licenses, which are a form of intellectual property law, which is modeled after property law. Earlier, I said that you can’t use licenses to regulate production, and that’s <em>technically</em> not true. For example, say that I own a brand, like McDonalds. I own the intellectual property surrounding that brand. I can licence that intellectual property to others, contingent on them producing hamburgers (and whatever else) in a certain way, according to my specification.</p>
<p>This doesn’t really work with the way that open source is set up. The entities are in reverse here; it’s the software developers that want to be able to dictate things, but it’s the project that sets the license terms.</p>
<p>That got me thinking about a different pattern for doing this kind of thing. Have you ever seen one of these?</p>
<p><img src="https://svbtleusercontent.com/wWhQCoC3mGGKwQ1nS6CC9H0xspap_small.gif" alt="https://svbtleusercontent.com/wWhQCoC3mGGKwQ1nS6CC9H0xspap_small.gif" /></p>
<p>This image on a product is part of a process called “certification.” The image itself is referred to as a “certification mark.” In order to use this image on your product, you apply to a “certification body”, in this case, the <a href="https://www.usda.gov/">USDA</a>. This body has set up some kind of tests, and if your product passes them, you gain the ability to say that you’ve passed the certification. I chose organic food on purpose here; most aspects of this certification are about the process by which the food is produced.</p>
<p>Technology is no stranger to these kinds of processes:</p>
<p><img src="https://svbtleusercontent.com/fMstP8GhZwpMhHxNdZ3aTn0xspap_small.png" alt="https://svbtleusercontent.com/fMstP8GhZwpMhHxNdZ3aTn0xspap_small.png" /></p>
<p>So in theory, one could imagine an organization that produces a different kind of document. Instead of a license for the source code, they would provide a way to say uh, let’s go with “Open Development Certified.” Projects could then submit for certification, they’d get accepted or rejected.</p>
<p>I’m not confident this solution would work, however.</p>
<p>For one, even though some parts of our industry have accepted certifications, I feel like software developers have a pretty big bias against them. Beyond that, there’s two other major questions: who would do this certification, and how would they determine the criteria? It’s not clear who has the moral authority to suggest that they are the arbiter of what is correct here, and that a majority of people would agree. And then they would have a big job of actually determining what those sets of rules would be. It does have a nice property of a built-in business model; you can charge for application for the certification. But that also has a host of issues. And even if you sort all of that out, it runs afoul of the same “boycott” problems that I talked about above with Free Software. This certification only makes sense if people demand that the software they use is Open Development Certified. I’m not sure that this would be the case.</p>
<p>Another option is some sort of “Developer Union,” which would put pressure on the companies that those developers work at to contribute back to open source projects. Many developers seem <em>rabidly</em> anti-union, and tech companies are as well. I’m not sure this is a viable path, today.</p>
<h2>So where do we go from here?</h2>
<p>I’m still, ultimately, left with more questions than answers. But I do think I’ve properly identified the problem: many developers conceive of software freedom as something larger than purely a license that kinds in on redistribution. This is the new frontier for those who are thinking about furthering the goals of the free software and open source movements. Our old tools are inadequate, and I’m not sure that the needed replacements work, or even exist.</p>
<p>Something to think about, though.</p></content:encoded>
</item>
<item>
<title>Rust has finally outgrown me</title>
<link>https://steveklabnik.com/writing/rust-has-finally-outgrown-me/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rust-has-finally-outgrown-me/</guid>
<pubDate>Fri, 29 Mar 2019 00:00:00 GMT</pubDate>
<content:encoded><p>Today is a very interesting day. It’s giving me a lot of feels. Today is <a href="https://rustlatam.org/">Rust Latam</a>, a Rust conference in Montevideo.</p>
<p>And I’m not there.</p>
<p>This is significant because, as far as we know, I’m the only person who has been to every Rust conference in existence so far. I went to RustCamp, all three RustConfs, all five RustFests so far, all three Rust Belt Rusts. One RustRush. Am I forgetting any? Thirteen Rust conferences in the past four years.</p>
<p>I really wanted to attend Rust Latam. The main organizer, Santiago, has been a friend for a very long time. It’s amazing that we’re getting a new conference in this part of the world. There’s been a lot of energy with Rust there for quite a while. It would have been nice to meet all those people whose tweets I’ve seen.</p>
<p>But in the end, I’m still between jobs (news on that soon, hopefully!), and spending that money would be irresponsible. I’m just getting settled in Austin, and it’s nice to stay in one place for a little while. I didn’t only attend Rust conferences last year, I attended many more. I missed Thanksgiving with my friends, because I was at a conference. I made five trips in October last year; Eastern Europe, then San Francisco, then Michigan, then New York (for a conf, not to stay at home), then Berlin. I did three more in November. I did about twenty conferences, and a few personal trips, last year. I am very tired.</p>
<p>Don’t get me wrong, I love travel, and plan to do more of it in the future. But I’ve been taking a bit of a breather; I’ve only done four trips so far this year, twice to Europe, twice in the US. But it’s been nice to take a break. For now.</p>
<p>More importantly, it’s time to let go of the pressure I felt to keep that streak up. Some of those trips really took a toll on me, mentally and physically. Nobody should ever fly from New York to Belarus for 30 hours, then immediately fly to San Francisco for three days, then back to New York. I’m not old, but I’m not as young as I once was, either.</p>
<p>It’s also time to acknowledge that I cannot do it all on my own. This conference streak is one thing, but Rust has outgrown me in other ways. I used to be on <code>#rust</code> every day, answering almost every question. Then it was both <code>#rust-beginners</code> and <code>#rust</code>. Not to mention all the other channels. I’m not even on IRC anymore. I was on the core team when the core team approved every RFC; now my team very rarely has RFCs. I used to read every single GitHub email on <code>rust-lang/rust</code>, and then <code>rust-lang/cargo</code>, and then all their other libraries. Now I skim <code>rust-lang/rust</code>, and have unsubscribed from a lot of the other repos. I used to know every single production user of Rust. Those times are long gone; I’m starting to get used to hearing “oh yeah our team has been using Rust in production for a while now, it’s great.” The first time that happened, it felt <em>very</em> strange. Exciting, but strange.</p>
<p>Back when I first decided I wanted to devote my time to Rust, I used to submit every single blog post about Rust to Hacker News and Reddit, and then answer any question anyone had about things. I diligently did this for <em>years</em>. I’m slowly letting go of this one, too. We’ll see.</p>
<p>All of these things are excellent. This is the future I wanted. This is why I spent so much time over the years doing all of this stuff. I did it, mission accomplished. It feels good. But it also feels a little weird. That Rust is bigger than <em>me</em> is an amazing, wonderful thing.</p>
<p>I wonder what the next stage of Rust’s growth will feel like.</p></content:encoded>
</item>
<item>
<title>Learning Ada</title>
<link>https://steveklabnik.com/writing/learning-ada/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/learning-ada/</guid>
<pubDate>Tue, 26 Mar 2019 00:00:00 GMT</pubDate>
<content:encoded><p>I decided to try and learn Ada. I love programming languages. This post documents my attempt, and explains what I learned. This is pretty stream of consciousness and basically is a bunch of random notes, so if you hate that, better close the tab. I typed my thoughts up pretty raw, so I may be incorrect in places too. Let me know if you know the answer to something I don’t!</p>
<hr />
<p>The first step, of course, is to google:</p>
<p><img src="https://svbtleusercontent.com/72NhELV1orufu8jasf5iiT0xspap_small.png" alt="https://svbtleusercontent.com/72NhELV1orufu8jasf5iiT0xspap_small.png" /></p>
<p>Great! I know that Adacore are sort of the stewards of Ada? Vaguely, that might be wrong. Let’s look at all three of those.</p>
<blockquote>
<p>Learn.adacore.com is an interactive learning platform designed to teach the Ada and SPARK programming languages.</p>
</blockquote>
<p>Ah, that’s awesome! No installation needed.</p>
<p>The second one looks <strong>extremely</strong> like spam. It does have a page of learning materials that look legit enough. I wonder if this is a low-effort attempt to make money through ads. Or maybe it’s actually good and real and feels outdated, it does have a high google ranking. Upon doing some more research, it seems like the latter. Oops…</p>
<p>The third one is a parked domain, it apparently expired. Oops.</p>
<p>Anyway, I’m going to pursue learn.adacore.com</p>
<h2>learn.adacore.com</h2>
<p>Here’s the initial code presented:</p>
<pre><code>with Ada.Text_IO; use Ada.Text_IO;
procedure Learn is
subtype Alphabet is Character range 'A' .. 'Z';
begin
Put_Line ("Learning Ada from " &amp; Alphabet'First &amp; " to " &amp; Alphabet'Last);
end Learn;
</code></pre>
<p>Okay! This clearly has some style to it. Seems like <code>Pascal_Snake_Case</code>. I’m not super hyped about this, but it’s fine. Syntax discussions are like the tabloids of programming language discussion. They’re fun, but they don’t really have any educational comment. One thing I will point out: are apostrophes used for method calls? Otherwise, this seems to be fairly Pascal like, which I think makes sense given what I know about Ada.</p>
<p>I can click run and see it execute right there! Awesome.</p>
<p>There seems to be three sections here: About, Courses, and Books. About is a single page of information that reinforces what I already know:</p>
<blockquote>
<p>The Ada programming language was designed from its inception to be used in applications where safety and security are of the utmost importance.The SPARK programming language is a formally verifiable subset of the Ada language which allows developers to mathematically prove program correctness through static means.Founded in 1994, AdaCore is the leading provider of commercial and open-source software solutions for Ada</p>
</blockquote>
<p>Cool.</p>
<p>There are two courses, introduction to Ada, and introduction to SPARK. I think SPARK is cool but I’ll learn that after I learn Ada itself. I also do like books, but let’s do that first.</p>
<p>I’m not going to write everything about the book as I read it, but I will say that I like the basic structure here: first some history, then imperative programming stuff, then talking about types. After that it gets into packages, more type stuff, privacy, generics, exceptions, tasks, interfacing with C, the standard library, and some appendices. This feels pretty logical. Time to do some reading, I’ll jump back here at random points with notes.</p>
<blockquote>
<p>The first Ada standard was issued in 1983; it was subsequently revised and enhanced in 1995, 2005 and 2012, with each revision bringing useful new features.</p>
</blockquote>
<p>I heard that Ada 2020 is a thing; seems like they’re on a roughly 7-8ish year timeframe generally. Cool.</p>
<blockquote>
<p>In terms of modern languages, the closest in terms of targets and level of abstraction are probably C++ and Rust.</p>
</blockquote>
<p>We got a shout-out! That’s nice.</p>
<blockquote>
<p>Readability is more important than conciseness. Syntactically this shows through the fact that keywords are preferred to symbols, that no keyword is an abbreviation, etc.</p>
</blockquote>
<p>This is interesting; it seems to suggest that readability and verbosity are synonymous. I often find concise things easier to read, personally. This is, of course, subjective, and I wonder how I’ll feel after I learn more of the language.</p>
<blockquote>
<p>Ada is a multi-paradigm language with support for object orientation and some elements of functional programming, but its core is a simple, coherent procedural/imperative language akin to C or Pascal.</p>
</blockquote>
<p>This makes a lot of sense, especially given the time it was created.</p>
<p>I’m finding it a <em>little</em> hard to understand what <code>main</code> is. The docs say</p>
<blockquote>
<p>Greet is a procedure, and the main entry point for our first program. Unlike in C or C++, it can be named anything you prefer. The builder will determine the entry point. In our simple example, gprbuild, GNAT’s builder, will use the file you passed as parameter.</p>
</blockquote>
<p>It will use it as … parameter? Like, is the fact that we called the file <code>greet.adb</code> the reason it calls <code>Greet</code>? It’s not clear to me at the moment.</p>
<blockquote>
<p>Integer’Image is a function that takes an Integer and converts it to a String. It is an example of a language construct known as an attribute, indicated by the “’” syntax, which will be covered in more detail later.</p>
</blockquote>
<p>Ah, it’s not a method call, it’s an attribute.</p>
<blockquote>
<p>In Ada, an integer type is not specified in terms of its machine representation, but rather by its range. The compiler will then choose the most appropriate representation.</p>
</blockquote>
<p>Very interesting. I wonder how well-defined these mappings are, for example, if you need to call a C function, you’re gonna need to give it the correct sized type… I’m sure I’ll get to that.</p>
<blockquote>
<p>Mainly for efficiency reasons, while machine level overflow always results in an exception, type level overflows will only be checked at specific boundaries, like assignment:</p>
</blockquote>
<p>Okay, <em>this</em> is the good stuff. Ranged integers are something people always say Ada is better at than Rust; I wondered if this was a static or dynamic check. It <em>is</em> a dynamic check. Cool. I’ve always wanted to know!</p>
<blockquote>
<p>As mentioned previously, every “built-in” type in Ada is defined with facilities generally available to the user.</p>
</blockquote>
<p>Pretty cool!</p>
<blockquote>
<p>While unconstrained arrays in Ada might seem similar to variable length arrays in C, they are in reality much more powerful, because they’re truly first-class values in the language. You can pass them as parameters to subprograms or return them from functions, and they implicitly contain their bounds as part of their value. This means that it is useless to pass the bounds or length of an array explictly along with the array, because they are accessible via the ’First, ’Last, ’Range and ’Length attributes explained earlier.</p>
</blockquote>
<p>Awesome. This is the way to do it, for sure.</p>
<blockquote>
<p>Here is how the string type is defined in Ada:type String is array (Positive range &lt;&gt;) of Character;</p>
</blockquote>
<p>Hmm, what’s a <code>Character</code> though? Let’s try to mess with the example:</p>
<pre><code>with Ada.Text_IO; use Ada.Text_IO;
procedure Greet is
Message : String (1 .. 2) := "😊";
-- ^ Pre-defined array type.
-- Component type is Character
begin
for I in reverse Message'Range loop
-- ^ Iterate in reverse order
Put (Message (I));
end loop;
New_Line;
end Greet;
</code></pre>
<p>Here’s the output:</p>
<pre><code>Run...
greet.adb:4:04: warning: "Message" is not modified, could be declared constant
greet.adb:4:33: warning: wrong length for array of subtype of "Standard.String" defined at line 4
greet.adb:4:33: warning: "Constraint_Error" will be raised at run time
raised CONSTRAINT_ERROR : greet.adb:4 length check failed
</code></pre>
<p>Hm, wrong length. What is the right length? This particular emoji is four bytes, so let’s try to set it to 5, since we start at 1. Nope that doesn’t work either. What about 4?</p>
<pre><code>Run...
the machine running the examples is not responding, please try again later
</code></pre>
<p>Very suspicious. Anyway, I’m guessing it’s ASCII, maybe it’ll tell me later.</p>
<blockquote>
<p>One last feature of Ada arrays that we’re going to cover is array slices. It is possible to take and use a slice of an array (a contiguous sequence of elements) as a name or a value.</p>
</blockquote>
<p>Hell yeah, I love slices.</p>
<blockquote>
<p>Ada has multidimensional arrays, which are not covered in this course. Slices will only work on one dimensional arrays.</p>
</blockquote>
<p>Ah, bummer. I guess I’ll have to read up on those separately.</p>
<blockquote>
<p>Returning variable size objects in languages lacking a garbage collector is a bit complicated implementation-wise, which is why C and C++ don’t allow it, prefering to depend on explicit dynamic allocation / free from the user.The problem is that explicit storage management is unsafe as soon as you want to collect unused memory. Ada’s ability to return variable size objects will remove one use case for dynamic allocation, and hence, remove one potential source of bugs from your programs.Rust follows the C/C++ model, but with safe pointer semantics. However, dynamic allocation is still used. Ada can benefit from an eventual performance edge because it can use any model.</p>
</blockquote>
<p>Hmmm, I wonder <em>how</em> it is doing this. Doesn’t that still dynamically allocate in Ada?</p>
<blockquote>
<p>Ada doesn’t have a tuple construct and does not allow returning multiple values from a subprogram (except by declaring a full-fledged record type). Hence, a way to return multiple values from a subprogram is to use out parameters.</p>
</blockquote>
<p>I love tuples, oh well.</p>
<blockquote>
<p>While reading an out variable before writing to it should, ideally, trigger an error, imposing that as a rule would cause either inefficient run-time checks or complex compile-time rules. So from the user’s perspective an out parameter acts like an uninitialized variable when the subprogram is invoked.</p>
</blockquote>
<p>Hmmmmmmmmmmmmmmmmmm. This seems <em>extremely</em> against Ada’s safety focus.</p>
<pre><code>with Ada.Text_IO; use Ada.Text_IO;
procedure Outp is
procedure Foo (A : out Integer) is
begin
Put_Line (Integer'Image (A));
end Foo;
A : Integer := 12;
begin
Foo(A);
end Outp;
</code></pre>
<p>gives</p>
<pre><code>Run...
outp.adb:4:19: warning: formal parameter "A" is read but never assigned
outp.adb:10:08: warning: "A" modified by call, but value might not be referenced
32514
2 errors.
</code></pre>
<p>Each time I run it, I get a different number. Seems bad?</p>
<blockquote>
<p>There are two ways in which Ada helps shield programmers from the dangers of pointers:One approach, which we have already seen, is to provide alternative features so that the programmer does not need to use pointers. Parameter modes, arrays, and varying size types are all constructs that can replace typical pointer usages in C.Second, Ada has made pointers as safe and restricted as possible, but allows “escape hatches” when the programmer explicitly requests them and presumably will be exercising such features with appropriate care.</p>
</blockquote>
<p>Seems great! I’m very here for this.</p>
<p>… but they have <code>null</code>. Of course they do, given the timeframe Ada was developed. Oh well.</p>
<p>Okay, so <code>new</code> is a keyword, and is in the language. But freeing is an unsafe standard library API? I guess this makes sense, similarly to how leaking memory isn’t unsafe in Rust. Feels weird, though.</p>
<p>I guess, reading more, the idea is that you should basically never need to do this yourself. I think this is where I was getting caught up earlier with the “return an unknown type” bit too. It <em>is</em> allocating, but since you’re not typing <code>new</code>, you’re not responsible, and so that’s safer. That makes sense. I hope my understanding is correct here, but I’m not totally sure.</p>
<p>This does mean that it feels like allocations exist, but are hidden from you, the user. That’s okay, but it’s very different than Rust.</p>
<blockquote>
<p>Ada’s variant records are very similar to Sum types in functional languages such as OCaml or Haskell. A major difference is that the discriminant is a separate field in Ada, whereas the ‘tag’ of a Sum type is kind of built in, and only accessible with pattern matching.There are other differences (you can have several discriminants in a variant record in Ada). Nevertheless, they allow the same kind of type modeling as sum types in functional languages.</p>
</blockquote>
<p>This is pretty cool.</p>
<p>The “Character Types” section still does not explain to me what valid characters are. Oh well. Googling it gives me <a href="https://www.adaic.org/resources/add_content/standards/05rm/html/RM-3-5-2.html">this</a> which says</p>
<blockquote>
<p>The predefined type Character is a character type whose values correspond to the 256 code positions of Row 00 (also known as Latin-1) of the ISO/IEC 10646:2003 Basic Multilingual Plane (BMP). Each of the graphic characters of Row 00 of the BMP has a corresponding character_literal in Character. Each of the nongraphic positions of Row 00 (0000-001F and 007F-009F) has a corresponding language-defined name, which is not usable as an enumeration literal, but which is usable with the attributes Image, Wide_Image, Wide_Wide_Image, Value, Wide_Value, and Wide_Wide_Value; these names are given in the definition of type Character in A.1, “The Package Standard”, but are set in italics.</p>
</blockquote>
<p>Ah. Yeah so that’s why the emoji didn’t work. Did that crash the compiler earlier, is that what was happening?</p>
<blockquote>
<p>Generic subprograms or packages can’t be used directly. Instead, they need to be instantiated, which we do using the new keyword, as shown in the following example:</p>
</blockquote>
<p>Wait, is this the same <code>new</code> keyword as before? Do generics require heap allocation? That is a reasonable design, I’m just a bit surprised.</p>
<blockquote>
<p>Ada exceptions are not types, but instead objects, which may be peculiar to you if you’re used to the way Java or Python support exceptions.</p>
</blockquote>
<p>I found this sentence confusing; aren’t the objects in Java or Python as well? When I was reading this sentence, halfway through, I thought to myself “oh, like the stuff I know, cool” and then the second half was “this is unlike the stuff you know”. Hrm.</p>
<blockquote>
<p>A task can be thought as an application that runs concurrently with the main application. In other programming languages, a task can be called a thread, and tasking can be called multithreading.</p>
</blockquote>
<p>Cool, this is a big topic I’ve been wondering about.</p>
<blockquote>
<p>As we’ve just seen, as soon as the main task starts, its subtasks also start automatically. The main task continues its processing until it has nothing more to do. At that point, however, it will not terminate. Instead, the task waits until its subtasks have finished before it allows itself to terminate.</p>
</blockquote>
<p>Ah interesting, this is different than Rust. Well, Rust’s standard library, anyway.</p>
<p>I wish it explained if tasks <em>are</em> system threads, or if they’re green threads.</p>
<p>I did some googling, and apparently this is implementation-defined. GNAT uses system threads for some of the same reason that Rust decided to expose system threads. Seems good.</p>
<blockquote>
<p>The pragma Assertion_Policy statement is used to force the compiler to generate code to check the precondition.</p>
</blockquote>
<p>Design by contract works at runtime. This is something I always thought was true, but was never actually sure of. Cool.</p>
<blockquote>
<p>To interface with C’s built-in types, we use the Interfaces.C package, which contains most of the type definitions we need. For example:</p>
</blockquote>
<p>Ah ha, this answers my question from earlier. Cool.</p>
<blockquote>
<p>Object-oriented programming (OOP) is a large and ill-defined concept in programming languages and one that tends to encompass many different meanings because different languages often implement their own vision of it, with similarities and differences from the implementations in other languages.However, one model mostly “won” the battle of what object-oriented means, if only by sheer popularity. It’s the model used in the Java programming language, which is very similar to the one used by C++. Here are some defining characteristics:</p>
</blockquote>
<p>Spicy! But not incorrect.</p>
<p>Dot notation exists! But only if you’re actually using objects, er, “tagged types”.</p></content:encoded>
</item>
<item>
<title>What's next for SemVer</title>
<link>https://steveklabnik.com/writing/what-s-next-for-semver/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/what-s-next-for-semver/</guid>
<pubDate>Mon, 11 Feb 2019 00:00:00 GMT</pubDate>
<content:encoded><p>On December 18, 2009<a>1</a>, the Semantic Versioning specification was announced. More commonly known as SemVer, it provided guidance for developers regarding software versions:</p>
<blockquote>
<p>I propose a simple set of rules and requirements that dictate how version numbers are assigned and incremented. For this system to work, you first need to declare a public API. This may consist of documentation or be enforced by the code itself. Regardless, it is important that this API be clear and precise. Once you identify your public API, you communicate changes to it with specific increments to your version number. Consider a version format of X.Y.Z (Major.Minor.Patch). Bug fixes not affecting the API increment the patch version, backwards compatible API additions/changes increment the minor version, and backwards incompatible API changes increment the major version.I call this system “Semantic Versioning.” Under this scheme, version numbers and the way they change convey meaning about the underlying code and what has been modified from one version to the next.</p>
</blockquote>
<p>The author came from the Ruby world, and around the same time, some big things were brewing. <a href="https://bundler.io/">Bundler</a> was being brought into the world. Bundler brought a few new things, but the most important features involved specifying dependencies for your projects. Normally, you’d specify which version of a gem your project needed. Instead, Bundler allowed you to specify a range of versions you’d accept, and it would help figure out which version you should use, according to the SemVer rules.</p>
<p>In January of 2010, npm, a similar program for the shiny new Node.js, also shipped, and also used SemVer.</p>
<p>Thanks to these tools, Ruby and Node enjoyed (and still do, to this day) a vibrant open source ecosystem. SemVer is the underlying concept that made this possible.</p>
<p>However, SemVer is not perfect. In some ways, I think of SemVer as the <code>gofmt</code> of versioning:</p>
<blockquote>
<p>Gofmt’s style is no one’s favorite, yet gofmt is everyone’s favorite.Go Proverbs</p>
</blockquote>
<p>SemVer has its flaws, but it empirically works better than previous attempts at such a thing. That such a thing exists is far more important than how perfect it is. Fundamentally, version numbers are a communication tool, and SemVer provides a framework for said communication.</p>
<p>That said, SemVer <em>does</em> have flaws. Bundler, <code>npm</code>, Rust’s Cargo, NuGet, and a variety of tools have implemented the specification. But there are edge cases, as well as gaping holes, in the spec, and so in some ways, these tools are incompatible.</p>
<p>Over the last few months, several of us<a>2</a> have been talking, and today, we have an announcement to make. We’ve formed a semver team, and we intend to start work on producing a new version of the spec.</p>
<p>Who is “we”? The team roster will be <a href="https://github.com/orgs/semver/teams/maintainers/members">in a GitHub team</a><a>3</a>, but for posterity’s sake, here’s the initial list of members, in the same order GitHub lists them:</p>
<ul>
<li>Anand Gaurav, NuGet</li>
<li>Dave Herman, Notion</li>
<li>André Arko, Bundler</li>
<li>isaacs, npm</li>
<li>Samuel Giddins, CocoaPods</li>
<li>Steve Klabnik, Cargo</li>
</ul>
<p>(I’m putting myself as “Cargo,” because Cargo is the main client of my SemVer implementation. In most of these projects, the person who maintains the SemVer implementation is the same as the project, but in Rust, they’re separate. Cargo is the higher-order bit here, though.)</p>
<p>We have not yet started the work on the next iteration of the specification, but we have agreed on a governance model for SemVer. Described in the new <code>[CONTRIBUTING.md](https://github.com/semver/semver/blob/master/CONTRIBUTING.md)</code>, we’re going with an RFC process, similar to the Rust programming language and many other projects. In short, “substantial” changes to the SemVer spec need to go through a process by which a written description of the changes are proposed, and the team reaches consensus on acceptance.</p>
<p>It also contains some principles we have for the future of SemVer. I’ve copied the important ones for the specification itself here:</p>
<ul>
<li>No RFC will be approved if it is deemed to cause significant breakage to any of the SemVer-using communities represented by the SemVer Maintainers group.</li>
<li>RFCs will be considered formally adopted only when they are approved by the SemVer Maintainers group, and implemented in a simple majority of represented implementations.</li>
<li>Implementations may add functionality in advance of an approved RFC but all such functionality must be flagged as “experimental”, so that users understand it may change in the future.</li>
</ul>
<p>In essence, we don’t intend to re-write the spec, but instead, fill in holes in the spec, find out where our implementations differ, and unify all of our implementations as much as is feasible. We believe that this will not only help all of us, but also help new tools as well. Being compatible with each other is an important part of the value of these tools.</p>
<p>I look forward to letting you know when our work is done! If you’d like to get involved, please watch that governance repository, and submit RFCs! While we’re all excited about the future of SemVer, it’s nobody’s job to do this work. As such, we’ll be moving fairly slowly. Please be patient. Thanks :)</p>
<hr />
<p>You may also want to read <a href="https://haacked.com/archive/2019/02/11/semver-collective/">Phil’s post</a>, reflecting on this transition from the other side. Thanks for being so much of a good steward of SemVer so far, Phil!</p>
<hr />
<ol>
<li>At least <a href="https://web.archive.org/web/20091218170740/http://semver.org/">according to the WayBack Machine</a><a>↩</a></li>
<li><a href="https://crates.io/crates/semver">I maintain Cargo’s <code>semver</code> implementation</a>.<a>↩</a></li>
<li>Ugh, I’m told apparently this can’t be made public? Even better that I put the list here, then.<a>↩</a></li>
</ol></content:encoded>
</item>
<item>
<title>thank u, next</title>
<link>https://steveklabnik.com/writing/thank-u-next/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/thank-u-next/</guid>
<pubDate>Sun, 06 Jan 2019 00:00:00 GMT</pubDate>
<content:encoded><p>I started working at 15, when I took a job as a pizza cook. Over the next seven years, I moved up the ranks, to a driver, shift manager, and then as part of the “new store opening team.” The franchise was growing, and we needed to help new franchisees open their new stores. I’d travel to where the new store was a week before they would open, help train the new staff, and then work their opening weekend. It was really fulfilling work; if pizza paid as well as tech, I’d seriously consider doing it forever.</p>
<p>One time, somewhere in Maryland, I got a bit ill. It wasn’t a good idea for me to work the rest of the week, but the company’s top brass would usually visit to cut the ribbon on the store. It was about a four hour drive back to home, and our COO graciously agreed to let me come back to the ’burgh with him, so I could recover at home. On the drive, I asked him what he did before working here, and the answer really surprised me. “Oh, I was the CEO of American Eagle. I grew them from 20 locations to 200, and then decided to move on.” To a kid from the suburbs, being in charge of one of the most popular clothing brands among my peers seemed like a great gig. Why’d he quit? The answer is something that’s really stuck with me; this happened about fifteen years ago. What he told me was this: at each stage of a company’s growth, they have different needs. Those needs generally require different skills. What he enjoyed, and what he had the skills to do, was to take a tiny company and make it medium sized. Once a company was at that stage of growth, he was less interested and less good at taking them from there. And that was what he was trying to do for our pizza chain.</p>
<p>I’ve mostly worked at small companies. I did a startup, then consulted, then worked for a series of companies with five to 15 people. I quit one of those jobs when I found Rust. What I saw in Rust was something that the world really <em>needed</em>. And I wanted to help it get there. Beyond that, the only real way to get a job working on Rust was to work at Mozilla. And independently of Rust, that was something I’m really excited about.</p>
<p>Here’s a poster on my wall:</p>
<p><img src="https://svbtleusercontent.com/x35Th9VqQFuuZ4SFBEsZkU0xspap_small.jpeg" alt="https://svbtleusercontent.com/x35Th9VqQFuuZ4SFBEsZkU0xspap_small.jpeg" /></p>
<p>I’ve been using Firefox before it was called Firefox. This poster is of a two-page ad that Mozilla took out in the New York Times to announce Firefox 1.0. I’ve long believed in Mozilla’s mission. This was an exciting opportunity!</p>
<p>Fast forward a few years. Rust has been growing, we’re doing great things. I love my team. But every time that this means interacting with Mozilla more broadly, I end up being frustrated. I could say a lot of things here, but I won’t get into the details. What’s really important is this: I’m not proud to be a Mozillian anymore. A variety of incidents contributed to this, but at the end of the day, it’s what’s true. Furthermore, I don’t have any personal opportunity at Mozilla; I recently discovered I’m the lowest-paid person on my team, and Mozilla doesn’t pay particularly well in the first place. In order to have any kind of career growth, I would have to not do the work that aligns with my skills, and what I’m doing now is really how I can have the most impact on Rust moving forward.</p>
<p>I don’t believe that Mozilla’s direction has any chance of changing; when I’ve tried to express my frustrations, I’ve only gotten disciplined. Mozilla is not interested in hearing what I have to say. And that’s fine, but when I take a step back and think about things, that means it’s time to go, for both my sake and Mozilla’s. So I’ve just put in my two weeks’ notice.</p>
<p>So what’s next? Well, the first thing is that I don’t plan to stop working on Rust. How much I’ll be able to depends on what’s next, but that’s the great part about open source; I can continue to help the thing I love, even if it might not be full-time anymore.</p>
<p>I’ve also been enamored with another technology recently: WebAssembly. 2019 is going to be a <em>huge</em> year for WebAssembly, even if many people don’t know it yet, and may not see the effects until 2020. Once again, I find myself seeing something that the world needs, and I think I have the skills to help make that happen. Unlike with Rust, I don’t think that WebAssembly needs my help to actually work on it itself; I think it’s not yet found its killer application. I’d like to help find out what that is, and help both wasm and that application grow.</p>
<p>There’s something similar in Google’s Fuchsia. I’ve worked with the web for a long time, but doing Rust for the last few years, I’ve also really enjoyed systems-level work. Fuchsia is very interesting and a bit of an enigma, but I can’t wait to see how it turns out. They use a lot of Rust, and plan to use more. I’m not really a kernel hacker, and I’m not sure that Fuchsia is far enough along to really need my skills, but it also really piques my interest.</p>
<p>Additionally, I’m always open to other things. Before Rust, I worked at Balanced Payments, which was one of my favorite jobs I’ve ever had. I would have never thought I’d want to work at a payments company, but by being open to it, I found an awesome role, with fantastic co-workers. Maybe what I do next will be something completely different!</p>
<p>In terms of the actual <em>work</em> I would like to do, I don’t think a traditional engineering role really suits me. Don’t get me wrong, I love to write some code, but I don’t think that those kinds of roles really play to my unique strengths. What I really love to do is teaching, evangelizing, and growing something. I’ve done it with Rust, and I can do it again. I could see several roles or hybrid ones here, with titles like “developer evangelist,” “product manager”, or possibly even “VP of Growth.” I want something that challenges me, and where I can grow my career.</p>
<p>So if you’re:</p>
<ul>
<li>A company that wants to invest in Rust, or at Rust in your company</li>
<li>A company that thinks they can take webassembly (and their product built on it) to the next level</li>
<li>The Fuchsia team at Google</li>
</ul>
<p>or something completely different that you think I’d be psyched about, please <a href="mailto:[email protected]">email me</a> and let’s see where things go!</p></content:encoded>
</item>
<item>
<title>Thoughts on Rust in 2019</title>
<link>https://steveklabnik.com/writing/thoughts-on-rust-in-2019/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/thoughts-on-rust-in-2019/</guid>
<pubDate>Thu, 27 Dec 2018 00:00:00 GMT</pubDate>
<content:encoded><p>Each year, <a href="https://blog.rust-lang.org/2018/12/06/call-for-rust-2019-roadmap-blogposts.html">we ask the community to write blog posts about what they want to see on Rust’s roadmap for the next year</a>. This is my post for Rust in 2019.</p>
<h2>Rust 2021: Maturity</h2>
<p>This year is also a bit special; in 2018, we introduced “editions” to Rust, on a roughly three-year schedule. So now is not just a good time to think about 2019, but about 2020 and 2021 as well. Rust 2015 was about “stability”. Rust 2018 was about “productivity.” I’d like Rust 2021 to be about “maturity.”</p>
<p>In order to get there, here’s what we need in 2019.</p>
<h2>No <em>new</em> features</h2>
<p>Emphasis on “new” here. What do I mean by this? Well, there are a few features that are in the pipeline that I <em>do</em> think should land:</p>
<ul>
<li><code>async</code>/<code>await</code></li>
<li>GATs</li>
<li><code>const</code> generics</li>
</ul>
<p>And <em>possibly</em></p>
<ul>
<li>Specialization</li>
</ul>
<p>None of these features are new; we’ve got their basic designs in place. These features are also <em>significant</em> and <em>foundational</em>; we need <code>async</code>/<code>await</code> (and to some degree, GATs) for a great networking story, we need <code>const</code> generics for an excellent numerics story.</p>
<p>But after that? I’d like to see a moratorium on major features until 2020, and I’d prefer if we limited 2020 to two or three, if any.</p>
<p>We’ve hit a sweet spot. We’ve always said that Rust 1.0 was about stability, not completeness. I think we’re fast approaching completeness.</p>
<p>That said, I don’t think that the language team should be disbanded or anything; I think their work should transition to working on <em>specifying</em> what we already have, in more detail. I’m not sure if we can have the reference be done in 2019 (more on that later), but I’d like it to be much further along. That can only happen with the help of the language team, and they can only do that work if they have the time.</p>
<h2>Refine the RFC process</h2>
<p>The RFC process needs to be significantly re-vamped. Niko wrote a great post on this <a href="http://smallcultfollowing.com/babysteps/blog/2018/06/20/proposal-for-a-staged-rfc-process/">back in June</a>, and I think it’s really, really, really important. I’d like to work on an RFC to propose this, so if you’re interested, we should talk.</p>
<p>Niko already makes the case and lays out some foundations, so I won’t say more there.</p>
<h2>Pay down organizational debt</h2>
<p>Consider this section an endorsement of <a href="https://boats.gitlab.io/blog/post/rust-2019/">everything boats said</a>. I cannot say it better, so I will just leave it at that.</p>
<h2>Figure out documentation sustainability</h2>
<p>This year was a bad year for the docs team. The book got shipped, and that was great. We had some people pitch in on the reference, and their work was amazing. Some of the other documentation writers moved on to work on <code>rustdoc</code>, which was important.</p>
<p>But a lot of the goals we wanted to do about writing more docs themselves never came to fruition. For example, contributing to major ecosystem crates? Never happened. The cookbook isn’t done. Rust by Example still languishes. The standard library could use a lot of love.</p>
<p>We just don’t have the people to make this happen. We’ve tried, but nothing has worked. It’s possible that this is simply unfixable, after all, most programmers don’t like writing docs. But I also don’t want to give up on it either. I’m not sure what to do here, but I do know that it is a major problem.</p>
<h2>Conclusion</h2>
<p>There’s a lot of work to do, and I’m very excited to do it. I think Rust is in an <em>excellent</em> place, and with some work, we can make Rust even more amazing in a year’s time.</p></content:encoded>
</item>
<item>
<title>Six years with Rust</title>
<link>https://steveklabnik.com/writing/six-years-with-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/six-years-with-rust/</guid>
<pubDate>Fri, 21 Dec 2018 00:00:00 GMT</pubDate>
<content:encoded><p>Today is six years since I first heard of Rust. <a href="https://words.steveklabnik.com/five-years-with-rust">I wrote a post last year</a> about it.</p>
<p>This past year was… intense. Rust 1.31 was basically Rust 2.0, at least in the marketing sense. I burned myself out getting the first edition of the book together for Rust 1.0, and I burned myself out getting the edition shipped.</p>
<p>Let’s talk about the bad and the good. Bad first so we end on the high notes.</p>
<h2>The Bad</h2>
<p>This year, the docs team sort of… died. Not entirely, but it’s… different now. We used to have a single docs team that handled most everything docs related. The core of that for the past few years has been me and two others. But early this year, we split the sub-teams into fewer individual teams. The old docs team became the docs team, the <code>rustdoc</code> team, the Rust By Example team, and the Reference team. But we didn’t really attract many more people. We did to some degree! The reference team did a <em>ton</em> of amazing reference work. RBE has still struggled. I stayed on the docs team, and the other two went to the <code>rustdoc</code> team. That’s fine, and they’re doing great work, but it’s a little lonely over here now. I can’t decide how much of this issue is “it’s really hard to get people to volunteer to write docs” and how much of this issue is “I mis-managed the team(s) somehow and so it’s my fault.”</p>
<p>Shipping the edition was a monumental effort that caused a lot of debt. We did a good job at avoiding <em>technical</em> debt, but we have a lot of social and organizational debt. <a href="https://boats.gitlab.io/blog/post/rust-2019/">See here for more</a>. This has to be paid off now…</p>
<p>There’s a few other things, but let’s stop there.</p>
<h2>The Good</h2>
<p>We shipped the edition! In fact, we invented the entire edition system and then shipped our first one. Whew! This is a <em>massive</em>, <em>massive</em> accomplishment. Figuring out how we are going to manage Rust’s evolution decades into the future is, frankly a pretty amazing thing to do in twelve months. Non-lexical lifetimes finally landed. Procedural macros finally landed. Rocket is almost on stable. <code>async</code>/<code>await</code> is almost here, and <code>Futures</code> are finally where they should be after two years of work. We did <em>so much</em> good work this year, technically speaking. I’m very proud.</p>
<p>We shipped the new website! This was met with some controversy, but every single website redesign in the history of redesigns has been met with controversy. Doing so in less than a year is also an incredible accomplishment. It almost always takes way longer. We now have the foundations for Rust web properties going forward. This will also pay off a ton later.</p>
<p>We made wasm a thing! I really really wanted wasm to be a thing. A lot of people put in a ton of work, and Rust’s wasm support is bar-none best-in-class. I think this is still a <em>massive</em> growth area for Rust, and we’re almost a full year ahead of almost everyone else on this.</p>
<p>Speaking of growth, Rust’s commercial uptake skyrocketed this year. I met someone for the first time who told me “I learned Rust because it’s what we use at work.” A ton of companies, big and small, have started deploying Rust in critical places, and report good things. Out of the FAANGs, we’ve got Facebook, Amazon, and Google. Three out of five ain’t bad. This tweet is wild:</p>
<blockquote>
<p>Introducing #HyperledgerSawtooth version 1.1, an enterprise-grade #blockchain platform with support for #WebAssembly smart contracts and great performance with @rustlang. Empower your business. https://t.co/wl9OYVnUsR #HyperledgerForum pic.twitter.com/5vurpbVMP1
— Intel Security (@IntelSecurity) December 12, 2018</p>
</blockquote>
<p>So is this one:</p>
<blockquote>
<p>Discover Rust, a C-like language that supports imperative, functional, and object-oriented programming. https://t.co/fCgY4x6zqc
— IBM Developer (@IBMDeveloper) June 10, 2018</p>
</blockquote>
<p>There’s still lots more to do, but things are headed in the right direction.</p>
<p>Speaking of heading in the right direction, tomorrow will be the compliment to this post; my entry for <code>#rust2019</code>. Catch you then!</p></content:encoded>
</item>
<item>
<title>Why is Clojure so stable?</title>
<link>https://steveklabnik.com/writing/why-is-clojure-so-stable/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/why-is-clojure-so-stable/</guid>
<pubDate>Tue, 18 Dec 2018 00:00:00 GMT</pubDate>
<content:encoded><p>There’s been some Programming Language Discourse lately, and I have some thoughts. It’s mostly centered around <a href="https://news.ycombinator.com/item?id=18702802">a comment on the orange website</a>:</p>
<blockquote>
<p>I would put it simply: Clojure was designed by a thinker, who creates when he is away from the keyboard, not in front of it. When one releases and breaks the code in his head first, very few breaking changes are left for the public releases.</p>
</blockquote>
<p>I think, regardless of this person being right or wrong, there’s an interesting question here. Let’s look at <a href="https://news.ycombinator.com/item?id=18701834">the comment this is ultimately responding to</a>:</p>
<blockquote>
<p>From the perspective of a (fairly large-scale at this point) app developer: I find it great that Clojure places such emphasis on backwards compatibility. In general, migration to newer Clojure versions is completely painless.The language has been designed by experienced and mature people and doesn’t go through “let’s throw everything out and start again” phases like so many other languages do.</p>
</blockquote>
<p>Clojure has a well-deserved reputation for stability. The argument is not about that, but about <em>why</em> Clojure is so stable.</p>
<h2>Individuals and their greatness</h2>
<p>A lot of people, like the original commentator, believe that Rich, the BDFL of Clojure, is responsible. I think Rich deserves a lot of credit, but at the same time, this explanation is a little bit too close to the “great man theory of history.” The Great Man theory was originally proposed in the mid 1800s by Thomas Carlyle, and goes something like this: "“the history of the world is but the biography of great men.”</p>
<blockquote>
<p>Don’t get me started about Carlyle, who was a pretty terrible guy even in his own day. A lot of the alt-right, “Dark Englightenment”, and Neo-reactionary folks consider him a really important thinker, and that’s all I’ll say about that.</p>
</blockquote>
<p>What this <em>means</em> is that Important Men make big decisions, and that the lives of the rest of us are irrelevant in the scheme of things. He cited people like Shakespeare, Martin Luther, and Napoleon. In Carlyle’s mind, history was changed by these men and them alone. The French Army didn’t win wars, Napoleon won wars.</p>
<p>I tend to subscribe to the opposite, that is, “history from below,” or “a people’s history,” which emphasizes the army over Napoleon, the worker over the CEO. A critic of Carlyle’s, Herbert Spencer, said this:</p>
<blockquote>
<p>You must admit that the genesis of a great man depends on the long series of complex influences which has produced the race in which he appears, and the social state into which that race has slowly grown. … Before he can remake his society, his society must make him.</p>
</blockquote>
<p>That is, we have to look at historical context to see what created these changes, not just a few famous individuals.</p>
<blockquote>
<p>Two sidebars here: first, Spencer is the person who coined the phrase “survival of the fittest.” Second, there is an irony in framing this particular philosophical debate about if individuals or history cause historical change by citing well-known opinions of two individual men. This idea was so popular, and still is, that it’s something many of us latch onto, sometimes subconsciously, even those who don’t believe it!</p>
</blockquote>
<p>So, beyond Rich, who as I said before, does deserve a lot of credit… why might Clojure be so stable? I can think of three reasons. There may be more, but these are what comes to my mind.</p>
<h2>Clojure is a Lisp</h2>
<p>First of all, Clojure is a Lisp. This has a wide array of reasons why stability is a bit easier than in other languages.</p>
<p>The first is Lisps’ lack of syntax. Most languages have updates that introduces new syntax. As this grows larger and larger, the design space gets more and more constrained, making updates harder and harder. Lisps, on the other hand, rarely grow syntactically. Clojure has more syntax than most Lisps, but it still has far less syntax than non-Lisp languages.</p>
<p>Another perspective of this is homoiconicity. This property means that “new syntax” for Lisps can be introduced through macros, and not through the base language. In most other languages, users cannot extend the language to provide new syntax, but in Clojure, they can.</p>
<p>Lesson: the smaller your core language, the more stable you can be.</p>
<h2>Clojure is dynamically typed</h2>
<p>There are many, many, many ways to debate the virtues of static typing against those of dynamic typing. However, one area in which dynamically typed languages clearly win out, in my mind, is that they have a wider set of tools at their disposal to make changes without breaking code. This cuts both ways; they also make it much, much harder to know if you’ve broken code as well. Those are able to be mitigated through a variety of techniques, and we can argue about the tradeoffs here, but I think the possibility of advantage is plainly clear.</p>
<p>Lesson: it can be easier to change a dynamic system than a static one. This is both good and bad, but for now we’re focused on the good.</p>
<h2>Clojure’s development is pretty slow</h2>
<p>Clojure famously optimizes contributions around Rich and his process, at the expense of others. Debating this has been a real sore spot lately, and I don’t mean to pile onto this part of the discussion. I’m not interested in suggesting, in this blog post, if this is good or bad. I’m interested in the effects it has.</p>
<p>If we ask <a href="https://github.com/clojure/clojure/compare/1.0...master">GitHub how many commits were between 1.0 and <code>master</code></a>, the answer is 1,957. If we look at <a href="https://github.com/clojure/clojure/compare/clojure-1.9.0...clojure-1.10.0">commits between 1.9.0 and 1.10.0</a>, we get 138 commits. Clojure 1.9 was released December 8, 2017. Clojure 1.10 was released December 17, 2018. That’s 138 commits for that release, for a year.</p>
<p>In contrast, <a href="https://github.com/rust-lang/rust/compare/1.30.0...1.31.0">between 1.30 and 1.31</a>, six weeks of development, Rust had 1,694 commits. that’s almost as many as Clojure has had in the last nine and a half years.</p>
<p>Lesson: It’s much easier to keep things stable when you don’t change as often.</p>
<hr />
<p>All of these factors tie into each other. Clojure doesn’t need to change as much, since the core language is a Lisp. When it does need to change, it has good tools at its disposal to change. Also, note that Rich was the one who made a lot of these decisions! He absolutely deserves credit for a job well done, even if I think that there are many other factors at play.</p>
<p>Furthermore, ascribing this entirely to Rich makes it harder for other languages to learn good lessons from Clojure’s successes; they can’t just get a Rich. By digging in a bit deeper, we can apply these lessons a bit more broadly. Good language design requires taking a sober look at various tradeoffs, and making the best decision you can make, regardless if you’re a BDFL or part of a language design team.</p></content:encoded>
</item>
<item>
<title>Borrow checking, escape analysis, and the generational hypothesis</title>
<link>https://steveklabnik.com/writing/borrow-checking-escape-analysis-and-the-generational-hypothesis/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/borrow-checking-escape-analysis-and-the-generational-hypothesis/</guid>
<pubDate>Wed, 10 Oct 2018 00:00:00 GMT</pubDate>
<content:encoded><p>There’s a fairly common argument that pops up on programming forums, and it’s about the nature of what “garbage collection” means. In the research world, <a href="http://www.memorymanagement.org/glossary/g.html#term-garbage-collection">this</a> is what GC means:</p>
<blockquote>
<p>Garbage collection (GC), also known as automatic memory management, is the automatic recycling of dynamically allocated memory(2). Garbage collection is performed by a garbage collector which recycles memory that it can prove will never be used again. Systems and languages which use garbage collection can be described as garbage-collected.</p>
</blockquote>
<p>Historically, there has been two major forms of GC: reference counting, and tracing. The argument happens because, with the rise of tracing garbage collectors in many popular programming languages, for many programmers, “garbage collection” is synonymous with “tracing garbage collection.” For this reason, I’ve been coming around to the term “automatic memory management”, as it doesn’t carry this baggage.</p>
<hr />
<p>I’ve often seen people make statements like this one, from the Rust subreddit this morning:</p>
<blockquote>
<p>Manual memory management requires more work than garbage collected. Its a trade off of course to be more performant or use lower resources. When and where should Rust be used or not used according to you?</p>
</blockquote>
<p>While I don’t <em>completely</em> disagree with this sentiment, it’s also never quite sat right with me. Yes, Rust is a bit harder <em>at the start</em>, but once you get over a hump, I don’t generally find writing Rust to be significantly harder than using a GC’d language. I’ve been trying to figure out why that is.</p>
<blockquote>
<p>The idea that it’s easy for me because I’m an amazing programmer is not an answer I’m willing to accept here, both because I don’t think that’s particularly true, but also because I’ve seen Rust have a pretty wide appeal to a fairly large group of people, and they can’t all be amazing programmers. It’s just math.</p>
</blockquote>
<hr />
<p>My first clue comes from my own programming history. Some of my first programming languages were C, C++, and Java. These languages are <em>statically</em> typed, that is, they require you to declare (or have inferred) the type of variables up front, and check that the type holds. That is, they’re known at compile time. I can remember the first time I came across Perl, and the concept of dynamic typing. Wow, this is <em>so much easier</em>! I could do <em>anything</em>. From then on, I pretty much focused on dynamically typed languages.</p>
<p>While doing my work in Ruby, one thing I worked hard on was testing. Rubyists are big on tests; with a language that’s <em>so</em> dynamic, you really need them to make sure your stuff is working properly.</p>
<p>When I decided to start writing a bunch of Rust, I had to get used to using static types again. But I also wrote less tests. Many of the things that I was testing for were things that were taken care of by the type system. In other words, my tests were sort of like “dynamic type checking.” Instead of being checked by a compiler at compile-time, these behaviors of my system were checked at run time, by my tests. This isn’t a novel insight; many others have stated this relationship between tests and types.</p>
<p>We often acknowledge that writing tests is a <em>skill</em>. When I first started with Ruby, I was bad at testing. I got better with practice. But I don’t think we collectively acknowledge that working with types is <em>also</em> a skill. At first, it’s tough. But as you develop those skills, it becomes easier and easier.</p>
<hr />
<p>Let’s switch gears a little, to garbage collection.</p>
<p>In garbage collection research, there’s a term, the <a href="http://www.memorymanagement.org/glossary/g.html#term-generational-hypothesis">“generational hypothesis”</a>. It states</p>
<blockquote>
<p>In most cases, young objects are much more likely to die than old objects.</p>
</blockquote>
<p>If that succinct description doesn’t make sense, here’s a slightly longer one: there’s a kind of GC called a “generational garbage collector.” There are a few options when implementing one, but I’m going to describe one form in particular. What it does is this: new objects are allocated in a section of memory called the “nursery,” an analogy to where human children go shortly after they’re born. When the GC runs, it looks through the nursery. Any objects that survive are promoted to a new section of memory. Some collectors may have multiple of these “generations.” The generational hypothesis states that most objects die in the nursery. The GC can then take advantage of this fact by scanning the nursery more often than the older generations, which allows you to not scan every object every collection.</p>
<p>Empirically, the generational hypothesis is true. But I haven’t seen much investigation into <em>why</em> it is true. That said, I’m also not a GC expert. If you have references here, please get in touch.</p>
<hr />
<p>There’s another concept related to GCs called “escape analysis.” This is an optimization which several languages do. The idea is fairly straightforward: if something is declared in a function, and never leaves (“escapes”) that function, you don’t need to put the object on the heap, you can stack allocate it instead.</p>
<blockquote>
<p>“It isn't that the generational hypothesis isn't true for Go, it's just that the young objects live and die young on the stack.The result is that generational collection is much less effective than you might find in other managed runtime languages” #golang https://t.co/aXEleY2yQ6
— Dave Cheney (@davecheney) July 18, 2018</p>
</blockquote>
<p>If you squint a little at this tweet, you can almost think of escape analysis as a <em>stronger</em> version of the generational hypothesis. The claim is that escape analysis applies so often to Go code that any object that reaches the heap is effectively in the older generation.</p>
<p>Why is this? I’m not totally sure, but I have a feeling that is has to do with the core concept of functions and a call stack. Additionally, the value/reference type distinction. The more you program with values in functions, rather than references, the less likely objects are to escape past the call stack. in generational hypothesis terms, values allocated lower in the call stack tend to live longer, and ones higher in the call stack tend to live shorter.</p>
<hr />
<p>There’s an interesting area of type research called “gradual typing.” The idea is that typing is not an either/or, static vs dynamic. It’s a spectrum. Some things are statically typed, some things are dynamically typed, and it all works together. The idea is that you can start off writing things that are dynamically typed, and then once you’re more sure of the behaviors you want, you can start to statically type them.</p>
<p>TypeScript is a great example:</p>
<pre><code>// person is dynamically typed
function greeter(person) {
return "Hello, " + person;
}
// person is statically typed
function greeter(person: string) {
return "Hello, " + person;
}
</code></pre>
<p>TypeScript is really, really growing in popularity. I think one of those factors is the gradual typing; you can slowly work on your type-using skills, and you don’t have to use them everywhere.</p>
<hr />
<p>Interestingly enough, many JIT compilers rely on the fact that in practice, programmers tend to pass the same types of things to functions, even if the language is dynamically typed. This is called <a href="https://hacks.mozilla.org/2017/02/a-crash-course-in-just-in-time-jit-compilers/">“type specialization”</a>. This is similar to the generational hypothesis to me, but in a different domain. And as I said about it, empirically, this is true. But I haven’t seen much investigation into <em>why</em> it is true. That said, I’m also not a JIT expert. If you have references here, please get in touch.</p>
<hr />
<p>So what’s all this have to do with Rust?</p>
<p>Similar to the dichotomy between static and dynamic typing, I think that Rust is proving out a new niche in the concept of garbage collection. That is, historically, we think of GC as something <em>dynamic</em>, that is, it involves a run-time component that does stuff at runtime. However, the idea of automatic memory management doesn’t <em>inherently</em> mean that it has to execute at runtime.</p>
<blockquote>
<p>I should mention that C++ paved a lot of the path here too, so maybe the above is worded too strongly. I do think that Rust is significantly different here, but you may not. If you don’t, please replace the above with “Rust is continuing to prove out a niche pioneered by C++,” and that’s fine by me.</p>
</blockquote>
<p>This also opens up two more things about why I don’t feel that “Rust is too hard and so you should use a GC’d language when you can afford it.” First of all, Rust’s strong, static checking of memory provides me a similar peace of mind to a runtime GC. To make an analogy to types and tests, when I program in a language like C, I have to think a lot about the details of memory management, because I don’t really have anything at all checking my work. Its memory management is completely unchecked, similar to (compile time) types in a Ruby program. But with automatic memory management, I don’t have to worry about this, as the memory management system has my back. I won’t get a use after free, I won’t get dangling pointers.</p>
<p>I further think this line of thought is on the right track because lifetimes are generic types, and the ownership system is affine types.</p>
<hr />
<p>This also relates to the “difficulty” of Rust. In the same way that learning to use static types takes time, learning to use static memory management takes time. But once you’ve developed that skill, you can rely on those static checks to free up your brain to instead focus on your program itself, and not the house-keeping tasks that these systems are checking for.</p>
<p>And, just like with static types, different people will make different choices of tradeoff for effort. Some people refuse to use dynamically typed languages, because they’ve invested the time into effectively using static types and prefer to have these things checked. Others don’t want to waste time on learning these kinds of things, and are more focused on shipping things. They have their own set of skills to ensure reliability. I think both approaches make sense depending on what you’re doing.</p>
<hr />
<p>In some ways, you can think of the borrow checker as an even stronger form of escape analysis; that is, instead of saying “oh, this escapes, so allocate it on the heap”, the borrow checker says “oh, this escapes, so error at compile time.” For example, this code:</p>
<pre><code>fn bar() -&gt; i32 {
5
}
fn foo() -&gt; &amp;'static i32 {
&amp;bar()
}
</code></pre>
<p>gives this error:</p>
<pre><code>error[E0716]: temporary value dropped while borrowed
--&gt; src/lib.rs:6:6
|
6 | &amp;bar()
| ^^^^^ creates a temporary which is freed while still in use
7 | }
| - temporary value is freed at the end of this statement
|
= note: borrowed value must be valid for the static lifetime...
</code></pre>
<p><code>bar()</code> produces a value, and so <code>&amp;bar()</code> would produce a reference to a value on <code>foo()</code>’s stack. Returning it would be a dangling pointer. In a system without automatic memory management, this would cause a dangling pointer. In a system with dynamic automatic memory management, this would heap allocate the return value of <code>bar()</code>, so that everything works out. But Rust’s automatic memory management is fully static, and so it has to error in this situation.</p>
<hr />
<p>Similar to gradual typing, this means that you can also have gradual GC. For example, Rust does have <code>Rc&lt;T&gt;</code> and <code>Arc&lt;T&gt;</code>, which are reference-counted types, clearly GC according to the literature. This moves the memory management from static to dynamic.</p>
<blockquote>
<p>I don’t think that the transition here is as seamless as in TypeScript, though that’s a whole other post… additionally, it’s not like gradual typing is perfect.</p>
</blockquote>
<hr />
<p>One last thing before I go: the lifetime elision rules are also an empiric statement about how programmers write code, but I’m not sure what the underlying reasons are. From <a href="https://github.com/rust-lang/rfcs/blob/master/text/0141-lifetime-elision.md">the RFC</a>:</p>
<blockquote>
<p>We found that of the 169 lifetimes that currently require annotation for libstd, 147 would be elidable under the new rules, or 87%.</p>
</blockquote>
<p>If it were 100%, we could never require you to write annotations. But with 87%, we can let you not write annotations, and most code that follows most normal coding practices never needs to write explicit lifetimes.</p></content:encoded>
</item>
<item>
<title>Should you learn C to "learn how the computer works"?</title>
<link>https://steveklabnik.com/writing/should-you-learn-c-to-learn-how-the-computer-works/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/should-you-learn-c-to-learn-how-the-computer-works/</guid>
<pubDate>Tue, 02 Oct 2018 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve often seen people suggest that you should learn C in order to learn how computers work. Is this a good idea? Is this accurate? I’m going to start with my conclusion right upfront, just to be crystal clear about what I’m saying here:</p>
<ul>
<li>C is not “how the computer works.”</li>
<li>I don’t think most people mean this phrase literally, so that is sort of irrelevant.</li>
<li>Understanding the context means that learning C for this reason may still be a good idea for you, depending on your objectives.</li>
</ul>
<p>I plan on making two follow-up posts as well, exploring more implications of this idea, but this is already quite a lot. I’ll update this post with links to them when I make them.</p>
<hr />
<p>I’ve often seen people suggest this:</p>
<blockquote>
<p>By learning C, you can learn how computers work.</p>
</blockquote>
<p>I think that this idea is not inherently wrong, but does come with some caveats. As long as you keep those caveats in mind, I think this can be a viable strategy for learning new and important things. However, I rarely see people discuss this slogan in detail, so I’m writing this post so that I can provide that context that I believe is sorely needed. If you’re thinking about learning C for this reason, this post is written for you. I hope it’ll assist you as you learn.</p>
<blockquote>
<p>Before we truly begin, I’d like to say one more thing: if you want to learn C, then learn C! Learning things is great. Learning C was quite important to my understanding of computing and my career. Learning C and about its place in programming language history will make you a better programmer. You don’t need a justification. Learn stuff for the sake of learning it. This post is intended to be a guide to your quest for understanding, not a suggestion that you should or should not learn C.</p>
</blockquote>
<p>First of all, let’s talk about who is generally recommended this idea. If you’re trying to “learn how computers work,” then it stands to reason that you don’t currently understand that. Which programmers don’t understand how computers work? I’ve mostly seen this sentiment come from people who primarily program in dynamically typed, “scripting” languages, like Ruby, Python, or JavaScript. They “don’t know how computers work” because these languages operate inside of a virtual machine, and therefore, the semantics of the virtual machine are what actually matters. After all, the whole idea of a virtual machine is to provide portability. The goal is to only require knowledge of what’s portable, rather than rely on the details of the hardware that the VM is running on.</p>
<p>There’s just one problem with this: C <em>also</em> operates inside of a virtual machine.</p>
<h2>The C abstract machine</h2>
<p>From the <a href="http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf">C99 spec</a>, Section 5.1.2.3, “Program Execution”:</p>
<blockquote>
<p>The semantic descriptions in this International Standard describe the behavior of an abstract machine in which issues of optimization are irrelevant.</p>
</blockquote>
<p>In my opinion, this is the most important concept to understand when learning C. C does not “describe how the computer works,” it describes how the “C abstract machine” works. Everything else of importance flows from this concept and its implications.</p>
<blockquote>
<p>A further note: I’ve chosen C99 here, which is not the latest C standard. Why? Well, MSVC has an …interesting amount of C support, and I’m a Windows user these days. Yes, you can run clang and gcc on Windows. There is not a ton of difference between C89, C99, and C11 with regards to what we’re talking about in these posts. At some point you have to pick. The version I’ve linked here includes some amendments to the initial spec.</p>
</blockquote>
<p>You may have heard another slogan when talking about C: “C is portable assembler.” If you think about this slogan for a minute, you’ll also find that if it’s true, C cannot be how the computer works: there are many kinds of different computers, with different architectures. If C is like a version of assembly language that works on multiple computers with multiple architectures, then it cannot function exactly how each of those computers work simultaneously. It <em>must</em> hide details, or else it wouldn’t be portable!</p>
<p>That said, I think this fact is sort of irrelevant, because I don’t think people mean the phrase “C is how the computer works” literally. Before we can talk about that, let’s talk about the C abstract machine, and why many people don’t seem to understand this aspect of the C language.</p>
<h2>Aside: why do people get this wrong?</h2>
<p>I can only relate my own experience, though I think it generalizes to many others.</p>
<p>I learned GW-BASIC, then C, then C++, then Java. I had heard about Java before I started writing it in roughly 1999, four years after it was introduced. Java’s marketing at the time was very anti-C++, and it focused on the JVM as a platform, and how the machine model was what differentiated it from C++, and therefore C. Sun Microsystems no longer exists, but a <a href="https://tech-insider.org/java/research/1996/0123.html">mirror of the press release</a> has this to say:</p>
<blockquote>
<p>Java-based applications are platform-independent; only the Java Virtual Machine needs to be ported to each platform. It acts as an interpreter between an end user’s computer and the Java-based application. An application written in the Java environment can run anywhere, ending the need for porting applications to multiple platforms.</p>
</blockquote>
<p>“Write once, run everywhere” was the big slogan. These two sentences were how I (and many others) came to understand Java, and how it differed from C++. Java has an interpreter, the Java Virtual Machine. C++ does not have a Virtual Machine.</p>
<p>With this huge marketing push, “virtual machine” became synonymous with “large runtime and/or interpreter” in the minds of many. Languages without this feature were too tied to the particular computer, and required porting between platforms, because they are not truly platform independent. Java’s very reason to exist was to change this aspect of writing applications in C++.</p>
<blockquote>
<p>“Runtime”, “virtual machine”, and “abstract machine” are different words for the same fundamental thing. But they’ve since gained different connotations, due to non-essential variance in different implementations of these ideas.</p>
</blockquote>
<p>I personally believe that this marketing, in 1995, is why programmers still misunderstand C’s nature today.</p>
<p>So is this claim false? Why would Sun Microsystems spend millions and millions of dollars promoting a lie? If C is also based on an abstract machine that offers portability between platforms, why did Java need to exist? I think this is key to understanding what people <em>really</em> mean when they say “C is how the computer works.”</p>
<h2>What do people actually mean?</h2>
<p>Even though C operates in the context of a virtual machine, it still is significantly different than something like Java. Sun was not lying. In order to understand why, you have to understand C’s history.</p>
<p>An operating system was written in assembly language for a computer called the “PDP-7” in 1969 at Bell Labs. In 1970, it was christened UNIX. As time went on, Bell Labs bought more and newer computers; including the PDP-11.</p>
<p>When it came time to port Unix to the PDP-11, they considered using a higher-level language, which was a pretty radical idea at the time. Imagine that I said to you “I’m going to write an OS in Java” today, you’d probably laugh, even though <a href="https://en.wikipedia.org/wiki/JavaOS">it is possible</a>. The situation (in my understanding, I wasn’t alive then) was pretty analogous. They considered a language called B, but it didn’t support some of the features that the PDP-11 had, and so they created a successor, naming it “C”, since C was the next letter in the alphabet.</p>
<blockquote>
<p>There was no “A”; B was a successor to BCPL, the “Basic Combined Programming Language.”</p>
</blockquote>
<p>In 1972, on a PDP-11, they wrote the first C compiler, and simultaneously re-wrote UNIX in C. Initially, portability wasn’t the actual goal, but C did resonate with a lot of people, and C compilers were ported to other systems.</p>
<p>In 1978, the first edition of “The C Programming Language,” a book on C, was published. Affectionately known as “K&amp;R,” after its authors, the book was very much not a specification, but provided enough of a description of the language that others attempted to write C compilers. This would later be called “K&amp;R C.”</p>
<p>As UNIX spread, so did C, and both were ported to many computers. In the 70s and 80s, the ports grew and grew. In the same way that C was created because B couldn’t support all of the features of the PDP-11, many compilers had extensions to the language. Since there was only K&amp;R, and not a spec, as long as they were close enough, that was considered acceptable. By 1983, the lack of any sort of standardization was causing issues, and so a group was created at ANSI to produce a spec. C89 was published in 1989, and is sometimes called “ANSI C.”</p>
<p>As such, the C specification was attempting to unify these diverse implementations, on diverse hardware. And so the C abstract machine is sort of the minimal possible specification that would allow the same code to run the same on all platforms. Implementations of C were compiled, not interpreted, and so there was no interpreter, so there was no “VM” in that 1995 sense. However, C programs are written against this abstract, non-existent machine, which is then translated into assembly that is specific to the actual computer that the program is running on. You could not rely on some specific details in order to write portable C code. This makes writing portable C very tricky, as you may have made a platform-specific assumption when writing the initial version of your code.</p>
<p>This is best illustrated with an example. One of the fundamental data types in C is a <code>char</code>, named after “character.” However, the C abstract machine does not define how many bits a <code>char</code> is. Well, it defines it, but not with a number; it defines it as <code>CHAR_BIT</code> big, which is a constant. Section 5.2.4.2.1 of the spec:</p>
<blockquote>
<p>The values given below shall be replaced by constant expressions suitable or use in #if preprocessing directives. … Their implementation-defined values shall be equal or greater in magnitude (absolute value) to those shown, with the same sign.CHAR_BIT: 8</p>
</blockquote>
<p>So in other words, you know that a <code>char</code> must be at least 8 bits in size, but implementations are allowed to be larger. In order to properly code against the “C abstract machine”, you must use <code>CHAR_BIT</code> instead of <code>8</code> as the size when doing <code>char</code>-by-<code>char</code> processing. But this isn’t some sort of interpreter feature, in the way that we think of virtual machines; it’s a property of how the compiler turns your source code into machine code.</p>
<blockquote>
<p>Yes, there are systems where CHAR_BIT isn’t 8.</p>
</blockquote>
<p>So this “abstract machine”, while still technically being the same idea as the Java VM, is more of a compile-time construct to guide compilers in the task of emitting assembly code than some sort of runtime check or property. The equivalent type in Java is <code>byte</code>, which is always 8 bits, and it’s up to the implementation of the JVM to decide how to make that happen on platforms where a byte is larger. (I am not sure if the JVM runs on any of those platforms, but that’s how it would work.) The C abstract machine was created to be a minimal papering over of various hardware, not to be some sort of platform created from whole cloth that’s written in software that your code targets.</p>
<p>So while Sun is technically incorrect, in practice, they mean something different than what they’re literally saying, and what they <em>mean</em> is accurate. It’s the same with “learn C to learn how computers work.”</p>
<h2>Learn C to learn <em>more</em> about how computers work</h2>
<p>What <em>do</em> people actually mean? In the context of “should a Rubyist learn C to learn about how computers work”, this desire to drop “down to the metal,” as it were, is an interest in understanding not only their program, and how it works inside the VM, but to understand how the combo of their program + their VM operates in the context of the machine itself.</p>
<p>Learning C <em>will</em> expose you to more of these kinds of details, because the abstract machine maps so much more closely to hardware, as well as abstractions provided by operating systems. C is very different than these sorts of languages, and so learning C can teach you a lot.</p>
<p>But it’s also important to remember that C is fundamentally an <em>abstraction</em> of hardware, and abstractions are leaky. Don’t conflate what C does or how it operates with the machine itself. If you do, you’re bound to run afoul of these differences, which can cause problems. Most learning resources for C, especially today, as hardware becomes more and more homogeneous, will promote the idea that this is how the computer works. So it can be hard, as a learner, to know what’s going on under the hood, and what’s an abstraction provided by C.</p>
<p>We haven’t even touched on other factors in this discussion, like that due to C’s tremendous popularity, hardware has become more homogeneous because it tends towards being similar to the semantics of C’s abstract machine. If your architecture deviates too much from C’s semantics, C programs may run much more slowly than others, and C programs are often how the speed of hardware is tested. This post is long enough…</p>
<p>For this reason, I think a more accurate version of this statement would be “By learning C, you can learn <em>more</em> about how computers work.” I do think that a rough familiarity with C can be useful to many programmers, and will serve them well even if they don’t write any C themselves. Learning more about C can also give you insight into the history of how our field has developed.</p>
<p>There are other ways to learn this stuff as well; C is not <em>inherently</em> the way to learn these topics, but it is a good option. My next two posts will explore some of the implications of this intersection between portability, the C abstract machine, and how people think about C programs.</p>
<p>There’s so many things to learn in computing. I wish you well on your journey.</p></content:encoded>
</item>
<item>
<title>You can't "turn off the borrow checker" in Rust</title>
<link>https://steveklabnik.com/writing/you-can-t-turn-off-the-borrow-checker-in-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/you-can-t-turn-off-the-borrow-checker-in-rust/</guid>
<pubDate>Fri, 14 Sep 2018 00:00:00 GMT</pubDate>
<content:encoded><p>Every once in a while, someone will talk about <code>unsafe</code> in Rust, and how it “turns off the borrow checker.” I think this framing leads to misconceptions about <code>unsafe</code> and how it interacts with safe code.</p>
<p>Here’s some code that causes a borrow checker error:</p>
<pre><code>fn main() {
let mut x = 5;
let y = &amp;x;
let z = &amp;mut x;
println!("{}", y);
}
</code></pre>
<p>And the error:</p>
<pre><code>error[E0502]: cannot borrow `x` as mutable because it is also borrowed as immutable
--&gt; src/main.rs:5:18
|
4 | let y = &amp;x;
| - immutable borrow occurs here
5 | let z = &amp;mut x;
| ^ mutable borrow occurs here
6 | }
| - immutable borrow ends here
</code></pre>
<p>Rust won’t let us have a <code>&amp;T</code> and a <code>&amp;mut T</code> to the same <code>T</code> at the same time.</p>
<p>If <code>unsafe</code> "turned off the borrow checker, we’d expect this code to work:</p>
<pre><code>fn main() {
let mut x = 5;
unsafe {
let y = &amp;x;
let z = &amp;mut x;
println!("{}", y);
}
}
</code></pre>
<p>However, we get the same error, plus a new warning:</p>
<pre><code>error[E0502]: cannot borrow `x` as mutable because it is also borrowed as immutable
--&gt; src/main.rs:6:22
|
5 | let y = &amp;x;
| - immutable borrow occurs here
6 | let z = &amp;mut x;
| ^ mutable borrow occurs here
7 | }
| - immutable borrow ends here
warning: unnecessary `unsafe` block
--&gt; src/main.rs:4:5
|
4 | unsafe {
| ^^^^^^ unnecessary `unsafe` block
|
= note: #[warn(unused_unsafe)] on by default
</code></pre>
<p>So what does <code>unsafe</code> actually do? Unsafe Rust is a <em>superset</em> of Safe Rust. <code>Unsafe</code> never changes the semantics of Rust code. It instead adds new features that you can only use inside an <code>unsafe</code> block.</p>
<p>One of those features is the ability to de-reference a “raw pointer”. This code <em>does</em> compile:</p>
<pre><code>fn main() {
let mut x = 5;
let y = &amp;x as *const i32;
let z = &amp;mut x as *mut i32;
unsafe {
println!("y: {} z: {}", *y, *z);
}
}
</code></pre>
<p><code>y</code> and <code>z</code> here have the types <code>*const i32</code> and <code>*mut i32</code>, both kinds of raw pointers. In order to print <code>y</code> and <code>z</code>, we must de-reference them, which means that we need an unsafe block.</p>
<blockquote>
<p>Incidentally, this code is safe. Raw pointers are allowed to alias. And we have no &amp;T or &amp;mut Ts here. So we’re good.</p>
</blockquote>
<p>Why does this matter? Well, a lot of people think that as soon as you drop into <code>unsafe</code>, you’re missing the point of Rust, and that you lose all of its guarantees. It’s true that you have to do a lot more work when writing unsafe, since you don’t have the compiler helping you in certain situations, but that’s only for the unsafe constructs.</p>
<p>For example, let’s look at Rust’s standard library, and the <code>[LinkedList&lt;T&gt;](https://doc.rust-lang.org/stable/std/collections/linked_list/struct.LinkedList.html)</code> that it contains. It looks like this:</p>
<pre><code>pub struct LinkedList&lt;T&gt; {
head: Option&lt;NonNull&lt;Node&lt;T&gt;&gt;&gt;,
tail: Option&lt;NonNull&lt;Node&lt;T&gt;&gt;&gt;,
len: usize,
marker: PhantomData&lt;Box&lt;Node&lt;T&gt;&gt;&gt;,
}
</code></pre>
<p>I’m not gonna go into <em>too</em> much detail here, but it has a head pointer, a tail pointer, a length. <code>[NonNull&lt;T&gt;](https://doc.rust-lang.org/stable/std/ptr/struct.NonNull.html)</code> is like <code>*mut T</code>, but asserts that it will never be null. This means that we can combine it with <code>Option&lt;T&gt;</code>, and the option will use the <code>null</code> case for <code>None</code>:</p>
<pre><code>fn main() {
println!("{}", std::mem::size_of::&lt;*mut i32&gt;());
println!("{}", std::mem::size_of::&lt;Option&lt;std::ptr::NonNull&lt;i32&gt;&gt;&gt;());
}
</code></pre>
<p>prints this on my system:</p>
<pre><code>8
8
</code></pre>
<p>This optimization is guaranteed due to guarantees on both <code>Option&lt;T&gt;</code> and <code>NonNull&lt;T&gt;</code>.</p>
<p>So now, we have a sort of hybrid construct: <code>Option&lt;T&gt;</code> is safe, and so we can do some operations entirely in safe code. Rust is now forcing us to handle the null checks, even though we have zero runtime overhead in representation.</p>
<p>I think a good example of how this plays out in practice is in the implementation of <code>[append](https://doc.rust-lang.org/stable/std/collections/linked_list/struct.LinkedList.html#method.append)</code>, which takes two <code>LinkedList&lt;T&gt;</code>s and appends the contents of one to the end of the other:</p>
<pre><code> pub fn append(&amp;mut self, other: &amp;mut Self) {
match self.tail {
None =&gt; mem::swap(self, other),
Some(mut tail) =&gt; {
if let Some(mut other_head) = other.head.take() {
unsafe {
tail.as_mut().next = Some(other_head);
other_head.as_mut().prev = Some(tail);
}
self.tail = other.tail.take();
self.len += mem::replace(&amp;mut other.len, 0);
}
}
}
}
</code></pre>
<p>Here’s some comments to help understand what’s going on:</p>
<pre><code> pub fn append(&amp;mut self, other: &amp;mut Self) {
// do we have a tail?
match self.tail {
// If we don't, then we have no elements, and so appending a
// list to this one means that that list is now this list.
None =&gt; mem::swap(self, other),
// If we do have a tail...
Some(mut tail) =&gt; {
// the first thing we do is, check if the other list has
// a head. if it does...
if let Some(mut other_head) = other.head.take() {
// we set our tail to point at their head, and we
// set their head to point at our tail.
unsafe {
tail.as_mut().next = Some(other_head);
other_head.as_mut().prev = Some(tail);
}
// We set our tail to their tail
self.tail = other.tail.take();
// finally, we add their length to our length, and
// replace theirs with zero, because we took all
// their nodes!
self.len += mem::replace(&amp;mut other.len, 0);
} // there's no else case for that if let; if they don't
// have a head, that list is empty and so there's
// nothing to do.
}
}
}
</code></pre>
<p>We only need <code>unsafe</code> when mutating the <code>next</code> of the <code>tail</code> of the first list, and the <code>prev</code> of the <code>head</code> of the second list. And that’s even only in the case where the <code>tail</code> exists in the first place. We can do things like “set our tail to their tail” in safe code, because we’re never de-referencing pointers. Safe Rust is helping us manage many, but not all, of our guarantees. We <em>could</em> move the <code>unsafe</code> block to encompass the entire body of this function, and Rust would still make us check those <code>Option&lt;T&gt;</code>s. It would still check that <code>self</code> and <code>other</code> are not aliased by <code>&amp;T</code> or <code>&amp;mut T</code>s in the body of the function.</p>
<p><code>unsafe</code> is a powerful tool, and should be used with care. But that doesn’t mean that you throw out many of the things that make Rust what it is.</p></content:encoded>
</item>
<item>
<title>Is WebAssembly the return of Java Applets & Flash?</title>
<link>https://steveklabnik.com/writing/is-webassembly-the-return-of-java-applets-flash/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/is-webassembly-the-return-of-java-applets-flash/</guid>
<pubDate>Wed, 25 Jul 2018 00:00:00 GMT</pubDate>
<content:encoded><p>In my <a href="http://words.steveklabnik.com/webassembly-is-more-than-just-the-web">last post on WebAssembly</a>, I made the following claim:</p>
<blockquote>
<p>Some have compared WebAssembly to Java applets; in some ways, they’re very right, but in some ways, they’re very wrong. Eventually I’ll write a post about the wrong, but for now, the right: in some sense, WebAssembly is a different way of accomplishing what the JVM set out to do: it’s a common virtual machine that can be used to build very cross-platform software.</p>
</blockquote>
<p>A lot of people expressed interest in me elaborating, so let’s get to it! For this post, I’m going to make three comparisons: to Flash, to Java Applets, and <em>occasionally</em> to PNaCL. Secondly, this post is going to focus on the <em>web</em> use-case for WebAssembly, even though the previous post was about non-web uses. We’ll make <em>that</em> comparison in the next post. Finally, this post is kind of like eating tapas, there’s a bunch of little sections. I feel like it’s a bit short, but at the same time, I’m trying to get back into the habit of blogging, and if I try to keep expanding it, it’ll take forever, so it is what it is.</p>
<hr />
<p>For what it’s worth, I think this comparison is very <em>natural</em>; that is, when you hear this pitch for WebAssembly:</p>
<blockquote>
<p>WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable target for compilation of high-level languages like C/C++/Rust, enabling deployment on the web for client and server applications.</p>
</blockquote>
<p>It sounds quite a bit like previous technologies. Asking how they relate, or assuming that they’re a slightly different take on the problem makes a lot of sense. But WebAssembly is significantly different, for a few reasons.</p>
<h2>Wasm won</h2>
<p>The first reason that WebAssembly is different is that it’s succeeded, and those technologies did not. When I say this, I mean it in a very specific way; if you look at the number of Flash applications out there, and compare it to the number of WebAssembly apps out there, Wasm loses, for sure. Those of us that are excited about WebAssembly still have a lot of work to do before it’s widely used.</p>
<p>When I say “WebAssembly won”, I mean that it’s become a part of the Web Platform. Flash, Java Applets, and PNaCL used the browser’s plugin functionality to work. They were, in a sense, <em>outside</em> of the web ecosystem. They were never standardized, and that matters a lot.</p>
<p>In some sense, this is the final point, really. But just saying that it’s true doesn’t explain <em>why</em>. The rest of this post digs into <em>why</em> this occurred, and those points sort of sum up to this one: other technologies didn’t make it, but WebAssembly did. There’s a <em>lot</em> of overlap here, but I’m trying to reasonably separate it out into individual points.</p>
<h2>Other technologies didn’t fit nicely into the platform</h2>
<p>Remember this?</p>
<p><img src="https://svbtleusercontent.com/gY9tj6D62jnC9Fc6RncFhD0xspap_small.gif" alt="https://svbtleusercontent.com/gY9tj6D62jnC9Fc6RncFhD0xspap_small.gif" /></p>
<p>Or this?</p>
<p><img src="https://svbtleusercontent.com/qaipxL2e5a4gLH5cAhRHDn0xspap_small.jpg" alt="https://svbtleusercontent.com/qaipxL2e5a4gLH5cAhRHDn0xspap_small.jpg" /></p>
<p>What about this?</p>
<p><img src="https://svbtleusercontent.com/9mGgBDzvwYYLGHFg4Pafzt0xspap_small.png" alt="https://svbtleusercontent.com/9mGgBDzvwYYLGHFg4Pafzt0xspap_small.png" /></p>
<p>If you built an applet in one of these technologies, you didn’t really build a <em>web application</em>. You had a web page with a chunk cut out of it, and your applet worked within that frame. You lost all of the benefits of other web technologies; you lost HTML, you lost CSS, you lost the accessibility built into the web. These platforms never gained the ability to interact with the rest of the platform provided by your browser. Well, <a href="https://docs.oracle.com/javase/tutorial/deployment/applet/manipulatingDOMFromApplet.html">technically they might have</a>, but that’s not how these technologies were used in practice.</p>
<p>How could the web sustain an ecosystem that doesn’t integrate with it? It was never going to happen. And users ended up solidly rejecting it. Outside of games, users <em>hated</em> Flash. Java Applets were heavy and slow. Enshrining these technologies into the web platform wasn’t going to work.</p>
<p>WebAssembly, on the other hand, is much closer to JavaScript. It doesn’t inherently require taking over a part of your screen. It doesn’t expect to be its own little closed off world. Via JavaScript today, and <a href="https://github.com/WebAssembly/host-bindings/blob/master/proposals/host-bindings/Overview.md">on its own in the future</a>, it’s able to interact with the surrounding environment. It just… fits.</p>
<h2>Other technologies were owned by companies</h2>
<p>Java was owned by Sun Microsystems, Flash was owned by Adobe. But why does this matter?</p>
<p>Corporate influence on the web is a complicated topic. But <em>in general</em>, the web runs on a psuedo-consensus model. Java and Flash, on the other hand, were controlled by their respective companies. These companies have significant motive to make profit, not to make the web better. And that’s partially what led to the above situation; these companies didn’t care that they didn’t integrate properly with the rest of the platform. Why would they? It’d be much better for business if you got locked into their platform, and abandoned the rest of the web entirely. The incentives are fundamentally misaligned.</p>
<p>WebAssembly is a joint venture between Mozilla, Google, Apple, Microsoft, and others. It’s not trying to push anyone’s particular platform, but instead represents the shared interests of a wide number of stakeholders, both corporate and individual.</p>
<h2>WebAssembly followed the web process</h2>
<p>This <em>also</em> meant that these technologies never really followed the process that we use to standardize the web. That being said, the web standards process is healthier than its ever been, and at the time these technologies were big, things worked a bit differently. Regardless, WebAssembly followed the way that we do things on the web.</p>
<p>First, <code>asm.js</code> was created, proving that we could do more impressive things with the web. Its performance was good enough to demonstrate what could be done, though it wasn’t fantastic. The secret sauce of <code>asm.js</code> was that it was just JavaScript; it was fully backwards compatible with the existing ecosystem. “Don’t break the web” is a very, very important rule for browser vendors.</p>
<p>WebAssembly was sort of <code>asm.js</code> done better. After building some degree of consensus around <code>asm.js</code>, a variety of people came together to make WebAssembly a reality. Given that it wasn’t just JavaScript, it had to go through the usual process for browsers to implement it, and it did so. Now a W3C spec, it went <em>with</em> the grain of web standards, not against them.</p>
<h3>Corollary: other technologies were too big and unstable</h3>
<p>There’s another reason Wasm succeeded: it’s tiny. Wasm takes the approach of other web technologies: start small, and build on those foundations. Stay backwards compatible. These technologies also didn’t follow that particular social convention. For Flash and Java Applets, you’d be downloading the particular runtime you needed, and so there was no need for compatibility. PNaCl built on top of LLVM IR, which is completely unstable. They were going to make it a stable subset, but then when LLVM introduces changes, it would have ended up diverging, which isn’t a great situation to be in.</p>
<p>These technologies were also far too large to ever make it through the stabilization process. Can you imagine the big four browser vendors specifying the JVM in full and then agreeing to those semantics for all time? For all of ActionScript, itself a similar-to-but-not-quite version of ECMAScript? For all of LLVM-IR?</p>
<p>Large technologies introduce an unacceptable amount of risk in this situation. It’s an all-or-nothing kind of deal, and the safe bet is to go with “nothing.” WebAssembly, on the other hand, does almost nothing. It’s pretty much math and calling into JavaScript. This means it’s <em>much</em> easier to develop consensus around.</p>
<h3>Collary: other technologies required an entire separate virtual machine</h3>
<p>I haven’t mentioned Dart in this post, but it sort of fits here too. One big issue with “let’s just put the JVM in every browser” is that browsers <em>already</em> contain a language runtime: JavaScript’s. It’s hard enough to maintain one runtime, let alone two. And then how do you integrate them?</p>
<p>WebAssembly, on the other hand, was designed to be a small extension to existing JavaScript virtual machines. While you <em>can</em> implement your own WebAssembly VM, and non-web use cases often do, for browsers, the maintenance cost is much, much lower.</p>
<h2>Other technologies were too specific</h2>
<p>WebAssembly is fundamentally language-agnostic. Flash and Java Applets were built first and foremost to run ActionScript and Java. They’re deeply tied to their relative semantics. Even PNaCl suffers from this to some degree; LLVM is really designed for C-like languages, though not quite the same amount.</p>
<p>Do we really want to christen one language as the next language of the web? We already have JavaScript. Are we someday going to introduce a third language? A fourth? An agnostic approach is significantly better for longevity’s sake.</p>
<h2>Wasm has a strong approach for safety</h2>
<p>Java Applets and Flash were security <em>nightmares</em>. Even if there were attempts to secure them, in reality, they constantly had problems.</p>
<p>WebAssembly, on the other hand, piggybacks on that JavaScript VM again; all of the effort going into creating its sandbox also applies to Wasm. Wasm is missing some features of regular assembly language that can cause security vulnerabilities, like stack smashing. Wasm is memory-safe, which is huge!</p>
<p>Additionally, WebAssembly was designed for <em>validation</em> in mind; its fully typed, and can be checked without running any of its code. It spec includes instructions on how to do validation. This stuff is quite useful!</p>
<h2>Conclusion</h2>
<p>One thing I’ll say about this post is that it’s quite <em>implementer</em> focused. But implementers are important; they control the web. Providing a solution that works with their goals is just as important as something that works for users. In a future post, I’d like to try to go over user concerns some as well. I have lots of posts to write!</p>
<p>At the end of the day:</p>
<ul>
<li>Other technologies were not integrated into the platform, and commercial interests didn’t point towards that happening</li>
<li>Other technologies required too much; too much to agree on, too much to implement</li>
<li>Wasm has a really strong sandboxing and verification story that others simply did not.</li>
</ul></content:encoded>
</item>
<item>
<title>WebAssembly is more than just the web</title>
<link>https://steveklabnik.com/writing/webassembly-is-more-than-just-the-web/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/webassembly-is-more-than-just-the-web/</guid>
<pubDate>Fri, 13 Jul 2018 00:00:00 GMT</pubDate>
<content:encoded><p>While WebAssembly was given its name for good reason, I also think it’s quite the misnomer. When you hear about WebAssembly, don’t just think of the web: think bigger.</p>
<p>Many programmers know that WebAssembly is a technology that lets you run non-JavaScript code in the browser. That is true, but there’s a lot more to it. You see, the spec is written in a very interesting way: there’s the core, what WebAssembly actually is. Here’s <a href="https://webassembly.github.io/spec/">the opening section</a>:</p>
<blockquote>
<p>To support the embedding of WebAssembly into different environments, its specification is split into layers that are specified in separate documents.</p>
</blockquote>
<blockquote>
<p>Core specification - Defines the semantics of WebAssembly modules and its instruction set, independent from a concrete embedding.API specifications - Define application programming interfaces enabling the use of WebAssembly modules in concrete embedding environments.</p>
</blockquote>
<p>It goes on to say:</p>
<blockquote>
<p>Currently, two APIs are specified: JavaScript API … Web API.</p>
</blockquote>
<p>So, WebAssembly itself, the core, is completely independent of the web. The web is a particular environment that Wasm is embedded <em>in</em>, but not actually required to use wasm. Other embedding APIs can be created, with their own interfaces, to do various things.</p>
<p>Let’s make this concrete: on the web, we can write a program that draws graphics using WebGL. But we could also write a desktop program that embeds a WebAssembly interpreter, and provides an OpenGL (or whatever else) API to render to the screen. And now our WebAssembly program is running on the desktop.</p>
<p>Why would you want to do this? Well:</p>
<blockquote>
<p>I wonder if wasm, in being sufficiently minimal, well defined, practical and widely supported, will wind up as an archival format of choice for emulation-based software preservation. https://t.co/33OUOrqiWg
— Graydon Hoare (@graydon_pub) July 11, 2018</p>
</blockquote>
<p>These emulators, with a teeny bit of work, could be made to run on the desktop as well. “Cross-platform” doesn’t just mean “Windows, Linux, and MacOS”, but can also mean running to the web as well.</p>
<p>There’s so many more things that can be done, too. All it requires is a bit of re-framing of WebAssembly in your mind:</p>
<p>WebAssembly is <em>not</em> a way to run code other than JavaScript on the web; WebAssembly is a tiny, embeddable language that you can use almost anywhere.</p>
<p>Some have compared WebAssembly to Java applets; in some ways, they’re very right, but in some ways, they’re very wrong. Eventually I’ll write a post about the wrong, but for now, the right: in some sense, WebAssembly is a different way of accomplishing what the JVM set out to do: it’s a common virtual machine that can be used to build very cross-platform software.</p>
<p>There’s a couple of other interesting applications of WebAssembly outside of the browser; one of my personal favorites is <a href="https://github.com/nebulet/nebulet">Nebulet</a>:</p>
<blockquote>
<p>A microkernel that implements a WebAssembly “usermode” that runs in Ring 0.</p>
</blockquote>
<p>I’d describe it like this:</p>
<blockquote>
<p>Nebulet is an operating system that runs WebAssembly programs.</p>
</blockquote>
<p>None of this has anything to do with a web browser; just like ELF is the standard binary format on many Linuxes, WebAssembly is the standard binary format on Nebulet. It’s a new implementation of a quite old idea, but I’m really excited to see where it goes.</p>
<p>A second interesting example is Parity. I’m a cryptocurrency skeptic, but from a tech perspective, this is quite cool. Parity is a company who is building a VM for the Ethereum cryptocurrency. Etherium’s VM has a programming language, Solidity, that lets you script their blockchain. Writing a programming language is a tough task on its own, and Solidity has had some issues, leading to pretty serious bugs. And those bugs are affecting people’s money.</p>
<p>So, rather than keep their custom language, Parity is experimenting with <a href="https://wiki.parity.io/WebAssembly-Design">embedding WebAssembly into the Ethereum VM instead</a>. This gets them a well-tested language that has pretty decent toolchain support, and lets them use a variety of other languages to do said scripting. This is very cool!</p>
<p>These are a few of the possibilities for WebAssembly, and I’m really excited about where it could go in the future.</p></content:encoded>
</item>
<item>
<title>"The Expressive C++17 Coding Challenge (in Rust)" revisited</title>
<link>https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust-revisited/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust-revisited/</guid>
<pubDate>Wed, 14 Feb 2018 00:00:00 GMT</pubDate>
<content:encoded><p>In October of last year, I wrote a post, <a href="http://words.steveklabnik.com/the-expressive-c-17-coding-challenge-in-rust">“The Expressive C++17 Coding Challenge (in Rust)”</a>. For various reasons, it got brought up again in the D world, and <a href="https://seb.wilzba.ch/b/2018/02/the-expressive-c17-coding-challenge-in-d/">seb has written a new post</a>. It’s good, you should check it out!</p>
<p>However, it links to my gist, not my blog post. As I said back then:</p>
<blockquote>
<p>I held myself to the same constraints as the original contest; no external packages is a bit painful in Rust, but it’s not too bad. Mostly it would let me eliminate boilerplate while also improving correctness, and making the code a bit shorter.</p>
</blockquote>
<p>So, that got me thinking: What <em>would</em> this look like if I could use external packages? I took about an hour, and knocked it out. I have two versions to show you today, one where I pay no attention to allocations, and one where it’s zero-allocation.</p>
<p>First, the “whatever just allocate” version:</p>
<pre><code>extern crate csv;
#[macro_use]
extern crate serde_derive;
extern crate serde;
#[macro_use]
extern crate structopt;
use std::error::Error;
use std::fs::File;
use structopt::StructOpt;
#[derive(Debug,Serialize,Deserialize)]
struct Record {
name: String,
surname: String,
city: String,
country: String,
}
#[derive(StructOpt, Debug)]
#[structopt(name = "basic")]
struct Opt {
filename: String,
column_name: String,
replacement: String,
output_filename: String,
}
fn run(args: &amp;Opt) -&gt; Result&lt;(), Box&lt;Error&gt;&gt; {
let input = File::open(&amp;args.filename)?;
let output = File::create(&amp;args.output_filename)?;
let mut rdr = csv::Reader::from_reader(input);
let mut wtr = csv::Writer::from_writer(output);
for result in rdr.deserialize() {
let mut record: Record = result?;
let replacement = args.replacement.clone();
match &amp;*args.column_name {
"name" =&gt; record.name = replacement,
"surname" =&gt; record.surname = replacement,
"city" =&gt; record.city = replacement,
"country" =&gt; record.country = replacement,
_ =&gt; panic!("incorrect column name"),
}
wtr.serialize(record)?;
}
wtr.flush()?;
Ok(())
}
fn main() {
let opt = Opt::from_args();
if let Err(err) = run(&amp;opt) {
eprintln!("error: {}", err);
std::process::exit(1);
}
}
</code></pre>
<p>This uses the <code>[csv</code> crate](https://crates.io/crates/csv/1.0.0-beta.5) to do the heavy lifting with regards to the CSV stuff, and the <code>[strucopt</code> crate](https://crates.io/crates/structopt) for argument parsing. Not only is this code much shorter than my “no crates” version, but it’s also got more features!</p>
<p>There’s a few things that aren’t the best here: first, the <code>match</code>. Rust doesn’t have reflection, exactly, so saying “hey I have this string and I want to index it in this struct” is not as nice. I’m just panic-ing here, but it’d be nicer to validate it up front, and possibly even do some custom derive shenanigans so that I don’t have to write it myself, but I wanted to present a fairly <em>realistic</em> version of what I’d write today. This is good enough.</p>
<p>Second, there’s that one <code>clone</code>. We need it because we’ve said that our <code>Record</code> type holds <code>String</code>s, and so has ownership over the string. We end up copying <code>London</code> each iteration of the loop. Is copying a city name a problem? Not <em>really</em>, in the context of this example. But what if it <em>was</em>? Fundamentally, there’s no reason to do any allocation here: We’re reading from a file, and we’re writing the same string over and over again.</p>
<p>Enter version 2:</p>
<pre><code>extern crate csv;
#[macro_use]
extern crate serde_derive;
extern crate serde;
#[macro_use]
extern crate structopt;
use std::error::Error;
use std::fs::File;
use structopt::StructOpt;
#[derive(Debug,Serialize,Deserialize)]
struct Record&lt;'a&gt; {
name: &amp;'a str,
surname: &amp;'a str,
city: &amp;'a str,
country: &amp;'a str,
}
#[derive(StructOpt, Debug)]
#[structopt(name = "basic")]
struct Opt {
filename: String,
column_name: String,
replacement: String,
output_filename: String,
}
fn run(args: &amp;Opt) -&gt; Result&lt;(), Box&lt;Error&gt;&gt; {
let input = File::open(&amp;args.filename)?;
let output = File::create(&amp;args.output_filename)?;
let mut rdr = csv::Reader::from_reader(input);
let mut wtr = csv::Writer::from_writer(output);
let mut raw_record = csv::StringRecord::new();
let headers = rdr.headers()?.clone();
while rdr.read_record(&amp;mut raw_record)? {
let mut record: Record = raw_record.deserialize(Some(&amp;headers))?;
match &amp;*args.column_name {
"name" =&gt; record.name = &amp;args.replacement,
"surname" =&gt; record.surname = &amp;args.replacement,
"city" =&gt; record.city = &amp;args.replacement,
"country" =&gt; record.country = &amp;args.replacement,
_ =&gt; panic!("incorrect column name"),
}
wtr.serialize(record)?;
}
wtr.flush()?;
Ok(())
}
fn main() {
let opt = Opt::from_args();
if let Err(err) = run(&amp;opt) {
eprintln!("error: {}", err);
std::process::exit(1);
}
}
</code></pre>
<p>This is <em>slightly</em> more involved, as you no longer get access to the nice iterator, and have to do things a little more manually. But it took me about five minutes to port this bit over, so it’s not <em>too</em> onerous.</p>
<p>So, as you can see, once you let people use external packages, the Rust gets <em>significantly</em> nicer. The code pretty much mirrors the task: open the two files, iterate over them, replace the column, write it out. Nice.</p>
<p>I’m not going to do any performance comparisons here because that’s not really what the contest was about. I am mildly curious… if I really wanted to optimize performance, I’d also skip the string stuff and go for bytes directly. This isn’t a super tough transformation, but moves things to be slightly lower level.</p>
<p>If you manipulate CSVs often, you should check out <code>[xsv](https://crates.io/crates/xsv)</code>, a command-line tool in Rust that lets you slice and dice these files with speed and ease.</p>
<hr />
<p>I should also mention that several people have been working on <code>quicli</code>, a package to make making CLIs super easy. One of its authors <a href="https://gist.github.com/killercup/049d759118e3b5029737eb77e157ea42">posted a gist of what this looks like with quicli</a>, you should check that out too!</p></content:encoded>
</item>
<item>
<title>0b100000</title>
<link>https://steveklabnik.com/writing/2-sup-5/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/2-sup-5/</guid>
<pubDate>Thu, 25 Jan 2018 00:00:00 GMT</pubDate>
<content:encoded><p>As someone who has <a href="https://www.inc.com/jeff-bercovici/peter-thiel-young-blood.html">significantly different beliefs than Peter Thiel</a>, today is likely my second-to-last power of two years alive.</p>
<p>Two years ago, <a href="http://words.steveklabnik.com/a-eulogy-for-my-20s">I wrote a post about turning 30</a>, and I pretty much feel the same way. Never enough time, tick tock, tick tock. I have today off (thanks Mozilla!), but am still considering working anyway, because I really care about what I’m working on, and am excited about it. We’ll see.</p>
<p>I feel like I have more to say here, but also don’t. So maybe let’s leave it at that.</p></content:encoded>
</item>
<item>
<title>Looking back at Rust in 2018</title>
<link>https://steveklabnik.com/writing/looking-back-at-rust-in-2018/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/looking-back-at-rust-in-2018/</guid>
<pubDate>Tue, 09 Jan 2018 00:00:00 GMT</pubDate>
<content:encoded><blockquote>
<p>A careful reader will note the date of this post; it’s fundamentally in response to this call for blog posts, with a rhetorical device of looking backwards as if my personal plans for Rust came true. If you’re reading this in 2019 or beyond, this may be a work of fiction.</p>
</blockquote>
<p>It’s been an exciting year for Rust. Well, it depends on what you mean by “exciting”, exactly. In a word, this year has been about <em>polish</em>. I find this kind of thing exciting! It’s a very different kind of exciting than the “new and shiny” exciting. It’s “take what we have and make it shiny.”</p>
<p>Let’s recap the major achievements of the year.</p>
<h2>Rust 2018</h2>
<p>First of all, we shipped our first epoch! If you type <code>cargo new</code> today, you’ll see a new shiny entry in your <code>Cargo.toml</code>:</p>
<pre><code>[package]
name = "new-program"
version = "0.1.0"
authors = ["steveklabnik &lt;[email protected]&gt;"]
epoch = "2018"
[dependencies]
</code></pre>
<p>What does this mean in practical terms? Well, a few things. First, let’s get the big one out of the way: Try to compile this code with no epoch, or with <code>epoch = "2015"</code>:</p>
<pre><code>fn main() { let catch = 5;}
</code></pre>
<p>You’ll see this:</p>
<pre><code>&gt; cargo build
Compiling new-program v0.1.0 (file:///C:/Users/steve/tmp/new-program)
warning: `catch` will become a keyword in the 2018 epoch
--&gt; src\main.rs:2:9
|
2 | let catch = 5;
| ^^^^^
|
= note: #[warn(new_keyword)] on by default
= note: to avoid this warning, consider using `_catch` instead
</code></pre>
<p>And if you change it to <code>epoch = "2018"</code>, you’ll see this:</p>
<pre><code>&gt; cargo build
Compiling new-program v0.1.0 (file:///C:/Users/steve/tmp/new-program)
error: expected pattern, found keyword `catch`
--&gt; src\main.rs:2:9
|
2 | let catch = 5;
| ^^^^^
error: aborting due to previous error
</code></pre>
<p>Success! However, the specifics here aren’t as interesting as what this change means:</p>
<p>First, <em>this is one of only two changes that relies on the epoch</em>. When we announced our plans for epochs, many people were skeptical that we would do as we said, and only included small, parser-level changes here. But we did, and will continue to do so. <code>catch</code> and <code>dyn Trait</code> are the only two changes here.</p>
<p>Second, <em>we’ve demonstrated that this doesn’t split the ecosystem</em>. All of the crates that had no epoch are treated as 2015, and everything still works. Mixing 2015 and 2018 code works just fine.</p>
<p>Finally, <em>mrustc has not had trouble implementing this split</em>. <code>[mrustc</code> is an alternative implementation of Rust](https://github.com/thepowersgang/mrustc), and some of the worries about epochs would be that they would unnecessarily burden alternative implementations. This fear has not borne fruit yet though.</p>
<h2>We did ship <em>some</em> new stuff</h2>
<p>Even though this year was about polish, that doesn’t mean that we had no new features. The key is, these features were all <em>designed</em> in 2017 or before. That is, features like non-lexical lifetimes, <code>impl Trait</code>, and Macros 2.0 didn’t land in stable until 2018, but they’ve been worked on for years. At the end of 2017, we had preliminary implementations already landed, it was just about polishing them up so we could ship.</p>
<p>Cargo, however, did gain some new abilities. Cargo was sort of neglected in 2017, or at least, wasn’t worked on as much as the rest of the project was. Cargo grew the ability to use alternative registries, and at least one company is offering “crates.io as a service” and “crates.io behind your firewall”. Cargo also became easier to use offline, and the “integrate Cargo into other build systems” work bore some more fruit.</p>
<h2>WebAssembly and Embedded platforms</h2>
<p>At the end of 2017, Rust was poised to take advantage of WebAssembly. And take advantage, we did. WebAssembly has not taken over the world yet, but that was also never its intention. It is a common aspect of the Web Platform these days, and Rust’s support for it is so excellent that it’s the primary language with which you write new code for WebAssembly today. Tons of JavaScript programmers use Rust for their various systems-level tasks on the web, and we’re just starting to see completely new kinds of applications form. Adding Rust to your Node or front-end project is trivial, thanks to excellent integration from JavaScript’s build tools, or as we now call them, “compilers”.</p>
<blockquote>
<p>An aside: in the 2014-2017 era, even though JavaScript was JIT compiled, many programmers used various “build tools” or “transpilers” to do things like minify and bundle their JavaScript together. This was useful for things like “let me use new ES2017 features today but transpile to ES5.” In 2017, lots of JavaScript programmers, through tools like Rollup, started to appreciate that what they’re doing, on a fundamental level, is compiling. Once these tools realized this, they started taking advantage of decades of existing compiler research, and these tools started doing amazing things. Oddly enough, this makes JavaScript feel more like Java: you compile to a thing that ends up being JIT-ed. And that’s not even half of it; stuff like Ember’s GlimmerVM took this approach to new heights.</p>
</blockquote>
<p>Furthermore, it’s considered standard practice to test your crate against the WebAssembly target, just like you would Windows, Linux, and macOS, so the vast majority of the crates.io ecosystem Just Works.</p>
<p>But this work on wasm has a good side effect: wasm is basically a very minimal embedded platform. By doing the legwork to make wasm awesome, we also made embedded platforms of various kinds awesome. There’s also specific work that had to be done, for example, remember <code>xargo</code>? Cargo just does that now. We’ve elevated ARM to a Tier 1 platform these days, and provide easy instructions for getting a new architecture going.</p>
<p>This also implies stabilizing some of the lower-level things that are needed to do OSdev work. You can’t write a “real” OS in stable Rust yet, but you can write toy ones, and real OSes should be usable by next year.</p>
<h2>Stabilization</h2>
<p>One way in which we did polish was to ship a ton of stuff that was previously in flight. I mentioned non-lexical lifetimes earlier, as well as some other features, but there’s also a sort of “long tail” of stabilizations that we addressed.</p>
<p>At the start of the year, the teams went through each of their feature gates, and voted: stabilize, put on hold, or remove. Things put on hold this year will get decided next year, but cannot be put on hold for another year. Of course, there were some small course-corrections over the year, but in general, it let us go “oh hey, slice patterns has been unstable since before Rust 1.0, let’s finally ship it.”</p>
<h2>Bugs &amp; new contributors</h2>
<p>Rust’s bugtracker was in a much better state in 2017 than in 2016, but got even better in 2018. 2017 saw the infrastructure team actually ensure that all bugs get triaged, that is, they all have tags within two days of being opened. In 2018, however, there was a focus to knock out a lot of the long-standing, papercut style bugs. At the start of 2018, there were ~3100 bugs on the tracker, but today, there are only ~1500. That’s four bugs per day! We couldn’t have gotten there without all of the new contributors and team members we’ve added this year; in 2017, we had about 75 people on our various teams, and now we have 100. Your average Rust release in 2017 had about 120 contributors, it’s 180 for 2018.</p>
<h2>Community and Job growth</h2>
<p>It’s not just the teams that have grown, but also the community. For example, in 2016, crates.io had ~7500 crates. In 2017, ~13,000. In 2018, we’re up to 35,000. That’s a lot of packages! One major driver of this growth is jobs; once <strong>REDACTED</strong> announced that they’d been using Rust in a production setting, and that compared to the old system written in <strong>REDACTED</strong>, it saved them millions of dollars in server costs per year, a ton of companies started using Rust. It’s not <em>super</em> easy to get a Rust job, but they’re out there. We’ve also seen our meetups continue to grow, RustConf had two tracks this year, and there’s a new RustFest in South America, with rumors of a conference in China next year.</p>
<h2>IDEs</h2>
<p>While I don’t really use IDEs, we put a lot of work into making them awesome, and 2018 was the first time that people who <em>do</em> use IDEs love using one. Specifically, they love using the new IDE that JetBrains has released for Rust. In 2017, JetBrains was sponsoring the plugin for IntelliJ, but in 2018, they’ve released a whole product. Other IDEs are coming along swimmingly; Visual Studio now has really excellent support, for example.</p>
<h2>Documentation</h2>
<p>I work on docs, so I’m going to give this a section, even though maybe other things are more “exciting”.</p>
<p>2018 saw the release of $REDACTED, a sort of “next-generation <code>rustdoc</code>.” It focused on not just API docs, but all of your docs holistically; you can publish guides as well, structure things how you’d like, etc. In some ways, it’s “<code>rustdoc</code> + <code>mdbook</code>” in a nice integrated package. This has led to even more people writing even better docs, since it’s so nice to use.</p>
<p>There were two other major initiatives this year: focusing on intermediate documentation, and helping major ecosystem crates get excellent docs. In 2017, there was sort of a “what do I do after TRPL” situation, but now we’ve got some great resources to help you grow as a Rust programmer. Additionally, many of the more popular crates have docs that rival the standard library’s, and smaller crates have decent docs.</p>
<p>Oh, and TRPL saw its initial paper release early in the year. Yay!</p>
<h2>Epilogue</h2>
<p>Whew, that’s a lot of stuff! What does 2019 hold for Rust? That post is for another time.</p></content:encoded>
</item>
<item>
<title>Five years with Rust</title>
<link>https://steveklabnik.com/writing/five-years-with-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/five-years-with-rust/</guid>
<pubDate>Thu, 21 Dec 2017 00:00:00 GMT</pubDate>
<content:encoded><p>Today is five years since I first heard of Rust. <a href="http://words.steveklabnik.com/four-years-with-rust">I wrote a post last year</a> about it. This year, I have something slightly different in mind.</p>
<p>Five years is a long time in tech. It’s also a long time for me to be focused on one specific thing; other than the pizza job I held in high school and college, I’ve never stayed in a single job for five years. (This is true today as well; it’s not like I got a job with Rust the first day I learned of its existence.) I started with Ruby and Rails in 2008/2009, and my final commits to Rails were in August/September of 2013. So at this point, my career has basically been split in half, rather than dominated by Ruby.</p>
<p>Five years ago tomorrow, I registered <a href="http://www.rustforrubyists.com/">http://www.rustforrubyists.com/</a>.</p>
<p>Yesterday, I had lunch with someone I’m very honored to call a friend. We talked about a lot of things, but also about Rust’s past, present, and future. One thing I’ve always felt a little awkward about is that there was a crucial point back in 2011/2012 where I wasn’t sure what to do with myself. I was very frustrated by the industry, and considered doing other things instead. But, as they and I talked about, sometimes, it’s not about your interests. If what you want to do is change the world, caring a lot isn’t enough; you have to analyze what you can actually <em>do</em>. While I think our industry is pretty broken, I at least have the ability to try and improve it, thanks to my stature. (God, even just typing that makes me feel gross.)</p>
<p>I decided to put my time into Rust because I believed, and still believe, that it’s an important thing. And I saw the work that needed to be done, and saw that it’s the kind of work that I could do. And so, it’s maybe not something bigger, but I resolved myself to doing the work.</p>
<p>Five years in, I think my journey has been successful so far. I’m proud of how far Rust has come. That said, there’s a lot more work to do. I’m still committed as ever to getting that work done, and feel proud to be able to work alongside people as amazing as the Rust community.</p>
<p>Here’s to five more years. ?</p></content:encoded>
</item>
<item>
<title>User-Agent: moz://a</title>
<link>https://steveklabnik.com/writing/user-agent-moz-a/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/user-agent-moz-a/</guid>
<pubDate>Tue, 12 Dec 2017 00:00:00 GMT</pubDate>
<content:encoded><p>Every six months, Mozilla gets all of its employees from around the world together in one place, for a week. This morning was the plenary session, with Mozilla leadership talking about our successes this year, as well as plans for the future. I noticed a theme that, while not super overt, I find very interesting, and bodes well for Mozilla’s future.</p>
<p>The words “user agent” kept coming up in various aspects of the plenary. The <code>User-Agent</code> header has a… shall we say <a href="https://webaim.org/blog/user-agent-string-history/">storied history</a>. When most people hear the term “user agent”, they mostly think of the “<a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent/Firefox">user agent string</a>”. This is passed through the <code>User-Agent</code> header to identify which browser you’re using.</p>
<p>Well, that’s how most people think of it, but that’s not the whole story. If it is supposed to identify which browser you’re using, why is it not the <code>Browser</code> header? What is this “user agent” term anyway?</p>
<p>The concept is a bit more broad: a “user agent” is a bit of software that acts on behalf of its user. A web browser is a “user agent” for the web. But an email program is also a “user agent” for email, for example. So “user agent” and “web browser” are not synonymous; a web browser is a specific form of user agent.</p>
<p>Words have power, however. Framing matters. The way you describe a problem affects the way you think about the solutions. So the shift in language I picked up on was, Mozilla is not building a web browser, Mozilla is building a user agent. So what’s the difference here?</p>
<p>A web browser is a dumb terminal. A user agent advocates and works for you, to better your experience.</p>
<p>There’s a lot more to say about what all this framing implies: for security, for privacy, for customization, for openness, for advocacy, for social impact. But I have some meetings to go to, so I’ll just leave you with a question:</p>
<p>What if your browser worked <em>for</em> you, rather than <em>against</em> you?</p>
<p>In my mind, this “user agent” framing is a very useful and interesting way to think about Mozilla’s mission, and the goals we pursue in the future.</p></content:encoded>
</item>
<item>
<title>"The Expressive C++17 Coding Challenge" in Rust</title>
<link>https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust/</guid>
<pubDate>Wed, 25 Oct 2017 00:00:00 GMT</pubDate>
<content:encoded><p>A few weeks ago, two C++ bloggers initiated a challenge (<a href="https://www.fluentcpp.com/2017/10/23/results-expressive-cpp17-coding-challenge/">here</a> and <a href="http://www.bfilipek.com/2017/09/the-expressive-cpp17-challenge.html">here</a>): use the shiny new features of C++17 to write a small program that manipulates CSV files. The link is to the results; you can see the examples people came up with.</p>
<hr />
<p>I’ve written <a href="http://words.steveklabnik.com/the-expressive-c-17-coding-challenge-in-rust-revisited">a follow-up post</a> too!</p>
<hr />
<p>I decided to implement it in Rust for fun. You should give it a try, in whatever language you use, it’s a nice sample problem. I held myself to the same constraints as the original contest; no external packages is a bit painful in Rust, but it’s not too bad. Mostly it would let me eliminate boilerplate while also improving correctness, and making the code a bit shorter.</p>
<p>Anyway, here it is:</p>
<p>https://gist.github.com/steveklabnik/ad0a33acc82e21ca3f763e4278ad31a5</p>
<p>I implemented <code>From</code> for everything so I could use <code>?</code>, but then <code>main</code> just has a ton of <code>unwrap</code>s. I normally would remove those too, but the challenge does not include good error handling, so I decided to just stick to the text of the challenge.</p>
<p>I’m not going to say that this is the <em>best</em> Rust code in the world, nor compare it directly to the C++ examples. But it was a fun hour or two to knock it out, and figured I’d at least share my results.</p></content:encoded>
</item>
<item>
<title>Want to join the Rust docs team?</title>
<link>https://steveklabnik.com/writing/want-to-join-the-rust-docs-team/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/want-to-join-the-rust-docs-team/</guid>
<pubDate>Fri, 24 Mar 2017 00:00:00 GMT</pubDate>
<content:encoded><p>One of Rust’s major goals this year is to <a href="https://github.com/rust-lang/rust-roadmap/issues/13">provide mentoring at all levels of the project</a>. This is a goal I’m very on-board with. There’s lots of ways to get involved with Rust, but I’d like to talk about the bit I’m most involved in: documentation.</p>
<p>There’s many ways to contribute to Rust’s docs. Some people contribute occasional PRs, some people end up contributing a lot. No matter what your interest, I’d like to help. But before I get into those details, <em>why</em> should someone get involved with documentation? I have my own reasons, but I also have a pitch for you:</p>
<blockquote>
<p>Working on documentation is a fantastic way to learn something in a deeper way.</p>
</blockquote>
<p>In order to document something, you have to understand it, even if only a little bit! It’s a good excuse to dig into the details. And I’ve found that many people are eager to help explain something to you in return for you writing that down and synthesizing it into documentation.</p>
<p>If that sounds intriguing to you, let’s talk! There’s a number of different ways to do this, and I’m interested generally in helping anyone work on Rust’s documentation, in any form. However, if you don’t have something in mind, I do have a suggestion.</p>
<p>Back in October of 2015, I made a <a href="https://github.com/rust-lang/rust/issues/29329">big GitHub issue</a> to track giving the whole standard library documentation a once-over. I then made sub-issues for each main module. Slowly, myself and others have been knocking them out, but there’s still a number of issues open.</p>
<p>Today, I went through all of the remaining ones, and made fine-grained bullet points of what needs to be done. I’ll admit that some of them may not be as explicit as they could be, but it at least gives a starting point to talk about. Here’s all the sections that are currently open:</p>
<ul>
<li><a href="https://github.com/rust-lang/rust/issues/29349">convert</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29354">ffi</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29381">macros</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29378">thread</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29377">sync</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29375">str</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29371">ptr</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29370">process</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29367">os</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29365">ops</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29363">net</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29357">hash</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29355">fmt</a></li>
</ul>
<p>Some of the work that needs to be done is very small: add a link here, re-word a sentence there. Other bits are large; “this whole thing needs re-written.”</p>
<p>So here’s my pitch to you: have you ever wanted to learn a bit more about Rust? Do any of those names sound interesting to you? Maybe you’ve wanted to learn a bit more about threads in Rust. Check out that ‘thread’ issue above, poke around, learn some Rust, and then let’s make the docs better! We can work together on the issues, through email, or on IRC. Whatever works. Maybe you’ll find it as addicting as I do. And I find that people who are non-experts write way better docs than experts. So don’t worry about that!</p>
<p>Worst case, you’ll have learned a bit of the standard library a bit better, and that’s always a good thing, too.</p></content:encoded>
</item>
<item>
<title>An overview of macros in Rust</title>
<link>https://steveklabnik.com/writing/an-overview-of-macros-in-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/an-overview-of-macros-in-rust/</guid>
<pubDate>Tue, 24 Jan 2017 00:00:00 GMT</pubDate>
<content:encoded><p>Rust has had a lot of stuff going on with “macros”. There are a few, interconnected aspects of these features, many different names for the same thing, and a lot of development work going on. This post will give an overview of the past, present, and future of macros in Rust.</p>
<p>Ultimately, macros allow you to write things like this:</p>
<pre><code>let map = hashmap!{
"a" =&gt; 1,
"b" =&gt; 2,
};
</code></pre>
<p>or</p>
<pre><code>#[derive(Serialize, Deserialize)]
struct S {
#[serde(rename = "a")]
f: i32,
}
</code></pre>
<p>That is, some macros look like functions: <code>foo!</code>, but allow you to write nearly arbitrary syntax inside of the <code>()</code>s. Some let you control the ability to write custom attributes. Both generate some other kind of valid Rust code for the compiler to interpret. In the first case, the <code>hashmap!</code> macro will expand to a call to <code>HashMap::new</code> and a bunch of <code>insert</code> calls, and the second case, the <code>Serialize</code> and <code>Deserialize</code> traits will be automatically implemented for this struct, along with a bit of custom logic where the field <code>f</code> maps to the name <code>a</code> in the data that’s being {de,}serialized.</p>
<p>These features are extremely powerful, but also must be done right: metaprogramming is very useful, but can also make code inscrutable, or give poor error messages.</p>
<p>Oh, and I’d like to give a shout-out to <a href="https://danielkeep.github.io/tlborm/book/index.html">The Little Book of Rust Macros</a>, which is basically <em>the</em> resource for doing this kind of work. If you are interested in writing macros, you should check it out.</p>
<h2>Pre-1.0</h2>
<p>As Rust developed, two major forms of macros evolved. One was called “macros” and the other was called “compiler plugins”. Both of these systems were pretty good, but both of them also had a lot of problems. We’ll get into the details of each of these kinds of macros below, but there’s one thing we should talk about first. On the way to 1.0, it was decided to stabilize one kind of macros, but not the other. From <a href="https://blog.rust-lang.org/2014/10/30/Stability.html">Stability as a Deliverable</a>: (also, this uses some terms that we will cover shortly)</p>
<blockquote>
<p>After extensive discussion, we plan to release globs and macros as stable at 1.0. For globs, we believe we can address problems in a backwards-compatible way. For macros, we will likely provide an alternative way to define macros (with better hygiene) at some later date, and will incrementally improve the “macro rules” feature until then. The 1.0 release will stabilize all current macro support, including import/export.On the other hand, we cannot stabilize syntax extensions, which are plugins with complete access to compiler internals. Stabilizing it would effectively forever freeze the internals of the compiler; we need to design a more deliberate interface between extensions and the compiler. So syntax extensions will remain behind a feature gate for 1.0.Many major uses of syntax extensions could be replaced with traditional code generation, and the Cargo tool will soon be growing specific support for this use case. We plan to work with library authors to help them migrate away from syntax extensions prior to 1.0. Because many syntax extensions don’t fit this model, we also see stabilizing syntax extensions as an immediate priority after the 1.0 release.</p>
</blockquote>
<p>We’ll discuss in each section how this ended up working out, but it’s important to remember that only <em>one</em> form of these macros was available on stable Rust for 1.0, and in fact, as of the time of this writing. That’s going to change soon…</p>
<p>It’s also important to note that we weren’t entirely happy with macros. They were fine, but they have some flaws. We wanted to fix them, but we also didn’t want to ship 1.0 without macros. Language design is hard. I’ll talk about this more in the appropriate section.</p>
<h2>Declarative Macros</h2>
<p>The first form of Macros in Rust, and the one that’s most widely used, are called “declarative macros”, sometimes “macros by example”, sometimes “macro_rules macros”, or sometimes just “macros”. At their core, declarative macros allow you to write something similar to a Rust <code>match</code> statement:</p>
<pre><code>match x {
4 =&gt; println!("four!"),
5 =&gt; println!("five!"),
_ =&gt; println!("something else"),
}
</code></pre>
<p>With <code>match</code>, <code>x</code>’s structure and value will be evaluated, and the right arm will execute based on what matches. So if <code>x</code> is five, the second arm happens. Etc.</p>
<p>These kinds of macros work in the same way: you set up some sort of pattern, and then, if that pattern matches, some code is generated. One important difference here is that in <code>match</code>, <code>x</code> gets evaluated. With macros, <code>x</code> does not get evaluated.</p>
<p>There are two kinds of declarative macros: ones defined with the <code>macro_rules!</code> keyword, and ones defined with the <code>macro</code> keyword.</p>
<h3><code>macro_rules!</code></h3>
<p>These are most commonly known as “macros” in Rust world, and are the only thing that’s stable in Rust at the time of this writing. Here’s a macro you’ll see when using Rust:</p>
<pre><code>let x: Vec&lt;u32&gt; = vec![1, 2, 3];
</code></pre>
<p>This macro creates a new vector, with three elements inside. Here’s what the macro could look like, written with <code>macro_rules!</code>:</p>
<pre><code>macro_rules! vec {
( $( $x:expr ),* ) =&gt; {
{
let mut temp_vec = Vec::new();
$(
temp_vec.push($x);
)*
temp_vec
}
};
}
</code></pre>
<p>Whew! That’s a bunch of stuff. The most important line is here:</p>
<pre><code> ( $( $x:expr ),* ) =&gt; {
</code></pre>
<p>This <code>pattern =&gt; block</code> looks similar to the <code>match</code> statement above. If this pattern matches, then the block of code will be emitted. Given that this is the only pattern in this macro, there’s only one valid way to match; any other will be an error. More complex macros will have more than one rule.</p>
<p>The <code>$x:expr</code> part matches an expression, and gives it the name <code>$x</code>. The <code>$(),*</code> part matches zero or more of these expressions. Then, in the body of the macro, the <code>$()*</code> part is generated for each part that matches, and the <code>$x</code> within is replaced with each expression that was matched.</p>
<p>These macros are fine, but there’s a number of bugs and rough edges. For example, there’s no namespacing: if a macro exists, it’s everywhere. In order to prevent name clashes, this means that you have to explicitly import the macros when using a crate:</p>
<pre><code>#[macro_use]
extern crate serde;
</code></pre>
<p>Otherwise, you couldn’t import two crates that had the same macro name. In practice this conflict doesn’t come up much, but as we grow, it becomes more likely. The hygiene of <code>macro_rules</code> is there, but not perfect. (Only local variables and labels are hygienic…) Et cetera. The plan is that with the new macro systems, you’ll be able to use the regular module system, and use macros just like you use functions today.</p>
<p>Even with these problems, macros were useful enough to include in Rust 1.0. But plans for a better macro system were already being worked on, and so, they were given the name <code>macro_rules!</code> so that we could eventually deprecate all of this. This is one of the few features that we decided to do this with Rust 1.0; a vestigial thing that was always planned for obsolescence. Doing it hurts, but not having any metaprogramming at all would hurt much more.</p>
<h3><code>macro</code></h3>
<p>As such, the <code>macro</code> keyword was reserved for a new macro system. We’re still not 100% sure if we’re calling these “macros by example” (which you’ll sometimes see with a “2.0” at the end) or “declarative macros”. I prefer the latter, because it pairs nicely with “procedural macros.” It seems like most agree with me. We’ll see.</p>
<p>These macros are an incremental improvement over the existing <code>macro_rules!</code> macros: they’re fundamentally the same thing, but with some nicer features.</p>
<p>At the time of this writing, <a href="https://github.com/rust-lang/rfcs/pull/1584">an initial RFC is in FCP</a>, laying out some high-level aspects of this feature. An initial implementation is underway, but it hasn’t landed yet, so you can’t use <code>macro</code> in real code yet.</p>
<h2>Procedural Macros</h2>
<p>In opposition to the pattern-based declarative macros, the second form are called “procedural macros” because they’re functions: they accept some Rust code as an input, and produce some Rust code as an output. I say “code” but I don’t mean that literally; we’ll get into the exact details in each section.</p>
<h3>Syntax Extensions</h3>
<p>Syntax extensions, also sometimes just called “procedural macros” or “compiler plugins”, were not stabilized for Rust 1.0. Here’s (part of) a syntax extension that implements roman numeral literals:</p>
<pre><code>#![feature(plugin_registrar, rustc_private)]
extern crate syntax;
extern crate rustc;
extern crate rustc_plugin;
use syntax::parse::token;
use syntax::tokenstream::TokenTree;
use syntax::ext::base::ExtCtxt;
use rustc_plugin::Registry;
fn expand_rn(cx: &amp;mut ExtCtxt, sp: Span, args: &amp;[TokenTree])
-&gt; Box&lt;MacResult + 'static&gt; {
// implementation goes here
}
#[plugin_registrar]
pub fn plugin_registrar(reg: &amp;mut Registry) {
reg.register_macro("rn", expand_rn);
}
</code></pre>
<p>You’d use it like this:</p>
<pre><code>#![feature(plugin)]
#![plugin(roman_numerals)]
fn main() {
assert_eq!(rn!(MMXV), 2015);
}
</code></pre>
<p>That is… a lot of code. And frankly, probably doesn’t even work on today’s Rust right now. Why not? Well, that’s the big problem with syntax extensions:</p>
<pre><code>extern crate syntax;
extern crate rustc;
extern crate rustc_plugin;
</code></pre>
<p>This literally loads up bits of the Rust compiler and uses it as a library. Then, it uses Rust’s internal AST to do the processing. This is extremely powerful, but also extremely unstable. We don’t and can’t freeze Rust’s internal AST, or any of its other internal APIs, at least not at this time. Not to mention that this means any alternative Rust compiler would need to either use our AST or map ours to theirs, which puts a damper on alternative Rust implementations. This means that syntax extensions haven’t had a good path towards stability. It’s also why they couldn’t be made stable as-of Rust 1.0. But “run arbitrary Rust code to produce more Rust code” is a useful feature, and so, some projects needed them. In addition, while declarative macros can produce function-style <code>foo!</code> calls, they cannot do the custom attribute stuff. And those are very useful. We <em>could</em> make this work, but there hasn’t been a real reason to, nor much demand.</p>
<p>A side note: there’s also the term “compiler plugin.” Syntax extensions are a particular kind of compiler plugin; there are others, like lint passes, LLVM pass plugins, MIR plugins… we’re only talking about syntax extensions in this post, however.</p>
<p>The current plan is to never stabilize syntax extensions, and instead, stabilize “procedural macros.” Again, this isn’t for all compiler plugins, only syntax extensions.</p>
<h3>Procedural Macros</h3>
<p>Accepted in <a href="https://github.com/rust-lang/rfcs/blob/master/text/1566-proc-macros.md">RFC 1566</a>, procedural macros will give you the same power as compiler plugins, without needing to reach into the guts of <code>rustc</code>. How? Well, they look like this:</p>
<pre><code>#[proc_macro]
pub fn foo(TokenStream) -&gt; TokenStream {
// do stuff to implement the foo macro
}
</code></pre>
<p>This creates a <code>foo!</code> procedural macro. You’ll note that it accepts a <code>TokenStream</code> and produces a <code>TokenStream</code>. To quote the RFC:</p>
<blockquote>
<p>The first argument is the tokens between the delimiters in the macro use. For example in foo!(a, b, c), the first argument would be [Ident(a), Comma, Ident(b), Comma, Ident(c)].The value returned replaces the macro use.</p>
</blockquote>
<p>Why tokens? Well, while an AST can change, the actual tokens will not. This approach will allow us the flexibility to change <code>rustc</code>’s internal AST, while giving us the full power of compiler plugins.</p>
<p>So, how do you work with one of these kinds of macros? You can of course work with the token stream directly, but we also expect to see libraries that will provide easier-to-use interfaces on top of this. You could imagine a package that converts a <code>TokenStream</code> to its own AST, and then provides an interface to that AST. This is nicer to use than working with tokens directly. There are already some libraries that do similar kinds of things, like <a href="https://github.com/dtolnay/syn">syn</a>. More on that in a moment.</p>
<h3>Function-like vs attribute like</h3>
<p>The above is for a macro that gives you a <code>foo!</code> macro, that is, one that looks like a function. What about one that gives you an attribute? That’d look like this:</p>
<pre><code>#[proc_macro_attribute]
pub fn foo(Option&lt;TokenStream&gt;, TokenStream) -&gt; TokenStream {
// implementation goes here
}
</code></pre>
<p>To quote the RFC:</p>
<blockquote>
<p>The first argument is a list of the tokens between the delimiters in the macro use. Examples:#[foo] =&gt; None#[foo()] =&gt; Some([])#[foo(a, b, c)] =&gt; Some([Ident(a), Comma, Ident(b), Comma, Ident(c)])The second argument is the tokens for the AST node the attribute is placed on.</p>
</blockquote>
<p>Internally, it’s the same thing: use a <code>TokenStream</code>, produce a <code>TokenStream</code>.</p>
<h3>“Macros 1.1”</h3>
<p>You may have also heard a lot of commotion about “Macros 1.1.” In fact, they will be stable as of the next release of Rust, 1.15. Macros 1.1 are the <a href="https://en.wikipedia.org/wiki/Pareto_principle">Pareto principle</a> applied to procedural macros: as it turns out, the vast majority of people who use procedural macros today use it for “custom derive”, that is, the ability to say <code>#[derive(MyTrait)]</code> and implement a trait automatically. As such, a plan was hatched: we could stabilize <em>just the very very very basics</em> of procedural macros, enough to get the ability to write custom derives, and stabilize that, without needing to implement everything. What does that look like? Here’s one that implements <code>#[derive(Double)]</code></p>
<pre><code>extern crate proc_macro;
use proc_macro::TokenStream;
#[proc_macro_derive(Double)]
pub fn double(input: TokenStream) -&gt; TokenStream {
let source = input.to_string();
// Parse `source` for struct/enum declaration, and then build up some new
// source code representing a number of items in the implementation of
// the `Double` trait for the struct/enum in question.
let source = derive_double(&amp;source);
// Parse this back to a token stream and return it
source.parse().unwrap()
}
</code></pre>
<p>You’ll notice that this still looks like <code>TokenStream</code> -&gt; <code>TokenStream</code>, but there’s something more subtle here: <code>input.to_string()</code>. That is, for Macros 1.1, <code>TokenStream</code> has two methods; all it has is the ability to be converted to a string, with the original Rust code inside of it, and a <code>parse</code> method, which can take a string of Rust code and produce another <code>TokenStream</code>. What you do with that <code>String</code> is your own business.</p>
<p>What most people will do is use <a href="https://github.com/dtolnay/syn">syn</a> to parse the Rust code, and <a href="https://github.com/dtolnay/quote">quote</a> to generate Rust code from that. These libraries will make it feasible to build real-world custom derive code, without us needing to stabilize anything we don’t want to. Of course, once full procedural macros are implemented, it’s not likely that people will want to turn <code>TokenStream</code>s into and parse <code>TokenStream</code>s from strings, but it’s not <em>totally</em> unheard of, and will be the tiniest wart. It’s worth it to move a significant chunk of the ecosystem onto the stable compiler.</p>
<p>For more details, see <a href="https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md">RFC 1681</a>.</p>
<h3>“Macros 1.2” ?</h3>
<p>There’s also been some small talk about possibly doing a “macros 1.2”, that is, another step towards full procedural macros without needing the whole thing. This would be “procedural macros without hygiene”, though it’s still early days for this idea; there isn’t even an RFC yet. We’ll see if this happens. There’s still some controversy around this idea.</p></content:encoded>
</item>
<item>
<title>Announcing Rust Contributors</title>
<link>https://steveklabnik.com/writing/announcing-rust-contributors/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/announcing-rust-contributors/</guid>
<pubDate>Thu, 12 Jan 2017 00:00:00 GMT</pubDate>
<content:encoded><p>When I was working with Ruby on Rails, someone once sent me this link:</p>
<p>http://contributors.rubyonrails.org/</p>
<p>In some ways, this link changed my life. You see, I respond <em>really</em> strongly to integers going upwards. I am a sucker for points, hits, whatever. That web page, however, puts that integer towards something useful: open source contributions. It’s fairly well known that open source contribution is a long tail, but that’s also <em>perfect</em> for this kind of gamification. One contribution, as of today, will put you in 2461th place. Your second? 1473! That’s almost a thousand places! Just <em>one</em> more contribution puts you at number 1114. Still three hundred spots. Your forth commit will let you get in the top 900, #893. You get the idea. (I ended up at #47.)</p>
<p>Today, I’m working on Rust. And one of the things that I do is write the release announcements, or at least, their first draft. And one of my favorite things to do is the last bit: thanking all of the people who have put in work toward this release. It’s one of my favorite things because I really do appreciate all of the patches, but also because I remember how hooked I was when people gave me a little bit of credit for my contribution. Thanking people is powerful.</p>
<p>So to write those blog posts, I have to run some <code>git</code> commands. And I always forget <em>exactly</em> what I have to write, given our branching strategy. So I wrote a little <code>Rakefile</code> almost a year ago that will generate a new post with the appropriate thank yous and statistics. But I never made it general enough; I have to tweak the numbers every release. It’s annoying. I’ve always wanted to fix it up. There’s one more way in which I don’t like this setup: we only thank people who contribute to <code>rust-lang/rust</code>. Even those with a commit to Cargo don’t make it in there. It’s a historical artifact of the release process, but I don’t like it. We should be thanking those people too.</p>
<p>While all of this has been rolling around in the back of my brain, I’ve been thinking about contributors.rubyonrails.org. I’ve said for a while I wanted it in Rust. So, over the last few days, I hacked one together.</p>
<p>https://rust-contributors.herokuapp.com/</p>
<p>Right now, it only replicates what we say in the blog posts. It doesn’t show commit counts, or order by it, or anything. But the foundation is there. Also, since this is a fun hack, I didn’t take too much care when writing it: there’s a number of low-hanging fruit kinds of things to do. I didn’t use a fancy web framework. I didn’t focus on refactoring relentlessly. I didn’t write any tests. The error handling is basically non-existent. I just knocked it out as fast as I can. But it’s a web app, written in Rust, and that’s fun.</p>
<p>If you’d like to help me hack on this, head over to https://github.com/steveklabnik/contributors and take a look at the issues. I’ve already identified a number of things, and if you do web work but have been meaning to mess with Rust, this might be a good chance to give it a shot.</p>
<p>Thank you to everyone who submits patches to Rust; you’re all great, and I’m lucky to be able to work with you all.</p></content:encoded>
</item>
<item>
<title>Fire Mario not fire flowers</title>
<link>https://steveklabnik.com/writing/fire-mario-not-fire-flowers/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/fire-mario-not-fire-flowers/</guid>
<pubDate>Thu, 29 Dec 2016 00:00:00 GMT</pubDate>
<content:encoded><p><a href="http://words.steveklabnik.com/rust-is-more-than-safety">My post yesterday</a> got a lot of great responses. Two of them are from people I greatly respect, <a href="https://thefeedbackloop.xyz/safety-is-rusts-fireflower/">Dave Herman</a> and <a href="http://graydon2.dreamwidth.org/247406.html">Graydon Hoare</a>. You should go read those two posts; this one is in response to them. I’m going to respond to both at the same time by mixing them together.</p>
<p>Graydon says:</p>
<blockquote>
<p>Safety in the systems space is Rust’s raison d’être. Especially safe concurrency (or as Aaron put it, fearless concurrency). I do not know how else to put it.</p>
</blockquote>
<p>“Fearless concurrency” is a different way to put it! That is, to use Dave’s metaphor, “safety” is the fire flower. But “fearless concurrency” is Fire Mario. This is where I think Dave misses the mark slightly as well. I agree with his metaphor, but disagree with his conclusion.</p>
<p>I’m not saying that we get rid of fire flowers. I’m saying that we focus on “Rust makes you Fire Mario,” not “Rust is made of fire flowers”, when talking to people about Rust.</p>
<p>Dave also misses my mark slightly, but in a different way:</p>
<blockquote>
<p>I don’t think the Rust community should worry about shying away from talking about safety, but I hope we all remember that it’s in service of something much more exciting: making more badass systems programmers!</p>
</blockquote>
<p>We are so close here that it feels like nitpicking, but I think the difference is important. I think one of Rust’s greatest potentials, like Dave does, is bringing a ton of new people into systems programming. But my point is this: “memory safety without garbage collection” doesn’t speak to that <em>at all</em>. That is, memory safety without garbage collection is the mechanism of how that’s accomplished, but it’s not the accomplishment itself. It’s the Fire Flower, not Fire Mario.</p>
<p>I’d like to make sure we’re talking about being Fire Mario, rather than being Fire Flower enthusiasts.</p></content:encoded>
</item>
<item>
<title>Rust is more than safety</title>
<link>https://steveklabnik.com/writing/rust-is-more-than-safety/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rust-is-more-than-safety/</guid>
<pubDate>Wed, 28 Dec 2016 00:00:00 GMT</pubDate>
<content:encoded><p>Rust is most well-known for its features around writing low-level code in a safer way than its predecessors. But it’s also a lot more than that. The Rust community is slowly evolving the way that we talk about Rust to people who haven’t yet given Rust a try, but it can take time. But how did we get here?</p>
<p>In Rust’s initial release announcement, Graydon used a phrase that is amazing:</p>
<blockquote>
<p>[Rust is] Technology from the past come to save the future from itself</p>
</blockquote>
<p>He goes on to elaborate a bit:</p>
<blockquote>
<p>Many older languages [are] better than new ones. We keep forgetting already-learned lessons.</p>
</blockquote>
<p>This is one of the many things that the “Rust” name evokes. Rust was not intended to be a cutting-edge PLT (programming language theory) research language. It was challenging an assumption that I think many of us hold dear: that “progress” means that things always get better. Reality is a bit more complex than that. Yes, we’ve improved on some things, but there are some older things that were actually “better” in some sense, but never quite made it for other reasons. I’ve always phrased this sentiment like this:</p>
<hr />
<p>Let’s look at the initial release dates of some of the top programming languages:</p>
<ul>
<li>C: 1972</li>
<li>C++: 1983</li>
<li>C#: 2000</li>
<li>Java: 1995</li>
<li>JavaScript: 1995</li>
<li>Python: 1991</li>
<li>PHP: 1995</li>
<li>Ruby: 1995</li>
</ul>
<p>The first thing you’ll notice is: what the hell were they putting in the water in 1995?</p>
<p>The second is that, other than C# in 2000, 1995 or earlier is where most of our programming languages come from. Have we really not made any advancements significant enough in the last 20 years to have gotten people’s attention?</p>
<hr />
<p>Now, there are a multitude of newer languages, and people do use them. And these languages have had more releases since then. But none of these languages has fundamentally changed. They’re all pretty much the same, but better. And specifically, you’ll notice that many of these languages are scripting or higher-level languages: if we restrict ourselves to “what most operating systems are programmed in”, we haven’t had a new generation of programming languages in thirty or forty years!</p>
<p>PLT research and industry live in two different worlds. Industry is effectively using ancient PLT tech. There’s lots of <em>good</em> reasons for this too, mind you. But if you’re looking to make a new hit programming language, you don’t actually have to be innovative in the PLT sense. You can take PLT tech from the 2005-2010 era and you’ll still be ten years ahead!</p>
<p>So in some senses, this is what Rust is trying to do: not be a cutting-edge programming language in a PLT sense, but advance the industry by mining “older” research.</p>
<p>However, plans don’t always work out the way you want to. <a href="https://www.youtube.com/watch?v=79PSagCD_AY">As Rust evolved</a>, many of its original bits were thrown out, and replaced with new ones. Rust <em>has</em> advanced a small bit of the universe in PLT with the ownership/borrowing system. Like most research, it’s built on top of older work. But that’s exciting!</p>
<p>Another thing about the borrow checker is that it’s “unique” about Rust as well. So if someone asks you, “hey, what’s cool about Rust?”, it’s extremely tempting to answer with the thing that makes it most clearly different than other programming languages. And for some people, this pitch is compelling! But not for everyone…</p>
<h2>Shifting priorities</h2>
<p>Marketing is hard. But there’s one thing that I’ve learned about it that I think applies to Rust:</p>
<blockquote>
<p>Your marketing should be focused on what problems your product solves for the consumer.</p>
</blockquote>
<p>Rust’s marketing, historically, hasn’t been as good at this as it could have been. That is, we’ve been too focused on the details, and not enough on the big picture. We’ve been worried about what’s interesting in a PLT aspect, and not what problems Rust solves for users. I can think of two big examples where this has happened:</p>
<ul>
<li>Rust is safer than C or C++. (“I never have problems with safety in the C++ code I write” is a common reply.)</li>
<li>Rust is as low-level of a programming language as C! (“I never write C. Rust is not for me.”)</li>
</ul>
<p>Both of these pitches for Rust focus on the <em>how</em>, and not on the problem itself. And that means that sometimes people draw the opposite inference as to what you were intending with the pitch in the first place.</p>
<p>I have some ideas on a better pitch for Rust, and I even had a draft of it here, but I’ve decided to delete it for now. Instead, I’d like to ask all of you who love Rust to give this some thought, and write your own blog posts about what problems Rust solves for you. I’ll edit this post and link to yours if you ask me to. I’ve seen several people do a good job at articulating this kind of thing before. And I’ll work on my own post as well, as a follow-up.</p>
<p>Let’s figure out the best way to share our love of Rust with the rest of the programming world.</p>
<hr />
<p>Here’s some of the responses generated from this:</p>
<p>I’ve written <a href="http://words.steveklabnik.com/fire-mario-not-fire-flowers">Fire Mario, not Fire Flowers</a>, as a response to two responses to this post.</p>
<p>The <a href="https://www.reddit.com/r/rust/comments/5krhr0/rust_is_more_than_safety/">/r/rust</a>, <a href="https://www.reddit.com/r/programming/comments/5krw7r/rust_is_more_than_safety/">proggit</a>, and <a href="https://news.ycombinator.com/item?id=13272474">orange website</a> threads are full of good replies.</p>
<ul>
<li><a href="https://medium.com/@Hisako1337/rust-in-2017-8f2b57a67d9b">Rust in 2017</a></li>
<li><a href="https://insanitybit.github.io/2016/12/28/why-rust-introduction">Why Rust?</a> and <a href="https://insanitybit.github.io/2016/12/28/why-rust-sum-types">Why Rust? Sum Types</a></li>
</ul></content:encoded>
</item>
<item>
<title>Four years with Rust</title>
<link>https://steveklabnik.com/writing/four-years-with-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/four-years-with-rust/</guid>
<pubDate>Wed, 21 Dec 2016 00:00:00 GMT</pubDate>
<content:encoded><p>Today is four years since I first learned about the existence of Rust. I know this because Rust 0.5 was the first release I used. Rust has changed a lot in that time. For a brief overview of its history, <a href="https://www.youtube.com/watch?v=79PSagCD_AY">watch this talk of mine</a>. But for today, I thought it would be fun to check out <a href="https://mail.mozilla.org/pipermail/rust-dev/2012-December/002787.html">the release announcement</a> and see what’s the same, and what’s changed.</p>
<h2>rust-dev</h2>
<p>First up, the announcement itself. The <a href="https://mail.mozilla.org/listinfo/rust-dev">rust-dev mailing list</a> used to be your one-stop shop for talking about Rust in a persistent form. But in <a href="https://mail.mozilla.org/pipermail/rust-dev/2015-January/011558.html">January of 2015</a>, we decided to close it down. Why? Let’s see what Brian said at the time:</p>
<blockquote>
<p>You likely have already noticed, but traffic to rust-dev has decreased dramatically in recent months. This was a result of natural changes in project coordination at first, and then an intentional effort to phase out the list.In the beginning of the project there were only two places to talk about rust: #rust and rust-dev, and they served us well for all purposes for several years. As Rust grew though the coordination of the project moved to different venues, conversations were happening in a number of other places, and the purpose of rust-dev became less clear. At the same time there were also a number of heated and essentially unmoderatable discussions that caused many to lose faith in the effectiveness of the mailing list.</p>
</blockquote>
<p>I love mailing lists, but I know I’m fairly unique in that. But they do have one big problem, which is that the tools for moderating them are extremely thin. You can pretty much ban someone, and that’s about it. No removing only certain posts, no cooldown periods, no shadow banning. While Rust has a generally positive reputation for community and moderation, that doesn’t mean things were always perfect. We learned some lessons the hard way.</p>
<p>More than that, mailman represents something of the “old guard” of open source. People see it as a sign of something backwards. Very few people want even more email. The culture is changing. We recognized this, and decided to move to <a href="http://www.discourse.org/">Discourse</a> instead. rust-dev was survived by two children, <a href="http://users.rust-lang.org/">users</a> and <a href="https://internals.rust-lang.org/">internals</a>. This created a clear split between “discussion about Rust” and “discussions about building Rust.” It’s still free/open source software, it has good moderation tools, and you can even have it send you email!</p>
<p>We also don’t make release announcements on these forums; we do it on <a href="https://blog.rust-lang.org/">our blog</a>.</p>
<p>Let’s dig into those notes!</p>
<h2>900 changes, numerous bugfixes</h2>
<p>900 changes is a lot! At the time, Rust was on a roughly six-month release schedule, so that’s about five patches merged per day. Rust 1.14 <a href="https://github.com/rust-lang/blog.rust-lang.org/pull/141">will have</a> 1230 patches over its six-week release cycle: that’s just over 29 per day. That’s a much, much higher velocity. And that only counts the main repository; we’ve since added many more, like <a href="https://github.com/rust-lang/cargo/">Cargo</a> and <a href="https://github.com/rust-lang/crates.io">crates.io</a>. This is actually something I’ve been worried about for a while now; we only credit people for commits to <code>rust-lang/rust</code> in release announcements, basically due to this history, but there’s a lot more work that goes on here these days. I want <a href="http://contributors.rubyonrails.org/">Rails Contributors</a>, but for the Rust project. Anyone want to work on it with me sometime?</p>
<h2>Syntax changes</h2>
<ul>
<li>Removed <code>&lt;-</code> move operator</li>
</ul>
<p>Rust went through several iterations of “when do you move, when do you copy” over its lifetime. Today, that distinction is made between types that implement <code>Copy</code> and ones that don’t. But older Rusts had various schemes. Given that this one was removed in the first version of Rust I used, I don’t <em>exactly</em> remember this, but I think it was</p>
<pre><code>x &lt;- y; // move
x = y; // copy
</code></pre>
<p>Rust also had this at one point in time, I believe:</p>
<pre><code>x = move y; // move
x = y; // copy
</code></pre>
<p>We decided to unify the two syntactically for two reasons: one, the annotations felt like busywork, the compiler would tell you what to put if you got it wrong. Secondly, at least in today’s Rust, move and copy are identical operations, with the exception of what you can do with the older variable after the move. Sharing the syntax reinforces that.</p>
<p>Niko’s blog has a lot of great history in it, and <a href="http://smallcultfollowing.com/babysteps/blog/2012/10/01/moves-based-on-type/">this topic is no exception</a>.</p>
<ul>
<li>Completed transition from <code>#fmt</code> extension syntax to <code>fmt!</code></li>
</ul>
<p>Long ago, Rust had a “no keywords can have more than five letters” rule, and so many things were abbreviated. This particular extension lives on today, but with the longer name <code>[format!](https://doc.rust-lang.org/stable/std/macro.format.html)</code>.</p>
<ul>
<li>Removed old fixed length vector syntax - <code>[T]/N</code></li>
</ul>
<p>Rust’s vector and array types, as we know them today, went through a <em>lot</em> of internal iteration, across many axes. The moral equivalent of <code>[T]/N</code> today is <code>[T; N]</code>, but I’m sure there are/were lots of subtle differences between the two. I can’t quite remember.</p>
<ul>
<li>New token quasi-quoters, <code>quote_tokens!</code>, <code>quote_expr!</code>, etc.</li>
</ul>
<p>These live on in some sense; they’re still not stable, so much so that they’re note even shown in the official docs. <a href="https://manishearth.github.io/rust-internals-docs/syntax/ext/quote/index.html">Manish hosts a copy though</a>. These are tools for “syntax extensions”, Rust’s most powerful form of metaprogramming. A final design for them was <a href="https://github.com/rust-lang/rfcs/pull/1566">just accepted eight days ago</a>, so it’ll be a while still before you see this on stable Rust.</p>
<ul>
<li>Macros may now expand to items and statements</li>
</ul>
<p>I don’t know why macros couldn’t do this before, but they still can today, and it’s very useful.</p>
<ul>
<li><code>a.b()</code> is always parsed as a method call, never as a field</li>
</ul>
<p>This is a fun edge case. Here’s the problem:</p>
<pre><code>struct Env&lt;F: Fn(i32)&gt; { f: F,}let e = Env { f: |i| println!("Hello, {}", i) };e.f(); // what does this do?
</code></pre>
<p>According to this entry, this would be a method call. In today’s Rust, this would be an error, you need to write <code>(e.f)();</code> At least the error tells you exactly what to do!</p>
<pre><code>error: no method named `f` found for type `Env&lt;[closure@&lt;anon&gt;:6:22: 6:50]&gt;` in the current scope
--&gt; &lt;anon&gt;:8:7
|
8 | e.f(); // what does this do?
| ^
|
note: use `(e.f)(...)` if you meant to call the function stored in the `f` field
--&gt; &lt;anon&gt;:8:7
|
8 | e.f(); // what does this do?
| ^
</code></pre>
<ul>
<li><code>Eq</code> and <code>IterBytes</code> implementations automatically generated with <code>#[deriving_eq]</code> and <code>#[deriving_iter_bytes]</code> respectively</li>
</ul>
<p>We now have a more generic way to derive traits: <code>#[derive(Eq)]</code>, for example. <code>Eq</code> is still here today, but <code>IterBytes</code> is not. I can’t recall exactly what it did.</p>
<p>Today, the list of traits you can derive is still only blessed ones by the compiler, and it’s a huge cause of people using nightly today, for better ergonomics with libraries like <a href="http://serde.rs/">Serde</a> and <a href="http://diesel.rs/">Diesel</a>. But as of Rust 1.15, this restriction will be lifted, and one of the largest blockers of people using stable Rust will be eliminated! ???</p>
<ul>
<li>Removed the special crate language for <code>.rc</code> files</li>
</ul>
<p>The moral equivalent of <code>.rc</code> files today are <code>.crate</code> files, which Cargo will upload to crates.io. But unlike in the 0.5 days, you almost never need to worry or think about them, since Cargo makes it Just Work (tm).</p>
<ul>
<li>Function arguments may consist of any irrefutable pattern</li>
</ul>
<p>This is true today, and is a little known fact! It works like this:</p>
<pre><code>struct Point { x: i32, y: i32,}fn takes_point(Point {x, y}: Point) { println!("({}, {})", x, y);}fn main() { let origin = Point { x: 0, y: 0 }; takes_point(origin);}
</code></pre>
<p>That is, arguments to functions are <code>PATTERN: TYPE</code>, not <code>NAME: TYPE</code>. Inside of <code>takes_point</code>, <code>x</code> and <code>y</code> are in scope, rather than a whole point.</p>
<h2>Semantic changes</h2>
<ul>
<li><code>&amp;</code> and <code>~</code> pointers may point to objects</li>
</ul>
<p>Rust did have objects, long ago, but I thought they were removed at this point. This one is unclear to me.</p>
<p>Worth mentioning that <code>~</code> is <code>Box&lt;T&gt;</code> now, though. That’s a long story…</p>
<ul>
<li>Tuple structs - <code>struct Foo(Bar, Baz)</code>. Replace newtype enums.</li>
</ul>
<p>These are <a href="https://doc.rust-lang.org/stable/book/structs.html#tuple-structs">still in Rust today</a>.</p>
<ul>
<li>Enum variants may be structs</li>
</ul>
<p>This is true today:</p>
<pre><code>enum Foo {
Variant { x: i32, y: i32 },
}
</code></pre>
<ul>
<li>Destructors can be added to all nominal types via Drop trait</li>
</ul>
<p>This is <a href="https://doc.rust-lang.org/stable/std/ops/trait.Drop.html">still true</a>!</p>
<p>I think the “nominal” here is alluding to something older in Rust’s type system; we don’t make these kinds of distinctions today, as IIRC, all types are nominal. Not 100% sure.</p>
<ul>
<li>Structs and nullary enum variants may be constants</li>
</ul>
<p>This is true today.</p>
<ul>
<li>Values that cannot be implicitly copied are automatically moved without writing <code>move</code> explicitly</li>
</ul>
<p>I alluded to this above with <code>&lt;-</code>.</p>
<ul>
<li><code>&amp;T</code> may now be coerced to <code>*T</code></li>
</ul>
<p><code>*T</code> is <code>*const T</code> or <code>*mut T</code> today, but this is still true:</p>
<pre><code>let x = &amp;5;
let y: *const i32 = x;
</code></pre>
<ul>
<li>Coercions happen in <code>let</code> statements as well as function calls</li>
</ul>
<p>We’ve gone through many different iterations of “what coerces and what doesn’t” over the years; I actually expected the example with <code>*const i32</code> above to require an <code>as</code>.</p>
<ul>
<li><code>use</code> statements now take crate-relative paths</li>
</ul>
<p>This is true today. I really like it, but a lot of people find it confusing. Basically, “use” always starts from the root of the crate. So</p>
<pre><code>mod bar { struct Foo; mod baz { use bar::Foo; // from the root use super::Foo; // if you want to start from the parent use self::super::Foo; // self makes it be relative. Not even sure if self + super works, but you get the idea. }}
</code></pre>
<ul>
<li>The module and type namespaces have been merged so that static method names can be resolved under the trait in which they are declared</li>
</ul>
<p>Not sure on this one, it’s too in the weeds.</p>
<h2>Improved support for language features</h2>
<ul>
<li>Trait inheritance works in many scenarios</li>
</ul>
<p>This has always been a bit of a misnomer, inheritance means:</p>
<pre><code>trait Foo: Bar {
}
</code></pre>
<p>If you implement <code>Foo</code>, you must also implement <code>Bar</code>. That’s it. This line makes me smile: “it works more often.”</p>
<ul>
<li>Support for explicit self arguments in methods - <code>self</code>, <code>&amp;self</code> <code>@self</code>, and <code>~self</code> all generally work as expected</li>
</ul>
<p>Today’s Rust doesn’t just support this; it’s required! However:</p>
<ul>
<li><code>self</code> is still <code>self</code></li>
<li><code>&amp;self</code> is still <code>&amp;self</code></li>
<li><code>@self</code> is <code>Rc&lt;self&gt;</code>, sorta kinda.</li>
<li><code>~self</code> is <code>Box&lt;self&gt;</code></li>
</ul>
<p><code>@self</code> was supposed to be a garbage-collected type, but was never really more than refcounting.</p>
<p>In addition, the last two don’t actually work like that: they would be <code>self: Rc&lt;Self&gt;</code> and <code>self: Box&lt;self&gt;</code>. The <code>Box</code> version will compile today, but the <code>Rc</code> version does not, because <code>Box</code> is magical. Eventually this will be rectified.</p>
<ul>
<li>Static methods work in more situations</li>
</ul>
<p>Another “yay more features work” line. :)</p>
<ul>
<li>Experimental: Traits may declare default methods for impls to use</li>
</ul>
<p>This works today, and is very useful. Consider <code>[Iterator](https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html)</code>: all of those methods have default implementations, so you only need to define <code>next()</code>, and you get the rest automatically.</p>
<h2>Libraries</h2>
<ul>
<li>New condition handling system in <code>core::condition</code></li>
</ul>
<p>Conditions were a Lisp way of handling errors that Rust supported for a while. They were really cool, but nobody really knew how to effectively use them, so they never got used, so they’re gone now.</p>
<ul>
<li>Timsort added to <code>std::sort</code></li>
</ul>
<p><code>std::sort</code> doesn’t exist anymore, though we do have <a href="https://doc.rust-lang.org/stable/std/primitive.slice.html#method.sort">a sort function on slices</a>. We don’t declare it to have a particular algorithm, though we do say “This sort is stable and O(n log n) worst-case but allocates approximately 2 * n where n is the length of self.”</p>
<p><a href="https://github.com/rust-lang/rust/pull/38192">Recently it was improved a ton</a>, by a first-time contributor, no less! Extremely well-done. In the PR, he explains the algorithm:</p>
<blockquote>
<p>However, if we take the main ideas from TimSort (intelligent merging strategy of sorted runs) and drop galloping, then we’ll have great performance on random inputs and it won’t be bad on partially sorted inputs either.</p>
</blockquote>
<p><code>TimSort</code> is still around, kicking.</p>
<ul>
<li>New priority queue, <code>std::priority_queue</code></li>
</ul>
<p>Today this is <code>[std::collections::binary_heap](https://doc.rust-lang.org/std/collections/binary_heap/)</code>.</p>
<ul>
<li>Pipes for serializable types, `std::flatpipes’</li>
</ul>
<p>No idea where this went.</p>
<ul>
<li>Serialization overhauled to be trait-based</li>
</ul>
<p>This is true today. See the above comments about Serde, though. <code>[rustc-serialize](https://crates.io/crates/rustc-serialize)</code> got to cheat though, and the compiler understands <code>RustcEncodable</code> and <code>RustcDecodable</code>. Can’t wait for Rust 1.15.</p>
<ul>
<li>Expanded <code>getopts</code> definitions</li>
</ul>
<p>We’ve <a href="https://github.com/rust-lang-nursery/getopts">moved <code>getopts</code> out of tree</a>, but it’s still there.</p>
<ul>
<li>Moved futures to <code>std</code></li>
</ul>
<p>Hoo boy! I forgot that we used to have futures in the standard library. Futures are one of the hottest topics in Rust today. <a href="https://github.com/tokio-rs">Tokio</a> is one of the most anticipated releases of all time in Rust world. For a taste, <a href="https://www.youtube.com/watch?v=bcrzfivXpc4">watch this talk</a>. A little birdie told me that Tokio should have a <code>0.1</code> release soon…</p>
<ul>
<li>More functions are pure now</li>
</ul>
<p>Rust no longer has a notion of purity, though <code>[const fn](https://github.com/rust-lang/rfcs/blob/master/text/0911-const-fn.md)</code> (still unstable) sorta feels like it in some senses. For some history on purity in Rust, see <a href="http://smallcultfollowing.com/babysteps/blog/2012/10/12/extending-the-definition-of-purity-in-rust/">Niko’s blog</a>, or <a href="https://mail.mozilla.org/pipermail/rust-dev/2013-April/003926.html">this explanation by Graydon on why it was removed</a>.</p>
<ul>
<li><code>core::comm</code> renamed to <code>oldcomm</code>. Still deprecated</li>
</ul>
<p>Very, very long gone.</p>
<ul>
<li><code>rustdoc</code> and <code>cargo</code> are libraries now</li>
</ul>
<p>Rustdoc still exists, though not <em>really</em> as a library. Cargo however, is not the cargo you’re thinking of. <a href="https://github.com/rust-lang/rust-wiki-backup/blob/73816d6f888c24fb8115d78078a1601805cbecb5/Doc-using-cargo-to-manage-packages.md">It’s a totally different one</a>. Rust went through many iterations of package managers before coming up with today’s Cargo; there was also <code>[rustpkg](https://github.com/rust-lang/rust/blob/3e39e3e80dcf726a96ec0fe778f96e2a9dde620b/doc/guide-rustpkg.md)</code>, for example, which was <em>after</em> the Cargo referenced here, but before the Cargo we’re using today.</p>
<h2>Misc</h2>
<ul>
<li>Added a preliminary REPL, <code>rusti</code></li>
</ul>
<p>We removed the repl from the tree, as it never <em>really</em> worked, and was a maintenance nightmare. <a href="https://github.com/murarth/rusti">This one</a> is the only one I know of today. Some people ask about one, but nobody has done the work to get a good one together yet. It’s tough!</p>
<ul>
<li>License changed from MIT to dual MIT/APL2</li>
</ul>
<p>This is still true today, with no plans on changing.</p>
<h2>Contributors to Rust 0.5:</h2>
<p>We had 41 people contribute to 0.5. 1.14 will have 144 contributors, and that’s around average for a release these days. Out of those 41 people, at a glance, I recognize the names of about 17 individuals, and roughly six of them are still involved with Rust and Servo today. What about the other eleven? Some were interns that got jobs at other places and can’t work on Rust anymore, some are contributors that left for various reasons, and then of course, there’s Graydon.</p>
<p>I personally opened <a href="https://github.com/rust-lang/rust/pull/4305">my first pull request</a> six days after discovering it. You’ll note this PR wasn’t merged due to a procedural issue: I sent it into the wrong branch! Today, GitHub lets you change this, but I had to <a href="https://github.com/rust-lang/rust/pull/4308">open a new PR instead</a>.</p>
<p>As I said in that PR:</p>
<blockquote>
<p>I’ve just started using Rust, and I really like it. One of the things that makes Rust hard to use is its lack of documentation, which is obviously fine, given that Rust isn’t production-ready yet.I’d like to help change that.To get started, I’ve just modified this one little description in the rustdoc. If this works out, and you all like my changes, I’ll start working on more docs like this for the rest of core, and maybe the stdlib after that.</p>
</blockquote>
<p>You might say it did work out. I’m now <a href="https://blog.rust-lang.org/2014/12/12/Core-Team.html">on the core team</a>, I’ve sent in 866 PRs since then, and that first patch landed in <a href="https://mail.mozilla.org/pipermail/rust-dev/2013-April/003427.html">Rust 0.6</a>. Rust 0.10 is the only release since then that I’ve not gotten at least one patch into, and today, I write the release announcements.</p>
<p>I hope you’ve enjoyed this little trip down memory lane! I’m still really loving Rust and its community. Here’s to many more years ?♥</p></content:encoded>
</item>
<item>
<title>What's new with "The Rust Programming Language"</title>
<link>https://steveklabnik.com/writing/whats-new-with-the-rust-programming-language/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/whats-new-with-the-rust-programming-language/</guid>
<pubDate>Mon, 15 Aug 2016 00:00:00 GMT</pubDate>
<content:encoded><p>It’s been a while since I’ve mentioned my work on “The Rust Programming Language”, and I wanted to give you all an update. Before we get into the news, though, I’d like to take a moment to reflect on where the book has been, and then we’ll talk about it’s future.</p>
<p>I <a href="http://words.steveklabnik.com/rusts-documentation-is-about-to-drastically-improve">previously wrote about my history with Rust and docs</a>. As I said back then:</p>
<blockquote>
<p>But my first area of focus will be on the area of Rust’s documentation that’s the most weak, and simultaneously the most important: the tutorial.</p>
</blockquote>
<p>“The tutorial” eventually turned into “The Rust Programming Language.” But there’s a funny thing about writing a book for a language that isn’t done yet: you don’t really know <em>how</em> to write a good book on something that doesn’t exist, because, well, you haven’t played with it yourself. I could make decent guesses, given that Rust was evolving incrementally, but that’s still very different. I think “technical debt” is a great metaphor here: the evolution of the language meant that I had to keep up with the work of many, many people to try and update the documentation. Furthermore, since nobody had tried to learn Rust 1.0 yet, since it didn’t exist, I couldn’t write a book that would do an amazing job of teaching them, since I hadn’t used it either!</p>
<p>In some ways, this mirrors the compiler itself, which accrued a <em>massive</em> amount of technical debt until 1.0. And so, the story since 1.0 has largely about been paying off this debt; culminating in <a href="https://blog.rust-lang.org/2016/04/19/MIR.html">MIR</a>. By the same token, since 1.0, a lot of people have read the book, and a lot of people have tried to learn Rust. I now know so much more about how people go about doing so, and so am much, much better situated to write something that’ll be great for them.</p>
<p>And so, now that time has passed, re-writing the book is what I’ve been spending a lot of my time on. I’ve been mostly quiet about this work, since I’ve wanted to get a chunk finished before starting to talk about it, and so today is finally that day.</p>
<p>Let’s talk about what’s going on with the book today.</p>
<h2>A new coauthor.</h2>
<p>The first bit of news is something that I’ve been wanting to tell you all for a few weeks now, and is the bit that I’m most excited about: I’m bringing <a href="https://twitter.com/carols10cents">Carol (Nichols || Goulding)</a> on as a coauthor. TRPL is an incredibly personal work for me, one that I’ve poured my life into over the last few years. Carol and I have been friends for a long time, and there’s nobody else I’d rather share an authorship with. Not only is Carol a friend, but she’s been doing Rust longer than most. You might know her from <a href="https://github.com/carols10cents/rustlings">Rustlings</a>, or the <a href="http://rust-belt-rust.com/">Rust Belt Rust conference</a>, or as a cofounder of <a href="http://integer32.com/">Integer32</a>, the first Rust-focused consulting company. Carol knows her stuff, and I’m proud to have her name next time mine on the cover. The book will be far better through her influence.</p>
<h2>A new beginning</h2>
<p>One of Carols’ first tasks was taking a bunch of the stuff I’d already written and checking it out herself, making improvements. She’s taken the first six chapters of the book and whipped them into shape. Some of the content you might recognize from the first edition of TRPL, but much of it is brand-new.</p>
<ul>
<li><a href="http://rust-lang.github.io/book/">Read the first six chapters online</a></li>
<li><a href="https://github.com/rust-lang/book">Source and issue tracker</a></li>
</ul>
<p>As an example of how much better this book is, can, and will be than the first edition, compare the <a href="http://rust-lang.github.io/book/ch04-00-understanding-ownership.html">Ownership chapter</a> to <a href="https://doc.rust-lang.org/stable/book/ownership.html">the current one</a> ( plus <a href="https://doc.rust-lang.org/stable/book/references-and-borrowing.html">borrowing</a> ). Instead of an abstract set of rules, we took one of the biggest things facing a new Rustacean, strings, and used them as a way to make these concepts concrete.</p>
<p>If you have the time, please check it out, and file issues with any suggestions!</p>
<h2>More to come</h2>
<p>If you’d like to see chapters as we work on them, just follow the repo: everything is open source. Initial PRs are already up for the start of the next three chapters, and hopefully we’ll get them merged and move forward soon. I’m really proud of how the book is shaping up, and hope you enjoy it too.</p></content:encoded>
</item>
<item>
<title>Structure literals vs. constructors in Rust</title>
<link>https://steveklabnik.com/writing/structure-literals-vs-constructors-in-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/structure-literals-vs-constructors-in-rust/</guid>
<pubDate>Tue, 26 Jul 2016 00:00:00 GMT</pubDate>
<content:encoded><p>Learning the basics of a language and its syntax is easy. Learning how all those bits fit together is a bit harder. There’s a neat intersection between three of Rust’s features that I’ve seen people use, but never seen written down. I was explaining this technique to someone in <code>#rust-beginners</code> the other day, and thought I’d write it down in case it helps you, too.</p>
<p>A small review: If you have a <code>struct</code> in Rust, like this:</p>
<pre><code>struct Point {
x: i32,
y: i32,
}
</code></pre>
<p>You can use ‘<code>struct</code> literal syntax’ to create a new instance of the <code>struct</code>:</p>
<pre><code>let origin = Point { x: 0, y: 0 };
</code></pre>
<p>However, this syntax only works if you have the proper access to the <code>struct</code> and its members via Rust’s privacy rules.</p>
<pre><code>mod foo {
pub struct Point {
x: i32,
y: i32,
}
pub fn foo(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y } // this is fine, as we're in the same module
}
}
fn main() {
let origin = foo::Point { x: 0, y: 0 }; // this is not fine
}
</code></pre>
<p>We can’t use <code>struct</code> literal synatx in <code>main</code> because <code>x</code> and <code>y</code> are also not public. But within the same module, we have access, so it works. So how would we let <code>main</code> instantiate a <code>Point</code> if we can’t use the literal syntax? Well, our <code>foo</code> function does this, so we could expose it. More conventionally, we’d make it an associated function and call it <code>new</code>:</p>
<pre><code>mod foo {
pub struct Point {
x: i32,
y: i32,
}
impl Point {
pub fn new(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y } // this is fine, as we're in the same module
}
}
}
fn main() {
let origin = foo::Point::new(0, 0);
}
</code></pre>
<p>Great. But what if we wanted <code>x</code> and <code>y</code> to be public for some reason, yet we still wanted to force people to use the <code>new</code> function to create <code>Point</code>s? Maybe the initial creation does some sort of side effect that’s important. If we changed our code to this:</p>
<pre><code>mod foo {
pub struct Point {
pub x: i32,
pub y: i32,
}
impl Point {
pub fn new(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y } // this is fine, as we're in the same module
}
}
}
fn main() {
let origin = foo::Point::new(0, 0);
// but so does this:
let origin = foo::Point { x: 0, y: 0 };
}
</code></pre>
<p>By making all of the elements of <code>Point</code> public, we’ve re-enabled the literal syntax, which isn’t what we wanted. So what do we do?</p>
<p>Fixing this requires two insights. The first is “zero-sized types”. In Rust, certain types only have values that don’t require any storage. Take, for example, <code>()</code>, the “unit” type. It only has one possible value, also <code>()</code>. Since it only has one value, there’s no need to actually store anything in memory to represent <code>()</code>; if we have a valid value, we already know what it is. That means that once we compile down to actual assembly, <code>()</code> just goes away entirely. So we can do this:</p>
<pre><code>mod foo {
pub struct Point {
pub x: i32,
pub y: i32,
_secret: (),
}
impl Point {
pub fn new(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y, _secret: () }
}
}
}
fn main() {
let origin = foo::Point::new(0, 0);
}
</code></pre>
<p>Now, we have a new, non-public field, <code>_secret</code>. I’ve given it a name that starts with an underscore because we don’t intend to use it for anything, and so Rust won’t warn us about it. <code>_secret</code> has the type of <code>()</code>, and so it’s entirely a compile-time construct; it doesn’t materially affect <code>Point</code>’s representation. But it being private <em>does</em> affect how we’re allowed to construct our <code>Point</code>s. <code>main</code> can no longer use the <code>struct</code> literal syntax, since not all of the fields are public.</p>
<p>However, remember that privacy is a module-level thing in Rust. Therefore, we can still use the <code>struct</code> literal syntax inside of the <code>foo</code> module:</p>
<pre><code>mod foo {
pub struct Point {
pub x: i32,
pub y: i32,
_secret: (),
}
impl Point {
pub fn new(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y, _secret: () }
}
}
fn foo() -&gt; Point {
Point: { x: 0, y: 0, _secret: () } // this is still allowed!
}
}
fn main() {
let origin = foo::Point::new(0, 0);
}
</code></pre>
<p>To prevent <code>foo</code> from being able to use the literal syntax, we need one more concept: <code>pub use</code>. Check this out:</p>
<pre><code>mod foo {
mod point {
pub struct Point {
pub x: i32,
pub y: i32,
_secret: (),
}
impl Point {
pub fn new(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y, _secret: () }
}
}
}
pub use foo::point::Point;
fn foo() -&gt; Point {
Point::new(0, 0) // must use `new` here, as we're no longer in the same module!
}
}
fn main() {
let origin = foo::Point::new(0, 0);
}
</code></pre>
<p>By giving <code>Point</code> its own module, everything that’s private to it is private. But typing <code>foo::point::Point</code> would be redundant and an ergonomic regression; <code>pub use</code> saves the day! We re-export the <code>Point</code> structure into <code>foo</code>, so we can still use <code>foo::Point</code>, but since one of its members is private, literal syntax isn’t allowed.</p>
<p>To me, understanding things like this is when I really start to feel like I’m getting to know a language: putting three or four disparate concepts together to achieve some goal. It’s when a language stops being a bunch of disjoint parts and starts becoming a cohesive whole.</p></content:encoded>
</item>
<item>
<title>A eulogy for my 20s</title>
<link>https://steveklabnik.com/writing/a-eulogy-for-my-20s/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/a-eulogy-for-my-20s/</guid>
<pubDate>Sun, 24 Jan 2016 00:00:00 GMT</pubDate>
<content:encoded><p>Today is the last day of my 29th year of existence. I’ve been thinking about it a lot, which is extra strange because I don’t generally care about birthdays. I’m not much of one for holidays in general. So why do I care about this one?</p>
<p>My 20s have been an extreme period of growth. At the start of it, I thought that I was going to live in my little farming town, get married to The Girl, go to mass every Sunday, and pretty much do exactly what everyone else I knew had done. If you’re reading this, well, you know a very different me. Before my 20s, I had only left the country once, a short trip to Toronto. I had barely even left my state. I was smart, but cocky, and incredibly sheltered. I was the very embodiment of a stereotype.</p>
<p>I’ve come a long way.</p>
<p>The root of my anxiety about being 30 comes from something that my brain is constantly whispering into my ear: “You don’t have enough time.” There’s sort of two sides to this; I think: the first is how much I’ve changed in this decade, and the second is a particularly toxic attitude of the industry I’ve immersed myself in.</p>
<p>I feel pretty dumb about this post overall, but whatever. Please just take this as a brain dump from this morning, not some sort of grand, planned essay about how I know all the things or something.</p>
<h2>Identity is a function</h2>
<p>Have you ever played around with <a href="http://www.utdallas.edu/~gupta/courses/apl/lambda.pdf">lambda calculus</a>? I myself only know the basics, I’m not a particularly math-y person. Here’s a short introduction: say we have a function. This function is very simple, it returns its single argument. In other words, the identify function:</p>
<pre><code>fn id(x) {
return x;
}
</code></pre>
<p>(Making this syntax up, though it will look sort of like Rust without types.)</p>
<p>We can write another function, <code>zero'()</code>, like this:</p>
<pre><code>fn zero'(f, g) {
return g();
}
</code></pre>
<p>In other words, <code>zero'()</code> takes two functions, <code>f</code> and <code>g</code>. It then calls <code>g()</code>, ignoring <code>f()</code>. This will make sense in a moment, we just need one more function: <code>succ()</code></p>
<pre><code>fn succ(f, g) {
return f(g());
}
</code></pre>
<p>This function, short for “successor”, applies <code>f()</code> to <code>g()</code>, one time.</p>
<p>With these tools, we can represent all of the natural numbers. This is called <a href="https://en.wikipedia.org/wiki/Church_encoding#Church_numerals">Church numerals</a>, and here’s how it works:</p>
<pre><code>
let zero = zero'(succ, id); // zero any function applied to id zero times.
let one = succ(succ, zero); // one is succ() applied to zero
let two = succ(succ, one); // two is succ() applied to one
let three = succ(succ, two); // three is succ() applied to two
</code></pre>
<p>And so on. Now I have this crippling fear that I’ve messed up the notation, trying to transform it into something that looks more like “normal” programming. I’m sure that someone on the Internet is going to find some kind of flaw and call me an idiot over it.</p>
<p>Ahem.</p>
<p>Anyway, this is the core idea: you can count the number of times you’ve applied a function to identity, and that encodes the number you’re trying to represent. We don’t need state to represent numbers and counting, we can do it all through functions.</p>
<p>The concept of “identity” is a really important one in philosophy. You can take all kinds of approaches here. One classic approach is called “Platonic idealism”, also known as “essentialism.” Essentialism states that you can tell the identity of something by the properties it has. For example, a chair is a chair because it has a platform for you to sit, it has four legs, and a back.</p>
<p>The problem with essentialism is that well, there are things that you or I would reasonably call a chair that has three legs, or five or six. Not all things that have a place to sit can be called a chair, either. In other words, while it sounds reasonable at first, the real world absolutely destroys this kind of categorization, in my opinion.</p>
<p>So what do you do? Another solution to this question is posed by a really broad family called ‘process philosophy.’ The solution to identity there is based around movement, change, becoming. Function application. A “chair” is a particular kind of object that participates in a world like other things that we call “chairs”. In other words, identity is something that comes out of <em>doing</em>, not <em>being</em>.</p>
<p>I am not the person I was ten years ago, quite literally. My body is made up of various kinds of cells, and they die at different rates. At a cellular level, I am quite literally dying all the time, in various microscopic places, and I am also regenerating. New cells are being produced. I am quite literally not my past self at a cellular level. And once I finally die, it’s not like my cells also instantly die either. Some of them will live for a period of time after I’m dead, though not for long. Bits of my body will actually out-live me.</p>
<p>Life is full of growth and change. But at this point, the function has been applied so many times that I barely even recognize my past self. It makes me feel like I’m starting <em>now</em>, so late into the game! I’ve wasted the earlier years of my life on all kinds of dumb and bad things.</p>
<p>I constantly feel that I am running out of time, that I have not done enough, that I will be a failure. That my work will not be meaningful enough to be remembered, or worse, will be remembered as bad. When I express this kind of feeling, people often point this out:</p>
<p><img src="https://svbtleusercontent.com/yteofannex78va_small.png" alt="https://svbtleusercontent.com/yteofannex78va_small.png" /></p>
<p>This doesn’t make me feel better, it makes me feel worse. There are so many things that I want to do that I’m <em>not</em> working on. That hypermedia book still isn’t done. I have been a bad maintainer of Resque and all my other Ruby stuff. I didn’t listen to my friends in Greece, who told me to relax, and then the exact situation they predicted happened. I just hope I’m around long enough to make an impact on people and be a positive change in the world. We’ll see if I make it.</p>
<p>There’s a whole ’nother rant here about Camus and Heidegger and absurd-ism and being-towards-death too. I will save that sophomoric rant for my “I’m turning 40” post.</p>
<h2>Youth Cults</h2>
<p>The second thing that’s making me feel terrible about turning 30 is the goddamn computer industry. It’s extra worse because historically, I have benefited from this particular terrible attitude. And that’s the worship of youth.</p>
<p>I used to teach new programmers, and often give talks to classrooms full of them. They always want to know how I got started, and I always preface it with something like</p>
<blockquote>
<p>My story is one I don’t like to share, because I’m the living embodiment of a stereotype that I think is harmful to new programmers.</p>
</blockquote>
<p>I started with computers when I was seven. That shouldn’t matter. I’ve learned more about software in the last five years than I did from seven till I was 18. But we hold up these young genius whiz-kids (and let’s be serious, it’s not ‘kids’, it’s ‘boys’) as the epitome of what our industry can do. The heroic youth who understands computers better than those silly old people. What do they know, anyway?</p>
<p>Age discrimination is real. It’s stupid, but it’s real. And even when you know that it’s stupid, it can still slip into your brain in a really terrible way. If I’m 30 and this is all I’ve done by now, well, why even bother?</p>
<p>This is dumb. The attitude is dumb. That I disagree with it, yet still feel it at times, is dumb. Ideology is a motherfucker.</p>
<p>So yeah, that’s it. My 20s are almost over. And I wasted some of the last bits of them writing this silly blog post. There’s no stopping it, just watching the clock tick forward, and doing my best as hard as I can until it’s all over. How many of these decades will I even get? My dad certainly thought he had a few more until that night he collapsed and found out that he had stage four, terminal cancer.</p>
<p>Let’s see what I can do in the next ten years.</p></content:encoded>
</item>
<item>
<title>Docember</title>
<link>https://steveklabnik.com/writing/docember/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/docember/</guid>
<pubDate>Tue, 01 Dec 2015 00:00:00 GMT</pubDate>
<content:encoded><p>There are few things I appreciate more than using software projects that have good documentation. When I’m new to the project, documentation is invaluable for helping me get going. When I’m not new, good documentation helps me get the information I need to :shipit:.</p>
<blockquote>
<p>We say programming is hard, and we hate bad code. So we practice writing code.</p>
</blockquote>
<p>We say writing docs is hard, and hate bad docs…</p>
<p>— Some(@steveklabnik) (@steveklabnik) November 30, 2015</p>
<blockquote></blockquote>
<p>About a month ago, I was tweeting about this topic, and <a href="https://mayeu.me/misc/2015/11/13/docember-one-month-to-improve-your-documentation.html">several of us came up with an idea</a>: similar to how November is <a href="http://nanowrimo.org/">National Novel Writing Month</a>, why not have a similar month for programming?</p>
<p>Here’s the idea: in December, let’s try to work on the documentation for the projects we work on. Make it 50k words like NaNoWriMo, make it <a href="http://24pullrequests.com/">24 pull requests</a>, whatever floats your boat. Your future self will love you for it.</p></content:encoded>
</item>
<item>
<title>Rescuing Resque (again)</title>
<link>https://steveklabnik.com/writing/rescuing-resque-again/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rescuing-resque-again/</guid>
<pubDate>Wed, 28 Oct 2015 00:00:00 GMT</pubDate>
<content:encoded><p>A long time ago, there was a big open source project, which was a huge part of the Rails world, but had almost no maintenance. This project was Resque, the go-to way to write background jobs for your Rails application. I attended a talk where the current maintainer was clearly getting burned out, and so, <a href="http://blog.steveklabnik.com/posts/2012-09-22-resque--let-s-do-this">offered to help</a>. The problem is, things have changed.</p>
<p>It wasn’t just me. It was a bunch of people. We all used Resque, and loved it, and wanted to make it better. We made plans for a big 2.0 to pay off a bunch of technical debt, simplify the codebase, and add more features. A ton of people chipped in, I had some awesome remote pairing sessions. Things were good.</p>
<p>But slowly, we all started to drift away. One contributor changed jobs. Another decided to spend more time with family. I got discouraged, and then, there was a strange situation in which I accidentally released an entirely straight-up broken version. So I got scared, released a fix, and basically never touched Resque again.</p>
<p>Resque is used by a <em>lot</em> of people:</p>
<p><img src="https://svbtleusercontent.com/rsoq8peusw08q_small.png" alt="https://svbtleusercontent.com/rsoq8peusw08q_small.png" /></p>
<p>Yet, it hasn’t seen a release in 18 months. The PRs are piling up. 2.0 is basically dead in the water. I’m not much of a fan of 2.0s anymore myself, at least, ones that don’t follow the “stability without stagnation” principles. I don’t use Resque myself anymore, and barely have enough time for my work, let alone my personal projects, let alone a project that I care about but only work on out of the goodness of my heart.</p>
<p>I’m here to tell you that that’s changing a little bit. I’ve been chatting with <a href="https://github.com/hoffmanc/">Chris Hoffman</a>, who’s interested in starting up on work again. I asked him what his intentions were, and what he answered with was exactly what I wanted to hear, as it turns out. So, I’m going to be doing some work again, this time, with the explicit intention of handing off maintenance to Chris and anyone else who wants to chip in. If you use Resque commercially, this is a great time to give some of your people the opportunity to give back. If you’ve never worked on an open source project before, but want to, I’d be more than happy to help mentor you with this. Resque is a big, real project, with a lot of impact. You can do a lot of good. That’s why I started working on it.</p>
<p>Without further ado, here is The Plan, from Chris and I:</p>
<h2>TL;DR</h2>
<ul>
<li>change <code>master</code> from unreleased 2.0 back to <code>1-x-stable</code></li>
<li>clean up tests - make the development experience enjoyable</li>
<li>integrate pending Pull Requests</li>
<li>2.0? Never heard of it. At least, not for a while.</li>
</ul>
<h2>Make <code>1-x-stable</code> master again</h2>
<p><a href="https://github.com/resque/resque/issues/1175">Many</a> <a href="https://github.com/resque/resque/issues/976">have</a> stated that the README on the master branch has led them astray, because the published gem is still at 1.25.2.</p>
<p>We will plan on making this switch in the next couple of weeks. If you feel this is the wrong move, please let us know. But since active development will in fact happen on 1.x, this feels like the correct thing.</p>
<h2>Fix up the test suite</h2>
<p>Tests need to be fast, and clean, so that there is more incentive to contribute.</p>
<p>One possible approach is using one of the available Redis mocking libraries (mock_redis, fakeredis) in non-forking tests, which could speed suite execution up considerably.</p>
<h2>Review Pending <code>1-x-stable</code> PRs</h2>
<p>This is the lion’s share of the work, as there are currently 56 (that’s <em><a href="https://github.com/resque/resque/pulls">fifty six</a></em>) open pull requests…</p>
<p><em>And that’s an awesome problem!</em> Let’s make use of them! These PRs need to be evaluated and closed, so that the project can benefit from the community.</p>
<h2>Decide What to do with 2.0</h2>
<p>The focus of Resque should be on stability, with an eye towards legacy support. 2.0 is off the table, at least in the near term.</p>
<p>If there is a 2.0, it will be a “garbage collection”-style release, where only deprecated functionality is removed, and the upgrade path is clear.</p></content:encoded>
</item>
<item>
<title>"The Rust Programming Language" will be published by No Starch Press</title>
<link>https://steveklabnik.com/writing/the-rust-programming-language-will-be-published-by-no-starch-press/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-rust-programming-language-will-be-published-by-no-starch-press/</guid>
<pubDate>Tue, 08 Sep 2015 00:00:00 GMT</pubDate>
<content:encoded><p>I’m happy to bring you some news: <a href="http://doc.rust-lang.org/stable/book/">“The Rust Programming Language”</a> is going to be published, in physical form, by <a href="https://www.nostarch.com/">No Starch Press</a>. I am really excited to work with No Starch on this: they’ve published some of my favorite programming books, like <a href="https://www.nostarch.com/hemingway">“If Hemingway Wrote JavaScript”</a>, <a href="https://www.nostarch.com/rum">“Ruby Under a Microscope”</a>, and <a href="https://www.nostarch.com/rubywizardry">“Ruby Wizardry”</a>.</p>
<p>Another reason I’m keen to work with No Starch on this is that they believe in publishing freely available material: we’re going to take the existing manuscript, whip it into shape, make a nice paper book out of it, and merge the changes back into the official documentation. So the improvements will get back to everyone, even those who can’t afford a physical copy.</p>
<p>The authorship credit is to “The Rust Project Developers,” as other people have contributed significantly: we even have an <a href="https://github.com/rust-lang/rust/pull/28301">active PR to re-write an entire chapter</a>. Rust is a team effort.</p>
<p>Oh, and one last thing: the profit from the book won’t go to me, it will go to <a href="http://openhatch.org/">OpenHatch</a>, “a non-profit dedicated to matching prospective free software contributors with communities, tools, and education,” to use their words about it.</p>
<p>I’ll let you know when we’re closer to actually shipping!</p></content:encoded>
</item>
<item>
<title>Community Versions for Rust</title>
<link>https://steveklabnik.com/writing/community-versions-for-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/community-versions-for-rust/</guid>
<pubDate>Sun, 23 Aug 2015 00:00:00 GMT</pubDate>
<content:encoded><p><a href="http://www.rust-lang.org/">Rust</a> has been through a <em>lot</em> of changes in recent years. As such, tutorials get out of date. We’ve occasionally said that including <code>rustc -v</code> in your blog posts would be helpful, but today, I saw this tweet go by:</p>
<blockquote>
<p>Tired of guessing what versions of Ember work with a given blog post? Check out http://t.co/6joJiNdJQ5 #feedbackplz pic.twitter.com/YToBduBVtx
— mixonic (@mixonic) August 23, 2015</p>
</blockquote>
<p>After some quick hacking, I’ve modified it to work with Rust. You can find the project here: <a href="https://github.com/steveklabnik/rust-community-versions">https://github.com/steveklabnik/rust-community-versions</a></p>
<p>TL;DR: Put this into your blog post, substituting the appropriate date and title:</p>
<pre><code>&lt;iframe
width="178" height="24" style="border:0px"
src="http://steveklabnik.github.io/rust-community-versions/2013/10/18/pointers-in-rust-a-guide.html"&gt;
&lt;/iframe&gt;
</code></pre>
<p>Then, make a <a href="https://github.com/steveklabnik/rust-community-versions/pulls">pull request</a> adding a new post with some front-matter:</p>
<pre><code>---
layout: post
url: http://words.steveklabnik.com/pointers-in-rust-a-guide
title: "Pointers in Rust: a Guide"
date: 2013-10-18
start_version: 0.8
end_version: 0.9
---
</code></pre>
<p>(<code>end_version</code> is optional)</p>
<p>And bam! You get a nice looking badge. Check out out on this blog post of mine: http://words.steveklabnik.com/pointers-in-rust-a-guide</p>
<p>What’s nice about this approach is that, if your post ever gets out of date, someone else can modify the YAML in the repo, and your badge will automatically update to include the end date. No remembering to go back and fix it.</p>
<p>Let’s see how this goes!</p></content:encoded>
</item>
<item>
<title>Ember Data: ‘DS’ is not defined</title>
<link>https://steveklabnik.com/writing/ember-data--ds-is-not-defined/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/ember-data--ds-is-not-defined/</guid>
<pubDate>Sun, 05 Jul 2015 00:00:00 GMT</pubDate>
<content:encoded><p>I’m doing a little Ember app, and if there’s one thing I’ve learned from writing software, it’s to blog about error messages. Two-years-later me has ended up finding my own posts when searching for help!</p>
<p>So today, when getting started with Ember Data 1.13, I was trying to use the new <code>JSONAPIAdapter</code>. I saw this code snippet:</p>
<pre><code>App.ApplicationAdapter = DS.JSONAPIAdapter.extend({
namespace: 'v1',
});
</code></pre>
<p>Using that gave me an error when <code>ember serve</code>-ing, though:</p>
<pre><code>app.js: line 16, col 26, ‘DS’ is not defined.
</code></pre>
<p>Turns out, <code>DS</code> isn’t in scope by default, even though <code>ember-cli</code> installs Ember Data by default.</p>
<p>Fixing this just means importing it at the top of <code>app/app.js</code>:</p>
<pre><code>import DS from 'ember-data';
</code></pre>
<p>Easy enough!</p></content:encoded>
</item>
<item>
<title>The language strangeness budget</title>
<link>https://steveklabnik.com/writing/the-language-strangeness-budget/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-language-strangeness-budget/</guid>
<pubDate>Fri, 26 Jun 2015 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve always loved programming languages. I’ve spent plenty of times with many of them, and am fortunate enough that <a href="http://blog.rust-lang.org/2014/12/12/Core-Team.html">language design is now part of my job</a>. In discussions about building Rust, I’ve noticed myself making a particular kind of argument often in design discussions. I like to call it ‘the language strangeness budget’.</p>
<p>When building anything, it’s important to understand why you are building it, and who you are building it for. When it comes to programming languages, <a href="https://github.com/steveklabnik/mojikun">building one is easy</a>, but getting people to use it is much, much harder. If your aim is to build a practical programming language with a large community, you need to be aware of how many new, interesting, exciting things that your language is doing, and carefully consider the number of such features you include.</p>
<p>Learning a language takes time and effort. <a href="http://doc.rust-lang.org/stable/book/">The Rust Programming Language</a>, rendered as a PDF, is about 250 pages at the moment. And at the moment, it really covers the basics, but doesn’t get into many intermediate/advanced topics. A potential user of Rust needs to literally read a book to learn how to use it. As such, it’s important to be considerate of how many things in your language will be strange for your target audience, because if you put too many strange things in, they won’t give it a try.</p>
<p>You can see us avoiding blowing the budget in Rust with many of our syntactic choices. We chose to stick with curly braces, for example, because one of our major target audiences, systems programmers, is currently using a curly brace language. Instead, we spend this strangeness budget on our major, core feature: ownership and borrowing.</p>
<p>On the other hand, if you’re trying to accomplish something different with your language, purposefully blowing this budget can be useful too. I really enjoy Haskell, and for a long time, their slogan has been ‘<a href="http://www.computerworld.com.au/article/261007/a-z_programming_languages_haskell/">avoid success at all costs</a>’:</p>
<blockquote>
<p>As for the second, I don’t know if you know this, but Haskell has a sort of unofficial slogan: avoid success at all costs. I think I mentioned this at a talk I gave about Haskell a few years back and it’s become sort of a little saying. When you become too well known, or too widely used and too successful (and certainly being adopted by Microsoft means such a thing), suddenly you can’t change anything anymore. You get caught and spend ages talking about things that have nothing to do with the research side of things.I’m primarily a programming language researcher, so the fact that Haskell has up to now been used for just university types has been ideal</p>
</blockquote>
<p>This is Simon Peyton-Jones knowing exactly what kind of language he wants to build, and in what way. Haskell’s embrace of strangeness in this sense is what makes Haskell Haskell, and actually furthers its goals.</p>
<p>If you include <em>no</em> new features, then there’s no incentive to use your language. If you include too many, not enough people may be willing to invest the time to give it a try. Language designers should give careful thought to how strange their language is, and choose the right amount to accomplish what they’re trying to accomplish.</p></content:encoded>
</item>
<item>
<title>Ouroboros</title>
<link>https://steveklabnik.com/writing/ouroboros/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/ouroboros/</guid>
<pubDate>Mon, 18 May 2015 00:00:00 GMT</pubDate>
<content:encoded><p>One of the things that I love about open source is its ability to send improvements across projects. Sometimes, an improvement in one project ends up improving its upstream. This kind of network effect really gets to the core of it for me: steady, constant improvement.</p>
<p>I saw an example of this today that makes me smile. Before <a href="http://blog.rust-lang.org/2015/05/15/Rust-1.0.html">Rust’s 1.0 release</a>, I wrote a <a href="http://doc.rust-lang.org/stable/book/dining-philosophers.html">chapter about the Dining Philosopher’s problem</a>. Incidentally, I’m really happy to use classic CS concepts in the Rust docs, hopefully exposing them to people who didn’t get the opportunity to get a degree in computer science. Anyway, towards that end, I decided to cite <a href="http://www.usingcsp.com/cspbook.pdf">Tony Hoare’s classic CSP paper</a>. Other funny aspect of that: I started my implementation using channels, but ended up using mutexes instead.</p>
<p>Anyway, today, a new contributor <a href="https://github.com/rust-lang/rust/pull/25575">spotted a typo</a>. The PR message made me smile:</p>
<blockquote>
<p>Looks like this was an issue in the source material. I’ve let the editor know and he has told me he is correcting it there as well.</p>
</blockquote>
<p>In my own small way, a thing I did led to someone else improving one of the most classic and well-known CS papers ever written. And it’s just Monday.</p></content:encoded>
</item>
<item>
<title>Hope</title>
<link>https://steveklabnik.com/writing/hope/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/hope/</guid>
<pubDate>Fri, 08 May 2015 00:00:00 GMT</pubDate>
<content:encoded><p>“The people are unstoppable / Another world is possible”</p>
<p>This has long been my favorite protest chant. For me, it gets to the heart of the matter: another world <em>is</em> possible. If I’m honest with myself, most days, I don’t really believe that the world will get better. We often present history as a kind of positive progression: the world has only been getting better, more tolerant, and less poor as the years go by. I don’t <em>feel</em> that, though. What I see is imperialism, consolidation of power, and rampant racism/sexism/etc. Liberalism presents itself as an end of history, the final stop on the line. That’s unacceptable to me, but I usually feel like the only one.</p>
<p>The start of this post sounds depressing, but I’m only writing it because I actually feel positive, invigorated. I’m writing this post from the Barcelona airport, about to board the plane home. It took me a while to figure this out, but what I realized is this: getting out of the US every once in a while really helps me feel better. This isn’t to say that things are actually <em>better</em> in other places, that’s not my point. My point is that It’s not something like “European-style parlamentary democracy is better than a bicameral legislature” or something like that, just look at the UK election results. It’s that the street signs are different. The money isn’t the same. Familiar foods have changes. I see 漢字 instead of the Latin alphabet. Whatever is unusual about whatever part of the world I am in, it’s the smallest differences that have the largest impact on me and my state of mind.</p>
<p>When I go somewhere else, I can <em>feel</em> the difference. And difference is possibility. And possibility is hope, for a better world, for a better humanity.</p></content:encoded>
</item>
<item>
<title>An introduction to economics under capitalism</title>
<link>https://steveklabnik.com/writing/an-introduction-to-economics-under-capitalism/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/an-introduction-to-economics-under-capitalism/</guid>
<pubDate>Tue, 10 Mar 2015 00:00:00 GMT</pubDate>
<content:encoded><p>The dismal science a difficult one. Many people have strong opinions about economics, and it determines many of their other political beliefs. As someone who has significantly different thoughts on this topic than many of my peers, I’m writing this post out to give a basic explanation as I see it. That doesn’t mean I’m <em>inventing</em> something here, just explaining the model that I currently prefer. I’ll give sources at the end, if you want to dig deeper. Speaking of ‘deeper,’ this is also an <em>introduction</em>, one could easily write a book or three on this topic. I’ll try to be clear where I’m making an assumption, and why I’m making it.</p>
<p>Finally, this is very much a draft, I may make small, or even maybe major errors. Nobody’s perfect. My spelling is terrible, and there’s a bug where my browser spellcheck isn’t working… anyway.</p>
<hr />
<p>Let’s start from the beginning. And that beginning starts with stuff. People trade things between them. Let’s be a bit more precise: people trade commodities. They trade commodities becuase they want something they do not currently have. In other words, if I’m hungry, and you have some $FOOD, and you’re cold, and see I have a coat, you might trade me your food for my coat and we’d both be better off. This ‘commodity’ is distinct from any useful thing because we trade for it, and because it satisfies some sort of need.</p>
<p>But how do we know how much food we should trade for a coat? What does that even mean? To make this trade equal, we need to settle on a value for food, and a value for coats. That value has two different parts to it. The first: that value represents how much something satisfies our needs. Something that’s really useful will have a lot of value to us. Something that’s not may have less value. We call this component of value ‘use value.’</p>
<p>The second component is more complicated. It’s called ‘exchange value,’ and it’s called that becuase this is the actual value that we trade each other for. These two kinds of value are distinct, but the distinction is important. While I may find a coat midly useful, and therefore, its use value is low to me, that doesn’t mean that if I trade it to someone else, they will agree in how useful it is. We might have to sell this coat for more or less than we expected to. While the usefulness affects the exchange value in a sense, it doesn’t mandate it.</p>
<p>Let’s give a more concrete example of this. If I can trade my single coat for four $UNITs of $FOOD, then the exchange value of one coat is four $FOODs, and the exchange value of a $FOOD is one quarter of a coat.</p>
<p>So if the exchange value of a commodity determines how we trade, and that’s based on a subjective preference for usefulness. So if I told you there was some underlying constant that did determine exchange value, you probably wouldn’t believe me. This is, in fact, one of the root areas in which economists disagree with me. Their first mistake. They claim that this exchange is entirely based on preference, with no underlying reasoning. I (and others) disagree. Consider the equation,</p>
<pre><code>4 * c = 1 * f
</code></pre>
<p>where <code>c</code> is coats and <code>f</code> is food. If we wanted to reduce these equations, we could end up with two variants:</p>
<pre><code>(4 * c) / 1 = f
c = (1 * f) / 4
</code></pre>
<p>Because <code>c</code> and <code>f</code> are defined in terms of each other, it’s hard to get at an underlying property of both things.</p>
<p>Now, remember why we’re trading food for coats: they satisfy some sort of need. Therefore, these equations make sense only in a certian scope, if you will: when we’re considering how to turn some sort of commodity into the satisfaction of our need. This is what produces this relationship between food and coats. But we have a layer of abstraction here: as we said, the exchange value isn’t the actual, underlying use values. Like all abstractions, this one leaks. We’ll get to the leak later.</p>
<p>But in order to turn some food into a satisfied need, you need to do some kind of labor. First of all, this food had to actually be produced by someone. We’re not ordering raw materials, we’re ordering a finished product. Someone, somewhere, had to obtain these raw materials, and then apply themselves. And if you’re buying raw materials, someone had to do the labor to cut down the trees, or mine the ore.</p>
<p>This is an area where I’m going to make a slight hand-wave for now, and argue that this aspect, the labor that’s needed to make and consume a commodity, is the underlying aspect missing from the equation. It might be a little bit of labor, it might be a lot of labor, but somewhere, someone is doing some kind of labor. We can make a better argument here, but for now, I’m just going to leave it as is, because I think it makes intutive sense. Many, many people go to work, and labor to produce things that get exchanged for many other things. The common thread is that we all work. We can dig into a more rigorous argument over this point later.</p>
<p>In other words, we’re trading an abstraction over people’s labor. When I say “I’ll trade you a coat for four food,” I’m making that estimate on how much labor I would need to aquire a coat, and estimating how much to get four food, and deeming them equal. But we’re not trading the hours directly: we’re trading objects. We objectify our labor, and then trade that.</p>
<p>Now, you might have a problem with this. And many economists do. However, their arguments are usually based on a misunderstanding of this point, which is a very important one: If value is based on how many hours of labor someone puts into something, than isn’t a slow worker making more valuable stuff than a fast worker? How does that make any sense? And if I were to do some labor that’s useless, like for example, I dig around in my backyard and make a bunch of pies out of the mud, I did some labor, and therefore, shouldn’t I get paid?</p>
<p>Early economists such as Adam Smith often had this problem with their models. And it’s indeed a problem. But there’s a solution: we’re not talking about just any old labor. We’re only talking about labor that produces something useful. “Useful” in this case is decided by the particular society in which this person is doing this labor. A carriage maker in 1890 does a lot of productive labor, but a carriage maker in 2015 does significantly less.</p>
<p>Our revised statement: the common thread between all of the commodities that get exchanged is that somewhere, sometime, someone had to do some sort of labor that was socially useful to produce that commodity. We exchange commodities so that we can gain the commodities we need to fulfill our desires. We exchange them at a ratio where they’re equal in terms of how much labor went into making them, a fair trade.</p>
<p>This leads to another problem: how do we qualify ‘how much labor’? What is this unit and magnitude? Well, many people have a 40 hour week, so naturally, we gravitate towards ‘hours’ as an answer for this question. Hours are a unit of time, so we can quantify our exchange values as ‘socially neccesary labor time.’ Time, becuse we work in hours. Labor, because that’s what we’re doing. “Socially neccesary” to explain the context of only labor that’s doing something society deems useful.</p>
<p>But back to one of the original problems: If I’m bad at making coats, and I take a hundred hours to make a coat, and you’re good at making coats, so it takes you an hour, is my coat worth one hundred times yours? That doesn’t make any sense. So we’re going to make an assumption. When we use ‘socially neccesary labor time’ in this way, we mean that it’s an average hour, by an average skilled worker, across all of society. Because averages are all you can really reason about here, it’s impossible to compare each individual worker. It’s an approximation. And understanding that it’s just an approximation has led to many successful businesses, as they’re more efficient than the average, and therefore, make more money. For now, we’re going to set that aside, however. We’ll get there.</p>
<p>So! To recap, we have needs and desires, and to fulfill those, we need certain commodities, which we can aquire through exchanging commodities we have with others in our society who have the commodities we want. We make equal trades, in a ratio roughly determined by the average amount of time it would take an average worker in our society to make the thing.</p>
<hr />
<p>Exchange values are a bit unweildy, though. For one thing, there’s a ton of them: they explode in a <code>O(n^2)</code> fashion. If we have coats and food, each commodity has a single exchange value, but if we add just one more item, each commodity can be traded for two others, and therefore has two exchange values, times the three commoddities, meaning our system has six exchange values, five more than before. Adding a fourth means each of our four commodities has three possible trades, meaning twelve exchange values, six more than before. So what’s a society that’s trying to produce use-value to do?</p>
<p>We can add more things to our equation. We have coats and food in our economy, let’s add a third, and call it ‘coins.’ We’ve established that one coat is worth four food, and let’s say that one coin is worth two food. This means that four food is worth two coins is worth one coat:</p>
<pre><code>4 * food = 2 * coins = 1 * coat
</code></pre>
<p>In another sense, one coat is worth two coins is worth four food:</p>
<pre><code>1 * coat = 2 * coins = 4 * food
</code></pre>
<p>We have coins in the middle of both of these equations, which means that we can express the two halves in one system:</p>
<pre><code>1 * coat = 2 * coins
4 * food = 2 * coins
</code></pre>
<p>And when we add another commodity to our equation, we can just say what it’s worth in coins, rather than what it’s worth in both coats and food. This makes it significantly easier to conduct trade, as we only have to know what our commodities are worth in terms of this particular commodity. This commodity plays a special role in our little society: it’s the commodity that we use to determine the rates of all the other commodities. The vocabulary term for this is ‘the money commodity.’ And we call the exchange value of a commodity to the money commodity a ‘price.’</p>
<p>At this point, we’re going to take a small detour. Because our commodity is an abstraction of socially neccsary abstract labor time, I want to talk for a moment about how that abstraction leaks. Becuase objects fulfill our needs, and trading them is such an important part of society, we place a ton of emphasis on this abstraction. In other words, we love to buy things. Why does this happen? What’s the weird obsession, or some might say, fetish, with commodities?</p>
<p>Let’s say you’re a cobbler, and you make shoes. Hopefully, you’re proud of the shoes you make, and so, to some degree, because you’ve put hard labor into those shoes, you see them as a reflection of your own self. And when every commodity is priced equivalant to the money commodity, it’s really easy to see that your shoes trade at $50, but mine may trade at $45. This means you make better shoes, and so you have a higher social standing than I do. Your labor is more socially useful. This sort of collective desire we all have that sets the price of commodities, and can make us quite obsessed. And not just in the classical materialist way, but in a way that shapes our worldview.</p>
<p>You can’t take a microscope to a coat and observe that the coat costs $2. There’s no ‘money atom.’ It’s not some inherent property of coats. It’s something that our society has created as a basic assumption. But it doesn’t have to be that way, we’ve just chosen it.</p>
<hr />
<p>Let’s talk a bit more about exchanging stuff. Let’s use the previous exchange as an example, I have a coat, you have four food. But now, we don’t know each other. What do we do? Well, I can take my coat to a market, and sell it for two money. You can take your food to market, and sell it for two money. I can then buy your food for two money, and you can buy my coat for two money. We’ve exchanged our food and coats succesfully. We both completed a trade in the same way:</p>
<pre><code>commodity -&gt; money -&gt; commodity
</code></pre>
<p>or</p>
<pre><code>C -&gt; M -&gt; C'
</code></pre>
<p>We both had a commodity, we traded it for the money commodity, and then we traded that money commodity for a different commodity. There’s two sides to these trades: a commodity for the money commodity, and then money commodity for the commodity.</p>
<p>It’s important we’re making an assumption here, but it’s one that virtually all economists make: we’re assuming that both of us know the value, and that the intermediary who is holding the money agrees. This simplifying assumption is also a place where many business opportunities are formed, charging a fee to facilitate transactions like this. See the pattern yet?</p>
<p>That’s also something new: there’s an intermediate here, ‘the market.’ We don’t have to know each other, we just have to know the market exists. Software engineers would say that by lowering coupling, we enable scale. When everyone doesn’t need to know each other, we can trade a lot more things. This is another reason why the money commodity is so damn useful.</p>
<p>Anyway, it’s important to recognize that this is four total exchanges, which form two little circuts, which form our overall economy here. It’s a system, like any other. And so we can model it like a system. This is something that we virtually all agree upon. But this assumption is treacherous too: we assume that both ends of the trade always happen. But there are four trades here, and only some of them may occur. This causes disruption in the system, which throws it into crisis. We’ll talk about this case more later.</p>
<p>A moment ago, I mentioned a business model for charging for transactions. This is a very natural reaction to systems like these: by intervening in each trade, by being the mediator, you can argue that you are doing socially neccesary work, and therefore, should be paid in compensation.</p>
<p>Furthermore, since the money commodity mediates all exchange, there’s lots of good reasons to control said mediation. This is why states print currency and then ensure that transactions occur in said currency, as it gives them control over all exchange, and therefore, of what is ‘socially neccesary.’</p>
<p>To summarize up to this point: people have needs. To satisfy those needs, some people do labor to make commodities, which have the ability to satisfy those needs. We call one of these commodities ‘money’, and people trade commodities for it, and it for other commodities. This gets them the things they want. The rate that all of this is exchanged at is based in how much time it takes to make, and how important it is to the society overall.</p>
<p>I have yet to say what any of this has to do with <em>capitalism</em>, however. This is just people trading things. Money, and exchange, have been around longer than capitalism. It’s something a bit different. Before we can get into that, though, we have to define the ‘capital’ in ‘capitalism.’</p>
<hr />
<p>There is a distinction between ‘money’ and ‘capital.’ Capital is a special kind of money. In other words, it holds a special, social distinction about how it’s used. Let’s examine our little trade again:</p>
<pre><code>commodity -&gt; money -&gt; commodity
</code></pre>
<p>Someone has to actually have that money in the middle. To simplify our story, let’s say this shop both buys and sells various commodities. I sell my coat to the shopkeeper, you sell your food to the shopkeeper, then I buy food from the shopkeeper, and you buy a coat from the shopkeeper. From the shopkeeper’s perspective, these trades are backwards:</p>
<pre><code>money -&gt; commodity -&gt; money
</code></pre>
<p>The keeper has four dollars. They give two to me in exchange for the coat, and then they sell the coat for two dollars on the other side. Their circut here is the opposite of yours and mine: they start and end with money, rather than with another kind of commodity.</p>
<p>It’s important to note that this difference is significant. While you are attempting to exchange a commodity for another to gain some kind of use, with the <code>M -&gt; C -&gt; M</code> variant, you’re attempting to gain more exchange value. This underlying need is at odds with yours and mine. No concern is given for use here, just how much something can be exchanged for.</p>
<p>However, as we know, a shopkeeper isn’t exactly happy when they start off with four dollars and end up with four dollars. While this system is balanced, they’re not being compensated for this social function, one of shopkeeper. And in the real world, shopkeers do make money, so something is missing from our model of equal exchanges. But before we get into that, consider the implications of this point: the shopkeeper has no concern for what is useful, just for what can be exchanged. If we optimize for the needs of shopkeepers, we may not be optimizing for the usefullness of society, but instead for the pursuit of exchanging the money commodity.</p>
<p>So, if we can engineer a situation in which we have some money, we exchange it for a commodity, and then we exchange that commodity for money, and we get <em>more</em> money out of it in the end, that extra money is called ‘capital’. In line with our earlier equations:</p>
<pre><code>M -&gt; C -&gt; M'
</code></pre>
<p>The difference between <code>M'</code> and <code>M</code> is how much money we’re making. This is very close to our <code>M -&gt; C -&gt; M</code> circut, and very different than our <code>C -&gt; M -&gt; C</code> circut. Only shopkeepers produce capital, not those who are simply exchanging a commodity in the hopes of attaining another.</p>
<p>But we’ve already established that all trades here are equal. So how can we get <code>M'</code>? Shoudn’t that be impossible?</p>
<p>This is where the mistake we mentioned before rears its ugly head. If all value is subjective, and exchanges are between equals, where can this extra money come from? And if all of these exchanges are equal, then why do both people feel like they’ve won?</p>
<hr />
<p>In order to make this happen, our prospective capitalist shopkeeper needs to find a commodity whose use-value, the act of consuming the commodity, produces value. If they can, they can purchase that commodity, and then use it to get more value than it was exchanged for, due to the extra production. In other words, they don’t need to purchase an object, but something more abstract: a potential source of value.</p>
<p>Previously, we mentioned that the common thread behind exchanges was labor, and so, that labor was the common element of value. Labor, when placed on a market, is called ‘labor power,’ and is a commodity that produces value. We call someone who possesses labor power a worker. They enter into the market, selling their labor power. This is colloquially known as ‘getting a job.’ In order to be willing to sell your labor on the market, though, you need to have some sort of need that is fulfilled. If you don’t need anything, there’s no reason to get a job. Furthermore, you can’t just do work yourself, you need objects to do your work, that will allow you to produce something. We call these objects the ‘means of production.’ A worker also needs enough commodities to keep themselves alive, to work another day. We call this the ‘means of subsistence.’ It’s also important to note that if a worker has other commodities, they could sell those to fulfill their needs, rather than selling their labor power. So someone selling their labor-power in the market needs something, but doesn’t have anything. The only thing they have is the labor-power their body possesses.</p>
<p>It’s important to note that “person with a pile of money” and “person possessing nothing but their ability to work” is not the state of nature. In other words, this stuation arises due to the way we’ve organized society. We like to think that this existance is somehow ‘natural’ or inevitable, but that’s not true. This doesn’t mean older forms of organization are better than what we have now, but the possiblity for change exists. Some people try to paint a picture that those who have the piles of money are virtuous, and those who have only their labor are at fault for their predicament. But the predicament wouldn’t even exist in the first place if not for certian historical developments.</p>
<p>So, if you’re trading labor-power in a market, how do you determine how much to sell it for? Like any other commodity, its value comes down to a socially neccesary labor time. In this case, it’s the amount of money you need for subsistence, the money to pay your rent, to eat food, to stay alive. Having some more is nice, but if you don’t go to work, you can’t pay your bills, and then bad things happen.</p>
<p>One odd thing about labor-power as a commodity is that you don’t just hand it over in the way that you hand over a coat. You put the work in first, and then, eventually, the thing you made gets sold at a totally different time. There’s a gap. Often, you don’t actually get to see the fruits of your labor, as someone else is actually doing the sales. This can lead to some angst, but I just want to make mention of this, we’ll talk more about it later.</p>
<hr />
<p>This commodity, labor-power, allows us to create the trick. Here’s how the story goes:</p>
<p>Let’s change our example slightly: a coat costs two coins, and four spindles of thread costs two coins:</p>
<pre><code>1 * coat = 2 * coin
4 * thread = 2 * coin
</code></pre>
<p>It takes two spindles of thread and an hour to make a coat. By this measure, given that a coat costs two coins, you’d imagine that cost of materials is a coin, so labor must be a coin. Which means an hour of work is worth a coin. if this is true, then the exchange looks like this:</p>
<p>Someone wants a coat. They’re willing to pay two coins. A prospective capitalist sees that the cost of materials is a coin, so they set the price of labor-power to one hour per coin:</p>
<pre><code>1 * coat = 2 * coin
2 * thread = 1 * coin
1 * hour of work = 1 * coin
</code></pre>
<p>I spend my one hour of labor and two threads to make a coat, which costs two coins and is sold for two coins. We’re back at an equilibrium. There’s still no capital being created here.</p>
<p>Now the capitalist is upset. “I provided the job, I provided the thread, yet I made no money. This worker couldn’t have made that coat without me. I deserve to be paid for having the initial capital to set up this business. That’s work!”</p>
<p>But here’s the thing: if I worked for half a coin instead of a full coin for an hour, and I worked eight of those hours a day, I’m still making more than enough money to pay my bills. Because that’s the outcome I’m shooting for, I’m happy to exchange my labor on the market for less than the labor time would be: I’m making a good salary, I can live a good life. So the <em>exchange value</em> of my labor-power is less than the value I create from it. It’s not that I’m being cheated with my wages, it’s just that the exchange value of my labor is worth less than the labor I create. With this set of equations:</p>
<pre><code>1 * coat = 2 * coin
2 * thread = 1 * coin
2 * hour of work = 1 * coin
</code></pre>
<p>the capitalist can extract that difference. He takes his initial four coins, invests two of them in four threads. He then invests a coin in two hours of a worker’s labor. That worker then uses the threads and their labor-power to make two coats. The capitalist then sells the two coats for four dollars. Looking back, he’s seemingly played a trick: he invested three coins, two in threads and one in labor, but gained four coins at the end, for the two coats.</p>
<p>We haven’t broken any rules of exchange here. The capitalist has paid full market price for all commodities, there’s no inherent need to pay less. Each exchange is between understanding, consenting parties. There’s no trickery. Yet an extra coin has moved from one party to another, the worker to the capitalist, just by following the rules.</p>
<p>Once you have a machine that you can put three dollars in and get four dollars out, you can live off of this extra, rather than doing labor yourself. By having the initial capital of three dollars, and by using it in this way, a capitalist can just make money out of thin air. As they say, it takes money to make money. This is why every worker isn’t a capitalist, you need to create this capital somehow, to be able to afford to use the machine. A machine that makes four dollars out of three isn’t much use if you dont have the inital three dollars.</p>
<hr />
<p>In summary: people come together at a market to exchange commodities to fulfill their needs. They exchange these commodities with each other in equal values. One of the commodities which can be exchanged is labor-power, which is a special commodity, because it creates more value. Capitalists purchase this commodity, and then use it to create capital, which they can re-invest to continue to accumulate money. Those who sell their labor-power on the market must because they don’t have the capital to do this themselves.</p>
<p>Now that we’ve established that this system is off-balance, we can figure out how that will eventually become its undoing.</p>
<hr />
<p>Anyway, this is the beginning. There’s obviously much more detail to go into here. If you’d like to read more, this post is basically my own little summary of <a href="https://www.marxists.org/archive/marx/works/1867-c1/">Capital, Volume I</a>, chapters 1-7, by Karl Marx. These basic building blocks of understanding are not without their own problems, but hopefully, I’ve given you a foothold into the topic. I may or may not do this for additional chapters, we’ll see….</p></content:encoded>
</item>
<item>
<title>A new introduction to Rust</title>
<link>https://steveklabnik.com/writing/a-new-introduction-to-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/a-new-introduction-to-rust/</guid>
<pubDate>Fri, 27 Feb 2015 00:00:00 GMT</pubDate>
<content:encoded><p>Lately, I’ve been giving a lot of thought to first impressions of Rust. On May 15, <a href="http://blog.rust-lang.org/2015/02/13/Final-1.0-timeline.html">we’re going to have a lot of them</a>. And you only get one chance at a first impression. So I’ve been wondering if our <a href="http://doc.rust-lang.org/intro.html">Intro</a> and <a href="http://doc.rust-lang.org/book/basic.html">Basics</a> are putting our best foot forward. At first I thought yes, but a few days ago, I had an idea, and it’s making me doubt it, maybe. So instead of re-writing all of our introductory material, I’m just going to write the first bit. A spike, if you will. And I’d like to hear what you think about it. This would take the same place as <a href="http://doc.rust-lang.org/book/variable-bindings.html">2.4: Variable bindings</a> in the existing structure: They’ve installed Rust and gotten Hello World working.</p>
<pre><code>$ rustc --version
rustc 1.0.0-dev (dcc6ce2c7 2015-02-22) (built 2015-02-22)
</code></pre>
<h2>Hello, Ownership</h2>
<p>Let’s learn more about Rust’s central concept: ownership. Along the way, we’ll learn more about its syntax, too. Here’s the program we’re going to talk about:</p>
<pre><code>fn main() {
let x = 5;
}
</code></pre>
<p>This small snippit is enough to start with. First up: <code>let</code>. A <code>let</code> statement introduces a <strong>variable binding</strong>. Bindings allow you to associate a name with some sort of value.</p>
<p>Why ‘variable binding’? Rust draws heavily from both systems languages and functional programming languages. The name “variable binding” is a great example of this. Many systems languages let you declare a variable. These variables are called by that name because they can change over time, they’re mutable. Many functional languages let you declare bindings. These bindings are called by that name because they bind a name to a value, and don’t change over time. They’re immutable.</p>
<p>Rust’s variable bindings are immutable by default, but can become mutable, allowing them to be re-bound to something else. In other words,</p>
<pre><code>fn main() {
let x = 5;
x = 6; // error: re-assignment of immutable variable
let mut y = 5;
y = 6; // just fine
}
</code></pre>
<p>You won’t be typing <code>mut</code> that often.</p>
<p>In any case, there’s one way in which <code>let</code> bindings work just like variables in other languages, but they’re the key insight into ownership. As you know, a computer program is executed line by line. At least, until you hit a control flow structure, anyway. Let’s give our program line numbers, so we can talk about it.</p>
<pre><code>// 1
fn main() { // 2
// 3
let x = 5; // 4
// 5
} // 6
// 7
</code></pre>
<p>Line one is before our program starts. The endless void. Not actually endless, though, as line two is where we start <code>main</code>. This is the first line that is actually executed, and kicks off our program. Great. Line three is blank, so nothing happens, just like one. Line four is where the first actually interesting thing occurs: we introduce a new variable binding, <code>x</code>. We set <code>x</code>’s initial value to five, allocated on the stack. If you don’t know what that means, we’ll talk about it right after ownership. For now, <code>x</code> is five. No big deal. Line six has a closing curly brace, and so <code>main</code>, and thus, our program, is over. Line seven, the void returns.</p>
<p>This is basically the same thing as many programming languages. But let’s point out an aspect you, as a programmer, probably take for granted. Scoping. If I asked you, “Is <code>x</code> valid on line one?” you would say “no.” “Three? Seven?” “Nope, nada. <code>x</code> is valid from line four, where it was declared, to line six, where it goes out of scope.” This illustrates the idea. There is a certain scope, a certain set of lines, where <code>x</code> is a valid identifier. That scope starts from where it was declared, and goes until the end of the block. We can look at this scope in two ways: for the first, imagine this source code printed on a piece of paper. You highlight lines four three six. In some sense, this is a distance: three lines of code, rather than three meters. But if we imagine the computer running this program, this scope represents a time: three statements of processor execution. Even though that number is actually different based on the assembly, but at our level of abstraction, three units.</p>
<p>In Rust, we have names for these concepts, which are implicit in other languages. The thing that introduces a new scope is called the ‘owner.’ It’s in charge of the data it’s bound to, so we say that it ‘owns’ that data. The length of a scope is called a ‘lifetime,’ taken from that idea of time passing as your program executes. But you can also think of it as a segment of lines of code.</p>
<p>So if other programs do this, why does this make Rust special? Sure, Rust gives these concepts specific names, but names themselves aren’t significantly different. The difference is that Rust takes this concept and cranks it up to 11. Where most programming languages only keep track of how long variables are in scope, Rust knows how to connect the scopes of variables that are pointing to the same thing, as well as how to know the scope of things that are more than just stack-allocated memory.</p>
<p>Let’s talk more about this connecting of scopes. Here’s another Rust program:</p>
<pre><code>fn main() { // 1
let x = 5; // 2
// 3
{ // 4
// 5
let y = &amp;x; // 6
// 7
} // 8
// 9
} // 10
</code></pre>
<p>In line four, we use an open curly brace to create a new scope. This scope, like in many languages needs to be closed before the scope of <code>main</code> gets closed. It’s nested inside of it.</p>
<p>In this new scope, on line six, we declare a new binding, <code>y</code>, and we set it equal to <code>&amp;x</code>. This reads as “a reference to <code>x</code>,” as <code>&amp;</code> is the reference operator in Rust. References are Rust’s version of ‘pointers’ from other systems programming languages. Here’s a quick introduction, if you haven’t had to deal with pointers before.</p>
<p>By default, in Rust, data is allocated on “the stack.” Your program is given a chunk of memory, and it’s able to allocate information there. The stack starts at the bottom address of memory, and then grows upwards. For example, at the start of our program, the stack has nothing in it:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20e35b5948ec7546b7a905f4db1ea869c6.csv">Untitled</a></p>
<p>At line2, we introduce <code>x</code>, and it stack-allocates the number five, like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20935c84280787447b8fefcaaefb4575c0.csv">Untitled</a></p>
<p>We’ve got an address in memory, the value at that address, and the name we’ve used in our program. It starts at the bottom because remember, the stack grows upward. Finally,on line six, we have <code>y</code>:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20f7b25102df1d41e884fdac5b761fe567.csv">Untitled</a></p>
<p>Instead of being a value itself, <code>y</code> is a pointer to another location in memory, which holds a value. Because <code>y = &amp;x</code>, we take the memory location of <code>x</code> and store it into <code>y</code>.</p>
<p>On line eight, the inner scope ends, and <code>y</code> goes out of scope. Its lifetime is over. It gets deallocated. Our memory is back to this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20daa49962f5c5408d82e28e9e3e073be7.csv">Untitled</a></p>
<p>Finally, on line ten, our program is over. <code>x</code> goes out of scope, and we look like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%2086c230f1a23e49d892a529d7988a272c.csv">Untitled</a></p>
<p>Everything is gone, all is well.</p>
<p>Pointers can point other places than the stack: they can also point to ‘the heap.’ The heap starts at the top of your program’s memory, and grows down:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20007add295d434d84a281a93e7358f0d5.csv">Untitled</a></p>
<p>The heap starts at <code>0x00</code> and grows down, and the stack starts at <code>0xff</code> and grows up. While the stack contains the data that the programming language has allocated for our bindings, our program can request dynamic amounts of memory from the heap. Here’s how to do this in C:</p>
<pre><code>#include&lt;stdio.h&gt;
#include&lt;stdlib.h&gt;
int main() {
int *x = malloc(sizeof(int)); // 1
*x = 5; // 2
// 3
printf("%i", *x); // 4
}
</code></pre>
<p>The call to <code>malloc</code> (‘memory allocate’) on line 1 requests some memory from the heap. We get a pointer, <code>int *x</code>, to that memory. Our memory looks like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%203e8198d0b0a5408bbcf399a9d682c779.csv">Untitled</a></p>
<p>We have <code>x</code>, which is a variable on the stack, as a pointer to our dynamically allocated memory, which is located on the heap at <code>0x00</code>. But at that location, we haven’t done anything, so who knows what the value is? This means that after line one is done executing, <code>x</code> will be pointing to garbage memory. Hopefully we set it equal to a value before we try to use it, or bad things could happen.</p>
<p>On line two, we set that memory location to five. Whew! Memory looks like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20ccd2c85eb0a34f6483acdbfafa4c841a.csv">Untitled</a></p>
<p>Everything looks good. On line four, we have to print out <code>*x</code>, not <code>x</code>: <code>x</code> is the address of the memory, so we’d get <code>0x00</code> instead of <code>5</code>. After line four, <code>x</code> goes out of scope, and gets deallocated. Memory looks like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%200aec19c2d6a645a3b860bf65cc922853.csv">Untitled</a></p>
<p>… wait a minute, we never took care of that <code>5</code>! We left some memory ‘dangling.’ In this program, that’s okay, because now that it’s over, the operating system reclaims all this memory, and so we don’t have any problems. But we <em>do</em> have a bug. We just don’t know it. This bug will sit there until something changes, and the bug is found. But we’ll get to that.</p>
<p>A slightly more responsible program looks like this:</p>
<pre><code>#include&lt;stdio.h&gt;
#include&lt;stdlib.h&gt;
int main() {
int *x = malloc(sizeof(int));
*x = 5;
printf("%i", *x);
free(x);
}
</code></pre>
<p>The call to <code>free</code> releases the memory pointed to by <code>x</code>, and all is good.</p>
<p>Let’s compare this to an equivalent Rust program:</p>
<pre><code>fn main() {
let x = Box::new(5);
println!("{}", x);
}
</code></pre>
<p>It looks similar at first, with <code>Box::new</code> filling in for <code>malloc</code>, But where’s the free? Does this have the same bug?</p>
<p>No. This Rust is in fact equivalent to the one with the <code>malloc</code> and <code>free</code>. You see, when we introduce our binding, <code>x</code>, Rust knows that <code>x</code> has ownership over the memory in the <code>Box</code>. and so, at the end of the program, when <code>x</code> goes out of scope, Rust also ensures to free the memory that the <code>Box</code> allocated. It happens automatically. The programmer can’t forget it. And the allocation itself was a little easier, because Rust used the type system to figure out how many bytes to allocate, you didn’t need to calculate it with <code>sizeof</code>. Allocation also set the value to 5 for you, which is convenient and less error-prone. When printing out <code>x</code>, you could just use the binding name itself, and not need to dereference it with <code>*</code>.</p>
<p>C++ programmers have ways of doing this kind of thing, with something called “RAII.” As promised before, Rust knows this concept at a deeper, language level. So it can be safe in places that RAII cannot. Here’s an example, in C++:</p>
<pre><code>#include&lt;iostream&gt;
#include&lt;vector&gt;
#include&lt;string&gt;
int main() {
std::vector&lt;std::string&gt; v;
v.push_back("a");
std::string&amp; x = v[0];
v.push_back("b");
std::cout &lt;&lt; x;
}
</code></pre>
<p>This program creates a vector of strings, called <code>v</code>:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20b1e4993d4fc14c9abbe2c2d7deb3be25.csv">Untitled</a></p>
<p><code>v</code> is a ‘fat pointer,’ three pointers in one. The first is a pointer to the data, the second is the length of the array, how many elements are in it. The third is the capacity of the array, how much memory is allocated for this vector. The amount of initial capacity is up to the implementation, I’ve chosen one here. Because this is a vector of strings, we need to allocate a single string, so in line with our capacity. Strings are also a fat pointer, and so we allocate three bytes, starting at <code>0x00</code>.</p>
<p>The program then calls <code>push_back()</code> on the vector, which pushes a string onto the end of the vector:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%205c80884c8d7d4ae290de9e2b9caa1f11.csv">Untitled</a></p>
<p>We’ve now replaced our initial, <code>GARBAGE</code>, string with a real one. Therefore, our data, length, and capacity are updated: <code>0x03</code> is the next free location in the heap, so it gets our signle character, <code>A</code>. We have a length of one, and a capacity of one, so the other two values are set appropriately.</p>
<p>Next, it creates a pointer to the <code>0</code>th element of that vector:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%2031184db665934f5ea7acaea0b98ce4b9.csv">Untitled</a></p>
<p><code>x</code> is allocated on the stack, and points to the first element of <code>v</code>, which, as we know, is located at address <code>0x00</code>, as its data pointer, <code>0xff</code>, shows us. So <code>x</code> has the initial value of <code>0x00</code>.</p>
<p>Next is where everything goes wrong. It tries to push a string onto the end of that vector. The problem is, we only allocated one element. So we need to allocate more. After this is done, memory looks like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20900c132e9e144b1e94ffb6c47d5fc068.csv">Untitled</a></p>
<p>Inside <code>push_back()</code>, there’s a call to <code>malloc</code>, and that is assigned to some temporary value. We’ve chosen <code>0xfb</code> and <code>(temp)</code> to identify it here. It points to the newly allocated data, at <code>0x04</code>.</p>
<p>Now that we’ve got space for two strings, let’s copy the data from the original place to the new one, from <code>0x00</code> to <code>0x04</code>:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%208311f6138fd34b39840f66e8b047db11.csv">Untitled</a></p>
<p>and then free that original string:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20885b35ff100341e6819740529f0d4701.csv">Untitled</a></p>
<p>Then, we have to update <code>v</code> to point at the newly allocated vector, at <code>0x04</code>:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20ec53dd46647143b58bb3c7b136d66a59.csv">Untitled</a></p>
<p>This step is where the bug is! You see, <code>x</code> was pointing at the same data that <code>v</code> was pointing at. This is called ‘aliasing.’ Because C++ does not have the concept of ownership and lifetimes, it doesn’t understand that the validity of <code>x</code> and the validity of <code>v</code> are closely intertwined. Thus, we’re allowed to do this reallocate-and-copy dance with what <code>v</code> points to, but <code>x</code> has no idea. Look at the diagram above: it still has <code>x</code> pointing to <code>0x00</code>. But we just deallocated that memory! This is when the bug occurs, but it’s a silent bug. A time bomb, ticking away.</p>
<p>Let’s see how it explodes. Our code goes on, oblivious.</p>
<p>Our new string at <code>0x04</code> still says that we have one capacity, one length, and that data is at <code>0x03</code>. That’s all still accurate, so we’re all good. Next, we need to deal with our new string, “B.” We have the space, so we do so:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20cc5b92de5ac545878c67a66a878863b6.csv">Untitled</a></p>
<p>We’ve placed our new string at <code>0x07</code> to <code>0x09</code>, with its data pointer, length, and capacity. Our data is at <code>0x0A</code>, which has the value <code>B</code> stored at it. All is good, and so we return from <code>push_back</code>. <code>(temp)</code> is deallocated, and memory looks like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20ee6def6517424e0fb6c9ad7c3ec0389e.csv">Untitled</a></p>
<p>On the final line, we print out the value that <code>x</code> points to. As we can see above, <code>x</code> points to <code>0x00</code>, which has nothing saved at it! <code>x</code> is invalid memory. Using <code>x</code> is a bad idea. But the programming language can’t save us, even though the bug happened many lines ago.</p>
<p>How are you supposed to deal with this problem? Here’s <a href="http://en.cppreference.com/w/cpp/container/vector/push_back">the documentation for <code>push_back</code></a>:</p>
<blockquote>
<p>If the new size() is greater than capacity() then all iterators and references (including the past-the-end iterator) are invalidated.</p>
</blockquote>
<p>In other words, when you call <code>push_back</code>, it’s your responsibility to make sure to not use anything that points to the underlying data, because they are now invalid. The documentation has explained our bug perfectly! And while this is a short example, these problems can crop up in huge codebases, and the errors can be very hard to track down.</p>
<p>Let’s look at the same problem, in Rust:</p>
<pre><code>fn main() {
let mut v = vec![];
v.push("Hello".to_string());
let x = &amp;v[0];
v.push("world".to_string());
println!("{}", x);
}
</code></pre>
<p>This program has identical semantics to the C++ version presented above. (If we weren’t trying to get the exact same semantics, we could leave off the <code>.to_string()</code> calls, and it would still work. But let’s ignore that for now.) So does it have the same bug? Well, it would, but it doesn’t compile:</p>
<pre><code>main.rs:8:5: 8:6 error: cannot borrow `v` as mutable because it is also borrowed as immutable
main.rs:8 v.push("world");
^
main.rs:6:14: 6:15 note: previous borrow of `v` occurs here; the immutable borrow prevents subsequent moves or mutable borrows of `v` until the borrow ends
main.rs:6 let x = &amp;v[0];
^
main.rs:11:2: 11:2 note: previous borrow ends here
main.rs:1 fn main() {
...
main.rs:11 }
^
</code></pre>
<p>When you call the method <code>push()</code> on a vector, it ‘borrows’ the vector. Just like we wrote <code>let y = &amp;x</code> before, to borrow the value of <code>x</code>. Rust now understands that the scope of <code>x</code> and the scope of <code>v</code> are connected. There’s just one problem: because <code>push</code> allocates more memory, it can’t just do a regular borrow, it needs a mutable borrow. If <code>&amp;x</code> borrows x, <code>&amp;mut x</code> mutably borrows <code>x</code>. Like we discussed previously, mutability means that you can reassign the value of that binding. But when we mutably <em>borrow</em> something, we make a guarantee: this is the only reference to this memory that exists. That way, we don’t have the problem of the C++ code, where a reference became outdated. If we’re the only reference, we are free to change what we’re pointing to, becuase that won’t cause any problems.</p>
<p>So when we create a borrow with <code>&amp;</code>, it lasts for the scope that it exists, like this:</p>
<pre><code>fn main() {
let mut v = vec![];
v.push("A");
let x = &amp;v[0]; // &lt;+
// |
v.push("B"); // |
// |
println!("{}", x); // |
} // &lt;+
</code></pre>
<p>The line indicates the scope, or lifetime, of the borrow. But when we call push with <code>B</code>, we have to get a mutable borrow, in order to allocate the new memory. But because <code>&amp;mut</code> is a promise that there are no other references, this is a violation of the rules. This is the point at which the bug happened in the C++ example, if you remember. And so it’s the place where the Rust compiler throws its error message. It might make more sense this time around:</p>
<pre><code>main.rs:8:5: 8:6 error: cannot borrow `v` as mutable because it is also borrowed as immutable
main.rs:8 v.push("world");
^
main.rs:6:14: 6:15 note: previous borrow of `v` occurs here; the immutable borrow prevents subsequent moves or mutable borrows of `v` until the borrow ends
main.rs:6 let x = &amp;v[0];
^
main.rs:11:2: 11:2 note: previous borrow ends here
main.rs:1 fn main() {
...
main.rs:11 }
^
</code></pre>
<p>We cannot borrow <code>v</code> as mutable, with <code>push()</code>, becuase it is also borrowed as immutable, by <code>x</code>. That’s exactly the problem, and Rust solves it.</p>
<p>There’s more to both ownership and Rust syntax, so let’s cover some syntactical things, and then talk more about ownership.</p>
<hr />
<p>I’m fairly confident with the memory layouts involved, though the addresses are of course not exact. I’m also not sure that this mega deep detail dive is the correct way to do an introduction either, it’s also thorough, but maybe in a bad way? It’s almost the exact opposite of the way that I’ve done things at present. Also, apparently I visuallize things in reverse, with <code>0x00</code> to <code>0xFF</code> as going down, but others think the opposite, which may be confusing. Please send me a <a href="http://twitter.com/steveklabnik">tweet</a> or <a href="mailto:[email protected]">an email</a> with your thoughts. Thanks!</p></content:encoded>
</item>
<item>
<title>Update GitHub Pages with Travis CI</title>
<link>https://steveklabnik.com/writing/update-github-pages-with-travis-ci/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/update-github-pages-with-travis-ci/</guid>
<pubDate>Tue, 16 Dec 2014 00:00:00 GMT</pubDate>
<content:encoded><p>Ever wanted to have your project automatically update Pages when you have a successful build?</p>
<p>I have a demo repository here: <a href="https://github.com/steveklabnik/automatically_update_github_pages_with_travis_example">https://github.com/steveklabnik/automatically_update_github_pages_with_travis_example</a></p>
<p>It, of course, uses Travis to update its own Github Pages, which is here: <a href="http://steveklabnik.github.io/automatically_update_github_pages_with_travis_example/">http://steveklabnik.github.io/automatically_update_github_pages_with_travis_example/</a></p>
<p>I hope that helps!</p></content:encoded>
</item>
<item>
<title>I'm going writeonly on Twitter for a while</title>
<link>https://steveklabnik.com/writing/im-going-writeonly-on-twitter-for-a-while/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/im-going-writeonly-on-twitter-for-a-while/</guid>
<pubDate>Wed, 03 Sep 2014 00:00:00 GMT</pubDate>
<content:encoded><p>This is something I’ve been joking about for a while, but I’ve decided for the near future, I’m just going to be posting things to Twitter, and not reading replies or anything.</p>
<p>So, if you don’t get a fav or a reply or something, it’s nothing personal.</p>
<p>There’s a lot going into this decision, but basically, I haven’t been actually doing a lot of things that I want to do because I’m super stressed out. Turns out Twitter is also a huge cause of stress in my life. Turns out Twitter is also a large part of my social interaction, so I’ve kept doing it even though I know it’s not great for me.</p>
<p>So, we’ll see what a post-read/write Twitter is like. You know how to get a hold of me otherwise.</p></content:encoded>
</item>
<item>
<title>Rust's documentation is about to drastically improve</title>
<link>https://steveklabnik.com/writing/rusts-documentation-is-about-to-drastically-improve/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rusts-documentation-is-about-to-drastically-improve/</guid>
<pubDate>Mon, 16 Jun 2014 00:00:00 GMT</pubDate>
<content:encoded><p>Historically, <a href="http://rust-lang.org/">Rust</a> has had a tough time with documentation. Such a young programming language changes a lot, which means that documentation can quickly be out of date. However, Rust is nearing a 1.0 release, and so that’s about to change. I’ve just signed a contract with Mozilla, and starting Monday, June 23rd, I will be devoting forty hours per week of my time to ensuring that Rust has wonderful documentation.</p>
<p>A year and a half ago, I was in my hometown for Christmas. I happened upon a link: <a href="https://mail.mozilla.org/pipermail/rust-dev/2012-December/002787.html">Rust 0.5 released</a>. I’ve always enjoyed learning new programming languages, and I had vaguely heard of Rust before, but didn’t really remember what it was all about. So I dug in. I loved systems programming in college, but had done web-based stuff my entire professional life, and hadn’t seriously thought about pointers as part of my day-to-day in quite some time.</p>
<p>There was just one problem: Rust was really difficult to learn. I liked what I saw, but there was very little of it. At the same time, I had been working on some ideas for a new toolchain for my <a href="http://www.designinghypermediaapis.com/">book on hypermedia APIs</a>, but wanted to try it out on something else before I took the time to port the content. And so, <a href="http://www.rustforrubyists.com/">Rust for Rubyists</a> was born. I decided that the best way to teach people Rust was to mimic how I learned Rust. And so, as I learned, I wrote. It ended up at about fifty pages of two weeks of work. I never contributed it to the main repository, because for me, it was really about learning, both about my ebook toolchain as well as Rust itself. I didn’t want the burden that came with writing an official tutorial, making sure that you cover every single feature, pleasing every single Github contributor…</p>
<p>After learning Rust, I decided that I really liked it. No other language provided such a promising successor to C++. And I really missed low-level programming. So I kept evangelizing Rust, and every so often, contributing official documentation. I figured that even if my low-level chops weren’t up to the task of writing actual patches, I could at least help with my writing skills. I’d previously contributed lots of documentation to Ruby and Rails, so it was something that was very familiar to me. I’ve often found that I start with documentation and then move into contributing code, once I get my head wrapped around everything. Writing is part of my own process of understanding.</p>
<p>Rust for Rubyists was a great hit, even amongst non-Rubyists (damn my love of alliteration!). Six months ago, on the eve of the first anniversary of the initial release of Rust for Rubyists, I <a href="https://air.mozilla.org/rust-meetup-december-2013/">gave a talk</a> at the Bay Area Rust meetup, specifically on the state of Rust’s documentation. In it, I laid out a plan for how I envisioned docs looking in the future. In the last six months, a lot has improved, but a lot hasn’t. But no more! I’m now going to be able to allocate a significant amount of my time on getting documentation done.</p>
<p>I’m also pleased in a meta-sense. You see, by contracting someone to work on documentation full-time, Mozilla is indicating that they take Rust and its future very seriously. You can (and I do) talk a lot of trash on Microsoft, but one of the reasons that the Microsoft platform is used by so many people around the world is that Microsoft products often have excellent documentation. I often find that open source ‘products’ are technically superior, but are often much harder to use, because they’re built by a community, for free, and very few people want to write documentation for free. Combined with the work that Tilde is doing on <a href="https://github.com/carlhuda/cargo">Cargo</a>, Mozilla is investing a significant amount of effort and dollars into ensuring that Rust will be a fantastic platform for those developing on it. Since I love Rust, this makes me very, very happy.</p>
<p>Forty hours a week is a significant amount of documentation, and I have a lot of work in front of me. But my first area of focus will be on the area of Rust’s documentation that’s the most weak, and simultaneously the most important: the tutorial. I tackled the first tip of that iceberg with <a href="http://doc.rust-lang.org/master/intro.html">my 30 minute introduction</a>, and I’d like to tweak it too. The main tutorial, however, is the first place where people go to <em>really</em> learn about Rust and how it works. And the current tutorial is largely the same as the 0.5 days, back when I first learned Rust. It suffers from receiving patchwork contributions from a variety of people, rather than having one unifying vision. It’s also much more of a list of features and how to use them than a coherent tutorial.</p>
<p>I’m really excited to get to work. Let’s all make Rust 1.0 a fantastic release.</p></content:encoded>
</item>
<item>
<title>SF</title>
<link>https://steveklabnik.com/writing/sf/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/sf/</guid>
<pubDate>Tue, 20 May 2014 00:00:00 GMT</pubDate>
<content:encoded><blockquote>
<p>@steveklabnik &lt;small&gt;including you?&lt;/small&gt;</p>
</blockquote>
<p>— Nick Quaranto (@qrush) March 5, 2014</p>
<blockquote></blockquote>
<p>I was always not quite sure about moving to San Francisco. I’ve heard about this place for my entire life: back in high school, during the first bubble, I would read Slashdot and wonder about life out west. But San Francisco is like a vortex for software developers. It’s got this gravitational pull that’s hard to escape. When I made the choice to move here, I wasn’t sure if I’d be able to handle it. Turns out I can’t.</p>
<p>So I’m leaving. My job is great, but it’s not worth staying in this city. I haven’t done anything outside of work in a few months. I go to work, work hard, go home, and then go to sleep. I don’t go home late. I go home at five. It’s not work/life balance that’s the issue. I’m just not psyched for the ‘life’ part. San Francisco has sucked out all my creativity, most of my passion, and has made me increasingly frustrated with things. Elaborating further would just be <a href="http://tomdale.net/2013/07/san-francisco-i-love-you-but-youre-bringing-me-down/">repeating Tom Dale</a> and <a href="http://programmingisterrible.com/post/50421878989/come-here-and-work-on-hard-problems-except-the-ones">tef</a> with not much difference, so I’ll leave it at that.</p>
<p>So, at the end of the month, I’m leaving Balanced and the city. I’ll be arriving in Brooklyn, a place I’ve been trying to live for the past few years. I’m going to deeply miss my friends and my (ex-)coworkers, but I have a lot of friends in NYC, too. Every hour I spend in New York, I feel excited, and every hour I spend in San Francisco, I feel drained.</p>
<p>So here’s to new adventures. I can’t wait to be happy again.</p></content:encoded>
</item>
<item>
<title>Hypermedia FizzBuzz</title>
<link>https://steveklabnik.com/writing/hypermedia-fizzbuzz/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/hypermedia-fizzbuzz/</guid>
<pubDate>Fri, 02 May 2014 00:00:00 GMT</pubDate>
<content:encoded><p>I read a really great blog post last week: <a href="http://smizell.com/weblog/2014/solving-fizzbuzz-with-hypermedia">Solving FizzBuzz with Hypermedia</a>. In it, Stephen Mizell builds a FizzBuzz service using <a href="https://github.com/kevinswiber/siren">Siren</a>, and then shows how client code evolves. I wanted to explain exactly why I think this example is amazing, because I’m not sure it’s exactly obvious.</p>
<h2>FizzBuzz? Really?</h2>
<p>The first thing that makes this post brilliant is that it uses FizzBuzz. While FizzBuzz has historically been a simple interview question to make sure that you can actually write some basic code, in this case, it works well because it’s a very, very simple programming problem. When writing examples, there’s always tension between real-world and straw-man examples. If you make it too real-world, all sorts of incidental details creep in and distract from your main point. If you make a trivial example, it can be claimed that it works for your simple example, but not for ‘real’ examples. Now, this may be true of FizzBuzz, but what I like about it is that it’s an example that everyone is already familiar with. It has <em>just enough</em> actual computing that it straddles that line.</p>
<h2>Haters gonna HATEOAS</h2>
<p>The real insight of this post, however, is demonstrating one of the hardest concepts to grok about the hypermedia approach: the way in which hypermedia reduces duplication, by moving application entirely to the server. This is the core of that dreaded “HATEOAS” concept, and it’s where people get tripped up. Here’s the sample client in the hypermedia style:</p>
<pre><code>from siren import SirenResource as Hyperclient
BASE_URL = "http://fizzbuzzaas.herokuapp.com"
def fizzbuzz(resource):
"""
Prints the fizzbuzz value and follows "next" links
"""
print resource.properties["value"]
if resource.has_link("next"):
fizzbuzz(resource.follow_link("next"))
def begin_fizzbuzz(resource):
"""
Follows the first link, then hands off to fizzbuzz
"""
if resource.has_link("first"):
fizzbuzz(resource.follow_link("first"))
root_resource = Hyperclient(BASE_URL, path="/")
begin_fizzbuzz(root_resource)
</code></pre>
<p>Let’s talk about what this client’s goal is: to print a list of numbers. This is a classic recursive list traversal algorithm: we start at the head of the list (<code>first</code>), and then follow the links until we get to the end. (<code>next</code>) This client <em>does not actually calculate fizzbuzz</em>. It just so happens that the service we’re pointing it at calculates FizzBuzz. If we pointed it at, say, a list of search results, where the same link relations and Siren were used, it would still work!</p>
<p>Compare this to the non-hypermedia client:</p>
<pre><code>import requests
BASE_URL = "http://fizzbuzzaas.herokuapp.com"
def fizzbuzz(params):
url = BASE_URL + "/fizzbuzz"
response = requests.get(url, params=params)
response_json = response.json()
return response_json["properties"]["value"]
for number in range(1, 101):
print fizzbuzz({"number": number })
</code></pre>
<p>This client doesn’t fully calculate FizzBuzz on its own, but it does have some of the logic for doing so embedded inside. We explicitly loop over the range FizzBuzz needs, and we print out each value.</p>
<p>Why does this matter? It seems like a pretty subtle distinction. Well, this logic already exists on the server. We’re duplicating that logic here. This duplication creates coupling across the client/server boundary. For example, if we pointed this client at a “search results” resource, we would get 100 search results, even if there are less. We’re not letting the server guide our interactions, and we’re coding business logic into the client.</p>
<p>This comes back to bite us when we try to change the client. As Stephen’s next example shows:</p>
<pre><code>import requests
BASE_URL = "http://fizzbuzzaas.herokuapp.com"
STARTS_AT = 4
ENDS_AT = 20
ADD = 2
def fizzbuzz(params):
url = BASE_URL + "/fizzbuzz"
response = requests.get(url, params=params)
response_json = response.json()
print response_json["properties"]["value"]
params["number"] += ADD
if params["number"] &lt;= ENDS_AT:
fizzbuzz(params)
fizzbuzz({ "number": STARTS_AT })
</code></pre>
<p>This version uses different start, end, and skip parameters. And so we had to throw out our entire old client, because it was too tightly coupled to the notion of how to calculate FizzBuzz. We’re now manually doing the skip and end calculations, even further coupling this client. Our fundamental algorithm even changed: the last client was iterative, but this client is recursive. We could have still made it iterative, but that still would have been a lot of change.</p>
<p>Now , the hypermedia version:</p>
<pre><code>from siren import SirenResource as Hyperclient
BASE_URL = "http://fizzbuzzaas.herokuapp.com"
def fizzbuzz(resource):
"""
Prints the fizzbuzz value and follows "next" links
"""
print resource.properties["value"]
if resource.has_link("next"):
fizzbuzz(resource.follow_link("next"))
def begin_fizzbuzz(resource):
"""
Follows the first link, then hands off to fizzbuzz
"""
if resource.has_link("first"):
fizzbuzz(resource.follow_link("first"))
def custom_fizzbuzz(root_resource, params):
"""
Submits actions for custom fizzbuzz
"""
resource = root_resource.take_action("custom-fizzbuzz", params)
begin_fizzbuzz(resource)
root_resource = Hyperclient(BASE_URL, path="/")
params = { "startsAt": 4, "endsAt": 20, "add": 2 }
custom_fizzbuzz(root_resource, params)
</code></pre>
<p>Very little has changed between the last client and this one. We just added an extra method to submit our custom start point, but the rest of the code is all identical: we’re still traversing the list, with the same code. This worked because we chose the right behavior for our client, and let the server handle all of the business logic.</p>
<h2>Traversal from the root</h2>
<p>The last thing this post does is demonstrate how an actual hypermedia API starts from the API root and progresses from there. Lots of people hear this, and assume that there will be way more API calls than in an API in a different style. That’s because they assume that the client will be written in the same way as they’re used to: function calls over HTTP. As you can see here, we don’t start each call from the root of the API, we start <em>our initial interaction with the service</em> from the root of the API. Each step is just one more call after that, just like in any other API.</p>
<p>This is the other part of the hypermedia constraint that trips people up. Navigating a state machine of your API’s business process has a very different feel than making function calls over HTTP. It requires a different kind of mindset and approach to building clients. This is the area of hypermedia research that’s been least published about. The server-side story has been fleshed out, but the real frontier in hypermedia theory and practice is client building guidelines, and that’s why I like this example so much.</p></content:encoded>
</item>
<item>
<title>How to be an open source gardener</title>
<link>https://steveklabnik.com/writing/how-to-be-an-open-source-gardener/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/how-to-be-an-open-source-gardener/</guid>
<pubDate>Mon, 14 Apr 2014 00:00:00 GMT</pubDate>
<content:encoded><p>I do a lot of work on open source, but my most valuable contributions haven’t been code. Writing a patch is the easiest part of open source. The truly hard stuff is all of the <em>rest</em>: bug trackers, mailing lists, documentation, and other management tasks. Here’s some things I’ve learned along the way.</p>
<p>It was RailsConf 2012. I sat in on a panel discussion, and the number of issues open on <a href="https://github.com/rails/rails">rails/rails</a> came up. There were about 800 issues at the time, and had been for a while. Inquiring minds wished to know if that number was ever going to drop, and how the community could help. It was brought up that there was an ‘Issues team,’ whose job would be to triage issues. I enthusiastically volunteered.</p>
<p>But what does ‘issue triage’ <em>mean</em>, exactly? Well, on a project as large as Rails, there are a ton of issues that are incomplete, stale, need more information… and nobody was tending to them. It’s kind of like a garden: you need someone to pull weeds, and do it often and regularly.</p>
<p>But before we talk about how to pull the weeds, let’s figure out what kind of garden we even have on our hands!</p>
<h2>What are Issues?</h2>
<p>The very first thing your project needs to do is to figure out what Issues are supposed to be for. Each project is different. For example, in Rails, we keep Issues strictly for bugs only. Help questions go to Stack Overflow, and new feature discussion and requests go to the rails-core mailing list. For Rust, we have issues for feature requests, meta-issues… everything. For some repositories, closing all of the issues is not feasible, and for others, you’re shooting for zero. (If you don’t believe that this is even possible, check out <a href="https://github.com/jeremyevans/sequel/issues">Sequel</a>. Issues are rarely even open for more than a few days!)</p>
<p>My personal favorite is to follow the Rails way. Ideally, you’d be at zero defects, and you can still have a place to discuss features. But really, having <em>some</em> plan is a necessary first step here.</p>
<h2>Regular tending</h2>
<p>So how do you tackle 800 issues? The only way I knew how: read all of them. Yep. Here’s what I did: I took a Saturday (and a Sunday), and I went to <a href="https://github.com/rails/rails/issues?state=open">the list of open Issues</a>, then control-clicked on each one in turn to open them in a new tab. Finally, I also control-clicked on page 2. Then I closed this tab. Now I had 31 open tabs: 30 issues, and the next page. I read through the whole issue, including comments. When I got to the last tab, I was ready to repeat the process: open 30 issues, open page 3, click close. Next!</p>
<p>See, people think working on open source is glamorous, but it’s actually not. Working on open source is reading 800 issues over the course of a weekend.</p>
<p>Anyway, once I read all of those issues, I was significantly more informed about the kinds of problems Rails was facing. I had a whole bunch of common questions, comments, and problems.</p>
<p>The next step was to do it all again.</p>
<p>Wait, again? Why? Well, now that I had a handle on things, I could actually take on the task of triage-ing the issues. If I’d tried to do it before I had the context, I might not have seen the duplicate issues, I wouldn’t know what the normal kinds of comments were on issues, I wouldn’t have known some common questions that maintainers had on pull requests, and in general, things would have just been worse.</p>
<p>This time, when reading the issue, I went through a little algorithm to sort them out. It looked a little like this:</p>
<ol>
<li>Is this issue a feature request? If so, copy/paste an answer I wrote that pointed them to the mailing list, and click close.</li>
<li>Is this issue a request for help? If so, copy/paste an answer I wrote that pointed them to StackOverflow, and click close.</li>
<li>Was this issue for an older version of Rails than is currently supported? If so, copy/paste an answer I wrote that asks if anyone knows if this affects a supported version of Rails.</li>
<li>Did this issue provide enough information to reproduce the error? If no, copy/paste an answer I wrote that asks if they can provide a reproduction.</li>
<li>If the issue has a reproduction, and it wasn’t on the latest Rails, try it against HEAD. If it still happened, leave a comment that it was still an issue.</li>
<li>If we got to this point, this issue was pretty solid. Leave a comment that I had triaged it, and cc the maintainer of that relevant sub-system of Rails, so they could find issues that pertain to the things they work on.</li>
</ol>
<p>At the same time I did this, I clicked this button on the GitHub interface:</p>
<p><img src="https://svbtleusercontent.com/e2rhsedvszk54q_small.png" alt="https://svbtleusercontent.com/e2rhsedvszk54q_small.png" /></p>
<p>And then set up a Gmail filter to filter all of the emails into their own tag, and to skip my inbox:</p>
<p><img src="https://svbtleusercontent.com/oyuljxmbfqieia_small.png" alt="https://svbtleusercontent.com/oyuljxmbfqieia_small.png" /></p>
<p>Why do this? Well, I didn’t do all 800 immediately. I decided to do one page per day. This kept it a bit more manageable, rather than taking up entire days of my time. I need these emails and filters for the important second part of the process: tending to the garden regularly.</p>
<p>Each morning, before I go to work, I pour a cup of coffee and check my emails. I don’t handle all of them before work, but I made an effort to tackle Rails’ emails first. There would usually be about 20 or 25 new emails each morning, and since it was largely just one new comment, they’d be pretty fast to get through. 15 minutes later, I was back to current on all issues. At lunch, I’d do it again: ten minutes to handle the ten or so emails by lunch, and then, before I’d go to bed, I’d do it again: 15 more minutes to handle the next 20 notifications. Basically, I was spending a little under an hour each day, but by doing it <em>every</em> day, it never got out of hand.</p>
<p>Once I got through all of the issues, we were down to more like 600. A whole fourth of the issues shouldn’t even have been open in the first place. Two weeks in is when the next big gain kicked in. Why two weeks? Well, two weeks is the grace period we decided before marking an issue as stale. Why two weeks? Well, that’s kind of arbitrary, but two weeks feels like enough time for someone to respond if they’re actively interested in getting an issue fixed. See, issues often need the help of the reporter to truly fix, as there just isn’t enough information in many bug reports to be able to reproduce and fix the problem.</p>
<p>So, after two weeks, I did one more thing each evening: I filtered by ‘least recently updated,’ and checked to see if any of those issues were stale. You just go back until they say ‘two weeks,’ and then, if you haven’t heard from the reporter, mention that it’s stale and give the issue a close. This is one of the other things I had to kind of let go of when working on a real project: closing an issue isn’t forever. You can always re-open the issue later if it turns out you were wrong. So when trying to get a handle on 800 open issues, I defaulted to ‘guilty until proven innocent.’ Terminate issues with extreme prejudice. Leaving old, inconclusive issues doesn’t help anyone. If it’s a real bug that matters to someone, they’ll come along and help reproduce it. If not, maybe someone else will later.</p>
<p>After a month or two, keeping on it, we got down to 450 or so issues. Members of the core team joked that they had to set up extra email filters from me, because they could tell exactly when I was doing triage. Slow and steady wins the race!</p>
<p>At this point, I knew enough about Rails to actually start writing some patches. And I happened to be familiar with basically every open bug. So it was easy to start picking some of them and try to reproduce them locally. So I’d do that, and then try to write a patch. If I couldn’t, I’d at least upload my reproduction of the issue, and then leave a note on the Issue, pointing to my reproduction. That way, another team member could simply clone my repository and get to it. The only thing better than reproduction instructions are when those instructions say <code>git clone</code>.</p>
<p>But I managed to get a few patches in, and then a few more. Doing all of this janitorial work directly led the way towards attaining a commit bit on Rails. It was a long slog at first, but it just got easier the more I did it. A lot of work in open source is this way: it’s really easy once you’ve done it a lot, but is hard for newbies. I’m not yet sure how to tackle this problem…</p>
<p>I’ve since taken this approach on basically every repository I’ve worked on, and it’s worked really well. But it only works if you keep at it: if you don’t tend your garden, you’ll get weeds. I haven’t had as much time for Rails over the last few months, and it’s back to 800 issues again. I’m not sure if these are real issues or not, as I’ve stopped tending. But without someone actively paying attention, it’s only a matter of time before things get unseemly. If you’re looking to help out an open source project, it’s not a glamorous job, but all it takes is a little bit of work, and developing a habit.</p>
<p>(Oh, and I should take a moment to mention <a href="http://www.codetriage.com/">Code Triage</a> here. It’s super useful, and can also help you find projects that need help.)</p></content:encoded>
</item>
<item>
<title>Is npm worth 26MM?</title>
<link>https://steveklabnik.com/writing/is-npm-worth-26mm/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/is-npm-worth-26mm/</guid>
<pubDate>Wed, 12 Feb 2014 00:00:00 GMT</pubDate>
<content:encoded><p>Yesterday, npm, Inc. announced that it had <a href="http://blog.npmjs.org/post/76320673650/funding">closed a $2.6MM round of funding</a>. This was met with a bunch of derision on Twitter. I feel this is not only unwarranted, but backwards: I think this investment signals a nascent trend within the VC community, and a more honest view of how open source actually works.</p>
<blockquote>
<p>Further disclaimer: I work for a company which has received investment from people mentioned in this article. I have never talked to those firms directly either, and I have no privileged information as to why they invested in us, either. It all happened before I came on board.</p>
</blockquote>
<p>Let’s talk about VC, and then about some odd investments, and then we’ll get around to npm. I apologize for the length of this post, I didn’t have time to make a shorter one.</p>
<h2>How VC investment works</h2>
<p>First, a quick refresher on the actual mechanics of VC investment: <a href="http://en.wikipedia.org/wiki/Betteridge's_law_of_headlines">the answer to this headline is ‘no.’</a>. npm, Inc (‘npm’ for the rest of this post) received an investment of $2.6MM, which means that their company is ‘worth’ some multiple of that. At least, it is to <a href="http://www.trueventures.com/">True Ventures</a> and friends.</p>
<p>As the saying goes, “follow the money.” A VC firm is, in many ways, a marketplace: they take money from investors (“limited partners,” or “lp’s”) and give it to entrepreneurs, and take a cut off the top. Let’s go through an example:</p>
<p>The Vulture Capital company is a venture capital firm. They announce a new fund. This fund, like many, has a mission statement: to invest in new SaaS technologies. They go to various wealthy individuals, insurance companies, funds-of-funds, endowments, and other companies, and pitch them on this investment: “The VC firm is going to invest $100MM into new SaaS technologies. Our last fund turned $10MM into $20MM. We think we can turn this $100MM into $300MM in the next ten years. We just need your money.” This motley band of the 1% scrapes together $100MM total out of their couch, and this new fund is good to go! The fund is now ‘closed.’ The VC firm now goes to work: the individuals who work for the firm now figure out who to actually give the money to. This generally takes three to five years. In this story, to make the math easy, VC invests in ten companies, $9.75MM each. For this, they get some stake in each company: we’ll call it 30% here. Since they purchased 30% of the company for $9.75MM, each company is ‘worth’ $29.25MM. In order to actually monetize this stake, generally, these companies need to experience a liquidity event: someone has to purchase them, or they need to go public. Because early-stage investing is exceedingly risky, when we check in on Vulture Capital ten years later, six companies have gone bankrupt, three sold for $67MM, and one went public and investors sold off $100MM. 3 * 67MM + 100MM = 300MM: success! The firm has turned $100MM into $300MM. The firm takes a chunk of this money, and the rest is given to those individuals that put up the money in the first place. A new fund is announced, and the cycle repeats itself.</p>
<p>There are variations at all points in this story, but that’s basically how it works. I’ve also ignored that VC firms generally have multiple funds going at once, that there are different kinds of employees in VC funds with different roles, and a whole ton of other things. This is a simplification, but a representative one.</p>
<p>Two big things matter here: It’s a numbers game. Over half of our example companies bit the dust, but the firm still made their money. Things have been successful overall. Secondly, companies <em>must</em> exit. A business that’s making $1MM a year in profit off of our $9.75MM investment will still make money eventually, but it won’t in our ten year period, which is when our limited partners want their money back. In another context, a million dollars of profit a year is a success, but for us, it’s a failure.</p>
<p>We’ll come back to these economics in a bit.</p>
<h2>Open Source and $$$</h2>
<p>It’s time for some #realtalk about open source. Open Source is big business. And it’s always been that way. Someday I will write more about this, but for now, I’ll just make that statement and not really back it up. For now, let’s trace the flow of labor in the open source world, and how it relates to businesses.</p>
<p>Open Source is pretty much the de-facto way to run a startup company these days. Even people on the Microsoft stack are increasingly utilizing open source. And there’s a good reason: the initial monetary cost of open source is zero. It’s also a form of DRY: a bug fixed in an open source component fixes a bug in every firm that uses that particular component.</p>
<p>Here’s a (sorta kinda) real world example:</p>
<p>A company, “<a href="http://adequatehq.com/">Adequate HQ</a>” builds a web application (<a href="http://www.recruiterspam.com/">RecruiterSpam</a>) in a hot new language, “<a href="https://github.com/tenderlove/phuby">Phuby</a>.” They decide to release a bunch of the code from this application as open source, and call it “<a href="http://www.youtube.com/watch?v=lsWKjS6Vufw">Phuby on Phails</a>.” There are a number of reasons to do this, but here’s a big one: bugs fixed in Phails will be fixed in RecruiterSpam, at no cost to Adequate! Furthermore, there’s a market opportunity here: by controlling Phuby on Phails, Adequate secures its reputation as a leader in the space, which leads to an invaluable amount of publicity, book deals, and eventually, pro subscriptions to RecruiterSpam. Adequate’s CEO, <a href="https://twitter.com/gorbypuff">Gorbachev “Puff Puff” Thunderhorse</a>, can just lay around all day and post photos of himself to Instagram. Other companies see the success of Adequate, and also decide to build their application on Phails. After all, Phails is pretty awesome, and is (truly) significantly better than all the other frameworks out there. Their new use exposes bugs in Phails, so they fix them and submit patches. Eventually, there’s an entire Phails ecosystem, with conferences, books, ninjas, and rockstars. Tons of gravy for everyone, and everyone rides their sports cars into their various sunsets.</p>
<p><img src="/img/2014-02-12/zonda-hh.jpg" alt="pagani zonda hh" /></p>
<p>This is a great story. Everyone wins. Everyone helps each other out. Costs go down, quality goes up. But there’s something that needs to be acknowledged here: Adequate had to exist. Someone had to do the initial work of actually building Phails (and all the other related tools I didn’t explicitly mention: deployment tools, testing tools, dependency management tools, the list goes on and on). In this case, it was because they had a kick-ass product and made boatloads of cash.</p>
<p>But fundamentally, programmers must pay rent and eat. Yes, they may not <em>directly</em> work on open source for free. But someone is paying them, and it has to come from <em>somewhere</em>. There is no such thing as a free lunch.</p>
<p>There are, however, different ways to get the money to buy lunch.</p>
<h2>Infrastructure investment</h2>
<p>So we have this trend of open source in general, and that for each language, we need an ecosystem to support building software. That ecosystem has to be bootstrapped somehow. Previous communities effectively got lucky, or put in long, long hours, only to be forgotten. I still cry tears for CPAN every once in a while. But I digress. The real question is this:</p>
<p>I’m a VC. I need a large number of companies to exist to make my money. I also need them to get as big as possible to make my money. I need costs to be low and profits to be high. I need a healthy open source ecosystem to make this happen. So what do I do?</p>
<p>So here’s what I’m seeing slowly happen: VCs are realizing this story, and are starting to invest in infrastructure. Remember, failure is built into their model. If you had to make ten investments, and I told you that I would take one of them, and you’d lose money on me, but I’d build an ecosystem that would help be a multiplier on all your other investments, would you take it? Remember, you just need one really, really big payday. And a bunch of your investments <em>will</em> already fail.</p>
<p>This is what I see happening with npm. And it’s not just npm: there’s a number of investments in the last few years which fit this pattern. I’m pretty sure that <a href="http://en.wikipedia.org/wiki/Andreessen_Horowitz">a16z</a> is the only VC firm that explicitly understands this model, and True Ventures may have stumbled upon it by accident.</p>
<p>Because Rails (If the sarcasm wasn’t heavy enough, Phails is Rails) isn’t exactly repeatable. It’s a fluke. A wonderful fluke that’s changed my life (I have a Ruby tattoo), but I wouldn’t bet on it happening that way again. If I told you, “I have a design firm. No programming skill. We’re going to hire an offshore contractor to build an application and let them do whatever they want.” Would you invest? I wouldn’t.</p>
<p>If you want to repeatably manufacture an open source ecosystem, you need capital to do so. And a firm that’s progressive enough to understand the indirect payoffs of investing in infrastructure is poised to have a huge advantage.</p>
<p>While my VC story above is a generic, average one, there are also oddball investments. Before we get to NPM exactly, I want to talk about a few others. The first one is not actually a VC investment itself, but it is a total curveball that caused a lot of waves. Let’s talk about 37signals and Jeff Bezos. In many ways, 37signals is the anti-VC. A boostrapped, incredibly profitable company. So why take money from an investor? Well, money isn’t the only things investors have to offer. From <a href="http://signalvnoise.com/archives2/bezos_expeditions_invests_in_37signals.php">the announcement</a>:</p>
<blockquote>
<p>Since we launched Basecamp we’ve been contacted by nearly 30 different VC firms. We’ve never been interested in the typical traditional VC deal. With a few exceptions, all the VCs could offer us was cash and connections. We’re fine on both of those fronts. We don’t need their money to run the business and our little black book is full. We’re looking for something else.What we’ve been looking for is the wisdom of a very special entrepreneur who’s been through what we’re going through. Someone who sees things a little differently and makes us feel right at home. Someone with a long term outlook, not a build-to-flip mentality. We found a perfect match in Jeff. Jeff is our kinda guy.</p>
</blockquote>
<p>37signals themselves have said many, many times that they would never sell the company. So where’s Bezos going to get his liquidity event from? Jeff doesn’t actually need Basecamp (which 37signals abruptly re-branded to last week, so let’s abruptly re-brand them here) to experience a liquidity event to get what he wants out of this investment. Bezos <a href="http://www.crunchbase.com/financial-organization/bezos-expeditions">invests in all kinds of things</a>. And, as Basecamp mentions, his advice and connections are invaluable.</p>
<p>The next two investments that fall in this category of “force multiplier for other things I’m doing that I don’t have to worry about getting my money back directly” are both from Andreessen Horowitz. a16z is in my opinion the most progressive and interesting VC firm in the valley. They make a lot of weird, oddball, long-shot investments, and they tend to pay off. There are two (and a half) that directly fall into the category of investment I’m talking about in this post, and that’s GitHub and Meteor (and CoinBase). Let’s talk Meteor first.</p>
<p><a href="https://www.meteor.com/blog/2012/07/25/meteors-new-112-million-development-budget">Meteor Development Group got $11.2MM in a round led by a16z</a>. While MDG claims they’ll eventually release some sort of commercial product, I don’t think that’s what really matters. If Meteor is the new Rails, then a16z gets the most inside track possible to that entire new ecosystem. And given that most VC investments fail anyway, this is a really good deal for a16z. They’re purchasing a lottery ticket, but it’s more than that: it’s a lottery ticket that makes future lottery tickets worth more money. They can invest in future Meteor-based startups, knowing that Meteor development will continue for a long time. They’ll always have their nose to the ground on any promising companies built around Meteor, earlier than everyone else. Knowledge and information is power.</p>
<p>Secondly is GitHub. <a href="http://peter.a16z.com/2012/07/09/software-eats-software-development/">GitHub raised $100MM from a16z, and a16z alone</a>. At the time, I was super mad, because I said exactly what everyone has been saying about the NPM investment. As time goes on, though, I don’t think that GitHub <em>needs</em> to experience a liquidity event. While we don’t know the valuation of GitHub, I <em>do</em> know that an acquisition doesn’t make any sense, and I’m increasingly of the opinion that an IPO doesn’t either. This is based primarily on subjective opinions that I’ve gleaned from being friends with so many GitHubbers over the years, and I don’t think an IPO is <em>impossible</em>, but unlikely. So why invest?</p>
<p><a href="http://online.wsj.com/news/articles/SB10001424053111903480904576512250915629460">a16z famously believes that software is eating the world</a>. They even reference this article in the title of the GitHub announcement. Investing in GitHub makes a ton of sense when viewed through this lens. In fact, if I had to guess, I’d bet that a16z got a small enough chunk that GitHub’s valuation is <em>insane</em>. (And I try not to use that word.) This means that a16z and GitHub management are protected from a acquisition, and can retain control of GitHub. Why does that matter? Well:</p>
<p>“Scumbag Steve image saying”Open source infrastructure should be free" / “pushes all code to GitHub”" Apparently memegenerator got hacked, and it was giving this page a warning.</p>
<p>Increasingly, GitHub is where all the open source code is hosted. And open source is the way business is going. And GitHub makes money off of the code that’s not open. In many ways, the open source stuff is a loss leader, so that you get hooked on the GitHub interface and then use it for everything, even your closed source code. This is a great business strategy, but it also places GitHub at the center of this new universe. So for similar reasons, a16z needs GitHub to be awesome to bootstrap <em>every</em> open source ecosystem that will exist into the future. It’s the infrastructure for the infrastructure. The primordial ooze of the 10x return. And a16z has the money to ‘throw away’ on something they won’t get a direct return out of, because they’re smart enough to invest some of their fund in ecosystem development.</p>
<p>The extra half-investment I’m talking about is CoinBase. <a href="http://techcrunch.com/2013/12/12/coinbase-raises-25m-from-andreessen-horowitz-to-build-its-bitcoin-wallet-and-merchant-services/">They got $25MM from a16z and others</a>, and <a href="http://dealbook.nytimes.com/2014/01/21/why-bitcoin-matters/">Andreesen himself has written about BitCoin</a>. By way of analogy:</p>
<p>“Software will eat the world” : GitHub :: “Why Bitcoin Matters” : CoinBase</p>
<p>I only give them a half in this case because it’s not clear to me that CoinBase will be dominant in the way that GitHub is dominant, and that Bitcoin will build a new payments ecosystem the way that open source has built a new ecosystem. But <em>some</em> entity like CoinBase is necessary for Bitcoin to succeed. More on that in a future post, someday.</p>
<h2>npm?</h2>
<p>So by now, I hope it’s clear how I view this NPM investment: True Ventures puts in money because the ecosystem around Node.js is still young, and a package manager is one of the biggest parts that <em>requires</em> money. Someone has to pay that S3 (I guess CouchDB) and EC2 bills. So they bootstrap the ecosystem, and also give a multiplier on their future investments. They get insight and access at unprecedented levels into the future of Node, at the cost of something they can already afford to lose.</p>
<p>We’ll see if I’m right in ten years.</p></content:encoded>
</item>
<item>
<title>Announcing Emoji 1.0</title>
<link>https://steveklabnik.com/writing/announcing-emoji-1-0/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/announcing-emoji-1-0/</guid>
<pubDate>Mon, 10 Feb 2014 00:00:00 GMT</pubDate>
<content:encoded><p>A long time ago, I wanted an emoji gem. So I set out to make one. Turns out, emoji are damn complex. <a href="http://words.steveklabnik.com/emoji-licensing">Emoji Licensing</a> was one of the results. Who’d have guessed the hard part of making a gem was the legal part! Next up was that the ‘emoji’ gem was waaaay out of date: it was a library for converting between Japanese telecoms’ emoji, which was important years ago, but not now that they’re just UTF-8. So I contacted the owner, who graciously gave me the gem.</p>
<p>Well, after doing the legal and social work, it turns out that some other people were interested in the idea as well! In the end, <a href="https://github.com/steveklabnik/emoji/commits?author=steveklabnik">I didn’t write very much actual code</a>, and <a href="https://github.com/wpeterson">Winfield</a> really stepped up and made the code side happen.</p>
<p>So today, I’m very happy to have just released version 1.0 of the emoji gem, and to officially hand off maintainership to Winfield. Thanks for all your hard work!</p>
<p>Install it with</p>
<pre><code>$ gem install emoji
</code></pre>
<p>And find the source on GitHub: <a href="https://github.com/steveklabnik/emoji">https://github.com/steveklabnik/emoji</a></p></content:encoded>
</item>
<item>
<title>A 30 minute introduction to Rust</title>
<link>https://steveklabnik.com/writing/a-30-minute-introduction-to-rust/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/a-30-minute-introduction-to-rust/</guid>
<pubDate>Sat, 11 Jan 2014 00:00:00 GMT</pubDate>
<content:encoded><p>I recently gave a <a href="https://air.mozilla.org/rust-meetup-december-2013/">proposal for Rust’s documentation</a>. An important component of my overall proposal is a short, simple introduction for people who may have heard of Rust, so that they can figure out if Rust is right for them. The other day, I saw <a href="http://www.youtube.com/watch?v=gfCtbGiHcg0">this excellent presentation</a>, and thought it might serve as a great basis for this introduction. Consider this an RFC for such an introduction. Feedback very welcome on <a href="https://mail.mozilla.org/pipermail/rust-dev/2014-January/007903.html">the rust-dev mailing list</a> or <a href="https://twitter.com/steveklabnik">on Twitter</a>.</p>
<h2>This tutorial has now <a href="http://static.rust-lang.org/doc/master/intro.html">become the official one</a>.</h2>
<p>Rust is a systems programming language that focuses on strong compile-time correctness guarantees. It improves upon the ideas other systems languages like C++, D, and Cyclone by providing very strong guarantees and explicit control over the life cycle of memory. Strong memory guarantees make writing correct concurrent Rust code easier than in other languages. This might sound very complex, but it’s easier than it sounds! This tutorial will give you an idea of what Rust is like in about thirty minutes. It expects that you’re at least vaguely familiar with a previous ‘curly brace’ language. The concepts are more important than the syntax, so don’t worry if you don’t get every last detail: the <a href="http://static.rust-lang.org/doc/master/tutorial.html">tutorial</a> can help you out with that later.</p>
<p>Let’s talk about the most important concept in Rust, “ownership,” and its implications on a task that programmers usually find very difficult: concurrency.</p>
<h2>Ownership</h2>
<p>Ownership is central to Rust, and is one of its more interesting and unique features. “Ownership” refers to which parts of your code are allowed to modify various parts of memory. Let’s start by looking at some C++ code:</p>
<pre><code>int *dangling(void)
{
int i = 1234;
return &amp;i;
}
int add_one(void)
{
int *num = dangling();
return *num + 1;
}
</code></pre>
<p>This function allocates an integer on the stack, and stores it in a variable, <code>i</code>. It then returns a reference to the variable <code>i</code>. There’s just one problem: stack memory becomes invalid when the function returns. This means that in the second line of <code>add_one</code>, <code>num</code> points to some garbage values, and we won’t get the effect that we want. While this is a trivial example, it can happen quite often in C++ code. There’s a similar problem when memory on the heap is allocated with <code>malloc</code> (or <code>new</code>), then freed with <code>free</code> (or <code>delete</code>), yet your code attempts to do something with the pointer to that memory. More modern C++ uses RAII with constructors/destructors, but it amounts to the same thing. This problem is called a ‘dangling pointer,’ and it’s not possible to write Rust code that has it. Let’s try:</p>
<pre><code>fn dangling() -&gt; &amp;int {
let i = 1234;
return &amp;i;
}
fn add_one() -&gt; int {
let num = dangling();
return *num + 1;
}
</code></pre>
<p>When you try to compile this program, you’ll get an interesting (and long) error message:</p>
<pre><code>temp.rs:3:11: 3:13 error: borrowed value does not live long enough
temp.rs:3 return &amp;i;
temp.rs:1:22: 4:1 note: borrowed pointer must be valid for the anonymous lifetime #1 defined on the block at 1:22...
temp.rs:1 fn dangling() -&gt; &amp;int {
temp.rs:2 let i = 1234;
temp.rs:3 return &amp;i;
temp.rs:4 }
temp.rs:1:22: 4:1 note: ...but borrowed value is only valid for the block at 1:22
temp.rs:1 fn dangling() -&gt; &amp;int {
temp.rs:2 let i = 1234;
temp.rs:3 return &amp;i;
temp.rs:4 }
error: aborting due to previous error
</code></pre>
<p>In order to fully understand this error message, we need to talk about what it means to “own” something. So for now, let’s just accept that Rust will not allow us to write code with a dangling pointer, and we’ll come back to this code once we understand ownership.</p>
<p>Let’s forget about programming for a second and talk about books. I like to read physical books, and sometimes I really like one and tell my friends they should read it. While I’m reading my book, I own it: the book is in my possession. When I loan the book out to someone else for a while, they “borrow” it from me. And when you borrow a book, it’s yours for a certain period of time, and then you give it back to me, and I own it again. Right?</p>
<p>This concept applies directly to Rust code as well: some code “owns” a particular pointer to memory. It’s the sole owner of that pointer. It can also lend that memory out to some other code for a while: the code “borrows” it. It borrows it for a certain period of time, called a “lifetime.”</p>
<p>That’s all there is to it. That doesn’t seem so hard, right? Let’s go back to that error message: <code>error: borrowed value does not live long enough</code>. We tried to loan out a particular variable, <code>i</code>, using Rust’s borrowed pointers: the <code>&amp;</code>. But Rust knew that the variable would be invalid after the function returns, and so it tells us that: <code>borrowed pointer must be valid for the anonymous lifetime #1... but borrowed value is only valid for the block</code>. Neat!</p>
<p>That’s a great example for stack memory, but what about heap memory? Rust has a second kind of pointer, a ‘unique’ pointer, that you can create with a <code>~</code>. Check it out:</p>
<pre><code>fn dangling() -&gt; ~int {
let i = ~1234;
return i;
}
fn add_one() -&gt; int {
let num = dangling();
return *num + 1;
}
</code></pre>
<p>This code will successfully compile. Note that instead of a stack allocated <code>1234</code>, we use an owned pointer to that value instead: <code>~1234</code>. You can roughly compare these two lines:</p>
<pre><code>// rust
let i = ~1234;
// C++
int *i = new int;
*i = 1234;
</code></pre>
<p>Rust is able to infer the size of the type, then allocates the correct amount of memory and sets it to the value you asked for. This means that it’s impossible to allocate uninitialized memory: Rust does not have the concept of null. Hooray! There’s one other difference between this line of Rust and the C++: The Rust compiler also figures out the lifetime of <code>i</code>, and then inserts a corresponding <code>free</code> call after it’s invalid, like a destructor in C++. You get all of the benefits of manually allocated heap memory without having to do all the bookkeeping yourself. Furthermore, all of this checking is done at compile time, so there’s no runtime overhead. You’ll get (basically) the exact same code that you’d get if you wrote the correct C++, but it’s impossible to write the incorrect version, thanks to the compiler.</p>
<p>You’ve seen one way that ownership and lifetimes are useful to prevent code that would normally be dangerous in a less-strict language, but let’s talk about another: concurrency.</p>
<h2>Concurrency</h2>
<p>Concurrency is an incredibly hot topic in the software world right now. It’s always been an interesting area of study for computer scientists, but as usage of the Internet explodes, people are looking to improve the number of users a given service can handle. Concurrency is one way of achieving this goal. There is a pretty big drawback to concurrent code, though: it can be hard to reason about, because it is non-deterministic. There are a few different approaches to writing good concurrent code, but let’s talk about how Rust’s notions of ownership and lifetimes can assist with achieving correct but concurrent code.</p>
<p>First, let’s go over a simple concurrency example in Rust. Rust allows you to spin up ‘tasks,’ which are lightweight, ‘green’ threads. These tasks do not have any shared memory, and so, we communicate between tasks with a ‘channel’. Like this:</p>
<pre><code>fn main() {
let numbers = [1,2,3];
let (port, chan) = Chan::new();
chan.send(numbers);
do spawn {
let numbers = port.recv();
println!("{:d}", numbers[0]);
}
}
</code></pre>
<p>In this example, we create a vector of numbers. We then make a new <code>Chan</code>, which is the name of the package Rust implements channels with. This returns two different ends of the channel: a channel and a port. You send data into the channel end, and it comes out the port end. The <code>spawn</code> function spins up a new task. As you can see in the code, we call <code>port.recv()</code> (short for ‘receive’) inside of the new task, and we call <code>chan.send()</code> outside, passing in our vector. We then print the first element of the vector.</p>
<p>This works out because Rust copies the vector when it is sent through the channel. That way, if it were mutable, there wouldn’t be a race condition. However, if we’re making a lot of tasks, or if our data is very large, making a copy for each task inflates our memory usage with no real benefit.</p>
<p>Enter Arc. Arc stands for ‘atomically reference counted,’ and it’s a way to share immutable data between multiple tasks. Here’s some code:</p>
<pre><code>extern mod extra;
use extra::arc::Arc;
fn main() {
let numbers = [1,2,3];
let numbers_arc = Arc::new(numbers);
for num in range(0, 3) {
let (port, chan) = Chan::new();
chan.send(numbers_arc.clone());
do spawn {
let local_arc = port.recv();
let task_numbers = local_arc.get();
println!("{:d}", task_numbers[num]);
}
}
}
</code></pre>
<p>This is very similar to the code we had before, except now we loop three times, making three tasks, and sending an <code>Arc</code> between them. <code>Arc::new</code> creates a new Arc, <code>.clone()</code> makes a new reference to that Arc, and <code>.get()</code> gets the value out of the Arc. So we make a new reference for each task, send that reference down the channel, and then use the reference to print out a number. Now we’re not copying our vector.</p>
<p>Arcs are great for immutable data, but what about mutable data? Shared mutable state is the bane of the concurrent programmer. You can use a mutex to protect shared mutable state, but if you forget to acquire the mutex, bad things can happen.</p>
<p>Rust provides a tool for shared mutable state: <code>RWArc</code>. This variant of an Arc allows the contents of the Arc to be mutated. Check it out:</p>
<pre><code>extern mod extra;
use extra::arc::RWArc;
fn main() {
let numbers = [1,2,3];
let numbers_arc = RWArc::new(numbers);
for num in range(0, 3) {
let (port, chan) = Chan::new();
chan.send(numbers_arc.clone());
do spawn {
let local_arc = port.recv();
local_arc.write(|nums| {
nums[num] += 1
});
local_arc.read(|nums| {
println!("{:d}", nums[num]);
})
}
}
}
</code></pre>
<p>We now use the <code>RWArc</code> package to get a read/write Arc. The read/write Arc has a slightly different API than <code>Arc</code>: <code>read</code> and <code>write</code> allow you to, well, read and write the data. They both take closures as arguments, and the read/write Arc will, in the case of write, acquire a mutex, and then pass the data to this closure. After the closure does its thing, the mutex is released.</p>
<p>You can see how this makes it impossible to mutate the state without remembering to aquire the lock. We gain the efficiency of shared mutable state, while retaining the safety of disallowing shared mutable state.</p>
<p>But wait, how is that possible? We can’t both allow and disallow mutable state. What gives?</p>
<h2>A footnote: unsafe</h2>
<p>So, the Rust language does not allow for shared mutable state, yet I just showed you some code that has it. How’s this possible? The answer: <code>unsafe</code>.</p>
<p>You see, while the Rust compiler is very smart, and saves you from making mistakes you might normally make, it’s not an artificial intelligence. Because we’re smarter than the compiler, sometimes, we need to over-ride this safe behavior. For this purpose, Rust has an <code>unsafe</code> keyword. Within an <code>unsafe</code> block, Rust turns off many of its safety checks. If something bad happens to your program, you only have to audit what you’ve done inside <code>unsafe</code>, and not the entire program itself.</p>
<p>If one of the major goals of Rust was safety, why allow that safety to be turned off? Well, there are really only three main reasons to do it: interfacing with external code, such as doing FFI into a C library, performance (in certain cases), and to provide a safe abstraction around operations that normally would not be safe. Our Arcs are an example of this last purpose. We can safely hand out multiple references to the <code>Arc</code>, because we are sure the data is immutable, and therefore it is safe to share. We can hand out multiple references to the <code>RWArc</code>, because we know that we’ve wrapped the data in a mutex, and therefore it is safe to share. But the Rust compiler can’t know that we’ve made these choices, so <em>inside</em> the implementation of the Arcs, we use <code>unsafe</code> blocks to do (normally) dangerous things. But we expose a safe interface, which means that the Arcs are impossible to use incorrectly.</p>
<p>This is how Rust’s type system allows you to not make some of the mistakes that make concurrent programming difficult, yet get the efficiency of languages such as C++.</p>
<h2>That’s all, folks</h2>
<p>I hope that this taste of Rust has given you an idea if Rust is the right language for you. If that’s true, I encourage you to check out <a href="http://static.rust-lang.org/doc/0.9/tutorial.html">the tutorial</a> for a full, in-depth exploration of Rust’s syntax and concepts.</p></content:encoded>
</item>
<item>
<title>2013: year in review</title>
<link>https://steveklabnik.com/writing/2013--year-in-review/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/2013--year-in-review/</guid>
<pubDate>Tue, 31 Dec 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Every year I try to do a “year in Review” blog post, and every year I realize I didn’t keep good notes. Sigh.</p>
<p>Overall, this was a pretty… meh year, to be honest. :/ Don’t get me wrong, a lot of really, really good things happened, but as far as my grand strategic life goals go, this year was pretty much treading water, and in some cases, regressive. Oh well.</p>
<p>So instead of doing as much of a review, I’m going to instead discuss what I’d like 2014 to be like:</p>
<h3>Better tracking</h3>
<p>In general, I don’t think I accomplished all I wanted to because I didn’t measure it at all. I’ve found that <a href="http://beeminder.com/">Beeminder</a> has helped me track one or two of the things that I’ve really wanted to accomplish, and so I plan on making lots of use of that. It’s not perfect for everything, but it helps.</p>
<h3>Reading</h3>
<p>I have so much I need to read. I think I’m going to start by recording chapters in Beeminder; books are too granular. I don’t think that a book every two weeks is an unreasonable goal, and I may turn it up if I get in the swing of things.</p>
<h3>Writing</h3>
<p>I really need to finish off the hypermedia book, and the Rails 4 in Action book. R4IA should be easy, the hypermedia book will be harder.</p>
<h3>Software</h3>
<p>I want to contribute to Tor this year. I’ve been putting off contributing to Tor due to my paranoia about NSA stuff, but really, at this point, there’s nothing to worry about. I’m probably on all of those lists anyway. They’re watching everyone.</p>
<p>In order to do that, I need to up my crypto game. I’d like to finish off the Matasano Crypto Challenge, and read a bunch more things.</p>
<p>I also want to make lots of contributions to Rust this year, if not the language itself, the ecosystem. There’s so much work to be done.</p>
<h3>Body</h3>
<p>I’ve been trying to stay relatively fit, and it’s been okay, but I really need to get back on that train. I got a copy of “Starting Strength” for Christmas.</p>
<h3>Languages</h3>
<p>I’m consistently embarassed that I only speak one language. I’ve been working on some German, and it’s been okay, but I need to take it more seriously. I’d also like to pick up the basic Japanese I had before.</p>
<h3>Conclusion</h3>
<p>I think that’s it, at least for now. Let’s hope this next year is a good one.</p></content:encoded>
</item>
<item>
<title>Rust is surprisingly expressive</title>
<link>https://steveklabnik.com/writing/rust-is-surprisingly-expressive/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rust-is-surprisingly-expressive/</guid>
<pubDate>Sat, 28 Dec 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Do you remember the first time you saw Rails’ ActiveSupport library? I do: it totally blew my mind. The only dynamic language I had used up to that point was Perl, and it had a similarly mind-blowing effect on my young self. I thought that dynamic typing was mostly a requirement towards making this kind of expressiveness possible, but it turns out Rust can be just as expressive, while retaining type safety and static dispatch.</p>
<p>There was a time in the early days of Rails when its evangelists would show off snippets of what it let you do. These were often ActiveSupport extensions to the core language itself. It’d go something like this:</p>
<blockquote>
<p>Hey, have you tried Rails? Check this out:</p>
</blockquote>
<pre><code>irb(main):002:0&gt; Time.now - 2.days
=&gt; 2009-12-26 09:57:02 -0800
irb(main):003:0&gt; 2.days.ago
=&gt; 2009-12-26 09:57:04 -0800
</code></pre>
<blockquote>
<p>Did I just blow your mind???</p>
</blockquote>
<p>At the time, I enthusiastically replied “Yes, yes you did!” and immediately typed <code>rails new foo</code>. And all was well and good in the world.</p>
<p>Fast forward a few years. I now know that static typing is not equivalent to <code>public static void main(String [] args) {</code>. I’ve seen the <a href="http://www.cafepress.com/mf/2498088/what-part-of-types-dont-you-understand_tshirt?shop=skicalc&amp;productId=6225368">Hindley-Milner</a> light. Yet, for all practical purposes, I’ve still been hacking in Ruby for these past few years. For the most part, I’d found the complaints about dynamic typing from the static typing camp to be very FUD-y. I very rarely get <code>TypeError</code>s in my Ruby code. Refactoring generally isn’t very difficult.</p>
<p>But slowly, this has started to change. I think that it’s that I tend to do more infrastructure and plumbing work now. I’ve run into more and more situations where types would be helpful, or picking the mutable state path has caused headaches.</p>
<p>Then, I wake up to an IM from <a href="http://twitter.com/wycats">Yehuda</a>:</p>
<blockquote>
<p>bro, did you see my tweets?</p>
</blockquote>
<p>Basically, this:</p>
<pre><code>➜ active_support git:(master) cat main.rs
extern mod active_support;
use active_support::Period;
use active_support::Time;
fn main() {
let time = Time::now();
println!("{:?}", time);
println!("{:?}", 2.days().from_now());
println!("{:?}", 2.weeks().from_now());
println!("{:?}", 2.months().from_now());
println!("{:?}", 2.years().from_now());
}
➜ active_support git:(master) ./main
active_support::time::Time{date: active_support::date::Date{year: 2013, month: 12u, day: 28u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 242647081u}
active_support::time::Time{date: active_support::date::Date{year: 2013, month: 12u, day: 30u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243287552u}
active_support::time::Time{date: active_support::date::Date{year: 2014, month: 1u, day: 11u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243347757u}
active_support::time::Time{date: active_support::date::Date{year: 2013, month: 2u, day: 28u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243388962u}
active_support::time::Time{date: active_support::date::Date{year: 2015, month: 12u, day: 28u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243427393u}
</code></pre>
<p>Whaaaat? Yup. To compare side-by-side:</p>
<pre><code># Ruby
2.days.from_now
// Rust
2.days().from_now()
</code></pre>
<p>Awesome. Please note that in Rust, this is fully type-checked, safe, and <em>statically dispatched</em>. No slow reflection techniques used here!</p>
<p>Now, I should say that I almost <em>never</em> use <code>2.days</code>, but the point is that it’s a benchmark of expressiveness: if you can write it, you can do all sorts of other flexible things. Thinking back to my ‘libraries vs. application code’ confusion over types, I’m reminded by something <a href="https://twitter.com/garybernhardt">Gary Bernhardt</a> once said about Ruby vs. Python. I’m paraphrasing, but as I remember it:</p>
<blockquote>
<p>Python is a beautiful, clean language. But the same restrictions that make it nice and clean mean that it’s hard to write beautiful, clean libraries. Ruby, on the other hand, is a complicated, ugly language. But that complexity allows you to write really clean, nice, easy-to-use libraries. The end result is that I prefer Python the language, but I find myself preferring programming with Ruby.</p>
</blockquote>
<p>I’m starting to wonder if maybe I feel the same way about static vs. dynamic typing: I like the simplicity and the speed of programming in a dynamic language, but maybe, given the right static language, I’ll end up preferring to program in $LANGUAGE better.</p>
<p>And that language might just be Rust.</p>
<hr />
<p>You can check out Yehuda’s <a href="https://github.com/wycats/rust-activesupport">rust-activesupport on GitHub</a>, if you’d like to see how this is accomplished. He’s also told me that he will write a post explaining its implementation in technical detail, I’ll link to it here when he’s written it.</p></content:encoded>
</item>
<item>
<title>How Dogecoin changed my perspective on cryptocurrency</title>
<link>https://steveklabnik.com/writing/how-dogecoin-changed-my-perspective-on-cryptocurrency/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/how-dogecoin-changed-my-perspective-on-cryptocurrency/</guid>
<pubDate>Sat, 21 Dec 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I have long been a Bitcoin hater, for many reasons. But then, a fortnight ago, something happened. <a href="http://dogecoin.com/">Dogecoin</a> came into existence, and for some reason, I just couldn’t get mad about it. After some thoughtful self-evaluation this week, I’ve realized that Dogecoin has actually reversed my position on cryptocurrency. Here’s why: basically, I’ve come to see Dogecoin as an absurdist art project, mocking the entire global financial system.</p>
<blockquote>
<p>Investment Disclaimer: This webpage is provided for general information only and nothing contained in the material constitutes a recommendation for the purchase or sale of any security. Although the statements of fact in this report are obtained from sources that I consider reliable, I do not guarantee their accuracy and any such information may be incomplete or condensed. Also views expressed in this Report are based on research materials available from sources considered reliable. Views are subject to change on the basis of additional or new research, new facts or developments. The investment risks described herein are not purported to be exhaustive, any person considering an investment should seek independent advice on the suitability or otherwise of the particular investment. Investment products are not insured by government or governmental agencies. Investment and Treasury products are subject to Investment risk, including possible loss of principal amount invested. Past performance is not indicative of future results: prices can go up or down. Investors investing in investments and/or treasury products denominated in foreign (non-local) currency should be aware of the risk of exchange rate fluctuations that may cause loss of principal when foreign currency is converted to the investors home currency. I personally only hold about 40,000 DOGE.</p>
</blockquote>
<p>Okay, maybe I’m taking something that’s silly way too seriously. Do I really believe that sentence? This question was key for my own understanding of Dogecoin and where it fits into the world. It’s the first question many people ask when they first hear about Dogecoin:</p>
<blockquote>
<p>Wait. Is Dogecoin a real thing?</p>
</blockquote>
<p>— horse &gt;&gt;= js (@jcoglan) December 18, 2013</p>
<blockquote></blockquote>
<p>Searching for <a href="https://twitter.com/search?q=is%20dogecoin%20real&amp;src=typd&amp;f=realtime">“is dogecoin real” on Twitter</a> reveals a lot of people who harbor the same confusion. Here’s the thing: if you are confused if Dogecoin is ‘real’ or not, you’re basically on a wild ride. Critically investigating the ‘realness’ of Dogecoin leads you to realize that it’s basically just as ‘real’ as ‘real money,’ whatever that means. And if Dogecoin is absurd, then so is the entire financial system.</p>
<blockquote>
<p>Dogecoin as symptom of the postmodern condition.</p>
</blockquote>
<p>— Steve Klabnik (@steveklabnik) December 19, 2013</p>
<blockquote></blockquote>
<p>Wait, let’s back up here. What do I mean when I say Dogecoin is ‘just as real’ as, let’s say, Euro? And in economics, most questions end up leading to “What is the meaning of ‘value’?” It’s one of the most fundamental questions of the field. And there are many different positions. For the purposes of this post, I’m going to explain things from the perspective of the “subjective theory of value,” which is arguably the most dominant explanation of value in the field today. I’m painting this concept with a pretty wide brush. Please note two things: I’m not an economist by training, I just enjoy reading lots about it, and I don’t subscribe to the subjective theory of value. That said, I am trying to represent it as accurately as possible in this post.</p>
<p>The simplest formulation of the subjective theory of value is this: something has whatever value a person says it has. Put another way, my coffee cost $3 because a bunch of people all agree that it’s worth $3. I, as an individual, have goals that I want to accomplish. In this case, it’s to write this blog post. To accomplish this, I made a rational decision: make some coffee at home, or go to the coffee shop. I have various resources that allow me to accomplish this goal, namely, that I have three United States dollars. So I take all the information that’s available to me, and make a decision: three dollars is a pretty small amount of money to me (I’m lucky in this regard), I don’t have anything else I need to do at home, the coffee shop hasn’t changed its prices at all in the last week, so I can be pretty sure it’s still $3. So I decide that getting out of the house is worth at least three dollars to me, and I got to exchange my 3 USD for one latte. You can see how endlessly complex this is: if I had chronic pain, for example, I may not want to walk down the steps, so that changes my decision. Maybe I needed to do laundry at home, so staying at home and drinking regular coffee makes more sense. I can’t make a latte at home, so maybe I get out of the house anyway because I’d prefer it to black coffee today.</p>
<p>Anyway, that’s it. Seems pretty simple, right? So what about Dogecoin?</p>
<p>Well, if you think about my example more, I actually have a number of commodities that I’m using to exchange with others to achieve my goal. For example, I also ate a sandwich here at the coffee shop, so I actually spent $6, and that’s partially because I didn’t have any food at home. The coffee shop I went to has good sandwiches, so I chose it over other places that may also have decent lattes. So in my example, I’m trading $6 USD for a latte and a sandwich, and for the right to sit here and use up a seat and some wifi. The woman working behind the counter is trading her time and labor for a wage paid by the company who owns the shop, and the company is trading some coffee beans, a bagel, and some other things with me, through the woman, in exchange for my money.</p>
<p>TL;DR: buying a cup of coffee is surprisingly complicated.</p>
<p>Okay, so now that you’ve seen there are way more commodities involved in this transaction than just USD, why is USD special? If this scenario played out in Paris, I would have been using Euro instead of USD. There’s no <em>inherent</em> reason that a piece of paper has a certain amount of value. There is a <em>subjective</em> reason, though: namely, that a large enough number of people agree that USD is a worthwhile commodity. They believe that for a number of reasons, including that our Army will go off and fucking kill anybody who threatens to break the spell that makes everyone agree that USD are worth your time to accept.</p>
<p>Anyway.</p>
<p>But really, that’s the issue, right? Everyone agrees that dollars are worth something, and we trade them all the time. They’re really easy to trade. I don’t know of any coffee shop near me that takes Euros, so even though 1€ is “worth more” than $1 USD, the Euro note I have in my backpack is basically useless for me to get some coffee. Economists call this property ‘liquidity,’ and it basically means ‘how easy is it for me to get what I want with this commodity.’ USD is very liquid. Euro is very not liquid, unless I deposit it with my bank, which will exchange those Euro for me for USD, which I can then trade for my coffee. Most discussions around the ‘realness’ of cryptocurrency are really arguments about its liquidity. Bitcoins aren’t real because I can’t buy anything with them. Really, liquidity is the aspect that matters to most people when it comes to basic financial transactions.</p>
<p>This brings me to <a href="https://www.mtgox.com/">MtGox</a> and <a href="https://coinbase.com/">CoinBase</a>. These two companies are the biggest things to happen in cryptocurrency, and that’s because they significantly improve the liquidity of Bitcoins. MtGox is a place where you can trade Bitcoins for USD. As soon as you have an exchange like this, your Bitcoins now become incredibly more liquid: you can always trade some BTC for USD, and then trade USD with… anything. Coinbase is an online wallet that allows you to store Bitcoins, but more importantly, it provides liquidity to the BTC market because it allows merchants to ‘accept Bitcoins’ without exposing themselves to the risk of Bitcoin’s price fluctuations (this is called ‘volatility’). As a merchant, I can integrate with Coinbase, and someone can, upon checkout, pay with their Coinbase account. It’s sort of similar, from the perspective of the buyer, as paying with your PayPal account. Anyway, here’s the crucial bit: Coinbase then instantly converts the Bitcoins to USD, and then gives the USD to the merchant. This is <em>incredibly</em> important for the future of Bitcoin. By allowing merchants to not worry about the fact that their customers are paying in BTC, it means that more of them will ‘accept Bitcoins,’ which increases Bitcoins’ liquidity in two ways: first, that they can buy more things in BTC ‘directly,’ as well as adding a healthy amount of orders on the various marketplaces, which helps stabilize the price.</p>
<p>I haven’t said the word “Dogecoin” in a while, so you’re probably wondering where I’m going with this. So here’s the thing: as long as BTC and USD can be exchanged easily, it allows for a significant opening for all other cryptocurrencies. This is because BTC is incredibly liquid when it comes to other cryptocurrencies: there are a number of exchanges that will trade whatever kind of the cambrian explosion of cryptocurrencies you want for BTC. Basically, BTC is sort of the ‘world reserve currency’ of cryptocurrencies. So as long as the link to the ‘real world currencies’ exists, they can cross the gap. Imagine a “Dogecoin VISA”, a credit card where I hold a balance in DOGE. When I go to buy something with my Dogecoin VISA, the backing bank would query the DOGE-&gt; BTC -&gt; USD rate, and then ship the merchant that many USD, and debit me that many DOGE. What if I want to trade EURO for DOGE? EURO -&gt; USD -&gt; BTC -&gt; DOGE. Done. The weakest and most tenuous link here is the BTC &lt;-&gt; USD bridge. As long as it’s maintained, cryptocurrency is liquid, and therefore, ‘real.’ Furthermore, the entire financial system can be re-built on the cryptocurrency side of the bridge, as well. Imagine a site that allowed you to “Dogecoin IPO,” listing your stock on the DSE (Dogecoin Stock Exchange). TWTR -&gt; USD -&gt; BTC -&gt; DOGE -&gt; MYCOMPANY. First step is on the <a href="https://www.google.com/finance?cid=32086821185414">NYSE</a>, second step is on MtGox, third step is on <a href="https://coinedup.com/">Coinedup</a>, fourth step is on the DSE. It all works. Here’s the truly scary thing: we heavily regulate the finance industry, because when we don’t, bad things happen. Hell, when we do, bad things happen. As long as cryptocurrency isn’t taken seriously, it too, will remain unregulated, which allows for a massive extraction of wealth. The cryptocurrency community is largely made up of people who have no idea what they’re doing, and people who <em>do</em> know what they’re doing are starting to notice. It’s all of the worst, most predatory aspects of the global capitalist system, without any consumer protection.</p>
<p>Regardless, there’s an initial ‘bootstrapping’ phase for anything, and I think Bitcoin is past that point now. It doesn’t matter if I think Bitcoin is a terrible idea, it’s going to be a thing regardless, so time to figure out what that actually means.</p>
<p>So the statement that Dogecoin poses to us is this: If things really are worth whatever anyone says they’re worth, so are memes. Do we really want to live in a world where a dumb joke about dogs that can’t talk has the same legitimacy as money? Because the more people that take Dogecoin seriously, the more serious it becomes… and the very nature of its silliness encourages us to think seriously about the nature of money and therefore value.</p>
<p>Before, I just laughed. Now, thanks to a joke, I’m scared.</p>
<hr />
<p>You can tip me in Dogecoin at this address: <code>DEJf4pmo7a87FeZP1fZAT8jPjDHnKL9r8C</code> or in Bitcoin at this address: <code>13BHd8tYA41JMuoBpLTGjyXfrnc6Xi6ma5</code></p></content:encoded>
</item>
<item>
<title>git, history modification, and libuv</title>
<link>https://steveklabnik.com/writing/git-history-modification-and-libuv/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/git-history-modification-and-libuv/</guid>
<pubDate>Sat, 30 Nov 2013 00:00:00 GMT</pubDate>
<content:encoded><p>There has been a recent kerfuffle over a <a href="https://github.com/joyent/libuv/pull/1015">pull request to libuv</a>: it was rejected, applied, reverted, and re-applied. There was some question as to the authorship of that commit, and I wanted to show you why that was, because it illustrates how git handles history.</p>
<p>At first, the pull request was closed. Then, it was committed in <a href="https://github.com/joyent/libuv/commit/47d98b64c45db8335bf7e065351e385cae32323d">47d98b6</a>. Then <a href="https://github.com/joyent/libuv/commit/804d40ee14dc0f82c482dcc8d1c41c14333fcb48">804d40ee</a> reverted that 47d98b6. But when you look at 804d40ee on GitHub, you’ll see no indication of which branch it’s on. That’s because it’s not on any branch. If you clone down libuv, and try to find the commit, you’ll see it’s not there:</p>
<pre><code>~/libuv(master)$ git log 804d40e
fatal: ambiguous argument '804d40e': unknown revision or path not in the working tree.
</code></pre>
<p>What gives?</p>
<p>Let’s make a test repository:</p>
<pre><code>$ mkdir test
$ cd test
$ git init
Initialized empty Git repository in /home/action/test/.git/
$ touch HELLO.md
$ git add HELLO.md
$ git commit -m "initial commit"
[master (root-commit) 646567b] initial commit
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 HELLO.md
</code></pre>
<p>Okay, now let’s make a second commit:</p>
<pre><code>$ touch WHATEVER.md
$ git add WHATEVER.md
$ git commit -m "adding whatever"
[master 7c232cc] adding whatever
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 WHATEVER.md
</code></pre>
<p>Actually, that commit was a mistake. Since we haven’t pushed anywhere yet, let’s just use <code>git reset --hard</code> to just throw it out.</p>
<pre><code>$ git reset --hard HEAD~1
HEAD is now at 646567b initial commit
</code></pre>
<p>But what about that commit? Where is it?</p>
<pre><code>$ git show 7c232cc
commit 7c232cceb94a2e7cdd95c526de785efe08da2325
Author: Steve Klabnik &lt;[email protected]&gt;
Date: Sat Nov 30 20:19:26 2013 +0000
adding whatever
diff --git a/WHATEVER.md b/WHATEVER.md
new file mode 100644
index 0000000..e69de29
</code></pre>
<p>It’s still in the repository. We can use <code>git reflog</code> to see our changes:</p>
<pre><code>$ git reflog
646567b HEAD@{0}: reset: moving to HEAD~1
7c232cc HEAD@{1}: commit: adding whatever
646567b HEAD@{2}: commit (initial): initial commit
</code></pre>
<p>Git calls these kinds of commits ‘unreachable’ because, while they exist in your repository, you can’t find them unless you know their SHA. They’re not connected to any tag or branch. We can use <code>git fsck</code> to find these for us automatically:</p>
<pre><code>$ git fsck --no-reflogs --unreachable
Checking object directories: 100% (256/256), done.
unreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5
unreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325
</code></pre>
<p>Some Git commands will run <code>git gc</code>, the Git garbage collector, as part of their normal operation. However, if we run <code>git gc</code> ourselves, it doesn’t look like anything happens:</p>
<pre><code>$ git gc
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (5/5), done.
Total 5 (delta 0), reused 0 (delta 0)
$ git fsck --no-reflogs --unreachable
Checking object directories: 100% (256/256), done.
Checking objects: 100% (5/5), done.
unreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5
unreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325
</code></pre>
<p>What gives? Well, <code>git gc</code> has some settings that indicate how long it will let an unreachable commit lay around in your repository. The <code>gc.pruneExpire</code> configuration variable controls this behavior, and it defaults to 14 days. So even if we try to throw away a commit, and even if we manually run the garbage collector, we’ll still have the commit for two weeks.</p>
<p>We can, of course, tell <code>git gc</code> to ignore the setting:</p>
<pre><code>$ git gc --prune=now
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (5/5), done.
Total 5 (delta 0), reused 5 (delta 0)
$ git fsck --no-reflogs --unreachable
Checking object directories: 100% (256/256), done.
Checking objects: 100% (5/5), done.
unreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5
unreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325
</code></pre>
<p>Uhhhh what? It turns out that <code>git gc</code> won’t touch commits that are still in our reflog. So let’s clear that:</p>
<pre><code>$ git reflog expire --expire=now --all
$ git reflog
$
</code></pre>
<p>Good. And now, let’s take out the garbage:</p>
<pre><code>$ git gc --prune=now
Counting objects: 3, done.
Writing objects: 100% (3/3), done.
Total 3 (delta 0), reused 3 (delta 0)
$ git fsck --no-reflogs --unreachable
Checking object directories: 100% (256/256), done.
Checking objects: 100% (3/3), done.
</code></pre>
<p>Easy! When you fetch from a remote repository, Git does not include unreachable commits. That’s why when we cloned down libuv earlier, we didn’t get the orphaned commit.</p>
<p>Anyway, as you can see, it’s actually really hard to lose your data with Git, even if you explicitly try to throw it away. This is why you can rebase with… well, maybe not reckless abandon, but you don’t have to worry about your data. All it takes is a quick <code>git reset --hard HEAD@{1}</code> (read: reset me to the last entry in the reflog) and you’re back to where you were before that operation you screwed up.</p>
<p>So, what happened with that commit on GitHub? Basically, one committer pushed 804d40ee, and then another did a <code>git reset --hard</code> to remove it, and then force pushed up to the repository. That commit still lives on GitHub, and is still viewable in their interface, but in some time, when GitHub runs <code>git gc</code> on the repository, it will go away.</p></content:encoded>
</item>
<item>
<title>Why I'm partnering with Balanced</title>
<link>https://steveklabnik.com/writing/why-im-partnering-with-balanced/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/why-im-partnering-with-balanced/</guid>
<pubDate>Thu, 21 Nov 2013 00:00:00 GMT</pubDate>
<content:encoded><p>My next venture will be joining up with <a href="https://www.balancedpayments.com/">Balanced</a>, an API for marketplace payments, and I’m really excited about it. To the tax man, of course, the title of this post is “Why I’m going to be an employee of Balanced,” but I like to think about it as a partnership.</p>
<h3>Why I Balanced</h3>
<p>I first heard of Balanced almost exactly a year ago. You see, my friend Chad had started this project called <a href="https://www.gittip.com/">Gittip</a>. He was using Stripe to handle payments, but Gittip is a marketplace, and Stripe didn’t support marketplaces. So they gave Gittip a month to find another processor. So <a href="https://github.com/gittip/www.gittip.com/pull/137">a Balanced employee came along and submitted a pull request</a>. I thought this was super awesome. So much so, that I tweeted it:</p>
<blockquote>
<p>@jordanmessina It was like ‘hmm we can't use stripe, who should we use?’ “oh hey, just swung by to integrate you with us. &lt;3”</p>
</blockquote>
<p>— Steve Klabnik (@steveklabnik) December 1, 2012</p>
<blockquote></blockquote>
<p>I still told people that story, but didn’t really think a whole lot more about Balanced than that. But every so often, I’d bump into them again. The next time Balanced came across my radar was when they <a href="http://blog.balancedpayments.com/benefits-open-company-balanced/">declared themselves an Open Company</a>. Whoah. Now, don’t get me wrong: it’s not exactly socialism, but it <em>is</em> super interesting. The idea of doing feature development and discussion out in the open is really interesting to me. For example, if you’re interested in seeing when Balanced supports international payments, you can check out <a href="https://github.com/balanced/balanced-api/issues/23">this Issue</a>, and when the feature happens, you’ll know. That feature is obvious, but for less obvious ones, like <a href="https://github.com/balanced/balanced-api/issues/204">Bitcoin support</a>, you can see that there’s real discussion, where the users who want a feature can help explain their use cases, Balanced will ask questions, and (hopefully) eventually it’ll get implemented.</p>
<p>Anyway, so when I was looking for something new, Balanced seemed like a good fit. I dropped by their offices a few times, and met with the team. I was impressed when they included their intern, Matthew, in one of our meetings. Most companies I’ve seen treat their interns really poorly. When I mentioned that I was a Python newbie, Balanced’s CTO, <a href="https://twitter.com/mahmoudimus">Mahmoud</a>, replied that that was perfect: if they only hired Python people, they’d only get Python answers, and monoculture is bad. During that discussion, I remembered my own startup, CloudFab, and remembered my cofounder’s issues getting us a bank account, because we were a marketplace. I had felt the pain Balanced is trying to solve before. At that point, I was in, but Mahmoud thanking me for my role in helping fight for equality in tech was extra icing on the cake.</p>
<h3>My role</h3>
<p>Anyway, that’s why I’m pumped about Balanced. So what exactly will I be doing? A few things, taken straight from my offer letter:</p>
<ol>
<li>Design Balanced’s API based on the needs of our customers and the feedback of the team</li>
<li>Lead Balanced’s path towards HTTP and JSON-API compliance</li>
<li>Move standards forward to support Balanced’s innovation</li>
<li>Build tools to allow Balanced and others to create standards compliant APIs</li>
<li>Promote and evangelize the benefits of the Open Company philosophy</li>
</ol>
<p>To expand on these five steps:</p>
<p>You can reach me at <a href="mailto:[email protected]">[email protected]</a>. I have uploaded <a href="http://pgp.mit.edu:11371/pks/lookup?op=vindex&amp;search=0x1782FE756ACC3706">a PGP key</a> to the MIT keyserver if that’s your jam.</p>
<p>If you’re a Balanced customer and you have thoughts, good or bad, about Balanced, please get in touch.</p>
<p>If you’re <em>not</em> a Balanced customer, I’d love to hear why. Send me an email.</p>
<p>I’ve joined the <a href="http://www.w3.org/2001/tag/">W3C TAG mailing list</a>, and will be getting even more involved in the web standards process than I already am, to make sure that Balanced is keeping abreast of the latest standards, and to make sure that Balanced’s use-cases are properly advocated for as part of that process.</p>
<p>I’ll be working hard on <a href="http://jsonapi.org/">JSON API</a>, both with Balanced’s customers, our team, and with others who want to use the standard. Some tickets have been a bit stagnant: it’s now a part of my job to resolve them.</p>
<p>I’ll be working on some tooling, hopefully with other companies that care about APIs, to build tooling that we all can use. If you’re in San Fransisco, I’d love to swing by your office and say hello. If not, email works too.</p>
<p>If you want me to come speak to your usergroup, meetup, or conference about APIs, web standards, Open Companies, or payment systems, let me know, I really enjoy it.</p>
<p>Expect to see a bunch of writing on the <a href="http://blog.balancedpayments.com/">Balanced blog</a> from me on all of those topics, as well.</p>
<p>I’m excited for the future. There’s so much work to do. Let’s all build some awesome APIs!</p></content:encoded>
</item>
<item>
<title>On leftist infighting</title>
<link>https://steveklabnik.com/writing/on-leftist-infighting/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/on-leftist-infighting/</guid>
<pubDate>Mon, 18 Nov 2013 00:00:00 GMT</pubDate>
<content:encoded><p>There used to be a post here: <a href="https://web.archive.org/web/20131118145002/http://words.steveklabnik.com/on-leftist-infighting">https://web.archive.org/web/20131118145002/http://words.steveklabnik.com/on-leftist-infighting</a></p>
<p>After having some conversations on Twitter, I was made aware of some context around ‘infighting’ that made it problematic. I decided that the post was unworkable, as I certainly didn’t mean some of the things it implied, and so decided it was better to simply delete it.</p>
<p>I’ve been told that some people have been trying to cite this post as me saying that I believe any and all dissent should not be public, or that I believe in covering up incidents of wrongdoing because they may ‘harm the movement’ or something. This is wrong. I wrote this post after trying a number of different tactics to get other men to stop tone policing several women, specifically Shanley. “If you don’t like the way she says things, why not distract from the issue at hand and instead say nothing, and focus on the misogyny instead” was one strategy that actually worked. Hence the post. But since I didn’t just straight-up come out and say that, it failed.</p>
<p>Wrongdoing should be called out as wrong, and tone policing is bad. End of story.</p></content:encoded>
</item>
<item>
<title>Abstract vs concrete</title>
<link>https://steveklabnik.com/writing/abstract-vs-concrete/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/abstract-vs-concrete/</guid>
<pubDate>Thu, 14 Nov 2013 00:00:00 GMT</pubDate>
<content:encoded><p>When I talk to people about systemic inequalities, one of the hardest things for them to understand is the concrete nature of how these relations affect the day-to-day lives of others. For many, this topic is an abstract discussion. For many others, this is life, day to day.</p>
<p>In the interest of helping bridge this gap, I want to share a story with you from a few weeks ago. The setting: the West Village, Manhattan. The night: Halloween. In the early morning hours of All Saint’s Day, I got a text message from someone I care about. Here’s some screenshots of the conversation. She has seen this post and said it’s okay to share, for the same reasons I’m interested in sharing this with you. Also, if reading a story about a bad situation or a slur or two will make you upset, you may want to read another blog post.</p>
<p>I’m not sure how to convince random people on Twitter that yes, in 2013, in the West Villiage, in the largest city in the country, on a holiday night when everyone is out, you still can’t let your guard down. Even if you’re not actually a gay man.</p>
<p>And this is just one story.</p></content:encoded>
</item>
<item>
<title>Pointers in Rust, a guide</title>
<link>https://steveklabnik.com/writing/pointers-in-rust-a-guide/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/pointers-in-rust-a-guide/</guid>
<pubDate>Fri, 18 Oct 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Rust’s pointers are one of its more unique and compelling features. Pointers are also one of the more confusing topics for newcomers to Rust. They can also be confusing for people coming from other languages that support pointers, such as C++. This tutorial will help you understand this important topic.</p>
<h1>You don’t actually need pointers</h1>
<p>I have good news for you: you probably don’t need to care about pointers, especially as you’re getting started. Think of it this way: Rust is a language that emphasizes safety. Pointers, as the joke goes, are very pointy: it’s easy to accidentally stab yourself. Therefore, Rust is made in a way such that you don’t need them very often.</p>
<p>“But tutorial!” you may cry. "My co-worker wrote a function that looks like this:</p>
<pre><code>fn succ(x: &amp;int) -&gt; int { *x + 1 }
</code></pre>
<p>So I wrote this code to try it out:</p>
<pre><code>fn main() {
let number = 5;
let succ_number = succ(number);
println!("{}", succ_number);
}
</code></pre>
<p>And now I get an error:</p>
<pre><code>error: mismatched types: expected `&amp;int` but found `&lt;VI0&gt;` (expected &amp;-ptr but found integral variable)
</code></pre>
<p>What gives? It needs a pointer! Therefore I have to use pointers!"</p>
<p>Turns out, you don’t. All you need is a reference. Try this on for size:</p>
<pre><code>fn main() {
let number = 5;
let succ_number = succ(&amp;number);
println!("{}", succ_number);
}
</code></pre>
<p>It’s that easy! One extra little <code>&amp;</code> there. This code will run, and print <code>6</code>.</p>
<p>That’s all you need to know. Your co-worker could have written the function like this:</p>
<pre><code>fn succ(x: int) -&gt; int { x + 1 }
fn main() {
let number = 5;
let succ_number = succ(number);
println!("{}", succ_number);
}
</code></pre>
<p>No pointers even needed. Then again, this is a simple example. I assume that your real-world <code>succ</code> function is more complicated, and maybe your co-worker had a good reason for <code>x</code> to be a pointer of some kind. In that case, references are your best friend. Don’t worry about it, life is too short.</p>
<p>However.</p>
<p>Here are the use-cases for pointers. I’ve prefixed them with the name of the pointer that satisfies that use-case:</p>
<ol>
<li>Owned: ~Trait must be a pointer, becuase you don’t know the size of the object, so indirection is mandatory.</li>
<li>Owned: You need a recursive data structure. These can be infinite sized, so indirection is mandatory.</li>
<li>Owned: A very, very, very rare situation in which you have a <em>huge</em> chunk of data that you wish to pass to many methods. Passing a pointer will make this more efficient. If you’re coming from another language where this technique is common, such as C++, please read “A note…” below.</li>
<li>Managed: Having only a single owner to a piece of data would be inconvenient or impossible. This is only often useful when a program is very large or very complicated. Using a managed pointer will activate Rust’s garbage collection mechanism.</li>
<li>Borrowed: You’re writing a function, and you need a pointer, but you don’t care about its ownership. If you make the argument a borrowed pointer, callers can send in whatever kind they want.</li>
</ol>
<p>Five exceptions. That’s it. Otherwise, you shouldn’t need them. Be skeptical of pointers in Rust: use them for a deliberate purpose, not just to make the compiler happy.</p>
<h2>A note for those proficient in pointers</h2>
<p>If you’re coming to Rust from a language like C or C++, you may be used to passing things by reference, or passing things by pointer. In some languages, like Java, you can’t even have objects without a pointer to them. Therefore, if you were writing this Rust code:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn main() {
let p0 = Point { x: 5, y: 10};
let p1 = transform(p0);
println!("{:?}", p1);
}
</code></pre>
<p>I think you’d implement <code>transform</code> like this:</p>
<pre><code>fn transform(p: &amp;Point) -&gt; Point {
Point { x: p.x + 1, y: p.y + 1}
}
// and change this:
let p1 = transform(&amp;p0);
</code></pre>
<p>This does work, but you don’t need to create those references! The better way to write this is simply:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn transform(p: Point) -&gt; Point {
Point { x: p.x + 1, y: p.y + 1}
}
fn main() {
let p0 = Point { x: 5, y: 10};
let p1 = transform(p0);
println!("{:?}", p1);
}
</code></pre>
<p>But won’t this be inefficent? Well, that’s a complicated question, but it’s important to know that Rust, like C and C++, store aggregate data types ‘unboxed,’ whereas languages like Java and Ruby store these types as ‘boxed.’ For smaller structs, this way will be more efficient. For larger ones, it may be less so. But don’t reach for that pointer until you must! Make sure that the struct is large enough by performing some tests before you add in the complexity of pointers.</p>
<h1>Owned Pointers</h1>
<p>Owned pointers are the conceptually simplest kind of pointer in Rust. A rough approximation of owned pointers follows:</p>
<ol>
<li>Only one owned pointer may exist to a particular place in memory. It may be borrowed from that owner, however.</li>
<li>The Rust compiler uses static analysis to determine where the pointer is in scope, and handles allocating and de-allocating that memory. Owned pointers are not garbage collected.</li>
</ol>
<p>These two properties make for three use cases.</p>
<h2>References to Traits</h2>
<p>Traits must be referenced through a pointer, becuase the struct that implements the trait may be a different size than a different struct that implements the trait. Therefore, unboxed traits don’t make any sense, and aren’t allowed.</p>
<h2>Recursive Data Structures</h2>
<p>Sometimes, you need a recursive data structure. The simplest is known as a ‘cons list’:</p>
<pre><code>enum List&lt;T&gt; {
Nil,
Cons(T, ~List&lt;T&gt;),
}
fn main() {
let list: List&lt;int&gt; = Cons(1, ~Cons(2, ~Cons(3, ~Nil)));
println!("{:?}", list);
}
</code></pre>
<p>This prints:</p>
<pre><code>Cons(1, ~Cons(2, ~Cons(3, ~Nil)))
</code></pre>
<p>The inner lists <em>must</em> be an owned pointer, becuase we can’t know how many elements are in the list. Without knowing the length, we don’t know the size, and therefore require the indirection that pointers offer.</p>
<h2>Efficiency</h2>
<p>This should almost never be a concern, but because creating an owned pointer boxes its value, it therefore makes referring to the value the size of the box. This may make passing an owned pointer to a function less expensive than passing the value itself. Don’t worry yourself with this case until you’ve proved that it’s an issue through benchmarks.</p>
<p>For example, this will work:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn main() {
let a = Point { x: 10, y: 20 };
do spawn {
println(a.x.to_str());
}
}
</code></pre>
<p>This struct is tiny, so it’s fine. If <code>Point</code> were large, this would be more efficient:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn main() {
let a = ~Point { x: 10, y: 20 };
do spawn {
println(a.x.to_str());
}
}
</code></pre>
<p>Now it’ll be copying a pointer-sized chunk of memory rather than the whole struct.</p>
<h1>Managed Pointers</h1>
<p>Managed pointers, notated by an <code>@</code>, are used when having a single owner for some data isn’t convenient or possible. This generally happens when your program is very large and complicated.</p>
<p>For example, let’s say you’re using an owned pointer, and you want to do this:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn main() {
let a = ~Point { x: 10, y: 20 };
let b = a;
println(b.x.to_str());
println(a.x.to_str());
}
</code></pre>
<p>You’ll get this error:</p>
<pre><code>test.rs:10:12: 10:13 error: use of moved value: `a`
test.rs:10 println(a.x.to_str());
^
test.rs:8:8: 8:9 note: `a` moved here because it has type `~Point`, which is moved by default (use `ref` to override)
test.rs:8 let b = a;
^
</code></pre>
<p>As the message says, owned pointers only allow for one owner at a time. When you assign <code>a</code> to <code>b</code>, <code>a</code> becomes invalid. Change your code to this, however:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn main() {
let a = @Point { x: 10, y: 20 };
let b = a;
println(b.x.to_str());
println(a.x.to_str());
}
</code></pre>
<p>And it works:</p>
<pre><code>10
10
</code></pre>
<p>So why not just use managed pointers everywhere? There are two big drawbacks to managed pointers:</p>
<ol>
<li>They activate Rust’s garbage collector. Other pointer types don’t share this drawback.</li>
<li>You cannot pass this data to another task. Shared ownership across concurrency boundaries is the source of endless pain in other languages, so Rust does not let you do this.</li>
</ol>
<h1>Borrowed Pointers</h1>
<p>Borrowed pointers are the third major kind of pointer Rust supports. They are simultaneously the simplest and the most complicated kind. Let me explain: they’re called ‘borrowed’ pointers because they claim no ownership over the data they’re pointing to. They’re just borrowing it for a while. So in that sense, they’re simple: just keep whatever ownership the data already has. For example:</p>
<pre><code>use std::num::sqrt;
struct Point {
x: float,
y: float,
}
fn compute_distance(p1: &amp;Point, p2: &amp;Point) -&gt; float {
let x_d = p1.x - p2.x;
let y_d = p1.y - p2.y;
sqrt(x_d * x_d + y_d * y_d)
}
fn main() {
let origin = @Point { x: 0.0, y: 0.0 };
let p1 = ~Point { x: 5.0, y: 3.0 };
println!("{:?}", compute_distance(origin, p1));
}
</code></pre>
<p>This prints <code>5.83095189</code>. You can see that the <code>compute_distance</code> function takes in two borrowed pointers, but we give it a managed and unique pointer. Of course, if this were a real program, we wouldn’t have any of these pointers, they’re just there to demonstrate the concepts.</p>
<p>So how is this hard? Well, because we’re igorning ownership, the compiler needs to take great care to make sure that everything is safe. Despite their complete safety, a borrowed pointer’s representation at runtime is the same as that of an ordinary pointer in a C program. They introduce zero overhead. The compiler does all safety checks at compile time.</p>
<p>This theory is called ‘region pointers,’ and involve a concept called ‘lifetimes’. Here’s the simple explanation: would you expect this code to compile?</p>
<pre><code>fn main() {
println(x.to_str());
let x = 5;
}
</code></pre>
<p>Probably not. That’s becuase you know that the name <code>x</code> is valid from where it’s declared to when it goes out of scope. In this case, that’s the end of the <code>main</code> function. So you know this code will cause an error. We call this duration a ‘lifetime’. Let’s try a more complex example:</p>
<pre><code>fn main() {
let mut x = ~5;
if(*x &lt; 10) {
let y = &amp;x;
println!("Oh no: {:?}", y);
return;
}
*x = *x - 1;
println!("Oh no: {:?}", x);
}
</code></pre>
<p>Here, we’re borrowing a pointer to <code>x</code> inside of the <code>if</code>. The compiler, however, is able to determine that that pointer will go out of scope without <code>x</code> being mutated, and therefore, lets us pass. This wouldn’t work:</p>
<pre><code>fn main() {
let mut x = ~5;
if(*x &lt; 10) {
let y = &amp;x;
*x = *x - 1;
println!("Oh no: {:?}", y);
return;
}
*x = *x - 1;
println!("Oh no: {:?}", x);
}
</code></pre>
<p>It gives this error:</p>
<pre><code>test.rs:5:8: 5:10 error: cannot assign to `*x` because it is borrowed
test.rs:5 *x = *x - 1;
^~
test.rs:4:16: 4:18 note: borrow of `*x` occurs here
test.rs:4 let y = &amp;x;
^~
</code></pre>
<p>As you might guess, this kind of analysis is complex for a human, and therefore hard for a computer, too! There is an entire <a href="http://static.rust-lang.org/doc/0.8/tutorial-borrowed-ptr.html">tutorial devoted to borrowed pointers and lifetimes</a> that goes into lifetimes in great detail, so if you want the full details, check that out.</p>
<h1>Returning Pointers</h1>
<p>We’ve talked a lot about funtions that accept various kinds of pointers, but what about returning them? Here’s the rule of thumb: only return a unique or managed pointer if you were given one in the first place.</p>
<p>What does that mean? Don’t do this:</p>
<pre><code>fn foo(x: ~int) -&gt; ~int {
return ~*x;
}
fn main() {
let x = ~5;
let y = foo(x);
}
</code></pre>
<p>Do this:</p>
<pre><code>fn foo(x: ~int) -&gt; int {
return *x;
}
fn main() {
let x = ~5;
let y = ~foo(x);
}
</code></pre>
<p>This gives you flexibility, without sacrificing performance. For example, this will also work:</p>
<pre><code>fn foo(x: ~int) -&gt; int {
return *x;
}
fn main() {
let x = ~5;
let y = @foo(x);
}
</code></pre>
<p>You may think that this gives us terrible performance: return a value and then immediately box it up?!?! Isn’t that the worst of both worlds? Rust is smarter than that. There is no copy in this code. <code>main</code> allocates enough room for the <code>@int</code>, passes it into <code>foo</code> as <code>x</code>, and then <code>foo</code> writes the value into the new box. This writes the return value directly into the allocated box.</p>
<p>This is important enough that it bears repeating: pointers are not for optimizing returning values from your code. Allow the caller to choose how they want to use your output.</p></content:encoded>
</item>
<item>
<title>New Beginnings</title>
<link>https://steveklabnik.com/writing/new-beginnings/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/new-beginnings/</guid>
<pubDate>Tue, 01 Oct 2013 00:00:00 GMT</pubDate>
<content:encoded><p>TL;DR:</p>
<blockquote>
<p>WANTED: Proletarian/petite bourgeoisie looking for capitalist/industrialist/haute bourgeoisie. I’ve got the labor power, you’ve got the money commodity. Let’s exchange commodities at full market value in accordance with the Law of Value; I know you’ll extract some surplus and alienate my species-being, but I really need to regenerate and reproduce myself so that I can be around to labor tomorrow.</p>
</blockquote>
<p>Yes, that’s right, I am officially looking for work. I loved my job at <a href="http://jumpstartlab.com/">Jumpstart Lab</a>, but all good things must come to an end. So here we are. There’s a <a href="http://jumpstartlab.com/news/archives/2013/10/01/a-fond-farewell">farewell post</a> on the JSL blog too.</p>
<p>So, what’s next? To be honest, I’m not fully sure. I’m open to suggestions, and I give bonus points for interesting.</p>
<p>Things I want out of life:</p>
<ol>
<li>Travel. I still like doing this. I just don’t want to do it <em>constantly</em>.</li>
<li>Related: some degree of location independence. I’ll only accept positions that require me to be in a seat that’s in San Francisco or New York City. Remote would be even better. Probably.</li>
<li>Piles of money. I have student loan debt I’d prefer not to have.</li>
<li>Side project ability. I am highly motivated by doing side projects that are different than I do all day.</li>
</ol>
<p>… I’m not saying that Ruby is a minus, but I’m not saying it’s a plus, either.</p>
<p>I’m also open to re-locating out of the country. That certainly counts as interesting.</p>
<p>So, if you think we might work well together, please, <a href="mailto:[email protected]">send me an email</a>.</p></content:encoded>
</item>
<item>
<title>Keep Saturdays sacred</title>
<link>https://steveklabnik.com/writing/keep-saturdays-sacred/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/keep-saturdays-sacred/</guid>
<pubDate>Sat, 28 Sep 2013 00:00:00 GMT</pubDate>
<content:encoded><p>There are obviously a lot of factors that have contributed to my career as a programmer, but I think the most important one can be summed up in this sentence: Keep Saturdays sacred.</p>
<p>You see, in college, my friends and I were moderately obsessed with writing software. We sorta stumbled onto a tradition of making sure that every Saturday, we got together and wrote code. At some point, we realized that this was the formula:</p>
<ol>
<li>If none of us set our alarms, we’d all wander into the computer lab at 1pm Saturday.</li>
<li>We then went and got some burritos from Chipotle.</li>
<li>Once we were appropriately stuffed, we’d go back to the lab and hack on whatever.</li>
<li>In our town, food was half off at certain restaurants after 11pm. So we’d code until then, and then go get some food, and then go home later.</li>
</ol>
<p>Many people sifted in and out of this group over time, but a core was <em>always</em> there. Every week. For two or three years. I was one of those people.</p>
<p>Looking back, I think that time was where I grew the most as a software developer. I’ve been coding since I was very young, but didn’t really know what I was doing until that point. It’s where I tried out new stuff. Argued about methodology. Learned something about something.</p>
<p>We weren’t even always working on the same projects. They were often quite different, and changed all the time. But it was a safe place to try things, a safe space to fail. A place to get support and help. To socialize and figure out just what the hell we were even doing.</p>
<p>It was also an incredible privilege. Over the last few years, I’ve been too busy to keep that going. Life gets in the way. It’s hard when you don’t have that group. Maybe the solution, then, isn’t that it’s all day on a Saturday. Maybe it’s just finding <em>some</em> kind of gap time, but doing it every week.</p>
<p>Today, I’m doing work on http://www.designinghypermediaapis.com/ and the Rust compiler. What about you?</p></content:encoded>
</item>
<item>
<title>Thankfulness</title>
<link>https://steveklabnik.com/writing/thankfulness/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/thankfulness/</guid>
<pubDate>Tue, 24 Sep 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I recently attended GoGaRuCo, which was a great conference. My favorite part, however, wasn’t directly attatched to the conf itself: it was a few moments of thankfulness.</p>
<p>A lot of what I’ve been experiencing lately has been profoundly negative. That is what it is; there’s a lot of realy negative things in the world. But I’ve been a bit… overloaded lately.</p>
<p>The two moments tie into each other:</p>
<p>I’m at the speaker dinner. I bumped into someone I hadn’t met or said hi to yet. “Oh hey Steve, I’m <a href="https://twitter.com/samsaffron">Sam</a>.” “Oh hi! You don’t know how many times in the last few weeks I’ve said ‘Sam Saffron makes me so happy.’” You see, Sam works on <a href="http://www.discourse.org/">Discourse</a>, and it’s a Rails app. Sam’s work on Discourse has identified some bugs and performance regressions in the Rails codebase, and Sam has been submitting some great pull requests upstream. Lately, I’ve been feeling like nobody who uses Rails actually cares how the sausage is made, and so Sam has been a bright spot in my day. So I told him so, straight to his face. It felt really good to say thanks, and I hope I made Sam’s day a bit brighter, as well.</p>
<p>A couple times at the conference, someone would walk up to me and say basically the same thing for some other reason. It felt really good to have my work appreciated; the day-to-day grind of GitHub issue triaging can really make you feel crappy. Basically:</p>
<p><img src="http://s3.amazonaws.com/theoatmeal-img/comics/making_things/14.jpg" alt="http://s3.amazonaws.com/theoatmeal-img/comics/making_things/14.jpg" /></p>
<p>negative comments</p>
<p><a href="http://theoatmeal.com/comics/making_things">The Oatmeal</a> is often problematic, but this one really rings true.</p>
<p>Anyway, I’m not trying to fish for compliments here. I have a fairly high profile, and I hear this more than most. What I <em>do</em> want to emphasize, though, is that I’m sure it’s not just me that feels this way.</p>
<p>So, if someone does something that you appreciate, please take a moment to say thank you. <em>Especially</em> if they’re not someone who happened to get up on a stage and give a talk. Be specific: a generic ‘thanks for what you do’ is nice, but ‘oh man I lost two hours on that bug and your patch saved me’ is even nicer.</p>
<p>You never know. They might have been having a bad day too, your thanks might turn their day around.</p></content:encoded>
</item>
<item>
<title>The CLOSURE companion</title>
<link>https://steveklabnik.com/writing/the-closure-companion/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-closure-companion/</guid>
<pubDate>Thu, 19 Sep 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Today is the fourth <a href="http://whyday.org/">_why day</a>. So I thought I’d say a few words about his final gift to the rest of us, <a href="http://words.steveklabnik.com/closure">CLOSURE</a>.</p>
<h2>Be Skeptical</h2>
<p>First, a few words about the words I’m about to write. Software developers (though you don’t have to be one to read this) love meta, so surely you’ll indulge me for these next few paragraphs. Don’t worry, it’s only a little bit of meta-data.</p>
<blockquote>
<p>And if you don’t understand some of my language, things like “Open GL” and “fighting NULL” and so on, you can skip it and be fine I think. - _why, CLOSURE</p>
</blockquote>
<blockquote>
<p>Nowadays people are actually somewhat jaded by the term ``postmodern’’. Well, perhaps jaded is an understatement. Nauseated might be more like it. - Larry Wall, “Perl, the first post-modern programming language”</p>
</blockquote>
<blockquote>
<p>I define postmodern as incredulity toward meta-narratives, - Lyotard, “The Postmodern Condition: a report on knowledge”</p>
</blockquote>
<p>_why’s work has always taken on a strong element of the postmodern, and CLOSURE is no different. So in order to really get CLOSURE, you need to get a little postmodern. So let’s breifly examine this statement by Lyotard.</p>
<p>A ‘narrative’, to Lyotard, is a story that organizes tribal knowledge, the structure of our life and society. Therefore, a meta-narrative is a narrative about a narrative. As an example, let’s say you want to make sense of programming languages. A meta-narrative about programming languages would be that in the beginning, we started off with assembly language. Then we moved forward to structured programming, with Pascal and C and such. Next came object oriented languages, which is where we’re mostly at today. We’re starting to finally lean functional.</p>
<p>Now, this narrative works. It’s not <em>wrong</em>. But it also isn’t really <em>true</em>, either. After all, Lisp existed waaaay before the contemporary moment, yet it’s often functional. Actually, presenting Lisp as functional is in of itself not exactly true. So this meta-narrative, while it helps us understand a bit about programming languages and their history, also has a bunch of lies. It’s not black and white, it’s quite gray.</p>
<blockquote>
<p>"Well, it doesn’t matter – no one’s going to listen to either of us. I can say whatever I want at this point. You can tell the truth and no one would care.“That’s amazing.”“I’m going to change my story weekly – the more confusion the better. The Jonathan story is locked.”_why, CLOSURE, HELLOYES (p56)</p>
</blockquote>
<p>So what I’m saying is this: I’m about to present you with a meta-narrative about CLOSURE. So be skeptical of me! What I’m about to tell you is a lie, I swear. Don’t take what’s about to come as the final word, some sort of actual truth. It’s just my own little anti-memory that I’d like to share with you.</p>
<h2>What CLOSURE is</h2>
<p>CLOSURE is a few different things. 90% of what you need to know is located in <a href="https://github.com/steveklabnik/CLOSURE/blob/master/PDF/DISCLAIMER.pdf">DISCLAIMER</a>, which is page 13 of CLOSURE. In it, _why mentions a few things:</p>
<ol>
<li>It’s ‘perilous to communicate this way’</li>
<li>Everything he’s done belongs to us</li>
<li>He likes what we’ve done so far</li>
<li>This will be the last thing from him.</li>
</ol>
<p>#4 is a big deal. No more waiting for _why to come back. That’s it. No more questions, no more wondering, no more guesses. This is all we have.</p>
<p>For me, #3 was the most important. It brought me to tears, to be honest. That’s what I personally really needed to hear. It’s very strange to take up the work of someone you’ve never met, and make their life’s work your own. I spent quite a bit of time really wondering if keeping some of his stuff alive was the correct thing to do, and so this was a big relief. It was all gravy after that. #2 is related, it’s all public domain. That’s good to know.</p>
<h2>Identity</h2>
<p>#1 is interesting. It could just be flavor, but I think it’s something more than that. One of the major themes of CLOSURE is identity, and _why / Jonathan ’s struggles with it. On page 19:</p>
<blockquote>
<p>Now, could you please tell me the point of this ridiculous anonymity exercise, hmm?As it turns out, oddly enough, your real self is just an unknown programmer from Utah. The myth is that easily dispelled. Why not making something of your real self? (Of course I know why and can tell you: Because your fear of the world has clouded your ability to do things to improve your situation. You are stuck there in Draper, Utah, until you can cut through the paranoia!)Please, Mr. Gillette, come on in. The water’s fine. ;)Emery Pestus, CLOSURE</p>
</blockquote>
<p>While many hold _why as some kind of superhero, it’s really important to remember that he’s human. We, the Ruby community, placed an indescribable amount of pressure on a human being to play a character for us, without consideration for his feelings. Think about this for a moment: could Jonathan ever write Ruby code? As Jonathan? What if it had unit tests? If it was just some old, normal, boring gem that added a little feature to Rails?</p>
<p>Could we let him just be him? I’m not sure. I’m not sure he could let him just be him, either.</p>
<p>Once, I met someone for dinner in Chicago. This was the first time we’d met, and so we were asking each other a lot of questions. One of my answers yielded an interesting response: “Oh, you’re like actually for-real. All that twitter stuff isn’t an act. That’s really you.”</p>
<h2>Impermanance</h2>
<p>Impermanence is possibly the biggest question raised in CLOSURE.</p>
<blockquote>
<p>kafka would be a lot harder to get into if the trial only ran on a power pc. - one of _why’s last tweets</p>
</blockquote>
<p>This tweet was really confusing, until CLOSURE. _why reveals that one of his biggest problems is what we call ‘bitrot’: you can’t just write a program, it must be updated. I have one of the first C programs I ever wrote, from when I was 12, and it wouldn’t compile on a modern system, due to a ‘clear screen’ function I wrote that interacted with the hardware that I owned at the time. I don’t have any of the GW-BASIC programs that I wrote as a child, because there was no way to transfer the source off of the computer: I didn’t have a disk drive, they were too expensive.</p>
<p>And so it is with Kafka. Right before he died, Kafka asked his friend Max Brod to burn everything he ever wrote. Brod published it instead. _why brings up that if The Trial was written for the PowerPC, he wouldn’t have needed Brod to burn it: it would have just naturally gone away.</p>
<p>Our industry is constantly changing, and that’s great. But we have no institutional memory. We keep making the same mistakes, over and over again. _why talks about fighting NULL, and how that was his biggest struggle as a programmer. The guy who invented null pointers, Tony Hoare, calls it <a href="http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare">“my billion dollar mistake”</a>. Yet Go has null pointers.</p>
<p>It’s really easy to burn out. I won’t lie, when _why deleted himself, I thought it was a terribly silly idea. But the more stuff I do, the bigger my projects and responsibilities get, and the more of a public person I am, the more it sounds appealing.</p>
<p>When I had a shitty band in high school, I wrote a dumb song after a girl dumped me called “The Only Constant is Change.” I thought it was clever at the time.</p>
<h2>The Software Industry</h2>
<p>Speaking of the industry, and burnout, that’s the last part of CLOSURE, and the hardest. The entire end of the book is about it, and it’s a very obvious, very ridiculous allegory.</p>
<p>There is a cult of Steve Jobs, and they speak incorrect French. They obsess over these flutes, and the flutes can make more flutes, and when one of them gets old and dies, they forget him immediately. When a new person is born, they start worshipping him, and induct him into the cult as quickly as possible, by clothing him in blue jeans and a black turtleneck. The cult members never eat any food.</p>
<p>The cult is programmers, the flutes are programming. It all falls out of that. Songs on the flute always start with C#, which I found amusing.</p>
<blockquote>
<p>It occured to me that they could use the flutes to describe a whole landscape, to transfer a map from one man’s mind to another._why, CLOSURE, page 84: SPAWN</p>
</blockquote>
<p>There’s also a really fantastic comparison between Steve Jobs and Bill Gates, as well as an angel investor who wants to invest in _why: I highly recommend that part.</p>
<blockquote>
<p>For instance, many people struggle with acceptance, feeling like they aren’t accepted by other people. But what I deal with is primarily hatred of entrepreneurs. But it’s something that I’m always working on and I’ve gotten much better.</p>
</blockquote>
<p>I’ve come to realize that I’m starting to become a grumpy old man. These damn kids never learn from their elders, we solved all the world’s problems back in the 60s and 70s, and they don’t know anything. I wish we had a better way to transfer institutional knowledge and stop repeating ourselves.</p>
<h2>Some final thoughts</h2>
<p>There is so much more that’s enjoyable in CLOSURE. Please check out the full thing, even if you don’t get all of it. For example, the part on Neil Gaiman is amazing.</p>
<p>I hope these jumping off points helps you in your journey. Let’s all have some fun with programming, okay?</p></content:encoded>
</item>
<item>
<title>Resque 1.25.0 has been released</title>
<link>https://steveklabnik.com/writing/resque-1-25-0-has-been-released/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/resque-1-25-0-has-been-released/</guid>
<pubDate>Mon, 16 Sep 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve just released Resque 1.25.0! Resque is the most stable, widely-used job queue for Ruby. It uses Redis as its backing store.</p>
<p>This release is thanks to <a href="https://github.com/adelcambre">adelcambre</a>, who took the time to get it over the finish line! &lt;3.</p>
<p>A total of 19 contributors helped out with this release, with 77 commits. Thank you to everyone!</p>
<p>This release contains no changes from the 1.25.0.pre release that was put out a few weeks ago, so if you’re using it, you can upgrade with no problems.</p>
<h2>CHANGES</h2>
<p>You can see the full CHANGELOG <a href="https://github.com/resque/resque/blob/v1.25.0/HISTORY.md">here</a>.</p>
<p>This release had a pre-release because of one big change: <a href="https://github.com/resque/resque/pull/1017/files">a refactor to the forking code</a>. This was introduced to fix a problem: the exit syscall raises a SystemExit Exception in ruby, its exit is treated as a failure, thus deregistering the worker. After the refactor + fix, things should be peachy. But please give the prerelease a try.</p>
<p>In addition, it has been discovered that we have accidentally introduced a SemVer violation back in v1.23.1: https://github.com/resque/resque/issues/1074</p>
<p>I take SemVer seriously, and apologize for this change. SemVer states that at this time, I should basically do whatever. Since this fixes a bug, I’m inclined to leave it in. If this introduces a significant problem for you, please let me know. Since others might now be relying on this new behavior, I’m not 100% sure what the best option is. Your feedback helps.</p>
<h2>INSTALLING</h2>
<p>To get 1.25.0 installed, just use <code>gem install</code>:</p>
<pre><code>$ gem install resque
</code></pre>
<p>Or include it in your <code>Gemfile</code>:</p>
<pre><code>gem 'resque', '~&gt; 1.25.0'
</code></pre>
<h2>SOURCE</h2>
<p>You can find the source <a href="https://github.com/resque/resque">on GitHub</a>.</p></content:encoded>
</item>
<item>
<title>Just the regularly scheduled apocalypse</title>
<link>https://steveklabnik.com/writing/just-the-regularly-scheduled-apocalypse/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/just-the-regularly-scheduled-apocalypse/</guid>
<pubDate>Tue, 10 Sep 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Every once in a while, I unfollow EVERYONE on Twitter. I did it again last night. Feels good, like a fresh start.</p>
<p>Part of the problem is that I feel like I should be following everyone who I know in real life. This causes Twitter to be absolutely unmanageable, as I’m following far too many. So then I just dip into the stream every so often, but then I miss some people’s tweets entirely. So why follow them all in the first place?</p>
<p>So, if I’ve unfollowed you, don’t worry, I still &lt;3 you. And even as I start following people back, I’m doing it slowly, and at semi-random. Trying to get some people I hadn’t seen in a while. People from outside my own community. People that’ll make me a little less caremad.</p>
<p>Yeah, like that’ll last.</p></content:encoded>
</item>
<item>
<title>Emoji licensing</title>
<link>https://steveklabnik.com/writing/emoji-licensing/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/emoji-licensing/</guid>
<pubDate>Tue, 06 Aug 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I recently decided to make an emoji gem. Of course, as with any project, I first look into the relevant licensing issues. What I found made me <code>:cry:</code>. Here’s the low-down on emoji and intellectual property law.</p>
<h2>A history</h2>
<p>So what <em>are</em> emoji, anyway? The Japanese spelling is 絵文字: 絵 (e) means ‘picture’ and 文字 (moji) means ‘letter.’ Picture letters. Simple.</p>
<p>Back in the day, three different Japanese phone carriers created their own versions of these smilies: docomo, au, and SoftBank. Codes would be sent down the (nonexistant) wire, which would then be turned into the final character. However, the three carriers used different characters and different codes.</p>
<p>Google and Apple petitioned the Unicode Consortium to include a number of these characters, and version 6.0 of the Unicode Standard ended up with 722 emoji.</p>
<h2>The font</h2>
<p>Apple ended up implementing a font named Apple Color Emoji that included these symbols. It came first with OS X Lion and iOS 5. These are the characters that you’re probably familliar with. Apple actually implements the font with a proprietary extension to OpenType, but that’s not particularly important. What <em>is</em> important is two things: typing a character that’s not on a normal keyboard, and displaying custom fonts on the web. These two problems have led to the current emoji IP situation.</p>
<h3>Typing extended characters</h3>
<p>On an iPhone, you can type emoji easily by enabling the extra keyboard. That’s not hard. But what about on a computer? Web sites that wanted to use emoji needed an easy way for you to type these characters in an input box.</p>
<p>I’m not sure who was first, but the first I remember is GitHub: with their <a href="https://github.com/blog/816-emoji">announcement post</a>, showed off a slick way to input emojis: use a set of characters that would get replaced with the emoji. So <code>:heart:</code> becomes a heart. Easy!</p>
<h3>Custom fonts in browsers</h3>
<p>That takes care of input, but what about output? If you don’t have the proper font installed, you’ll get those terrible boxes. That’s not very useable. Also, even if you do, <a href="http://code.google.com/p/chromium/issues/detail?id=90177">Chrome won’t display them</a>.</p>
<p>So what do you do? You use your good buddy <code>&lt;img&gt;</code>.</p>
<p>If you’re building a web application, you are already telling users to type <code>:heart:</code> rather than the character, you can just make <code>:heart:</code> be rendered as <code>&lt;img src="/assets/heart.png"&gt;</code>. So that’s easy…. but where do you get the PNGs?</p>
<h2>Apple Color Emoji</h2>
<p>Remember that extension? <a href="https://color-emoji.googlecode.com/git/specification/v1.html">The spec was published</a>, and is being worked in to <a href="http://google-opensource.blogspot.de/2013/05/open-standard-color-font-fun-for.html">FreeType</a>.</p>
<p>Anyway, the extension works by embedding PNGs into the font. I think that you can see where this is going… just rip the images out of the font, and you’re golden. Well, other than that pesky IP law…</p>
<h2>Extensions</h2>
<p>Furthermore, now that you’re not constrained by the unicode standard, you can make anything between two colons an emoji. Like <code>:shipit:</code>. Several services do this. <a href="https://github.com/arvida/emoji-cheat-sheet.com/blob/master/LICENSE">http://emoji-cheat-sheet.com lists the licenses as such</a>:</p>
<pre><code>octocat, squirrel, shipit
Copyright (c) 2012 GitHub Inc. All rights reserved.
bowtie
Copyright (c) 2012 37signals, LLC. All rights reserved.
neckbeard
Copyright (c) 2012 Jamie Dihiansan. Creative Commons Attribution 3.0 Unported
feelsgood, finnadie, goberserk, godmode, hurtrealbad, rage 1-4, suspect
Copyright (c) 2012 id Software. All rights reserved.
trollface
Copyright (c) 2012 whynne@deviantart. All rights reserved.
All other emoji images
Copyright (c) 2012 Apple Inc. All rights reserved.
</code></pre>
<p>I made some inquries into all of these extended emoji, and GitHub said they’d get back to me, but didn’t. Therefore, their three are All Rights Reserved.</p>
<p>37Signals had said <code>:neckbeard:</code> is CC licensed, and in an email, said they’d be happy to license <code>:bowtie:</code> the same way.</p>
<p>The inventor of Trollface has yet to return my email. Go figure. Also All Rights Reserved.</p>
<p>Finally, while DOOM had its code released under the GPL, the assets were very clearly not. All Rights Reserved there too.</p>
<p>I didn’t even bother emailing Apple.</p>
<h2>What’s this mean for me?</h2>
<p>Basically, <strong>if you’re distributing the images from Apple Color emoji, you are violating copyright law</strong>. And if you’re not one of the above mentioned entities, you’re violating their licenses, as well.</p>
<p>So that’s a big downer. What to do?</p>
<h2>Phantom Open Emoji</h2>
<p>At some point, there was a kickstarter for an open emoji set. And it worked! And so <a href="https://github.com/Genshin/PhantomOpenEmoji">Phantom Open Emoji</a> was born.</p>
<p>Phantom Open Emoji basically gives you images to use as all of the glyphs used in Apple Open Emoji, but you have a license to use them however you’d like. So that’s pretty awesome.</p>
<hr />
<p>I wrote this post because I was writing a <a href="http://rubygems.org/gems/emoji">Ruby gem for emoji</a>, and needed to know if I could distribute the emoji with the gem. It’s not done yet (it’s barely started), but if you want to use emoji with your app, it will be the simplest and safest way to make sure you don’t run afoul of this kind of thing.</p></content:encoded>
</item>
<item>
<title>redis-namespace 1.3.1 security release</title>
<link>https://steveklabnik.com/writing/redis-namespace-1-3-1--security-release/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/redis-namespace-1-3-1--security-release/</guid>
<pubDate>Sat, 03 Aug 2013 00:00:00 GMT</pubDate>
<content:encoded><p>TL;DR: if you use <a href="https://rubygems.org/gems/redis-namespace">redis-namespace</a> and you use <code>send</code>, <code>Kernel#exec</code> may get called. Please upgrade to 1.0.4, 1.1.1, 1.2.2, or 1.3.1 immediately.</p>
<p>Link to the fix: <a href="https://github.com/resque/redis-namespace/commit/6d839515e8a3fdc17b5fb391500fda3f919689d6">https://github.com/resque/redis-namespace/commit/6d839515e8a3fdc17b5fb391500fda3f919689d6</a></p>
<h2>The Problem</h2>
<p>Redis has an EXEC command. We handle commands through <code>method_missing</code>. This works great, normally:</p>
<pre><code> r = Redis::Namespace.new("foo") r.exec # =&gt; error, not in a transaction, whatever
</code></pre>
<p>Here’s the problem: <code>Kernel#exec</code>. Since this is on every object, when you use <code>#send</code>, it skips the normal visibility, and calls the private but defined <code>Kernel#exec</code> rather than the <code>method_missing</code> verison:</p>
<pre><code> r = Redis::Namespace.new("foo") r.send(:exec, "ls") # =&gt; prints out your current directory
</code></pre>
<p>We fix this by not proxying <code>exec</code> through <code>method_missing</code>.</p>
<p>You are only vulnerable if you do the always-dangerous ‘send random input to an object through <code>send</code>.’ And you probably don’t. A cursory search through GitHub didn’t find anything that looked vulnerable.</p>
<p>However, if you write code that wraps or proxies Redis in some way, you may do something like this.</p>
<p>The official Redis gem does not use <code>method_missing</code>, so it should not have any issues: https://github.com/redis/redis-rb/blob/master/lib/redis.rb#L2133-L2147</p>
<p>I plan on removing the <code>method_missing</code> implementation in a further patch, but since this is an immediate issue, I wanted to make minimal changes.</p>
<p>Testing this is hard, <code>#exec</code> replaces the current running process. I have verified this fix by hand, but writing an automated test seems difficult.</p>
<p>:heart::cry:</p></content:encoded>
</item>
<item>
<title>Beware subclassing Ruby core classes</title>
<link>https://steveklabnik.com/writing/beware-subclassing-ruby-core-classes/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/beware-subclassing-ruby-core-classes/</guid>
<pubDate>Wed, 24 Jul 2013 00:00:00 GMT</pubDate>
<content:encoded><p>TL;DR: Subclassing core classes in Ruby can lead to unexpected side effects. I suggest composition over inheritance in all these cases.</p>
<h2>Subclassing Review</h2>
<p>If you’re familiar with the concept of subclassing, skip down to “The Problem.”</p>
<p>In Ruby, you can make your own classes:</p>
<pre><code>class List
end
</code></pre>
<p>You can also make subclasses of those classes:</p>
<pre><code>class OrderedList &lt; List
end
puts OrderedList.new.kind_of?(List) # =&gt; true
</code></pre>
<p>Now, subclassing represents an “is a” relationship. This means that our <code>OrderedList</code> should be a <code>List</code> in every respect, but with some added behavior. The <a href="http://en.wikipedia.org/wiki/Liskov_substitution_principle">Liskov Substitution Principle</a> is one formulation of this idea.</p>
<h2>The Problem</h2>
<p>Ruby has two major bits of code that it provides for your use: the core library and the standard library. <a href="http://www.ruby-doc.org/core-2.0/">The core library can be found here</a>, and contains cllasses that you know and love, like <code>String</code>, <code>Hash</code>, and <code>Array</code>. <a href="http://www.ruby-doc.org/stdlib-2.0/">The standard library can be found here</a>, and contains your favorite hits, like <code>CSV</code>, <code>JSON</code>, and <code>Logger</code>.</p>
<p>One way to think about the difference between core and the standard library is that core is written in C, while the standard library is written in Ruby. Core are the classes that are used the most, so they’re implemented in as low-level a fashion as possible. They’ll be in every single Ruby program, so might as well make them fast! The standard library only gets pulled in by bits and pieces; another way of thinking about the difference is that you need to <code>require</code> everything in the standard library, but nothing in core.</p>
<p>What do you think this code should do?</p>
<pre><code>class List &lt; Array
end
puts List.new.to_a.class
</code></pre>
<p>If you said “it prints <code>Array</code>,” you’d be right. This behavior really confuses me, though, because <code>List</code> is already an <code>Array</code>; in my mind, this operation shouldn’t suddenly change the class.</p>
<p>Why does this happen? Let’s check out the implementation of <a href="https://github.com/ruby/ruby/blob/trunk/array.c#L2064-L2082">Array#to_a</a>:</p>
<pre><code>static VALUE
rb_ary_to_a(VALUE ary)
{
if (rb_obj_class(ary) != rb_cArray) {
VALUE dup = rb_ary_new2(RARRAY_LEN(ary));
rb_ary_replace(dup, ary);
return dup;
}
return ary;
}
</code></pre>
<p>If the class is not an <code>Array</code>, (represented by <code>rb_cArray</code>), then we make a new array of the same length, call <code>replace</code> on it, and then return the new array. If this C scares you, here’s a direct port to pure Ruby:</p>
<pre><code>def array_to_a(ary)
if ary.class != Array
dup = []
dup.replace(ary)
return dup
end
return ary
end
array_to_a(List.new).class # =&gt; Array
</code></pre>
<p>So why do this? Well, again, this class will be used all over the place. For example, I made a brand new Rails 4 application, generated a controller and view, and put this in it:</p>
<pre><code>ObjectSpace.count_objects[:T_ARRAY]: &lt;%= ObjectSpace.count_objects[:T_ARRAY] %&gt;
</code></pre>
<p>ObjectSpace allows you to inspect all of the objects that exist in the system. Here’s the output:</p>
<p><img src="http://i.imgur.com/PVwdvyY.png" alt="http://i.imgur.com/PVwdvyY.png" /></p>
<p>rails arrays</p>
<p>That’s a lot of arrays! This kind of shortcut is generally worth it: 99.99% of the time, this code is perfect.</p>
<p>That last 0.01% is the problem. If you don’t know exactly how these classes operate at the C level, you’re gonna have a bad time. In this case, this behavior is odd enough that someone was kind enough to document it.</p>
<p>Here’s the Ruby version of what I’d expect to happen:</p>
<pre><code>def array_to_a2(ary)
return ary if ary.is_a?(Array)
dup = []
dup.replace(ary)
dup
end
array_to_a2(List.new).class # =&gt; List
</code></pre>
<p>This has the exact same behavior except when we’re already dealing with an Array, which is what I’d expect.</p>
<p>Let’s take another example: reverse.</p>
<pre><code>l = List.new
l &lt;&lt; 1
l &lt;&lt; 2
puts l.reverse.class # =&gt; Array
</code></pre>
<p>I would not expect that calling <code>#reverse</code> on my custom <code>Array</code> would change its class. Let’s <a href="https://github.com/ruby/ruby/blob/trunk/array.c#L2138-L2161">look at the C</a> again:</p>
<pre><code>static VALUE
rb_ary_reverse_m(VALUE ary)
{
long len = RARRAY_LEN(ary);
VALUE dup = rb_ary_new2(len);
if (len &gt; 0) {
const VALUE *p1 = RARRAY_RAWPTR(ary);
VALUE *p2 = (VALUE *)RARRAY_RAWPTR(dup) + len - 1;
do *p2-- = *p1++; while (--len &gt; 0);
}
ARY_SET_LEN(dup, RARRAY_LEN(ary));
return dup;
}
</code></pre>
<p>We get the length of the array, make a new blank array of the same length, then do some pointer stuff to copy everything over, and return the new copy. Unlike <code>#to_a</code>, this behavior is <em>not</em> currently documented.</p>
<p>Now: you could make the case that this behavior is expected, in both cases: after all, the point of the non-bang methods is to make a copy. However, there’s a difference to me between “make a new array with this stuff in it” and “make a new copy with this stuff in it”. Most of the time, I get the same class back, so I expect the same class back in these circumstances.</p>
<p>Let’s talk about a more pernicious issue: Strings.</p>
<p>As you know, the difference between interpolation and concatenation is that interpolation calls <code>#to_s</code> implicitly on the object it’s interpolating:</p>
<pre><code>irb(main):001:0&gt; "foo" + 2
TypeError: no implicit conversion of Fixnum into String
from (irb):1:in `+'
from (irb):1
from /opt/rubies/ruby-2.0.0-p195/bin/irb:12:in `&lt;main&gt;'
irb(main):002:0&gt; "foo#{2}"
=&gt; "foo2"
irb(main):001:0&gt; class MyClass
irb(main):002:1&gt; def to_s
irb(main):003:2&gt; "yup"
irb(main):004:2&gt; end
irb(main):005:1&gt; end
=&gt; nil
irb(main):006:0&gt; "foo#{MyClass.new}"
=&gt; "fooyup"
</code></pre>
<p>So what about a custom <code>String</code>?</p>
<pre><code>class MyString &lt; String
def to_s
"lol"
end
end
s = MyString.new
s.concat "Hey"
puts s
puts s.to_s
puts "#{s}"
</code></pre>
<p>What does this print?</p>
<pre><code>$ ruby ~/tmp/tmp.rb HeylolHey
</code></pre>
<p>That’s right! With <code>String</code>s, Ruby doesn’t call <code>#to_s</code>: it puts the value in directly. How does this happen?</p>
<p>Well, dealing with string interpolation deals with the parser, so let’s check out the bytecode that Ruby generates. Thanks to <a href="http://twitter.com/tenderlove">Aaron Patterson</a> for suggesting this approach. &lt;3</p>
<pre><code>irb(main):013:0&gt; x = RubyVM::InstructionSequence.new(%q{puts "hello #{'hey'}"})
=&gt; &lt;RubyVM::InstructionSequence:&lt;compiled&gt;@&lt;compiled&gt;&gt;
irb(main):014:0&gt; puts x.disasm
== disasm: &lt;RubyVM::InstructionSequence:&lt;compiled&gt;@&lt;compiled&gt;&gt;==========
0000 trace 1 ( 1)
0002 putself
0003 putstring "hello hey"
0005 opt_send_simple &lt;callinfo!mid:puts, argc:1, FCALL|ARGS_SKIP&gt;
0007 leave
=&gt; nil
irb(main):015:0&gt; x = RubyVM::InstructionSequence.new(%q{puts "hello #{Object.new}"})
=&gt; &lt;RubyVM::InstructionSequence:&lt;compiled&gt;@&lt;compiled&gt;&gt;
irb(main):016:0&gt; puts x.disasm
== disasm: &lt;RubyVM::InstructionSequence:&lt;compiled&gt;@&lt;compiled&gt;&gt;==========
0000 trace 1 ( 1)
0002 putself
0003 putobject "hello "
0005 getinlinecache 12, &lt;ic:0&gt;
0008 getconstant :Object
0010 setinlinecache &lt;ic:0&gt;
0012 opt_send_simple &lt;callinfo!mid:new, argc:0, ARGS_SKIP&gt;
0014 tostring
0015 concatstrings 2
0017 opt_send_simple &lt;callinfo!mid:puts, argc:1, FCALL|ARGS_SKIP&gt;
0019 leave
=&gt; nil
</code></pre>
<p>You can see with a string, the bytecode actually puts the final concatenated string. But with an object. it ends up calling <code>tostring</code>, and then <code>concatstrings</code>.</p>
<p>Again, 99% of the time, this is totally fine, and much faster. But if you don’t know this trivia, you’re going to get bit.</p>
<p><a href="https://rails.lighthouseapp.com/projects/8994/tickets/6023-unable-to-find-by-subclasses-of-string-in-activerecord#ticket-6023-5">Here</a> is an example from an older version of Rails. Yes, you might think “Hey idiot, there’s no way it will store your custom <code>String</code> class,” but the whole idea of subclassing is that it’s a drop-in replacement.</p>
<p>I know that there’s some case where Ruby will not call your own implementation of <code>#initialize</code> on a custom subclass of <code>String</code>, but I can’t find it right now. This is why this problem is so tricky: most of the time, things are fine, but then occasionally, something strange happens and you wonder what’s wrong. I don’t know about you, but my brain needs to focus on more important things than the details of the implementation.</p>
<p>Since I first wrote this post, <a href="http://twitter.com/jeg2">James Edward Gray II</a> helped me remember what this example is. One of the early exercises in <a href="http://exercism.io/">http://exercism.io/</a> is based on making a DNA type, and then doing some substitution operations on it. Many people inherited from <code>String</code> when doing their answers, and while the simple case that passes the tests works, this case won't:</p>
<pre><code>class Dna &lt; String
def initialize(*)
super
puts "Building Dna: #{inspect}"
end
end
result = Dna.new("CATG").tr(Dna.new("T"), Dna.new("U"))
p result.class
p result
</code></pre>
<p>This prints:</p>
<pre><code>Building Dna: "CATG"
Building Dna: "T"
Building Dna: "U"
Dna
"CAUG"
</code></pre>
<p>It never called our initializer for the new string. Let's check <a href="https://github.com/ruby/ruby/blob/trunk/string.c#L5484-L5525">the source of <code>#tr</code></a>:</p>
<pre><code>static VALUE
rb_str_tr(VALUE str, VALUE src, VALUE repl)
{
str = rb_str_dup(str);
tr_trans(str, src, repl, 0);
return str;
}
</code></pre>
<p><code>rb_str_dup</code> has a pretty simple definition:</p>
<pre><code>VALUE
rb_str_dup(VALUE str)
{
return str_duplicate(rb_obj_class(str), str);
}
</code></pre>
<p>and so does <code>str_duplicate</code>:</p>
<pre><code>static VALUE
str_duplicate(VALUE klass, VALUE str)
{
VALUE dup = str_alloc(klass);
str_replace(dup, str);
return dup;
}
</code></pre>
<p>So there you have it: MRI doesn't go through the whole initialization process when duplicating a string: it just allocates the memory and then replaces the contents.</p>
<p>If you re-open <code>String</code>, it's also weird:</p>
<pre><code>class String
alias_method :string_initialize, :initialize
def initialize(*args, &amp;block)
string_initialize(*args, &amp;block)
puts "Building MyString: #{inspect}"
end
end
result = String.new("CATG").tr("T", "U") # =&gt; Building MyString: "CATG"
p result.class # =&gt; String
p result # =&gt; "CAUG"
</code></pre>
<p>Again, unless you know exactly how this works at a low level, surprising things happen.</p>
<h2>The Solution</h2>
<p>Generally speaking, subclassing isn’t the right idea here. You want a data structure that <em>uses</em> one of these core classes internally, but isn’t exactly like one. Rather than this:</p>
<pre><code>class Name &lt; String
end
</code></pre>
<p>do this:</p>
<pre><code>require 'delegate'
class Name &lt; SimpleDelegator
def initialize
super("")
end
end
</code></pre>
<p>This allows you to do the same thing, but without all of the pain:</p>
<pre><code>class Name
def to_s
"hey"
end
end
"#{Name.new}" # =&gt; "hey"
</code></pre>
<p>However, this won’t solve all problems:</p>
<pre><code>require 'delegate'
class List &lt; SimpleDelegator
def initialize
super([])
end
end
l = List.new
l &lt;&lt; 1
l &lt;&lt; 2
puts l.reverse.class # =&gt; Array
</code></pre>
<p>In general, I’d prefer to delegate things manually, anyway: a <code>Name</code> is not actually a drop-in for a <code>String</code> it’s something different that happens to be a lot like one:</p>
<pre><code>class List
def initialize(list = [])
@list = list
end
def &lt;&lt;(item)
@list &lt;&lt; item
end
def reverse
List.new(@list.reverse)
end
end
l = List.new
l &lt;&lt; 1
l &lt;&lt; 2
puts l.reverse.class # =&gt; List
</code></pre>
<p>You can clean this up by using <code>Forwardable</code> to only forward the messages you want to forward:</p>
<pre><code>require 'forwardable'
class List
extend Forwardable
def_delegators :@list, :&lt;&lt;, :length # and anything else
def initialize(list = [])
@list = list
end
def reverse
List.new(@list.reverse)
end
end
l = List.new
l &lt;&lt; 1
l &lt;&lt; 2
puts l.reverse.class # =&gt; List
</code></pre>
<p>Now you know! Be careful out there!</p></content:encoded>
</item>
<item>
<title>Resque 1.25.0.pre has been released</title>
<link>https://steveklabnik.com/writing/resque-1-25-0-pre-has-been-released/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/resque-1-25-0-pre-has-been-released/</guid>
<pubDate>Tue, 23 Jul 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve just released Resque 1.25.0.pre! Resque is the most stable, widely-used job queue for Ruby. It uses Redis as its backing store.</p>
<p>This release is thanks to <a href="https://github.com/adelcambre">adelcambre</a>, who took the time to get it over the finish line! &lt;3</p>
<p>A total of 19 contributors helped out with this release, with 77 commits. Thank you to everyone!</p>
<h2>CHANGES</h2>
<p>You can see the full CHANGELOG <a href="https://github.com/resque/resque/blob/v1.25.0.pre/HISTORY.md">here</a>.</p>
<p>I am releasing this version as ‘pre’ due to one big change: <a href="https://github.com/resque/resque/pull/1017/files">a refactor to the forking code</a>. This was introduced to fix a problem: the exit syscall raises a SystemExit Exception in ruby, its exit is treated as a failure, thus deregistering the worker. After the refactor + fix, things should be peachy. But please give the prerelease a try.</p>
<p>In addition, it has been discovered that we have accidentally introduced a SemVer violation back in v1.23.1: https://github.com/resque/resque/issues/1074</p>
<p>I take SemVer seriously, and apologize for this change. SemVer states that at this time, I should basically do whatever. Since this fixes a bug, I’m inclined to leave it in. If this introduces a significant problem for you, please let me know. Since others might now be relying on this new behavior, I’m not 100% sure what the best option is. Your feedback helps.</p>
<h2>INSTALLING</h2>
<p>To get 1.25.0.pre installed, just use <code>gem install</code>:</p>
<pre><code>$ gem install resque --pre
</code></pre>
<h2>SOURCE</h2>
<p>You can find the source <a href="https://github.com/resque/resque">on GitHub</a>.</p></content:encoded>
</item>
<item>
<title>Removing Turbolinks from Rails 4</title>
<link>https://steveklabnik.com/writing/removing-turbolinks-from-rails-4/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/removing-turbolinks-from-rails-4/</guid>
<pubDate>Tue, 25 Jun 2013 00:00:00 GMT</pubDate>
<content:encoded><p>If you don’t want to use Turbolinks with your Rails 4 application, it’s easy! Just do this:</p>
<ol>
<li>Remove the <code>gem 'turbolinks'</code> line from your Gemfile.</li>
<li>Remove the <code>//= require turbolinks</code> from your <code>app/assets/javascripts/application.js</code>.</li>
<li>Remove the two <code>"data-turbolinks-track" =&gt; true</code> hash key/value pairs from your <code>app/views/layouts/application.html.erb</code>.</li>
</ol>
<p>Done!</p></content:encoded>
</item>
<item>
<title>Twitter ghosts</title>
<link>https://steveklabnik.com/writing/twitter-ghosts/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/twitter-ghosts/</guid>
<pubDate>Sat, 22 Jun 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve been thinking about death more than usual lately; my father died just before Father’s Day a year ago, and so I remembered, then was constantly reminded again on that day. But there’s another death that’s recently happened that really bothers me. The Internet is a really strange place.</p>
<p>A few days ago, I was reading Twitter as I often do, and I saw a tweet that mentioned a death. I don’t remember which one it was, specifically, but I didn’t think a whole lot of it: death happens all the time, and this one referred to an accident. It basically just said something like “Whoah, Michael, it was way too soon to lose you. :(” I didn’t think much of it, but a few minutes later, I saw a few more tweets: “@mmhastings, amazing reporter, what a great guy, how tragic.” If a few of my followers all knew this dude, then he must be a good guy. So I clicked through to his Twitter profile:</p>
<p><img src="http://i.imgur.com/uy8cRbB.png" alt="http://i.imgur.com/uy8cRbB.png" /></p>
<p>Michael Hastings’ Twitter Profile</p>
<p>Here’s what I noticed:</p>
<ol>
<li>At least his last tweet is a nice one.</li>
<li>He has 20,000 followers. Nice. And he’s only following 1,400 people, so it’s probably a legit number, not spammy.</li>
<li>Oh, how weird, he’s from Hollywood. I lived there for half of last year, and am still in LA.</li>
<li>Wait, he’s following <em>me</em>? And I’m not following him?</li>
</ol>
<p>As the day went on, more and more articles came out, and I read them all. Apparently, Michael Hastings was a really awesome guy. Apparently he was one of the few real reporters, the ones who dig in deep, aren’t afraid to speak about what they find, and never backed down. <a href="http://www.rollingstone.com/politics/news/michael-hastings-rolling-stone-contributor-dead-at-33-20130618">The story about his death in Rolling Stone</a> says “Hastings’ hallmark as reporter was his refusal to cozy up to power.”</p>
<p>So here’s what weirds me out about this situation:</p>
<p>First of all, wow, life is short. At any given time, if I just got hit by a car and died, I probably wouldn’t be all that proud of my last tweet. Isn’t it weird, that when someone’s life gets cut short, we can see what they were up to right before their death? I can now read the words of a man I’d never spoken to, and see what he was up to and what he cared about. This isn’t new news, there’s even a 4chan meme (from back in the early days of /b/) about a kid who committed suicide and all of his friends writing eulogies on his MySpace page. This is one of the first times it’s (in)directly affected me, though, and so it’s taken on a different tone.</p>
<p>Second, why didn’t I know who this was? Apparently this was a man who was a really great person, the kind I would have liked. And apparently he saw enough value in my random ramblings on Twitter to pay attention to me. But why didn’t I reciprocate? We lived in the same place. It wouldn’t have been hard to go grab a beer sometime. I walked past the intersection that he died many, many times. Would I have been friends with this person in some alternate universe?</p>
<p>One of the things that I always said about my father’s death was that at least he had two months where he knew he was going to die. While it was an incredible stress on everyone involved, at least he was able to say everything to everyone he wanted to say something to, wrap up all his unfinished business, and make peace with the God he deeply believed in. Sudden deaths have their own entire element of terrible, where you don’t get that chance. I mean, I’ve been thinking about this situation basically nonstop for the last few days, and I didn’t even really know this person: I can’t imagine how horribly tragic this is for all of the people who were actually close to Michael. I saw what a toll my father’s death took on my mother, I can’t even imagine the grief his wife must be dealing with right now.</p>
<p>So many questions. So few answers. So sad. All we can do is look at these digital remnants, wonder, and remember.</p></content:encoded>
</item>
<item>
<title>Returning to Free Software, a guide</title>
<link>https://steveklabnik.com/writing/returning-to-free-software-a-guide/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/returning-to-free-software-a-guide/</guid>
<pubDate>Sat, 15 Jun 2013 00:00:00 GMT</pubDate>
<content:encoded><p>A month or two ago, I made a conscious decision that I would return to using as much Free Software as possible. The PRISM debacle of the last week confirmed my fears and reasoning, and so I made the decision to accelerate the schedule.</p>
<p>Here’s what I did, and how you can, too.</p>
<h2>My GNU/Linux Background</h2>
<p>The first computer I ever used was an Apple machine, and at various times in my life I’ve used GNU/Linux as my main OS. So I’m already pretty familiar with what’s involved.</p>
<p>That said, it’s only gotten better over the years. Especially if you buy hardware that is known to work well, using GNU/Linux as your main OS shouldn’t be a big deal. One of the nice things about the rise of OS X in the developer community is that they’ve come to rely on the standard GNU tools, and so much of what you’re already working with will probably translate straight over. Of course, if you work on iOS or something, this won’t be the case, but for your run-of-the-mill web developer, there isn’t a whole lot of difference.</p>
<p>Well, other than that whole ‘knowing exactly what’s on your machine’ thing.</p>
<h2>Hardware: X1 Carbon</h2>
<p>IBM’s ThinkPad series of laptops have always enjoyed fantastic Linux support. When they were purchased by Lenovo, some were worried that they’d ruin this fantastic hardware line. That didn’t really happen, though, and they’re still fantastic computers to this day. When asking my Twitter followers what they thought, everyone who owned a ThinkPad loved it, whichever model they had.</p>
<p>I happened to get an <a href="http://shop.lenovo.com/us/en/laptops/thinkpad/x-series/x1-carbon/index.html">X1 Carbon</a>. I was coming from a 13" MacBook Air 13“, and the Carbon is very similar. It’s 14” though, which is a really nice size; it still fits in the same pocket in my bag as the MBA did, but I get a little more screen real estate. A note for those who give presentations: the X1 Carbon uses a MiniDisplay port, so save your Apple connectors!</p>
<p>All of the hardware I use has Just Worked; I hear that the thumbprint reader may not, but I don’t plan on using it, so that doesn’t bother me.</p>
<p>If you’re not in the market for new hardware, you can generally run GNU/Linux on your Apple hardware as well, which is a starting point, at least.</p>
<h2>Operating System: Crunchbang GNU/Linux</h2>
<p>I’ve used a lot of different GNU/Linux distros, and have been a happy Arch Linux user for years. However, this time around, I decided to go with <a href="http://crunchbang.org/">CrunchBang</a>. Crunchbang is a Debian derivative, but includes <a href="http://openbox.org/">OpenBox</a> as the default window manager. The result is an OS that’s lightweight, yet very, very functional. I’ve been really happy so far with Crunchbang’s level of polish.</p>
<p>Okay, so actually <em>installing</em> Crunchbang: the X1 Carbon comes with Windows by default, and so we need to get Crunchbang on there somehow. Luckily, you can just use USB. <code>dd</code> is your friend here, which you can use on your Mac, of course. You can grab a copy of Crunchbang <a href="http://crunchbang.org/download/">from their download page</a>, and they have a link to making the USB stick right on that page.</p>
<p>After you’ve made the USB stick, just reboot your computer with it on, and press down all of your F keys. I was simply too lazy to look up which one would actually let me choose a boot device. ;) The Crunchbang installer was very straightforward, just follow the prompts and you’ll be good to go.</p>
<p>One thing that I did choose was the ‘encrypted lvm’ option. Basically, your entire disk is encrypted except a small boot partition. On boot, it asks you for a passphrase to decrypt the disk. It’s just a matter of choosing the right option, in practice, this has worked and worked well for me.</p>
<p>EDIT: One of the reasons I chose Crunchbang was because I thought that it was solely configuration on top of Debian, which includes 100% Free Software. Checking my <code>sources.list</code>, there is a Crunchbang-specific repository, and I haven’t evaluated it for Free-ness. So if you’re trying for 100% purity, straight-up Debian may be better.</p>
<h2>Software</h2>
<p>Most of the tools that I use every day are already either Free Software or open source, and so my core experience hasn’t changed: <code>bash</code>, <code>vim</code>, <code>git</code>, and friends aren’t any different.</p>
<p>I did, however, install <a href="https://www.torproject.org/">tor</a>. Tor is not a privacy panacea, however, it does help in many cases.</p>
<p>I’ve added <a href="http://www.ghostery.com/">Ghostery</a>, <a href="http://noscript.net/">NoScript</a>, and <a href="http://adblockplus.org/">AdBlock Plus</a> to the built-in Iceweasel browser, which is for all intents and purposes Firefox without the Mozilla branding. These allow me to control exactly what information leaks out a little bit more. They also have the nice side effect of making pages load a bit faster, since a lot of the usual bullshit doesn’t even get loaded. (EDIT: it’s been brought to my attention that Ghostery isn’t actually free. Oops!)</p>
<p>I plan on setting up backups with <a href="http://www.tarsnap.com/">Tarsnap</a> later in the week, which is an excellent backup service that encrypts all of your data locally before saving it off in The Cloud. It has some pretty good (I hear) de-duplication features, and you only pay for the bytes used, so it’s pretty inexpensive. The author, Colin Percival, has said that he has some customers doing a muti-gig daily backup of their home directories, and they’re paying ~$10/month. (I should mention that <a href="http://www.tarsnap.com/legal.html">Tarsnap is not Free</a>, but it is shared source. I don’t plan on using it for anything else, so I’m okay with this tradeoff at this time.)</p>
<p>Most of my development work at the moment is in Ruby, and getting it set up was a breeze. I use the <a href="https://github.com/sstephenson/ruby-build">ruby-build</a> tool to compile my Rubies, and the most excellent <a href="https://github.com/postmodern/chruby">chruby</a> to switch between them.</p>
<p>One thing that I have yet to explore (but I’d like to) is to use a tool like Chef to set up my personal machine, so that I can periodically wipe everything and re-build from scratch. That’s still on the to-do list, though.</p>
<p>I personally re-map my caps lock key to control. To do this, make a <code>.Xmodmap</code> file in your home directory, and put this in it:</p>
<pre><code>keycode 66 = Control_L
clear Lock
add control = Control_L
</code></pre>
<p>I figured this out via a few simple searches. There’s a small chance that your caps lock is not code 66, and if it’s not, you can run the <code>xev</code> command to check it out. Just press your caps lock and make note of the number that shows up in the console.</p>
<p>I had to install <code>grandr</code> in order to share my screen to give a presentation. It’s easy enough to get: <code>sudo apt-get install grandr</code>.</p>
<p>I started giving presentations with <a href="http://rabbit-shocker.org/">rabbit</a>, which is what many Japanese Rubyists use. It’s pretty fun; you write plain text files and it makes your presentation for you, as cairo/pango application. You can also use it to generate PDFs of slides. It’s not for everyone, but I’m enjoying it so far.</p>
<p>Oh, and <em>incredibly</em> important: I typed <code>sudo apt-get install ttf-ancient-fonts</code> to get Emoji support. They’re not as pretty as the Apple ones, but they’re not empty boxes either.</p>
<h2>Conclusion</h2>
<p>I’m really, really happy with this setup. Very little of my day-to-day has changed, but I have a bit more privacy, and can feel good that I know exactly what software is on my machine. No NSA backdoors here!</p>
<p>I still have improvements to make, but what I have works well so far. Please let me know what you think, and suggest ways to make this setup even better.</p></content:encoded>
</item>
<item>
<title>Rust-y Scraps: iterating with a step</title>
<link>https://steveklabnik.com/writing/rust--y-scraps--iterating-with-a-step/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rust--y-scraps--iterating-with-a-step/</guid>
<pubDate>Thu, 30 May 2013 00:00:00 GMT</pubDate>
<content:encoded><p>This blog post works with <a href="http://rust-lang.org/">Rust</a> v 0.6, and may change in the future :).</p>
<p>A common need in various algorithms is to iterate with some sort of step. A simple way to do this in Rust is with <code>unit::range_step</code>:</p>
<pre><code>let nums = [1,2,3,4,5,6,7];
for uint::range_step(0, bytes.len() - 1, 2) |i| {
println(fmt!("%d &amp; %d", nums[i], nums[i+1]));
}
</code></pre>
<p>This yields</p>
<pre><code>1 &amp; 2
3 &amp; 4
5 &amp; 6
</code></pre>
<p>Of course, this is awkward: I wish I had some code to do something like:</p>
<pre><code>let nums = [1,2,3,4,5,6,7];
for nums.in_groups |i, j| {
println(fmt!("%d &amp; %d", i, j));
}
</code></pre>
<p>But I think this is just out of my abilities. The issue is that the closure needs to take the same number of arguments as the step, and I’m not sure that I have the Rust-fu to make it work yet.</p>
<p>After a discussion in the IRC room, Rust doesn’t have the Rust-fu yet either; varargs aren’t there, so this style is a no-go for now.</p>
<hr />
<p>Oh, and since there were no examples of this yet, <a href="https://github.com/mozilla/rust/pull/6841">I added some to the docs</a>.</p></content:encoded>
</item>
<item>
<title>Rails 4.0.0-beta1 to Rails 4.0.0-rc1</title>
<link>https://steveklabnik.com/writing/rails-4-0-0-beta1-to-rails-4-0-0-rc1/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rails-4-0-0-beta1-to-rails-4-0-0-rc1/</guid>
<pubDate>Tue, 21 May 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I’m writing <a href="http://www.manning.com/bigg2/">Rails 4 in Action</a> along with Ryan Bigg and Yehuda Katz. In updating the book to Rails 4, I started early: the sample app (<a href="https://github.com/steveklabnik/ticketee">ticketee</a>) was done with <code>4.0.0.beta1</code>.</p>
<p>Then <code>4.0.0.rc1</code> came out. I didn’t upgrade immediately, but last night, I took the plunge. There were only three errors, and the whole process took me something like 40 minutes.</p>
<p>Here’s what I had to do:</p>
<h2>Cookie Stores</h2>
<p>Error:</p>
<pre><code>/Users/steve/.gem/ruby/2.0.0/gems/railties-4.0.0.rc1/lib/rails/application/configuration.rb:144:in `const_get': uninitialized constant ActionDispatch::Session::EncryptedCookieStore (NameError)
</code></pre>
<p>The EncryptedCookieStore went away. Now, the regular cookie store handles these kinds of things.</p>
<p>Fix:</p>
<pre><code>diff --git a/config/initializers/session_store.rb b/config/initializers/session_store.rbindex c512cda..b4681fd 100644--- a/config/initializers/session_store.rb+++ b/config/initializers/session_store.rb@@ -1,3 +1,3 @@ # Be sure to restart your server when you modify this file.-Ticketee::Application.config.session_store :encrypted_cookie_store, key: '_ticketee_session'+Ticketee::Application.config.session_store :cookie_store, key: '_ticketee_session'
</code></pre>
<h2>Routes</h2>
<p>Error:</p>
<pre><code>/Users/steve/.gem/ruby/2.0.0/gems/actionpack-4.0.0.rc1/lib/action_dispatch/routing/route_set.rb:408:in `add_route': Invalid route name, already in use: 'signin' (ArgumentError)
You may have defined two routes with the same name using the `:as` option, or you may be overriding a route already defined by a resource with the same naming. For the latter, you can restrict the routes created with `resources` as explained here:
http://guides.rubyonrails.org/routing.html#restricting-the-routes-created
</code></pre>
<p>You can’t name two routes the same thing any more. In my case, just dropping the <code>:as</code> allowed me to get to where I needed to go:</p>
<p>Fix:</p>
<pre><code>diff --git a/config/routes.rb b/config/routes.rbindex f010adb..d573e89 100644--- a/config/routes.rb+++ b/config/routes.rb@@ -2,8 +2,8 @@ Ticketee::Application.routes.draw do root to: "projects#index"- get "/signin", to: "sessions#new", as: "signin"- post "/signin", to: "sessions#create", as: "signin"+ get "/signin", to: "sessions#new"+ post "/signin", to: "sessions#create" delete "/signout", to: "sessions#destroy", as: "signout"
</code></pre>
<h2>Clean dbs</h2>
<p>Error:</p>
<p>Tons of test failures relating to what was essentially “I don’t delete the db between test runs.”</p>
<p>Fix:</p>
<p>Upgrade to rspec-rails 2.13.2. Done!</p>
<h2>has_secure_password</h2>
<p>Error:</p>
<pre><code> 1) User passwords needs a password and confirmation to save
Failure/Error: expect(u).to_not be_valid
expected #&lt;User id: 1, name: "steve", email: "[email protected]", password_digest: "$2a$04$9D7M2iPvYavvvpb3rlLzz.vqMJdGBAQVBSJ6L.UIoXP6...", created_at: "2013-05-21 23:59:44", updated_at: "2013-05-21 23:59:44", admin: false&gt; not to be valid
</code></pre>
<p>This test failed. The test looked like this:</p>
<pre><code>it "needs a password and confirmation to save" do u = User.new(name: "steve", email: "[email protected]") u.save expect(u).to_not be_valid u.password = "password" u.password_confirmation = "" u.save expect(u).to_not be_valid u.password_confirmation = "password" u.save expect(u).to be_validend
</code></pre>
<p>Turns out, <a href="https://github.com/rails/rails/pull/10694">it’s a regression</a>! This will get fixed for RC2, but for now, just commenting out that middle section works. The final test is good enough for now.</p></content:encoded>
</item>
<item>
<title>Ditching Google Chat with XMPP</title>
<link>https://steveklabnik.com/writing/ditching-google--chat-with-xmpp/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/ditching-google--chat-with-xmpp/</guid>
<pubDate>Thu, 16 May 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Last week, I decided that it was time to migrate one of my services away from Google: XMPP for IM. The nice thing about XMPP is that you don’t have to totally drop support for your old identity: since Google still uses it (<a href="http://eschnou.com/entry/whats-next-google--dropping-smtp-support--62-24930.html">for now</a>), you can get online with both.</p>
<p>Here’s how:</p>
<hr />
<p>I set mine up with <a href="http://digitalocean.com/">Digital Ocean</a>, because my friends told me it was rad. It was pretty damn easy.</p>
<p>Once you have a server, you need to set up all the usual stuff: a non-privledged user account, yadda, yadda. I’m assuming you can do that.</p>
<h2>Install ejabberd</h2>
<p>I picked <a href="http://www.ejabberd.im/">ejabberd</a> for my server software because it’s used by a lot of people.</p>
<p>EDIT: I just got a tweet from <a href="http://twitter.com/jamesgolick">@jamesgolick</a>, who I trust to know stuff about stuff. He says:</p>
<blockquote>
<p>ejabberd is buggy. Use the erlangsolutions fork instead. - <a href="https://twitter.com/jamesgolick/status/335136677364396032">@jamesgolick</a></p>
</blockquote>
<p>So, take that for what it’s worth.</p>
<p>Anyway, since I use Arch Linux, installing was as easy as</p>
<pre><code>$ sudo pacman -S ejabberd
</code></pre>
<p>After that was over, time to edit the config. <code>ejabberd</code> is written in Erlang, and so it uses Erlang for its config. Don’t worry! It’s easy. Just copy and paste these bits in, or change them if there’s something like them in your config:</p>
<pre><code>{hosts, ["localhost", "xmpp.steveklabnik.com"]}.{acl, admin, {user, "steve", "xmpp.steveklabnik.com"}}.
</code></pre>
<p>These change my host to <code>xmpp.steveklabnik.com</code> as well as <code>localhost</code> and set my admin user to <code>[email protected]</code>.</p>
<p>Next up, we boot the node:</p>
<pre><code>sudo ejabberdctl start
</code></pre>
<p>And then register a way to administer things via the web interface:</p>
<pre><code>sudo ejabberdctl register steve xmpp.steveklabnik.com $password
</code></pre>
<p>Of course, change those to be your values, and change $password to your password. Now that you’ve typed a password into the command line, it’s good to clean out your bash history. When I did this, typing <code>history</code> showed this:</p>
<pre><code> 35 sudo ejabberdctl register ....
</code></pre>
<p>So, do this:</p>
<pre><code>$ history -d 35$ history -w
</code></pre>
<p>Or, to nuke <em>everything</em>:</p>
<pre><code>$ history -c$ history -w
</code></pre>
<p>Anywho, now that that’s over, you need to go to your domain registrar and insert an A record pointing at your server. I use <a href="https://www.namecheap.com/">NameCheap</a>.</p>
<p>Now that that’s done, hit this page in your browser:</p>
<pre><code>http://xmpp.steveklabnik.com:5280/admin/
</code></pre>
<p>The trailing slash is important, and obviously, you need to change it to be your server. Given that the DNS isn’t cached wrong, you should get a popup asking you to log in. Do so with the credentials you typed in on the command line up there. If you can log in, great! Everything should be working.</p>
<p>Let’s turn off registration, though. Go back to your <code>ejabberd</code> config, <code>/etc/ejabberd/ejabberd.cfg</code>:</p>
<pre><code>{access, register, [{deny, all}]}.
</code></pre>
<p>You probably have one that says <code>allow</code>, so just change it. Check in your admin panel on the site that it’s the same.</p>
<hr />
<p>Bam! That’s it. This whole process took me about an hour, including writing this blog post. I do have some experience with VPSes, though.</p>
<p>If I can help free you of Google in any way, ping me at <a>[email protected]</a>.</p></content:encoded>
</item>
<item>
<title>The profile link relation and you</title>
<link>https://steveklabnik.com/writing/the-profile-link-relation-and-you/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-profile-link-relation-and-you/</guid>
<pubDate>Mon, 06 May 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I was quite pleased when <a href="http://tools.ietf.org/html/rfc6906">RFC 6906</a> was finalized. It’s a really useful pattern that people are using to enhance documentation of their APIs.</p>
<p>Let’s start with an example. I tweet something like this:</p>
<blockquote>
<p>“Oh man, the example.com API is super awesome. You should check it out!” - <a href="https://twitter.com/steveklabnik">@steveklabnik</a> seconds ago from web</p>
</blockquote>
<p>You don’t know anything about this API or what it offers. So you fire up <code>curl</code>:</p>
<pre><code>$ curl -i http://example.com
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 273
{
"wtl": "MjAxMy0wNS0wNiAxMjo1Nzo1MyAtMDcwMA==\n",
"grobb34s": [
{
"flog": "Top 100 foobars",
"zilch": "http://example.com/foo/bar?baz=qux"
},
{
"flog": "Mega Troll Title",
"zilch": "http://example.com/exploit.exe/foobar"
}
]
}
</code></pre>
<p>To be blunt, this makes absolutely no sense. You tell me so, and the next day, I tell you to check it out again. You grumble, and get back to the <code>curl</code>:</p>
<pre><code>$ curl -i http://example.com
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 273
Link: &lt;http://example.com/profile&gt;; rel="profile"
{
"wtl": "MjAxMy0wNS0wNiAxMjo1Nzo1MyAtMDcwMA==\n",
"grobb34s": [
{
"flog": "Top 100 foobars",
"zilch": "http://example.com/foo/bar?baz=qux"
},
{
"flog": "Mega Troll Title",
"zilch": "http://example.com/exploit.exe/foobar"
}
]
}
</code></pre>
<p>Oh, wait. “Profile”. Let’s see what that’s about:</p>
<pre><code>$ curl -i http://example.com/profile
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 548
The Example.com API
===================
Example.com provides access to our blog through an API.
In the API, you'll see two major things of interest: `wtl` and `grobb34s`.
## wtl
The value provided under the `wtl` key is the time the latest blog post
was posted, in "%Y-%m-%d %H:%M:%S %z" format. This value is then Base64
encoded.
## grobb34s
The `grobb34s` key will hold an array of blog posts. These posts are
represented by a JSON object with two keys. `flog` has the title, suitable for
display, and `zilch` contains a link to the post.
</code></pre>
<p>Oh. You don’t care about some blog with such terrible titles, so you go about your day.</p>
<h2>self-descriptive-ness</h2>
<p>You may consider this a highly contrived example, but it’s not as contrived as you think. In Fielding’s thesis, he describes a ‘self-described messages’ constraint, which means that any API call should be able to be understood alone, without additional context.</p>
<p>In HTTP, this information is generally provided via the <code>Content-Type</code> header. It describes a media type (such as <code>application/json</code> in this example) that describes the rules for processing the response. When writing a HTTP client, you fetch the response, check the <code>Content-Type</code>, and then invoke the correct parser based on the type provided.</p>
<p>But many times are very general. Take <code>application/json</code>, for example. It says nothing about blog posts. A user-agent that only knows about <code>application/json</code> is very much like you as a human seeing gibberish keys and values; it’s not smart enough to make assumptions based on the text of keys and values it may see in JSON. However, with the added context of a profile, we have enough information to make sense of this strange API. And any user-agent that sees a profile that it recognizes can act on those new semantics, too.</p>
<h2>JSON API</h2>
<p>This is one of the reasons that we’re working on <a href="http://jsonapi.org/">JSON API</a>, a standard media type for APIs that use JSON. Generic JSON has no standard semantics that are useful to API authors, since it’s a very generic format. By declaring a new media type with common features to many APIs, we can write generic tools that handle the 80% of cases that come up during API development.</p>
<p>And if the generic case doesn’t suit your requirements, you can always extend it with profiles. In fact, it’d be nice if many people just added a <code>Link</code> header to their existing API documentation; that’d start alleviating this problem.</p>
<hr />
<p>If you enjoyed this article, you might want to check out my in-progress book on building APIs that respect standards and HTTP, <a href="http://www.designinghypermediaapis.com/">Designing Hypermedia APIs</a>. This post also serves as <a href="http://www.designinghypermediaapis.com/blog/the-profile-link-relation-and-you.html">its first blog post</a>.</p></content:encoded>
</item>
<item>
<title>Closure</title>
<link>https://steveklabnik.com/writing/closure/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/closure/</guid>
<pubDate>Fri, 19 Apr 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I spent six weeks of last December/January in my hometown. It was the first Christmas without my father, and it was cheap to stay, so I went back. It was my first time back in Pittsburgh since I’d moved away from the only place I’d ever lived.</p>
<p>The trip was incredibly emotional for many reasons, but one of the best was when I decided to walk down the street to get a pizza. I had to wait ten minutes for it to cook, and so I decided to open up my web browser. Then I saw this:</p>
<p><img src="https://svbtleusercontent.com/steveklabnik_24595149844116_small.png" alt="https://svbtleusercontent.com/steveklabnik_24595149844116_small.png" /></p>
<p>Whoah.</p>
<p>You see, I’ve always had a complex relationship with _why. I had been dabbling in Ruby when he was still alive, and planned on dropping by this ART &amp;&amp; CODE thing I’d heard about over at CMU, but I was out of town that weekend, so I shrugged and didn’t think twice.</p>
<p>Then he disappeared.</p>
<p>I started looking into _why and what he was actually about. What I found was amazing, and you already know that part of the story. I thought Hackety Hack was a masterpiece, and decided to help maintain it. Unfortunately for me, nobody else did. I hadn’t done a whole lot of open source work before, and now I was maintaining a big, important project by one of the most well-known Rubyists of all time. Whoah.</p>
<p>I cared about Hackety enough and felt inspired enough by _why that I actually quit my startup to spend more time working on it.</p>
<p>Here’s the weird thing about someone disappearing: you can’t ask them questions. I had so many things that I wanted to ask, so many unresolved questions, so much I wanted to know. It’s really weird to take someone’s life’s work and make it your own. I spent a lot of that first year with crippling doubts about what I was doing, if _why would like it, if that even mattered, wondering why he’d abandoned something so important, why he’d abandoned <em>us</em>. And I felt terrible that I <em>needed</em> this from him. Why does he owe us all that? Who am I to impose?</p>
<p>So I marked my calendar for the 18th, and waited. That was yesterday.</p>
<p><img src="http://distilleryimage6.s3.amazonaws.com/a8767b30a88f11e2896922000a1fbe1a_7.jpg" alt="http://distilleryimage6.s3.amazonaws.com/a8767b30a88f11e2896922000a1fbe1a_7.jpg" /></p>
<p><img src="http://distilleryimage11.s3.amazonaws.com/c22b37d2a8bc11e287e222000aaa0aa2_7.jpg" alt="http://distilleryimage11.s3.amazonaws.com/c22b37d2a8bc11e287e222000aaa0aa2_7.jpg" /></p>
<p>It was… great. Just great. The stuff _why sent is basically everything I’d ever hoped. It’s got so much buried inside. An IF game, commentary on Jobs vs. Gates, a very person struggle with identity, a comment that he likes what we’ve done… it’s great. And it’s finally brought me some closure.</p>
<p><a href="https://github.com/steveklabnik/CLOSURE">So, I’ve saved it all for <em>you</em>. &lt;3</a></p>
<p><img src="https://svbtleusercontent.com/steveklabnik_24595264609650_small.jpg" alt="https://svbtleusercontent.com/steveklabnik_24595264609650_small.jpg" /></p></content:encoded>
</item>
<item>
<title>Upgrading from Rust 0.5 to 0.6</title>
<link>https://steveklabnik.com/writing/upgrading-from-rust-0-5-to-0-6/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/upgrading-from-rust-0-5-to-0-6/</guid>
<pubDate>Sun, 07 Apr 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Here are some error messages that I got when updating <a href="http://www.rustforrubyists.com/">Rust for Rubyists</a> from Rust 0.5 to the new 0.6, and how to fix those errors.</p>
<hr />
<p>Error:</p>
<pre><code>rust.rs:1:4: 1:16 error: unresolved name
rust.rs:1 use task::spawn;
^~~~~~~~~~~~
rust.rs:1:4: 1:16 error: failed to resolve import: task::spawn
rust.rs:1 use task::spawn;
^~~~~~~~~~~~
error: failed to resolve imports
error: aborting due to 3 previous errors
make: *** [all] Error 101
</code></pre>
<p>or</p>
<pre><code>rust.rs:1:4: 1:16 error: unresolved name
rust.rs:1 use io::println;
^~~~~~~~~~~~
rust.rs:1:4: 1:16 error: failed to resolve import: io::println
rust.rs:1 use io::println;
^~~~~~~~~~~~
error: failed to resolve imports
error: aborting due to 3 previous errors
make: *** [all] Error 101
</code></pre>
<p>The fix:</p>
<p>Imports are now more explicit, so you can’t just import things from <code>core</code> without specifying that you want to any more.</p>
<pre><code>+ use core::task::spawn;- use task::spawn;
</code></pre>
<p>or</p>
<pre><code>+ use core::io::println;- use io::println;
</code></pre>
<p>etc.</p>
<hr />
<p>Error:</p>
<pre><code>error: main function not found
error: aborting due to previous error
</code></pre>
<p>The fix:</p>
<hr />
<p>When I used to write TDD’d Rust, I wouldn’t often write a <code>main</code> function until I had done a bunch of tests. This was okay, but now, you need one.</p>
<pre><code>+ fn main() {
+ }
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rustc rust.rs --test
rust.rs:5:2: 5:6 error: unresolved name: `fail`.
rust.rs:5 fail;
^~~~
error: aborting due to previous error
make: *** [build_test] Error 101
</code></pre>
<p>The fix:</p>
<p><code>fail</code> was turned into a macro. It now needs to be passed an owned pointer to a string:</p>
<pre><code>- fail;
+ fail!(~"Fail!");
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:5:18: 5:26 error: unresolved name: `int::str`.
rust.rs:5 println(int::str(num))
^~~~~~~~
error: aborting due to previous error
make: *** [build] Error 101
</code></pre>
<p>The fix:</p>
<p><code>int::str</code> is now <code>int::to_str</code></p>
<pre><code>- int::str
+ int::to_str
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:5:12: 5:18 error: cannot determine a type for this local variable: unconstrained type
rust.rs:5 let mut answer;
^~~~~~
error: aborting due to previous error
make: *** [build] Error 101
</code></pre>
<p>The fix:</p>
<p>You have to tell it what kind it is. In my case, it was a string:</p>
<pre><code>- let mut answer;
+ let mut answer = "";
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:3:11: 3:21 error: expected `;` or `}` after expression but found `is_fifteen`
rust.rs:3 assert is_fifteen(15)
^~~~~~~~~~
make: *** [build_test] Error 101
</code></pre>
<p>The fix:</p>
<p>This happens because <code>assert</code> is now a macro.</p>
<pre><code>- assert is_fifteen(15)
+ assert!(is_fifteen(15))
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rustc rust.rs
rust.rs:11:10: 11:24 error: the type of this value must be known in this context
rust.rs:11 chan.send(10);
^~~~~~~~~~~~~~
error: aborting due to previous error
make: *** [build] Error 101
</code></pre>
<p>The fix:</p>
<p>Rust 0.6 got rid of ‘capture clauses.’ Don’t worry about it.</p>
<pre><code>- do spawn |chan| {
+ do spawn {
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:10:17: 10:21 error: expected `,` but found `chan`
rust.rs:10 do spawn |move chan| {
^~~~
make: *** [build] Error 101
</code></pre>
<p>The fix:</p>
<p><code>move</code> is gone. Just remove it.</p>
<pre><code>- do spawn |move chan| {
+ do spawn |chan| {
</code></pre>
<p>Note that this diff wouldn’t actually make it work, you’d still run into the issue above. But it gets rid of the <code>move</code> error.</p>
<hr />
<p>Error:</p>
<pre><code>rust.rs:13:6: 13:12 error: unexpected token: `static`
rust.rs:13 static fn count() {
^~~~~~
make: *** [build] Error 101
</code></pre>
<p>The fix:</p>
<p><code>static</code> was removed. Any method that doesn’t take <code>self</code> is static.</p>
<pre><code>- static fn count() {
+ fn count() {
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:11:30: 11:37 error: obsolete syntax: colon-separated impl syntax
</code></pre>
<p>The fix:</p>
<p>As it says, the colon syntax is gone. Replace it with <code>for</code>:</p>
<pre><code>- impl float: Num
+ impl Num for float
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:5:18: 5:22 error: use of undeclared type name `self`
rust.rs:5 fn new() -&gt; self;
^~~~
</code></pre>
<p>The fix:</p>
<p>This happens when you’re making a trait, and you want the implementations to return whatever type they are. The type name is now <code>Self</code>:</p>
<pre><code>- fn new() -&gt; self;
+ fn new() -&gt; Self;
</code></pre>
<hr />
<p>I hope you found that informative! Happy hacking!</p></content:encoded>
</item>
<item>
<title>Deleuze for Developers: will smooth space/open source suffice to save us?</title>
<link>https://steveklabnik.com/writing/deleuze-for-developers-will-smooth-spaceopen-source-suffice-to-save-us/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/deleuze-for-developers-will-smooth-spaceopen-source-suffice-to-save-us/</guid>
<pubDate>Thu, 14 Mar 2013 00:00:00 GMT</pubDate>
<content:encoded><p>If you truly want to understand technology today, then you should at least be familiar with the philosophy of Gilles Deleuze. Unfortunately for technologists, Deleuze is rooted firmly in a philosophical tradition and a writing style that they probably find opaque. In this blog series, I plan on explaining Deleuze’s philosophy in terms that programmers can understand. This is the third in the series. You can find the first <a href="/deleuze-for-developers-assemblages">here</a>. Enjoy.</p>
<hr />
<p>Deleuze and Guattari employ the notion of ‘smooth space’ quite a bit in <em>A Thousand Plateaus</em>. I saw this quote about it the other day:</p>
<blockquote>
<p>never believe that a smooth space will suffice to save us</p>
</blockquote>
<p>I’ve recently been giving a lot of thought to <a href="http://en.wikipedia.org/wiki/Cultural_hegemony">‘cultural hegemony’</a> and its applicability to the web, specifically along the lines of Google. Basically, cultural hegemony is the idea that you don’t just have dominance through laws or economics, but through culture. So, for example, Google submitted SPDY to become HTTP 2.0: it’s going through the standards process, it’s ‘open,’ but it also gives Google a lot of cultural leverage.</p>
<p>So there’s this connection there between smooth space and open source, and the battle between the Nomads and the State Apparatus. We’ll cover that second bit in a while, but for now: what is ‘smooth space’?</p>
<h2>The spatial metaphor</h2>
<p>The first thing to understand about these concepts is that Deleuze and Guattari is (I think of them as a singular entity…) a big fan of spatial metaphors. Essentially, we can project our problem domain onto a 2 or 3 (or more) dimensional space, and then use tools to analyze that space. I was first introduced to this technique by Manuel de Landa, who has a great lecture called “Deleuze and the use of the genetic algorithm in archetecture” (<a href="http://www.youtube.com/watch?v=50-d_J0hKz0">video</a>)(<a href="http://www.cddc.vt.edu/host/delanda/pages/algorithm.htm">text</a>). A quick rundown:</p>
<p>So, let’s talk about the <a href="http://en.wikipedia.org/wiki/Eight_queens_puzzle">N queens problem</a>. We have a chess board with one space, and we place one queen on it:</p>
<p>(warning, lots of ASCII (+ unicode) art to follow)</p>
<pre><code>.-.
|♕|
.-.
</code></pre>
<p>This board is legal. It’s a solution to the ‘1 queen problem’: one queen, a 1x1 board. What about 2 queens? We have no solutions. (2 and 3 don’t. :/ All other natural numbers do.) First we place a queen:</p>
<pre><code>.---.
|♕| |
-----
| | |
.---.
</code></pre>
<p>… but there’s no legal place for the second one. Bummer. We can determine that there’s no solution by using a simple brute force with backtracking: place the queen in the upper right, then try to place the second below it. That’s not legal, so we backtrack and place it on the right. That’s not legal, so we backtrack and put it in the bottom corner. Oh no! That didn’t work, so let’s move our first queen: now it’s in the bottom right, and we try to place the second in the upper right: fail! So we backtrack….</p>
<p>As you can see, this is a lot of steps for a small board. Once you get up to a ‘real’ chess board, there are 4,426,165,368 possible placements, but only 92 solutions. A needle in a haystack! And we’re using a lot of steps to determine if there’s even one possible placement. We need something better.</p>
<p>One answer is genetic algorithms. So we can take our board’s current state and assign it a score. When we place a new queen, if it makes our score better, we keep it, and if it doesn’t, we lose it. Now we’ve taken our 2x2 board and projected it onto a 2-axis linear space. Imagine a plot where queens are on the x axis and score is on the y axis:</p>
<pre><code> .dAHAd.
.adAHHHAbn.
.dAHAd. .adAHHHHHHAbn.
.adAHHHAbn. .adAHHHHHHHHAbn.
dHHHHHHHHHHHHHHHHHHHHHHHHHHHHHb
dHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHb
</code></pre>
<p>So we want to keep going with our solution as long as it slopes up: we’ve found that our answers are at the maximum of each curve. Yay calculus! But here’s the thing: this curve only has two possible maxima: we will have more. And it might not be in 2-d space, it might be in 5-d space. And we can only ‘use math’ to find the maximum if we generate the <em>entire</em> curve, which seems computationally out of our reach. So how do we find the maximum without generating the whole curve? Genetic algorithms!</p>
<p>One such is ‘<a href="http://en.wikipedia.org/wiki/Simulated_annealing">simulated annealing</a>.’ Without getting too into it, let’s just say that there’s a ‘cooling factor’ that controls how tolerant we are of going back down a slope. So at first, we wildly go all over the search space, but then, as we progress, we tighten up our cooling factor, and we stop being so wild. Eventually, we’ll arrive at a solution that’s very likely to be the true global maxima/minima. Neat!</p>
<h2>Discrete vs. Continuous</h2>
<blockquote>
<p>If we know that the enemy is open to attack, and also know that our men are in a condition to attack, but are unaware that the nature of the ground makes fighting impracticable, we have still gone only halfway towards victory.Sun Tzu, “the Art of War”</p>
</blockquote>
<p>Another interesting feature of this particular projection is that it transforms our problem from a discrete problem into a continuous one. One great tactic for when you’re losing: change the battlefield. Our genetic algorithm tool needs a continuous space to operate, but our problem is that our chess board is discrete. What do I mean by this?</p>
<p>“<a href="http://en.wikipedia.org/wiki/Discrete_space">discrete space</a>” is one in which the points are separated from one another in some way. As an example, ‘integers’ form a discrete topology: there’s a ‘gap’ between 1, 2, 3, and 4. You can see this by drawing a <a href="http://en.wikipedia.org/wiki/Number_line">number line</a>:</p>
<pre><code>&lt;--(-2)--(-1)--(0)--(1)--(2)--&gt;
</code></pre>
<p>The real numbers form a <a href="http://en.wikipedia.org/wiki/Continuity_%28topology%29">continuous topology</a> rather than a discrete one, there is no space between them. A ‘<a href="http://en.wikipedia.org/wiki/Real_line">real line</a>’:</p>
<pre><code>&lt;-------------------------&gt;
</code></pre>
<p>Our ‘scoring’ mechanism allows us to change the battlefield, it forms a function (isomorphism) to convert our discrete topology into a continuous one. We can now bring our continuous tooling to bear on a problem that was previously inaccessible.</p>
<h2>Striated vs Smooth Space</h2>
<blockquote>
<p>Military tactics are like unto water; for water in its natural course runs away from high places and hastens downwards. So in war, the way is to avoid what is strong and to strike at what is weak. Like water, taking the line of least resistance. Water shapes its course according to the nature of the ground over which it flows; the soldier works out his victory in relation to the foe whom he is facing. Therefore, just as water retains no constant shape, so in warfare there are no constant conditions.Sun Tzu, “the Art of War”</p>
</blockquote>
<p>Okay. NOW we’re ready to talk about smooth and striated space. They have a number of names for this concept, and the one that’s most direct from where we currently are is ‘Riemann space / Euclidean space’. Smooth -&gt; Riemann, Euclidean -&gt; Striated. Another isomorphism. ;)</p>
<p><a href="http://en.wikipedia.org/wiki/Euclidean_geometry">Euclidean geometry</a> is pretty much the foundation of a metric ton of our math. It’s what’s now known as algebra and geometry: lots of integers, discrete spaces. This cube starts at (1,2,4) and has a side length of 5. Cartesian coordinates.</p>
<p>Along comes <a href="http://en.wikipedia.org/wiki/Carl_Friedrich_Gauss">Gauss</a>, who was incredibly intelligent. He had this student named <a href="http://en.wikipedia.org/wiki/Bernhard_Riemann">Riemann</a>, who ran with the ideas Gauss had and started <a href="http://en.wikipedia.org/wiki/Riemannian_geometry">Riemannian Geometry</a>. Riemann’s geometry is <a href="http://en.wikipedia.org/wiki/Non-Euclidean_geometry">non-Euclidean</a>: if Euclid described lines, Riemann described curves. Einstein would later base the theory of relativity on Riemannian geometry.</p>
<p>So, who cares? Well, think about it this way: You’re in a car, driving down a straight road. It goes for miles, with no turns or other streets. You’re operating in a smooth space: you can easily go from one place to another. Now, someone comes along and puts up stop signs every block. You must (for now) stop at these signs. Now you’re operating in a striated space: your movement is restricted. Now you’re moving from point to point on a line. It’s a very different experience, and you also didn’t have anything to do with the stop signs….</p>
<p>According to D&amp;G, ‘the state machine’ (one instance of which is the State) attempts to take space that is smooth and striate it. This is often a means towards imperialism, as the striation happens through quantification. Take, for example, the sea: a smooth space, you can impose latitude and longitude on top of it, simultaneously quantifying and striating it. This allows you to navigate the waters, and conquer the sea.</p>
<p>This is also the position that many web startups take: ‘friendship’ was a smooth space before social networks came along. They quantified and striated that space, turning “I have many friends” into “I have 200 friends and you have 100 friends.” This quantification allows for commodification: now Facebook can sell ads. It’s also why startups don’t often have a ‘business model’ at first: they first need to conquer and striate their space before they find the commodity. Sometimes you just hit dirt and go bust, sometimes you find a more precious commodity, then refine and sell it.</p>
<p>‘nomads’ are entities which navigate and live in smooth spaces. Maybe my twitter bio makes a bit more sense now. ;)</p>
<h2>How do you create a smooth space?</h2>
<p>You might remember Riemann from the ‘<a href="http://en.wikipedia.org/wiki/Riemann_sum">Riemann sum</a>’ in Calc I. The brilliance of the Riemann sum is that it first striates, then re-smooths the space. You start off first with a curve, and then approximate the area under it by dividing it into a number of evenly-sized bits. This first bit was necessary in a world without calculus: we didn’t have the tooling or concepts to actually tackle the real area, so we mapped that problem to one we did know how to solve. As the number of bits goes up, and their width goes down, more and more of the space under the curve is captured by our algorithm. Finally, once we’re able to take that first step away from (striated) algebra and move into (smooth) calculus, we’re able to move about on the curve for reals. We’re back to smooth space again.</p>
<p>This interplay between smooth and striated spaces often happens, and there’s really no part of our world today that’s completely smooth or entirely striated. D&amp;G posit that the left should be attempting to create as much smooth space as possible, and that capitalism is constantly attempting to striate space. That said, smooth space is necessary, but not sufficient: capitalism, in its hunger and lust for acquisition and totalitization, has managed to navigate some smooth spaces. I’ll just quote straight from ‘a thousand plateaus’, 492. It’s a little hard, but this is getting long enough as is. ;)</p>
<blockquote>
<p>Not only does the user as such tend to be an employee, but capitalism operates less on a quantity of labor than by a complex qualitative process bringing into play modes of transportation, urban models, the media, the entertainment industries, ways of perceiving and feeling – every semiotic system. It is though, at the outcome of the striation that capitalism was able to carry to an unequaled point of perfection, circulating capital necessarily recreated, reconstituted, a sort of smooths pace in which the destiny of human beings is recast. … at the level of world capitalism, a new smooth space is produced in which capital reaches its ‘absolute’ speed, based on machinic components rather than the human component of labor. The multinationals fabricate a kind of deterritorialized smooth space in which points of occupation as well as poles of exchange become quite independent of the classical paths to striation. .. the essential thing is instead the distinction between striated capital and smooth capital, and the way in which the former gives rise to the latter through complexes that cut across territories and States, and even the different types of States.</p>
</blockquote>
<p>The first part about ‘users’ is very much drawing a parallel to “If you’re not paying for the product, you are the product.” When they talk about the ‘speed’ of capital, think of HFT. Multinational corporations have managed to overcome the striation that a State has imposed. McDonalds is only partially affected by US laws. They operate ‘outside’ of states.</p>
<p>In this way, simply being smooth will not suffice to save us.</p>
<hr />
<p>If you liked this blog post, you may enjoy <a href="https://groups.google.com/forum/#!forum/philosophy-in-a-time-of-software">"Philosophy in a time of software</a>, a moderated Google Group to discuss the intersection of philosophy and technology. This post started out as an email to that list.</p></content:encoded>
</item>
<item>
<title>Travis build matrix for Rails</title>
<link>https://steveklabnik.com/writing/travis-build-matrix-for-rails/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/travis-build-matrix-for-rails/</guid>
<pubDate>Sun, 10 Mar 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Do you have a gem that needs to test against multiple versions of Rails? Doing it all can be complex, and while I have lots to say about this topic, but here’s one of the pieces: Travis build matrix.</p>
<p>Here’s what you want:</p>
<pre><code>language: ruby
rvm:
- 1.8.7
- 1.9.2
- 1.9.3
- 2.0.0
- ruby-head
env:
- "RAILS_VERSION=3.2"
- "RAILS_VERSION=3.1"
- "RAILS_VERSION=3.0"
- "RAILS_VERSION=master"
matrix:
allow_failures:
- rvm: ruby-head
- env: "RAILS_VERSION=master"
exclude:
- rvm: 2.0.0
env: "RAILS_VERSION=3.0"
- rvm: 2.0.0
env: "RAILS_VERSION=3.1"
- rvm: 1.8.7
env: "RAILS_VERSION=master"
- rvm: 1.9.2
env: "RAILS_VERSION=master"
- rvm: ruby-head
env: "RAILS_VERSION=3.0"
- rvm: ruby-head
env: "RAILS_VERSION=3.1"
</code></pre>
<p>Here’s what all this does:</p>
<ol>
<li>1.8.7 and 1.9.2 is supported across all of the 3.x series, but not master</li>
<li>2.0.0 is supported on Rails 3.2 (as of the upcoming 3.2.13, but it will probably work with your gem with 3.2.12) and master, but not 3.0 or 3.1.</li>
<li>We want to allow failures on ruby-head, because it’s not likely to be stable</li>
<li>We also want to allow failures for Rails master, because sometimes it’s not stable either</li>
<li>ruby-head should only be built against master</li>
</ol>
<p>What do you think? <a href="http://twitter.com/steveklabnik">Tweet at me</a> if you have suggestions for improving this matrix.</p></content:encoded>
</item>
<item>
<title>Announcing security_release_practice</title>
<link>https://steveklabnik.com/writing/announcing-securityreleasepractice/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/announcing-securityreleasepractice/</guid>
<pubDate>Fri, 08 Mar 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Security is hard. One of the skills any OSS maintainer needs is how to do releases, and security releases are a special kind of release that needs special git-fu to make it work.</p>
<p>I’ve created a repository so that you can practice this particular skill. It’s called ‘security_release_practice’.</p>
<h2>The problem</h2>
<p>‘security_release_practice’ provides a binary, <code>omg_insecure</code>, which has a security issue. Basically, the <code>super_secure_calculation</code> method converts user input to a symbol. Since Ruby does not garbage collect symbols, the longer you run <code>omg_insecure</code>, the more memory it will use, until your computer runs out of memory and the box grinds to a halt. Seems bad.</p>
<p>So, just fix <code>super_secure_calculation</code> and release, right? Well, here’s the problem: the last release of <code>security_release_practice</code> was 1.0.0. Since then, we’ve had a new feature, and a backwards incompatible change with <code>super_secure_calculation</code>. You can see the two commits <a href="https://github.com/steveklabnik/security_release_practice/compare/v1.0.0...master">here</a>.</p>
<p>This is a problem: if we fix the issue and release, people who are relying on the <code>+ 5</code> behavior can’t upgrade: they’ll now be getting <code>+ 6</code>. Also, the new feature (<code>another_new_calculation</code>) may have conflicts or weirdness with their code. That’s bad! So what we really want is a relase that’s exactly the same as 1.0.0, but with the security fix applied.</p>
<p>Let’s give that a shot.</p>
<h2>The answer</h2>
<p>If you think you’re good with <code>git</code>, you can try this out right now. If you’ve done it correctly, you should end up with the following:</p>
<ol>
<li>A 1-0-stable branch</li>
<li>That branch should contain a new commit that fixes the issue</li>
<li>That branch should contain a new tag, v1.0.1 that fixes the issue</li>
<li>Master should have a backported version of the commit in #2.</li>
</ol>
<p>The repository <a href="https://github.com/steveklabnik/security_release_practice">as it exists</a> has all of this stuff, so check your work against it!</p>
<h2>Practice!</h2>
<p>If you <em>don’t</em> know how to do this, or you get stuck, you’ve come to the right place! Here’s what you need to do:</p>
<p>First, some setup work. Fork the repository and clone it down. Or, just clone mine, whatever:</p>
<pre><code>$ git clone https://github.com/steveklabink/security_release_practice
$ cd security_release_practice
</code></pre>
<p>Next, since this repository has the backported fix involved, you need to remove that commit:</p>
<pre><code>$ git reset --hard HEAD~1
</code></pre>
<p>This basically backs our branch out by one commit. Now we’re ready to go.</p>
<p>The first thing in actually doing the work is to check out the tag that we last released from. In our case, that tag is <code>v1.0.0</code>. So let’s do that now:</p>
<pre><code>$ git checkout v1.0.0
</code></pre>
<p><code>git</code> will give you a message:</p>
<pre><code>Note: checking out 'v1.0.0'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b new_branch_name
HEAD is now at 47a8bfb... Initial release.
</code></pre>
<p>We want to take <code>git</code>’s advice: let’s make a new branch. Since it’s going to be all our fixes for <code>1.0.x</code>, let’s call it <code>1-0-stable</code>:</p>
<pre><code>git checkout -b 1-0-stable
</code></pre>
<p>Now we have our stable branch. Awesome! Master is the work that will go into <code>1.1.x</code>, and this branch will be for <code>1.0.x</code>.</p>
<p>Next, we need to fix our bug. You need to remove this one line:</p>
<pre><code>--- a/lib/security_release_practice.rb+++ b/lib/security_release_practice.rb@@ -3,7 +3,7 @@ require "security_release_practice/version" module SecurityReleasePractice def super_secure_calculation(input)- input.to_sym input.to_i + 5 end def another_new_calculation(input)
</code></pre>
<p>We don’t even use that symbol, what a waste! Commit this, with a descriptive message. You can <a href="https://github.com/steveklabnik/security_release_practice/commit/168d5f756221ed43b0c67569ac82429f0b391504">find mine here</a>. Note the first few characters of the hash: mine is <code>168d5f756221</code>.</p>
<p>Next, we need to release. Go ahead and increment the version number in <code>lib/security_release_practice/version.rb</code>, commit that, and then try <code>rake install</code>. Everything should work. Great! If you were actually releasing this gem, you’d be running <code>rake release</code> instead, but it’s my gem, not yours. ????</p>
<p>Okay, now we’ve released a version with our fix, but <code>master</code> still has a vulnerability: we need to port the fix. So let’s go back to master:</p>
<pre><code>$ git checkout master
</code></pre>
<p>And then cherry-pick our fix over:</p>
<pre><code>$ git cherry-pick 168d5f756221
</code></pre>
<p>There will be a conflict. The diff is a little weird, such is life. Go ahead and fix the conflict, then commit:</p>
<pre><code>$ git commit
</code></pre>
<p>And you’re done!</p>
<h3>Other considerations</h3>
<p>If we had a CHANGELOG, we’d need to udpate that as appropriate, including creating new sections for our <code>1.0.1</code> release.</p>
<p>Sometimes it’s easier to fix things on master first, then backport to the new branch. I prefer to do it the way I showed here.</p>
<p>You should probably try to get as many people to know that you’ve fixed the bug. Tweet, blog, rabble-rouse, and possibly <a href="https://groups.google.com/forum/#!forum/ruby-security-ann">post to the ruby-security-ann mailing list</a>, which was created to help Rubyists know about security releases of their gems.</p>
<p>If your gem is really widely used, you may want to actually register a CVE. You can find information on this process <a href="https://groups.google.com/forum/#!forum/ruby-security-ann">here</a>. I made one <a href="https://groups.google.com/d/msg/ruby-security-ann/TDXOlIVjS54/Ty_9PXYNr3AJ">here</a>.</p></content:encoded>
</item>
<item>
<title>Going vimgan</title>
<link>https://steveklabnik.com/writing/going-vimgan/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/going-vimgan/</guid>
<pubDate>Wed, 06 Mar 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I’m going vim-gan. Basically, I don’t want to consume any source code which doesn’t come from vim, or products that contain non-vim source code.</p>
<p>While I won’t, some people I’ve talked to will also consume code produced in Eclipse, since recent scientific studies show that Eclipse users can’t actually feel pain. This is enough of an ethical gray area that I’m staying away from it, but I can see that perspective.</p>
<p>I just can’t stay silent while the software industry brutalizes source code like this. Not cool, everyone.</p></content:encoded>
</item>
<item>
<title>Using puma on Heroku</title>
<link>https://steveklabnik.com/writing/using-puma-on-heroku/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/using-puma-on-heroku/</guid>
<pubDate>Sun, 24 Feb 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I’m a big fan of <a href="http://puma.io/">Puma</a> these days. Today, I converted my blog over to use Puma, and I figured I’d share how I did it. This blog uses Sinatra.</p>
<p>If you want to look ahead, <a href="https://github.com/steveklabnik/blog/commit/3cd0f04ed29a25df6d6dfeeffccd2e12548c05cf">here</a> is the commit.</p>
<h2>Gemfile</h2>
<p>Installing Puma is really easy: just add <code>gem 'puma'</code> to your Gemfile. Done!</p>
<p>This section is way too easy. I’m sorry Puma isn’t more complicated. ;)</p>
<h2>Procfile</h2>
<p>Heroku attempts to guess how you want to run your site, but sometimes it guesses wrong. I prefer to be a bit more explicit, and in Puma’s case, it will guess wrong and use Webrick, so we need to do this step.</p>
<p>Heroku uses a gem called <code>foreman</code> to run your application. Basically, <code>foreman</code> gives you a ‘Procfile’ that lets you specify different processes and their types, and then foreman manages running them for you.</p>
<p>To use <code>foreman</code>, add it to your Gemfile:</p>
<pre><code>gem 'foreman'
</code></pre>
<p>And then make a <code>Procfile</code>:</p>
<pre><code>web: rackup -s puma -p $PORT
</code></pre>
<p>This says “I want my web processes to run this command.” Simple. We tell <code>rackup</code> to use Puma with the <code>-s</code> flag, for ‘server.’</p>
<p>To start up your application, normall you’d run <code>rackup</code> yourself, but now, you use <code>foreman</code>:</p>
<pre><code>$ bundle exec foreman start
</code></pre>
<p>It’ll print out a bunch of messages. Mine looks like this:</p>
<pre><code>$ bundle exec foreman start
15:05:05 web.1 | started with pid 52450
15:05:06 web.1 | Puma 1.6.3 starting...
15:05:06 web.1 | * Min threads: 0, max threads: 16
15:05:06 web.1 | * Environment: development
15:05:06 web.1 | * Listening on tcp://0.0.0.0:5000
</code></pre>
<p>Now you can <code>open http://localhost:5000</code> in another terminal, and bam! Done.</p>
<h2>Other Rubies</h2>
<p>If you really want Puma to scream, you should run with Rubinius or JRuby. I haven’t done this yet, but once I do, I’ll update this post with instructions.</p>
<h2>Rails</h2>
<p>If you’re using Rails, it’s the same process, but the Procfile line should be</p>
<pre><code>web: bundle exec puma -p $PORT
</code></pre>
<p>Easy!</p></content:encoded>
</item>
<item>
<title>I'm writing "Rails 4 in Action"!</title>
<link>https://steveklabnik.com/writing/i-m-writing--rails-4-in-action--/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/i-m-writing--rails-4-in-action--/</guid>
<pubDate>Wed, 13 Feb 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Last night, I took a big step forward in my writing career: I’ll be building upon the work Ryan Bigg was doing to take “Rails 3 in Action” and move it to Rails 4. I signed the contract a few weeks ago, but last night I actually did the first bit of editing, and successfully got everything through the system.</p>
<p>Here’s the <a href="http://www.manning.com/bigg2/">early access edition</a> link, where you can check out what I’m doing. Of course, Rails 4 isn’t actually out yet, but since I’m on the Rails team, I already keep abreast of changes daily. I’ll be doing a first pass to take the book from 3.2 to edge over the next few weeks, and then keeping it current until the actual release of Rails 4.</p>
<p>Of course, updating the content is much easier than writing it in the first place. Thanks for all of your hard work on previous editions, Yehuda and Ryan!</p>
<hr />
<p>Of course, this doesn’t mean I’ll stop working my my two self-published books, <a href="http://www.designinghypermediaapis.com/">Designing Hypermedia APIs</a> and <a href="http://www.rustforrubyists.com/">Rust for Rubyists</a>. It just means I’ll be doing even more writing!</p></content:encoded>
</item>
<item>
<title>The next iteration of "Designing Hypermedia APIs"</title>
<link>https://steveklabnik.com/writing/the-next-iteration-of-designing-hypermedia-apis/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-next-iteration-of-designing-hypermedia-apis/</guid>
<pubDate>Tue, 12 Feb 2013 00:00:00 GMT</pubDate>
<content:encoded><p>I sent out an email today to everyone who’d previously purchased <a href="http://www.designinghypermediaapis.com/">Designing Hypermedia APIs</a>. Here’s the text:</p>
<pre><code>Hey there,
First of all, I want to apologize for sending you an email. I try to
keep these to a minimum. But this one is important.
I haven't been uploading new content lately for a few reasons.
Holidays are complicated, I wasn't feeling inspired... but this is the
real reason: I've totally moved the site to something different. I
just updated it, so the DNS might need an update, but
http://www.designinghypermediaapis.com/ is all-new. Here's the deal:
1. Gone is logging in and reading it online. PDF/ePUB/MOBI now.
2. Because you purchased a copy before, you get the big bundle. Thank
you for your support over the last year.
3. You'll be getting another email with a link to download the files.
When I send an update, you'll get a new email with a new link.
4. The current written content is basically an appendix: I'm going to
be writing a more traditional book, from start to finish. It'll walk
you through building an application, both client and server.
5. I want this to be of the highest quality, so I'm not going to
commit to a schedule.
If you aren't a part of the mailing list, we're doing a reading club
over the next few months, "Building Hypermedia APIs with HTML5 and
Node." You can join by emailing [email protected].
Thank you once again for your support of this project, I really
appreciate it. As always, any and all feedback welcome: I bet I can do
some nicer formatting of the PDF/ePUB/MOBI now that I have them
generating.
</code></pre>
<p>Wooo!</p></content:encoded>
</item>
<item>
<title>Ruby on Rails maintenance policy</title>
<link>https://steveklabnik.com/writing/ruby-on-rails-maintenance-policy/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/ruby-on-rails-maintenance-policy/</guid>
<pubDate>Mon, 11 Feb 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Recently, the Rails team has committed to a specific policy related to release maintenance. Due to the rapid pace of recent releases, it’s good to understand how your apps relate to this policy.</p>
<p>The policy was originally posted on Google Groups, here: <a href="https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-security/G4TTUDDYbNA">https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-security/G4TTUDDYbNA</a>.</p>
<p>Here’s a copy if you don’t want to read Google Groups:</p>
<hr />
<p>Since the most recent patch releases there has been some confusion about what versions of Ruby on Rails are currently supported, and when people can expect new versions. Our maintenance policy is as follows.</p>
<p>Support of the Rails framework is divided into four groups: New features, bug fixes, security issues, and severe security issues. They are handled as follows:</p>
<h2>New Features</h2>
<p>New Features are only added to the master branch and will not be made available in point releases.</p>
<h2>Bug fixes</h2>
<p>Only the latest release series will receive bug fixes. When enough bugs are fixed and its deemed worthy to release a new gem, this is the branch it happens from.</p>
<p>Currently included series: 3.2.x</p>
<h2>Security issues:</h2>
<p>The current release series and the next most recent one will receive patches and new versions in case of a security issue.</p>
<p>Currently included series: 3.2.x, 3.1.x</p>
<h2>Severe security issues:</h2>
<p>For severe security issues we will provide new versions as above, and also the last major release series will receive patches and new versions. The classification of the security issue is judged by the core team.</p>
<p>Currently included series: 3.2.x, 3.1.x, 2.3.x</p>
<h2>Unsupported Release Series</h2>
<p>When a release series is no longer supported, it’s your own responsibility to deal with bugs and security issues. We may provide back-ports of the fixes and publish them to git, however there will be no new versions released. If you are not comfortable maintaining your own versions, you should upgrade to a supported version.</p>
<p>You should also be aware that Ruby 1.8 will reach End of Life in June 2013, no further ruby security releases will be provided after that point. If your application is only compatible ruby 1.8 you should upgrade accordingly.</p>
<p>– Cheers,</p>
<p>Koz</p></content:encoded>
</item>
<item>
<title>Ember.js and accessibility</title>
<link>https://steveklabnik.com/writing/emberjs-and-accessibility/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/emberjs-and-accessibility/</guid>
<pubDate>Thu, 07 Feb 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Hey everyone! I made a video today about Ember.js and accessibility. I had always been repeating what I’d heard about screen readers: they can’t work with JavaScript. Turns out that’s not exactly true. The video is the best way to show you:</p>
<p><a href="http://vimeo.com/59124303">VoiceOver, Ember.js, and WAI-ARIA</a> from <a href="http://vimeo.com/steveklabnik">Steve Klabnik</a> on <a href="http://vimeo.com/">Vimeo</a>.</p>
<p>Here’s a rough transcript, if you don’t want to watch it:</p>
<hr />
<p>Hey everyone. I recently was checking out <a href="http://emberjs.com/">Ember.js</a>, and more specifically, <a href="http://discourse.org/">Discourse</a>. Discourse is supposed to be a next-generation discussion platform, and it’s an Ember app backed by a Rails-based JSON API. I mentioned to Ember co-creator <a href="http://twitter.com/wycats">Yehuda Katz</a> that I wished Ember had ‘a better accessibility story’. He asked me to elaborate, and I mentioned how my friends on Twitter were saying that Discourse wouldn’t able to be inclusive because JavaScript-heavy apps can’t be used by screenreaders. So he said, “Hit ⌘-F5.”</p>
<p>Whoah.</p>
<p>Hitting that key combination activates <a href="http://www.apple.com/accessibility/voiceover/">VoiceOver</a>, a feature of OS X I didn’t know existed. Basically, VoiceOver is able to interact with applications and allow keyboard navigation for those who are blind or sight-impaired. Neat! But what about in-browser? Turns out VoiceOver handles JavaScript heavy pages just fine.</p>
<p>I tried it out on <a href="http://try.discourse.org/">http://try.discourse.org/</a>, and it works pretty well! Not perfect, but you can manage. Yehuda also pointed me at <a href="http://www.w3.org/WAI/intro/aria.php">WAI-ARIA</a>, which is a W3C initiative. I’ll let the site explain:</p>
<blockquote>
<p>WAI-ARIA, the Accessible Rich Internet Applications Suite, defines a way to make Web content and Web applications more accessible to people with disabilities. It especially helps with dynamic content and advanced user interface controls developed with Ajax, HTML, JavaScript, and related technologies.</p>
</blockquote>
<p>Neat! Basically, you can add markup to your HTML, and it will help applications like VoiceOver allow people to use your site more easily, even if it uses a lot of Ajax. This is super cool, and for me, mitigates one of my main complaints against using SPAs. Of course, WAI-ARIA isn’t really Ember’s job; you’ll have to make the HTML be output in the right way yourself. But it makes me really happy to know that people with disabilities don’t need to be left out of the rich-client portion of the web.</p></content:encoded>
</item>
<item>
<title>How to not rely on rubygems.org for development</title>
<link>https://steveklabnik.com/writing/how-to-not-rely-on-rubygemsorg-for-deployment/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/how-to-not-rely-on-rubygemsorg-for-deployment/</guid>
<pubDate>Thu, 31 Jan 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Due to the recent <a href="https://status.heroku.com/incidents/489">situation with Rubygems.org</a>, a lot of people noticed that they rely on Rubygems.org when deploying. A lot of people advocate <a href="http://ryan.mcgeary.org/2011/02/09/vendor-everything-still-applies/">“vendor everything”</a>, and while that’s one approach, I actually don’t think it’s necessary. I think a lot of people <em>think</em> they rely on Rubygems.org more than they actually do, or, at least, aren’t deploying with Bundler correctly. So let’s talk about it.</p>
<h2>Deploying with Bundler</h2>
<p>Bundler has a <code>--deployment</code> flag to help you deploy code. <a href="http://gembundler.com/v1.2/deploying.html">Here’s the documentation</a>. If you use Bundler, and you don’t use <code>--deployment</code> when deploying, you are probably Doing It Wrong. Let’s try a brand new app, and see what happens:</p>
<pre><code>$ rails _3.2.11_ new foo --skip-bundle
create
create README.rdoc
create Rakefile
create config.ru
create .gitignore
create Gemfile
create app
create app/assets/images/rails.png
create app/assets/javascripts/application.js
create app/assets/stylesheets/application.css
create app/controllers/application_controller.rb
create app/helpers/application_helper.rb
create app/views/layouts/application.html.erb
create app/mailers/.gitkeep
create app/models/.gitkeep
create config
create config/routes.rb
create config/application.rb
create config/environment.rb
create config/environments
create config/environments/development.rb
create config/environments/production.rb
create config/environments/test.rb
create config/initializers
create config/initializers/backtrace_silencers.rb
create config/initializers/inflections.rb
create config/initializers/mime_types.rb
create config/initializers/secret_token.rb
create config/initializers/session_store.rb
create config/initializers/wrap_parameters.rb
create config/locales
create config/locales/en.yml
create config/boot.rb
create config/database.yml
create db
create db/seeds.rb
create doc
create doc/README_FOR_APP
create lib
create lib/tasks
create lib/tasks/.gitkeep
create lib/assets
create lib/assets/.gitkeep
create log
create log/.gitkeep
create public
create public/404.html
create public/422.html
create public/500.html
create public/favicon.ico
create public/index.html
create public/robots.txt
create script
create script/rails
create test/fixtures
create test/fixtures/.gitkeep
create test/functional
create test/functional/.gitkeep
create test/integration
create test/integration/.gitkeep
create test/unit
create test/unit/.gitkeep
create test/performance/browsing_test.rb
create test/test_helper.rb
create tmp/cache
create tmp/cache/assets
create vendor/assets/javascripts
create vendor/assets/javascripts/.gitkeep
create vendor/assets/stylesheets
create vendor/assets/stylesheets/.gitkeep
create vendor/plugins
create vendor/plugins/.gitkeep
steve at thoth in ~/tmp
$ cd foo
steve at thoth in ~/tmp/foo
$ cat .bundle/config
cat: .bundle/config: No such file or directory
</code></pre>
<p>We have no configuration for bundler. Makes sense, we never bundled. If we try to use <code>--deployment</code> now, we get an error:</p>
<pre><code>steve at thoth in ~/tmp/foo
[1] $ bundle install --deployment ✘
The --deployment flag requires a Gemfile.lock. Please make sure you have checked your Gemfile.lock into version control before deploying.
</code></pre>
<p>This is because <code>--deployment</code> checks the lock. So let’s make one:</p>
<pre><code>steve at thoth in ~/tmp/foo
[16] $ bundle ✘
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/..
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.5.0)
Using activesupport (3.2.11)
Using builder (3.0.4)
Using activemodel (3.2.11)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.4)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.11)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.11)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.11)
Using activeresource (3.2.11)
Using bundler (1.3.0.pre.5)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Installing rack-ssl (1.3.3)
Using json (1.7.6)
Using rdoc (3.12)
Using thor (0.17.0)
Using railties (3.2.11)
Using coffee-rails (3.2.2)
Using jquery-rails (2.2.0)
Using rails (3.2.11)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
</code></pre>
<p>This of course does hit the network, and does all the usual things. Your application normally has a lock already and has it checked in. We still don’t have a config:</p>
<pre><code>steve at thoth in ~/tmp/foo
$ cat .bundle/config
cat: .bundle/config: No such file or directory
</code></pre>
<p>So, this is the state our app would be in before deploying. At this point, it’s just like we did a <code>git pull</code> or an <code>scp</code> to our server. So let’s install for deployment:</p>
<pre><code>steve at thoth in ~/tmp/foo
$ bundle install --deployment
Fetching gem metadata from https://rubygems.org/.........
Fetching gem metadata from https://rubygems.org/..
Installing rake (10.0.3)
Installing i18n (0.6.1)
Installing multi_json (1.5.0)
Installing activesupport (3.2.11)
Installing builder (3.0.4)
Installing activemodel (3.2.11)
Installing erubis (2.7.0)
Installing journey (1.0.4)
Installing rack (1.4.4)
Installing rack-cache (1.2)
Installing rack-test (0.6.2)
Installing hike (1.2.1)
Installing tilt (1.3.3)
Installing sprockets (2.2.2)
Installing actionpack (3.2.11)
Installing mime-types (1.19)
Installing polyglot (0.3.3)
Installing treetop (1.4.12)
Installing mail (2.4.4)
Installing actionmailer (3.2.11)
Installing arel (3.0.2)
Installing tzinfo (0.3.35)
Installing activerecord (3.2.11)
Installing activeresource (3.2.11)
Installing coffee-script-source (1.4.0)
Installing execjs (1.4.0)
Installing coffee-script (2.2.0)
Installing rack-ssl (1.3.3)
Installing json (1.7.6)
Installing rdoc (3.12)
Installing thor (0.17.0)
Installing railties (3.2.11)
Installing coffee-rails (3.2.2)
Installing jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Installing rails (3.2.11)
Installing sass (3.2.5)
Installing sass-rails (3.2.6)
Installing sqlite3 (1.3.7)
Installing uglifier (1.3.0)
Your bundle is complete! It was installed into ./vendor/bundle
Post-install message from rdoc:
Depending on your version of ruby, you may need to install ruby rdoc/ri data:
&lt;= 1.8.6 : unsupported
= 1.8.7 : gem install rdoc-data; rdoc-data --install
= 1.9.1 : gem install rdoc-data; rdoc-data --install
&gt;= 1.9.2 : nothing to do! Yay!
</code></pre>
<p>We still hit Rubygems.org, but note now that it said “Installing” rather than using. That’s because we now have everything vendored:</p>
<pre><code>$ cat .bundle/config
---
BUNDLE_FROZEN: '1'
BUNDLE_PATH: vendor/bundle
BUNDLE_DISABLE_SHARED_GEMS: '1'
steve at thoth in ~/tmp/foo
$ ls vendor/bundle/ruby/1.9.1
bin cache doc gems specifications
steve at thoth in ~/tmp/foo
$ ls vendor/bundle/ruby/1.9.1/gems
actionmailer-3.2.11 multi_json-1.5.0
actionpack-3.2.11 polyglot-0.3.3
activemodel-3.2.11 rack-1.4.4
activerecord-3.2.11 rack-cache-1.2
activeresource-3.2.11 rack-ssl-1.3.3
activesupport-3.2.11 rack-test-0.6.2
arel-3.0.2 rails-3.2.11
builder-3.0.4 railties-3.2.11
coffee-rails-3.2.2 rake-10.0.3
coffee-script-2.2.0 rdoc-3.12
coffee-script-source-1.4.0 sass-3.2.5
erubis-2.7.0 sass-rails-3.2.6
execjs-1.4.0 sprockets-2.2.2
hike-1.2.1 sqlite3-1.3.7
i18n-0.6.1 thor-0.17.0
journey-1.0.4 tilt-1.3.3
jquery-rails-2.2.0 treetop-1.4.12
json-1.7.6 tzinfo-0.3.35
mail-2.4.4 uglifier-1.3.0
mime-types-1.19
</code></pre>
<p>Okay! Neat. So we’ve Vendored Everything… what happens if we <code>bundle</code> again?</p>
<pre><code>steve at thoth in ~/tmp/foo
$ bundle
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.5.0)
Using activesupport (3.2.11)
Using builder (3.0.4)
Using activemodel (3.2.11)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.4)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.11)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.11)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.11)
Using activeresource (3.2.11)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using json (1.7.6)
Using rdoc (3.12)
Using thor (0.17.0)
Using railties (3.2.11)
Using coffee-rails (3.2.2)
Using jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Using rails (3.2.11)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! It was installed into ./vendor/bundle
</code></pre>
<p>The <code>bundle</code> command uses our config settings in <code>.bundle/config</code>, which re-runs with the vendored bundle.</p>
<h2>Wait, I thought you said NO to Vendor Everything!!!</h2>
<p>Well, here’s the deal: you can Vendor Everything <em>on your server</em>, which means that we’re not committing gems into source control, and then pushing that huge mess over the network.</p>
<p>Let’s re-examine <code>--deployment</code> in the context of our two strategies: <code>scp</code> and <code>git</code>.</p>
<h3>git</h3>
<p>If you deploy with git, it’s two ways: <code>ssh</code>ing into the server and running a <code>git pull</code>, or by doing a <code>git push</code>. In both cases, you’ll have some sort of post-deploy hook that manages the rest of the deploy.</p>
<p>This scenario is exactly as shown above: just make sure that your first <code>bundle</code> on deploy is using <code>--deployment</code>, and you’re Good To Go. Each next deploy won’t hit the network. Rock on.</p>
<h3>scp</h3>
<p>If you use scp in some way to deploy, then you’re getting a new copy of the application every time, so that bundle full of gems won’t work. You need one more flag: <code>--path</code></p>
<pre><code>steve at thoth in ~/tmp/foo
$ rm -rf vendor
steve at thoth in ~/tmp/foo
$ bundle install --deployment --path ../vendor/bundle
Fetching gem metadata from https://rubygems.org/.........
Fetching gem metadata from https://rubygems.org/..
Installing rake (10.0.3)
Installing i18n (0.6.1)
Installing multi_json (1.5.0)
Installing activesupport (3.2.11)
Installing builder (3.0.4)
Installing activemodel (3.2.11)
Installing erubis (2.7.0)
Installing journey (1.0.4)
Installing rack (1.4.4)
Installing rack-cache (1.2)
Installing rack-test (0.6.2)
Installing hike (1.2.1)
Installing tilt (1.3.3)
Installing sprockets (2.2.2)
Installing actionpack (3.2.11)
Installing mime-types (1.19)
Installing polyglot (0.3.3)
Installing treetop (1.4.12)
Installing mail (2.4.4)
Installing actionmailer (3.2.11)
Installing arel (3.0.2)
Installing tzinfo (0.3.35)
Installing activerecord (3.2.11)
Installing activeresource (3.2.11)
Installing coffee-script-source (1.4.0)
Installing execjs (1.4.0)
Installing coffee-script (2.2.0)
Installing rack-ssl (1.3.3)
Installing json (1.7.6)
Installing rdoc (3.12)
Installing thor (0.17.0)
Installing railties (3.2.11)
Installing coffee-rails (3.2.2)
Installing jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Installing rails (3.2.11)
Installing sass (3.2.5)
Installing sass-rails (3.2.6)
Installing sqlite3 (1.3.7)
Installing uglifier (1.3.0)
Your bundle is complete! It was installed into /Users/steve/tmp/vendor/bundle
Post-install message from rdoc:
Depending on your version of ruby, you may need to install ruby rdoc/ri data:
&lt;= 1.8.6 : unsupported
= 1.8.7 : gem install rdoc-data; rdoc-data --install
= 1.9.1 : gem install rdoc-data; rdoc-data --install
&gt;= 1.9.2 : nothing to do! Yay!
steve at thoth in ~/tmp/foo
$ cat .bundle/config
---
BUNDLE_FROZEN: '1'
BUNDLE_PATH: ../vendor/bundle
BUNDLE_DISABLE_SHARED_GEMS: '1'
steve at thoth in ~/tmp/foo
$ ls vendor
ls: vendor: No such file or directory
steve at thoth in ~/tmp/foo
[1] $ ls ../vendor ✘
bundle
steve at thoth in ~/tmp/foo
$ bundle
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.5.0)
Using activesupport (3.2.11)
Using builder (3.0.4)
Using activemodel (3.2.11)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.4)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.11)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.11)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.11)
Using activeresource (3.2.11)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using json (1.7.6)
Using rdoc (3.12)
Using thor (0.17.0)
Using railties (3.2.11)
Using coffee-rails (3.2.2)
Using jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Using rails (3.2.11)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! It was installed into /Users/steve/tmp/vendor/bundle
</code></pre>
<p>The <code>--path</code> flag controls where the bundle is located. In this case, we store it one directory up. Now, when we copy new versions of the code over, it will use the bundle location that stays the same, and all is peachy keen.</p>
<p>I am told by several people that this is what the Capistrano/bundler recipe does by default, so if you’re using that, you’re already doing this.</p>
<h2>One Tiny Weakness</h2>
<p>There is one small weakness of this approach compared to Vendor Everything: as an engineer, it’s your tradeoff to make.</p>
<p>This way of using bundler <em>will</em> hit the network <em>when you deploy for the first time after updating your bundle</em>. The cache on the server has to update in this case, so it will go fetch the new gems. So here’s the scenario:</p>
<ol>
<li>I update my bundle. It gets the new versions of the gems.</li>
<li>Rubygems.org goes down.</li>
<li>I need to deploy a branch that has the new versions in it.</li>
</ol>
<p>In this case, if you had Vendored Everything, the hit to Rubygems.org would have happened during step 1, and so things would work. If you used this strategy, it would have hit locally, so you could have developed, but then when deploying, it’d hit again to update the bundle on the server, and so it wouldn’t.</p>
<p>In these situations, you can temporarily switch to Vendor Everything, since you have the bundle installed locally: just copy your local gems over to <code>vendor/bundle</code> and you’re done. This may or may not be too much of a hassle. When I examine the downtime of Rubygems.org, I think it’s worth it to not mess up my git repository with all that gem code. You might not. Do whatever you need to do, but now you know how to not rely on Rubygems.org for every deployment.</p>
<hr />
<h3>An addendum</h3>
<p>I got an email mentioning one more issue with the ‘vendor everything’ strategy:</p>
<pre><code>$ ls vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/*.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_document.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_element_description.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_entity_lookup.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_sax_parser_context.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_sax_push_parser.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/nokogiri.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_attr.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_attribute_decl.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_cdata.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_comment.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_document.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_document_fragment.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_dtd.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_element_content.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_element_decl.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_encoding_handler.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_entity_decl.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_entity_reference.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_io.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_libxml2_hacks.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_namespace.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_node.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_node_set.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_processing_instruction.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_reader.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_relax_ng.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_sax_parser.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_sax_parser_context.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_sax_push_parser.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_schema.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_syntax_error.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_text.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_xpath_context.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xslt_stylesheet.o
</code></pre>
<p>Basically, your compiled files get vendored, so if you’re, say, developing on Mac and deploying on Linux, you might end up with compiled native extensions for the totally wrong architecture.</p></content:encoded>
</item>
<item>
<title>Node</title>
<link>https://steveklabnik.com/writing/node/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/node/</guid>
<pubDate>Wed, 23 Jan 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Soooo there’s this thing: <a href="http://harthur.wordpress.com/2013/01/24/771/">http://harthur.wordpress.com/2013/01/24/771/</a></p>
<p>First of all, unequivocally, I said it before, and I’ll say it again: I’m sorry, and feel terrible that I made someone feel terrible. @harthur, I felt bad yesterday, and I’m feeling bad again. Not that it helps you that I feel bad. But I don’t like making other people feel bad. I am sorry.</p>
<h2>tl;dr</h2>
<p>TL;DR: I am sorry. I feel terrible that I made someone else feel terrible. I feel even worse because I didn’t mean to say that they were terrible, but accidentally did.</p>
<h2>the why</h2>
<p>Sooooooo I wrote up <em>why</em> I said what it was that I said, because what I was trying to say was totally unrelated to <code>replace</code>. I was trying to explain why I don’t like Node, and it basically came out wrong. But, explaining what I said had the affect of feeling like I was trying not to apologize. So, I’ll just cut to the chase and say this, cut straight from the chat log:</p>
<pre><code>SK: i dont want people to think i'm saying "i got caught being an asshole so i'm saying sorry"
SK: i want to say "I was accidentally being an asshole so i'm sorry"
</code></pre>
<p>So, I’ll just leave it at that. Twitter makes it so hard not to accidentally be an asshole.</p></content:encoded>
</item>
<item>
<title>_why is a role, not a person</title>
<link>https://steveklabnik.com/writing/why-is-a-role-not-a-person/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/why-is-a-role-not-a-person/</guid>
<pubDate>Wed, 16 Jan 2013 00:00:00 GMT</pubDate>
<content:encoded><p>http://whytheluckystiff.net/ has re-ignited a discussion about _why, and while I have tons and tons and tons of things to say on the subject, this is all I can give you for now: _why is a role, not a person.</p>
<p>For a long time, I was talking about a separation between _why and the person who played him. I still have not yet figured out how to refer to that person, and so for now, I’ve settled on Subject. So I’ll be using that description for the rest of this post.</p>
<h2>All the world’s a stage</h2>
<blockquote>
<p>All the world’s a stage, And all the men and women merely players:Shakespeare, “As You Like It,” Act II Scene VII</p>
</blockquote>
<p>One of the things I’ve learned as I’ve gotten a little older (21 days till 27!) is that I’m a different person in different situations. This may seem painfully obvious, but let’s expand on this a bit: when I’m on stage at a Ruby conference, I’m one person, when I’m by myself at home, I’m a different person, online, I’m a different person, and when I’m spending time with someone I care about, I’m someone else. Each of these contexts produces different behavior, mannerisms, and vocabulary. When I’m speaking with someone who’s read French Postmodernism, I use very different terms than when I’m speaking with my Republican ’Murican relatives.</p>
<p>There’s absolutely nothing wrong with this, but it really suggests that ‘essentialism’ is not a thing that can properly model the world. “Essentialism” is a position that states that there is some set of attributes that one can possess to ‘be’ something. So, for example, let’s take a Zebra: some essential attributes may be:</p>
<ol>
<li>Four legs</li>
<li>Black with White Stripes</li>
</ol>
<p>http://www.flickr.com/photos/kliski/4816615762/</p>
<p>Shit.</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24435086611428_raw.jpg" alt="https://svbtleusercontent.com/inline_steveklabnik_24435086611428_raw.jpg" /></p>
<p>Double shit.</p>
<p>So what makes a zebra a zebra then? I don’t want to get into it, but basically, I take the position that you can’t define something by what it is, you have to define it by what it <em>does</em>. I’m punting on why this is, again, I want to talk about _why, so please give me a charitable reading and take this as axiomatic for now.</p>
<p>If we’re defined by what we do, then it makes sense that we can contextualize what we do: we can take on different roles at different times. When I program in Ruby, I am a Rubyist; when I ride a bike, I am a cyclist, when I have a child, I am a father, when I love and am loved, I am a lover.</p>
<p>When you treat programming as an art, when you treat software as an artistic medium, when you spread joy and whimsy with what you do, you are _why.</p>
<p>It just so happens that Subject was the person who created this particular role, as was the one who best played it. That doesn’t mean that we can’t also try to take that role on from time to time, try out out, see if it fits, wiggle our toes a bit.</p>
<h2>_why vs. Subject</h2>
<p>This is one of the reasons I think the concept of _why is important: Subject was trying to teach us something. They were creating an abstraction, or, maybe, an indirection. By keeping themselves private, they put an emphasis on the distance between _why-as-role and _why-as-person, since the second doesn’t exist. The focus was, as it should be, on the role.</p>
<p>This is also why it’s wrong to ‘track _why down.’ This doesn’t make any sense. Subject is not _why, they just played them on TV. I also think that’s why it’s not wrong to keep talking about _why, as long as we make sure that we’re actually talking about _why, and not about Subject. Subject deserves their privacy.</p>
<p>I also don’t mean that I want someone to literally ‘take up the mask’ of _why and publish under that name. What I mean is that if we want to take on the lessons that Subject taught us through their persona, we should periodically have becoming-_why moments.</p>
<h2>On Sloppiness and blog posts written in haste</h2>
<blockquote>
<p>@steveklabnik hand write it, take a picture of it. leave the mistakes in.</p>
</blockquote>
<p>— tef (@tef) January 6, 2013</p>
<blockquote></blockquote>
<h2>Further reading</h2>
<p>To send you further down a rabbit-hole: http://en.wikipedia.org/wiki/Body_without_organs</p>
<p>Enjoy.</p></content:encoded>
</item>
<item>
<title>Draper 1.0.0 released</title>
<link>https://steveklabnik.com/writing/draper-1-0-0-released/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/draper-1-0-0-released/</guid>
<pubDate>Mon, 14 Jan 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Today, I released Draper 1.0.0! Draper is a gem that gives you decorators/view models for Rails.</p>
<ul>
<li>Rubygems: <a href="https://rubygems.org/gems/draper">https://rubygems.org/gems/draper</a></li>
<li>GitHub: <a href="https://github.com/drapergem/draper">https://github.com/drapergem/draper</a></li>
<li>README: <a href="https://github.com/drapergem/draper/blob/master/README.md">https://github.com/drapergem/draper/blob/master/README.md</a></li>
</ul>
<p>You can get it with Rubygems:</p>
<pre><code>$ gem install draper
</code></pre>
<p>or by adding this to your Gemfile:</p>
<pre><code>gem "draper", "~&gt;1.0.0"
</code></pre>
<p>Draper supports all versions of Rails 3, and follows <a href="http://semver.org/">SemVer</a>, so you can be confident that your application will work well with Draper in the future. Thank you to everyone who tried out our six (six!!!11) beta releases and gave us patches and feedback.</p></content:encoded>
</item>
<item>
<title>Rails has two default stacks</title>
<link>https://steveklabnik.com/writing/rails-has-two-default-stacks/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rails-has-two-default-stacks/</guid>
<pubDate>Mon, 14 Jan 2013 00:00:00 GMT</pubDate>
<content:encoded><p>Rails’ greatest strength is “Convention over Configuration.” Basically, Rails makes a ton of default choices for you about how things should be named, where they should go, and other things. This is why working with Rails is so productive for an experienced developer: just follow the Golden Path and everything comes together quickly. It’s second greatest strength is its ‘full stack’ nature: Rails includes everything that you need to get going. It does not attempt to be ‘minimal’ in any way. Put these two things together, and a Rails app makes lots of assumptions about <strong>everything</strong> in your app.</p>
<p>But what happens when a significant number of people don’t use those defaults?</p>
<p>First, let’s talk about the actual default stack. Since <a href="http://david.heinemeierhansson.com/2012/rails-is-omakase.html">Rails is omakase</a>, I’m going to call this the ‘Omakase Stack.’ Roughly, this stack contains:</p>
<ul>
<li>ERB for view templates</li>
<li>MySQL for databases</li>
<li>MiniTest for testing</li>
<li>Fat Models, Skinny Controllers</li>
</ul>
<p>There has been a ‘second default stack’ brewing for a while now. I’m going to call this the “<a href="http://en.wikipedia.org/wiki/Prime_(symbol)%23Use_in_mathematics.2C_statistics.2C_and_science">Prime</a> stack”. This stack’s approximate choices are:</p>
<ul>
<li>Haml for view templates</li>
<li>PostgreSQL for databases</li>
<li>Rspec/Cucumber for testing</li>
<li>Skinny models, controllers, and a service layer</li>
</ul>
<p>This is great! As David says:</p>
<blockquote>
<p>That doesn’t mean patrons can’t express exceptions. Substitutions are allowed, within reason. So you don’t like test/unit? No problem, bring your own rspec. Don’t care for CoffeeScript as a dessert? Delete a single line from your Gemfile.</p>
</blockquote>
<p>A considerable minority uses a stack like this. It’s important that the Prime Stack isn’t exact: you might not use Cucumber at all, for example, or maybe you don’t have a service layer. The important part, though, is that this is the new default you build preferences off of: a big group of people assumes that you don’t use MiniTest, you use Rspec, and the choice is Cukes or Capybara. This creates the ‘secondary default’ effect.</p>
<h2>Dual Stack Ecology</h2>
<p>So where’s the beef? Well, before I get into it, I want to make one thing explicit: I am <em>not</em> taking a position on which stack, if any, is ‘better.’ I’m not interested, at least in this post, at placing them into any kind of order. That’s for another time. What I want to talk about here is about the effect that a second default stack has on the overall Rails ecosystem.</p>
<p>Looking at history, the Prime Stack was born from the Omakase Stack. A certain group of people noticed that they were changing a significant amount of the stack most of the time, and so the Prime Stack came into existence. Since this is a historical progression, many blog posts got written describing the delta: take “<a href="http://robots.thoughtbot.com/post/159807023/waiting-for-a-factory-girl">Waiting for a Factory Girl</a>”, for example.</p>
<blockquote>
<p>Here at thoughtbot, we’ve had it with fixtures.</p>
</blockquote>
<p>But wait! Then, years later, you get a discussion on how factories are bad. My own “<a href="http://blog.steveklabnik.com/posts/2012-07-14-why-i-don-t-like-factory_girl">Why I don’t like factory_girl</a>” is an example:</p>
<blockquote>
<p>So that’s what it really boils down to: the convenience of factories has set Rails testing strategies and software design back two years.</p>
</blockquote>
<p>Maybe a bit strongly worded. Anyway, the people who have created the Prime Stack also happen to be those who are vocal, blog a lot, and speak at a lot of conferences. Everybody loves a polemic. If I submitted a talk to RailsConf called “Why Fixtures are :metal:” this year, people would assume that I was being sarcastic. People always assume that things are slowly getting better: If the original advice was “use fixtures” and the new advice is “use Factories” and the <em>newest</em> advice is “use neither”… there’s no way that the “right answer” is “use fixtures,” right? Isn’t that so 2007?</p>
<p>For those of us who’ve been around Rails for a long time, this discussion is good, and interesting. I was able to search for that Factory Girl post by memory, because I remember reading it when it was posted. I’ve written tons of Rails apps in the past 6ish years, and formed my own opinions on what tools I want to use due to that experience.</p>
<p>But what about someone starting Rails today?</p>
<h2>Beginners lose out</h2>
<p>Someone told me earlier today that “Starting Rails today is like starting to watch a soap opera in the 7th season.” Like most witty statements, there’s both truth and falsehood here. But, as a newcomer, how do you pick between the Omakase Stack and the Prime Stack?</p>
<p>This is the crux of the matter for me. If today, you tweeted that you were building a Rails app with fixtures, people would tell you to <a href="http://www.youtube.com/watch?v=n-Tej0297wk">Stop It</a>. I saw it happen this morning. This is again, because a very vocal minority of Rails developers use the Prime Stack, and are really passionate about their choice over the Omakase Stack. That’s great.</p>
<p>The problem is this: it puts a pretty heavy burden on a new developer: they have to learn the Omakase Stack <strong>and</strong> the delta. At the moment when someone is new, and barely knows anything, we throw them into making <em>more</em> choices. This is the moment when they’re least likely to be able to properly evaluate which option they need to choose. It’s not just “You’re new to Ruby, now learn Rails,” it’s now “You’re new to Ruby, now learn Rails, and RSpec, and Cucumber, and use this advanced modeling technique that we agree Rails discourages. Oh, and your only help is all these blog posts. Have fun!”</p>
<p>As I said at the beginning of this post, Convention over Configuration is Rails’ greatest strength. So when a new person starts, the first thing we do is tell them to do a bunch of configuration. Any reasonable developer could easily ask “If these options really are so superior, why do I have to include them? And which do I choose? I’m drowning!”</p>
<h2>Helping People Build Better Apps</h2>
<p>We all want to write better, more maintainable apps. And many people (including me) are trying to push the state of the art forward in this regard. But we’re leaving the majority in the dust. Rather than trying to push the boundaries by choosing new techniques, maybe we should be helping people get better at existing ones, too. Because we need both approaches. There’s already so much to learn when building a Rails application, adding more gems and techniques to the pile just isn’t going to help.</p>
<p>Once those beginners become intermediate, then they can figure out how new techniques solve the particular pains that they feel while going with the default flow. But skipping straight to the tool that solves my pain without feeling that pain the first place doesn’t help, it just confuses.</p>
<p>Let’s keep pushing those boundaries, but let’s not <em>require</em> someone to catch up on a few hundred episodes before they can understand what’s going on.</p>
<hr />
<p>If you read this post previously, the “Omakase Stack” was called the “37signals stack,” but David has pointed out <a href="https://twitter.com/dhh/status/293108811219230720">that even if he didn’t work with 37signals, he’d use this stack, so calling it that would be wrong</a>. I’ve since changed the name.</p></content:encoded>
</item>
<item>
<title>2012: year in review</title>
<link>https://steveklabnik.com/writing/2012-year-in-review/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/2012-year-in-review/</guid>
<pubDate>Sat, 29 Dec 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I always enjoy reading <a href="http://www.kalzumeus.com/2012/12/29/bingo-card-creator-and-other-stuff-year-in-review-2012/">patio11’s year in review posts</a>, and while I didn’t take enough notes for this year to do a GOOD one, I figure that I should at least leave some notes so that when I do one next year, I’ll have something to benchmark against.</p>
<h2>Ruby</h2>
<p>This has been a whirlwind year for my career. My work with Jumpstart Lab is incredibly fulfilling, and makes sure my bills get paid.</p>
<ul>
<li>I flew all over the world. I’ve always wanted to travel, and while I complain about it a lot, it’s truly wonderful to be able to go places. I now have the second highest Delta frequent flyer tier memebership. If you’d told me that three years ago, I’d have laughed at you.</li>
<li>I spoke at a <em>lot</em> of conferences, and gave my first keynote. More lined up for next year already. Speaking is something I love to do.</li>
<li>Teaching classes has been fantastic, and I’d love to do even more of it next year.</li>
<li>I now have commit access to Rails. I put in a lot of work over the last few months, and this means a lot to me. Last step: core. We’ll see if I ever get there.</li>
<li>I now have 22 gems, with Draper being my largest project, at almost 200,000 downloads.</li>
</ul>
<p>Unrelated to Ruby, I was able to contribute to a wide variety of projects in a few languages this year, and now have <a href="http://github.com/steveklabnik">116 repos on GitHub</a>. A lot of those are abandoned or old, but still, they’re all something I was doing <em>something</em> with this year.</p>
<h2>Writing</h2>
<p>I finally made the <a href="http://designinghypermediaapis.com/">hypermedia book</a> become a reality, and it’s done really well. Many people have told me that it has opened their eyes to a new way of doing things, and a bunch of startups have told me they’ll be releasing hypermedia APIs in 2013. Sales have been brisk: I don’t want to disclose exact numbers, but I will tell you that it’s in the 5 figure range. This has allowed me to start taking chunks out of my student loans: once they’re paid off, I will be super lean financially.</p>
<p>I made it to the Svbtle blog network, which is invite-only. I’ve gotten mostly good feedback about everything I’ve done there, which is pleasing.</p>
<p>I started <a href="http://www.rustforrubyists.com/">a book to learn Rust programming</a> last week, as a way of helping myself learn Rust, making some more money, and testing out a publishing workflow that might influence the next iteration of Designing Hypermedia APIs. I like it a lot.</p>
<h2>Reading</h2>
<p>I read a lot, though not as much as I’d like. I want to read 30 books this next year, and finally got into the habit of using GoodReads.</p>
<h2>Etc</h2>
<p>I had a really good year in other regards: fulfilling relationships, fun times, good food… not everything was spectacular, but overall, really good. I moved out of my hometown, finally, and that’s been really great. I’m back now for the holidays, and it feels so small…</p>
<p>The only really bad thing was the death of my father. That still has interesting effects on my life, but mostly in non-obvious ways. That’s all I’ll say about that.</p></content:encoded>
</item>
<item>
<title>Most things I do are interconnected</title>
<link>https://steveklabnik.com/writing/most-things-i-do-are-interconnected/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/most-things-i-do-are-interconnected/</guid>
<pubDate>Fri, 28 Dec 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I work on a <em>lot</em> of different things. Sometimes I get asked <a href="http://words.steveklabnik.com/how-do-you-find-the-time">how I find the time</a>, but there’s another aspect not related to time management: most of my projects build off of each other.</p>
<h2>Draper and RequestStore</h2>
<p><a href="https://github.com/drapergem/draper/issues/390">This issue</a> got filed in Draper. What Draper does isn’t important, but it did have this code in it:</p>
<pre><code>def self.current
Thread.current[:current_view_context] ||= build_view_context
end
def self.current=(context)
Thread.current[:current_view_context] = context
end
</code></pre>
<p>Basically, storing stuff in a <code>Thread</code> local variable. This is great, normally, except when you run one of those new hot servers like Thin or Puma. They have a different concurrency model, and so while you’d normally expect this <code>Thread</code> local to be <code>nil</code> on each request, it will persist between requests.</p>
<p>I <em>could</em> have just fixed this bug in Draper, but I figured that it might be useful to someone else, so I whipped up a little gem to do it: <a href="https://rubygems.org/gems/request_store">request_store</a>, and used the gem in Draper. I tweeted about it.</p>
<blockquote>
<p>@steveklabnik Holy shit. I love you.</p>
</blockquote>
<p>— Dominic Dagradi (@dddagradi) December 17, 2012</p>
<blockquote></blockquote>
<p>Turns out that one of my friends had <em>exactly</em> this problem, and wasn’t sure how to solve it. Now a bunch of people at Heroku are using my gem, and it’s helping them in their work.</p>
<p>I wouldn’t have bothered building it if it didn’t scratch my itch, but by making it easy for other people to use, I helped them scratch their itch too.</p>
<h2>Rust for Rubyists and Designing Hypermedia APIs</h2>
<p>I’ve been working on a new iteration of <a href="http://designinghypermediaapis.com/">Designing Hypermedia APIs</a> that is more linear and like a traditional book. The issue is that I’d built the entire platform for the project myself, and now spitting out ePUB and PDFs would be hard.</p>
<p>Enter <a href="http://www.rustforrubyists.com/">Rust for Rubyists</a>: I’d been learning Rust, and there are very few docs since it’s a new programming language. I had also seen Zed Shaw present at RuPy recently, and he showed off <a href="http://orkestrix.org/">Orkestrix</a>, which let him generate a site and a book for music stuff.</p>
<p>So I decided to write up what I’d learned as <a href="http://www.rustforrubyists.com/">a tutorial on programming in Rust</a>, and built it using the Orkestrix gear. I also tried out <a href="http://getdpd.com/">DPD</a> after seeing others use it. It’s only been up for two days, but I really like the combo, and today I pushed my first update out to the people who bought a copy already.</p>
<p>This has given me the confidence that the tooling process is good enough to use for DHAs, so I’ll be doing all of that soon.</p>
<h2>Rails and Resque</h2>
<p>I started working on Resque because we needed to have a few production implementations of the new ActiveQueue system going before Rails 4 was released. I was already working on Rails to help get 4 out the door, and Terence was obviously swamped by all the things he works on. So helping out with Resque would help out with Rails.</p>
<h2>bring_back_snowman and configuration</h2>
<p>When I don’t know how to do something, I often look at a gem that does what I’m interested in doing, and copy how it does it. I had never built a gem that allowed you to configure it using the <code>config.something</code> pattern before, so I peeked at a few gems to learn how.</p>
<p>I had also been thinking a lot about the Rails 4 release, and what was new since Rails 3. I was teaching a new batch of people Rails with <a href="http://jumpstartlab.com/">Jumpstart Lab</a>, my employer, and was explaining why we have <code>form_for</code> generate a ✔. This gave me an idea: What if you could configure what Rails used in this case?</p>
<p>So I built <a href="https://github.com/steveklabnik/bring_back_snowman">bring_back_snowman</a>. It’s not very <em>useful</em>, but it is cute and fun, and more importantly, <a href="https://github.com/steveklabnik/bring_back_snowman/blob/master/lib/bring_back_snowman.rb#L5-L13">it serves as a concise example of how to build a configurable option into your gems</a>. I can always look at that code if I forget how this works, or point others to it if they ask.</p>
<h2>Many, many more</h2>
<p>These are just the most recent examples I can think of. I’d really encourage you to try this sometime: if you want to learn something, make a new little project. Make it small. Make it focused. Make it help you out with something bigger.</p>
<p>Pretty soon, you’ll feel mega-productive too.</p></content:encoded>
</item>
<item>
<title>I invented hypermedia APIs by accident</title>
<link>https://steveklabnik.com/writing/i-invented-hypermedia-apis-by-accident/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/i-invented-hypermedia-apis-by-accident/</guid>
<pubDate>Fri, 21 Dec 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Long long ago, I got an internship in college. Back then, I didn’t know anything about web development. My college professor said “GET and POST are the same thing, it’s just that GET requests show parameters in the URL bar.” But a friend of mine was working at this post-acquisition startup. They still operated as an autonomous unit within the parent company, so they were still pretty fun to work for. People would upload us MP3s and we’d give them a podcast. They’d get a blog, the necessary RSS, we’d help get them listed in iTunes, all that jazz.</p>
<p>Then the iPhone came out, and you could make apps for it.</p>
<p>Bizdev decided that we should sell apps to our clients, who could then sell them to their audience and help finance the show. We’d do all the dev work, we’d all make money, it’d be a good time. So they asked me to write an iPhone app generator. We picked a big feature set, and people could give us some images, pick their individual features, and set some other config options. We’d take that file, and compile in them as defaults. All seemed good.</p>
<p>Then the first app got rejected: it turns out that Apple has some restrictions on the bitrate of MP3s that you could download. I suspected that AT&amp;T wasn’t very happy with you using lots of data back then. Anyway, we had to get around this issue: we wanted to serve up quality audio. What to do?</p>
<p>I don’t remember who came up with it, but we decided that the first thing the app would do would be to fetch the config file from the server, to see if any of the defaults were changed. It was just XML, so the iPhone could parse it easily, and then change whatever was different. So we’d have something like this:</p>
<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;config&gt; &lt;link href="http://example.com/low.rss" rel="podcast_url" /&gt;&lt;/config&gt;
</code></pre>
<p>I don’t remember the exact schema, but you get the idea.</p>
<p>Anyway, so we’d link to the low quality feed while the app was in review, and then later, we’d change the response to this:</p>
<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;config&gt; &lt;link href="http://example.com/high.rss" rel="podcast_url" /&gt;&lt;/config&gt;
</code></pre>
<p>Now the client would fetch the high quality feed. No client code needed to change! We’d managed to sneak around a restriction in the review process.</p>
<p>But why stop here? Once we saw this flexibility, we started taking full advantage. People could set up a bunch of different configuration options, and that would change the UI based on their choices. So, for example, they could choose to be contacted by email, they could put in a website or two, and the page would change. Here’s a mockup of the web page:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24409213486218_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24409213486218_raw.png" /></p>
<p>This would cause our app to serve up the following XML:</p>
<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;config&gt; &lt;link href="http://example.com/high.rss" rel="podcast_url" /&gt; &lt;about&gt; &lt;links&gt; &lt;link href="http://example.com"&gt;Homepage&lt;/link&gt; &lt;link href="http://example.com/about"&gt;About Us&lt;/link&gt; &lt;/links&gt; &lt;phone&gt;5558675309&lt;/phone&gt; &lt;email&gt;[email protected]&lt;/email&gt; &lt;/about&gt;&lt;/config&gt;
</code></pre>
<p>And that would make this appear in the “about” section of the app:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24409223801964_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24409223801964_raw.png" /></p>
<p>Neat, eh? We’d turn the email into a <code>mailto:</code> link and the phone into a <code>tel:</code> one, as well.</p>
<p>Anyway, maybe later they’d go back and unset their phone number, and the about us link. So then we’d generate this XML:</p>
<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;config&gt; &lt;link href="http://example.com/high.rss" rel="podcast_url" /&gt; &lt;about&gt; &lt;links&gt; &lt;link href="http://example.com"&gt;Homepage&lt;/link&gt; &lt;/links&gt; &lt;email&gt;[email protected]&lt;/email&gt; &lt;/about&gt;&lt;/config&gt;
</code></pre>
<p>But check out what the app would do:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24409247735934_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24409247735934_raw.png" /></p>
<p>Whoah! Isn’t that rad? The UI would change based on the config.</p>
<p>Here’s what’s even rad-er: because the app would read the config on each load, changes would get propagated very quickly. If you took your email away, everyone’s apps would no longer have it, as if by magic. Next time they loaded up the app, it just wouldn’t be there any more.</p>
<p>Here’s what’s <strong>even rad-er</strong>: You wouldn’t need to go through the App Store approval process to push this change out to the users. It Just Worked. If you’re an app developer, and you’ve been forced to sit through long waits to push out a release, you know how painful it can be. I don’t know how fast it is these days, but back then, it could take a month. With this approach, we could, say, remove a feature, and it’d be gone immediately. No oversight. If we added a new feature, older apps would still work, because they’d just ignore the new part of the config, and people who got the new version would get the newest features.</p>
<p>Years later, after doing tons of research on Hypermedia APIs, I realized that that thing I did long ago was a form of it, and it provided us with a pretty massive benefit. So of course I wasn’t the first one to come up with it; my point is that I implemented a system in line with hypermedia principles because it made sense to do so, even though I had no idea that Roy Fielding was a PhD candidate. And I certainly had no way of articulating it as a ‘style,’ it was just a neat hack.</p>
<hr />
<p>If you liked this story, you may care to learn about what else I’ve learned along the way with Hypermedia APIs. I wrote an e-‘book’ about it called <a href="http://designinghypermediaapis.com/">Designing Hypermedia APIs</a>. It’s only $20, and you get updates forever! I’m actively working on adding new content, and I’ll be doing a total redux in the new year.</p></content:encoded>
</item>
<item>
<title>An early Christmas present for you</title>
<link>https://steveklabnik.com/writing/an-early-christmas-present-for-you/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/an-early-christmas-present-for-you/</guid>
<pubDate>Wed, 19 Dec 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Normally, I’m a bit of a grinch. I’ll admit it, I really hate Christmas. It reminds me of all of the worst parts of America: You know, how the baby Jesus commands us to <a href="http://www.ajc.com/news/news/crime-law/alleged-shoplifter-dies-after-being-subdued-by-wal/nTFPx/">literally shop until we kill each other</a>…</p>
<p>Anyway, today, I was teaching a bunch of brand new Rails devs, and I had to explain the mysterious hidden input that <code>form_for</code> generates:</p>
<pre><code>&lt;input name="utf8" type="hidden" value="✔"&gt;
</code></pre>
<p>That, combined with the impending release of Rails 4, made me think about the charming little Snowman that Rails originally had: ☃.</p>
<h2>So?</h2>
<p>And what happened, then? Well, in Whoville they say - that @steveklabnik’s small heart grew three sizes that day. So he decided to make you a present:</p>
<pre><code>gem install bring_back_snowman
</code></pre>
<p>Just do it, and your forms will submit <code>_snowman=☃</code> to ensure that your data is encoded in UTF-8.</p>
<p>It is configurable, if you prefer something else. Add this to your <code>application.rb</code>:</p>
<pre><code>config.snowman = {:emoji =&gt; "😢"}
</code></pre>
<p>You may also need to add the ‘magic comment’ at the top of the file:</p>
<pre><code># encoding: UTF-8
</code></pre>
<p>Please remember that if you don’t use something that’s UTF-8, this won’t actually work to fix the bug.</p>
<p>You can find the <a href="https://github.com/steveklabnik/bring_back_snowman">source of bring_back_snowman on GitHub</a>, of course.</p>
<p>Merry Christmas!</p></content:encoded>
</item>
<item>
<title>Announcing request_store</title>
<link>https://steveklabnik.com/writing/announcing-requeststore/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/announcing-requeststore/</guid>
<pubDate>Mon, 17 Dec 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Last night I had some insomnia, so I wrote a gem.</p>
<p>Here it is: <a href="https://github.com/steveklabnik/request_store">https://github.com/steveklabnik/request_store</a>.</p>
<p>TL;DR:</p>
<p>If you’re using <code>Thread.current</code> in your Rails app to store global-ish data, don’t do it! If you use Thin or Puma or a threaded web server, it won’t get reset between requests, and you’ll end up with subtle bugs. So do this instead:</p>
<pre><code>gem "request_store"
</code></pre>
<p>and replace</p>
<pre><code>Thread.current[:foo] = 1
</code></pre>
<p>with</p>
<pre><code>RequestStore.store[:foo] = 1
</code></pre>
<p>And things will be peachy keen.</p>
<h3>No Rails?</h3>
<p>If you’re not using Rails, <code>use RequestStore::Middleware</code> to make it work. If you are using Rails, a Railtie takes care of it for you.</p></content:encoded>
</item>
<item>
<title>Getting started with chruby</title>
<link>https://steveklabnik.com/writing/getting-started-with-chruby/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/getting-started-with-chruby/</guid>
<pubDate>Thu, 13 Dec 2012 00:00:00 GMT</pubDate>
<content:encoded><p>If you’re looking for crazy simplicity in your ‘switch between multiple Rubies’ life, you may want to check out <a href="https://github.com/postmodern/chruby">chruby</a>. Written by Postmodern, it’s basically the simplest possible thing that can work. As in, <a href="https://github.com/postmodern/chruby/blob/master/share/chruby/chruby.sh">76 lines of shell script</a>.</p>
<p>For that, you get:</p>
<h3>Features</h3>
<ul>
<li>Updates <code>$PATH</code>.
<ul>
<li>Also adds RubyGems <code>bin/</code> directories to <code>$PATH</code>.</li>
</ul>
</li>
<li>Correctly sets <code>$GEM_HOME</code> and <code>$GEM_PATH</code>.
<ul>
<li>Users: gems are installed into <code>~/.gem/$ruby/$version</code>.</li>
<li>Root: gems are installed directly into <code>/path/to/$ruby/$gemdir</code>.</li>
</ul>
</li>
<li>Additionally sets <code>$RUBY</code>, <code>$RUBY_ENGINE</code>, <code>$RUBY_VERSION</code> and <code>$GEM_ROOT</code>.</li>
<li>Optionally sets <code>$RUBYOPT</code> if second argument is given.</li>
<li>Calls <code>hash -r</code> to clear the command-lookup hash-table.</li>
<li>Fuzzy matching of Rubies by name.</li>
<li>Defaults to the system Ruby.</li>
<li>Supports [bash] and [zsh].</li>
<li>Small (~80 LOC).</li>
<li>Has tests.</li>
</ul>
<h3>Anti-Features</h3>
<ul>
<li>Does not hook <code>cd</code>.</li>
<li>Does not install executable shims.</li>
<li>Does not require Rubies be installed into your home directory.</li>
<li>Does not automatically switch Rubies upon login or when changing directories.</li>
<li>Does not require write-access to the Ruby directory in order to install gems.</li>
</ul>
<p>Kinda crazy, eh?</p>
<h2>Installing</h2>
<p>Most of the time, I install things from <code>homebrew</code>, but I actually prefered to run the <code>setup.sh</code> script:</p>
<pre><code>wget https://github.com/downloads/postmodern/chruby/chruby-0.2.3.tar.gz
tar -xzvf chruby-0.2.3.tar.gz
cd chruby-0.2.3/
./scripts/setup.sh
</code></pre>
<p>You can see the source for that script <a href="https://github.com/postmodern/chruby/blob/master/scripts/setup.sh">here</a>. You’ll end up with an MRI, JRuby, and Rubinius all installed in <code>/opt/rubies</code>, and after restarting your shell, you should be good to go!</p>
<h2>Usage</h2>
<p>To see what Rubies you’ve got, just <code>chruby</code>:</p>
<pre><code>$ chruby
ruby-1.9.3-p327
jruby-1.7.0
rubinius-2.0.0-rc1
</code></pre>
<p>to pick one:</p>
<pre><code>$ chruby 1.9.3
</code></pre>
<p>Pretty damn simple.</p>
<h2>Getting more Rubies?</h2>
<p>The setup script will install <a href="https://github.com/sstephenson/ruby-build#readme">ruby-build</a>, so just use that to get more rubies:</p>
<pre><code>ruby-build 1.9.3-p327 /opt/rubies/ruby-1.9.3-p327
</code></pre>
<p>You’ll then need to update your list of Rubies in your <code>.bashrc</code> or <code>.zshrc</code>:</p>
<pre><code>RUBIES=(
# other rubies here
/opt/rubies/ruby-1.9.3-p327
)
</code></pre>
<p>If you find this tedius, you can just glob them:</p>
<pre><code>RUBIES=(/opt/rubies/*)
</code></pre>
<h2>Automatic switching?</h2>
<p><a href="https://github.com/postmodern/chruby/issues/40">Not yet</a>. This is pretty much the only thing keeping me from saying ‘zomg use this for sure now.’</p>
<h2>How good is it?</h2>
<p>I don’t know, I’ve been using it for less than 24 hours. Seems good.</p></content:encoded>
</item>
<item>
<title>Deleuze for developers: deterritorialization</title>
<link>https://steveklabnik.com/writing/deleuze-for-developers-deterritorialization/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/deleuze-for-developers-deterritorialization/</guid>
<pubDate>Fri, 07 Dec 2012 00:00:00 GMT</pubDate>
<content:encoded><p>If you truly want to understand technology today, then you should at least be familiar with the philosophy of Gilles Deleuze. Unfortunately for technologists, Deleuze is rooted firmly in a philosophical tradition and a writing style that they probably find opaque. In this blog series, I plan on explaining Deleuze’s philosophy in terms that programmers can understand. This is the second in the series. You can find the first <a href="/deleuze-for-developers-assemblages">here</a>, and the next one <a href="http://words.steveklabnik.com/deleuze-for-developers-will-smooth-spaceopen-source-suffice-to-save-us">here</a>. Enjoy.</p>
<hr />
<p>Let’s re-examine this diagram of the assemblage:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388447125816_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388447125816_raw.png" /></p>
<p>What’s up with this line?</p>
<p>Well, just because assemblages are amorphous doesn’t mean that there’s no way to demarcate what’s in the assemblage and what is not. This particular assemblage has a ‘territory,’ of sorts: everything that’s inside the line is part of the territory, and everything that’s not is outside. Think of a fort, or a castle:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388459420680_raw.jpg" alt="https://svbtleusercontent.com/inline_steveklabnik_24388459420680_raw.jpg" /></p>
<p>Or, since we’re not just talking about physical spaces, a social circle or group:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388506430020_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388506430020_raw.png" /></p>
<p>Of course, this would extend around all the members of Rails Core, hence it being cut off at the bottom. But the pink line would designate the boundaries of the assemblage that makes up Rails Core.</p>
<h2>Deterritorialization</h2>
<p>So what happens when these boundaries get crossed? Well, your castle gets invaded. A new member joins the team. New servers are brought online. This process is called ‘deterritorialization.’ Healing the lines, repairing them, and re-containing the boundary is called ‘reterritorialization.’ I recently came across a really interesting symbol of deterritorialization: the Open Source logo:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388521022692_raw.gif" alt="https://svbtleusercontent.com/inline_steveklabnik_24388521022692_raw.gif" /></p>
<p>Check it out: visually, this logo communicates the deterritorializing effect of opening up your source code: the private internals have now been exposed internally. The walls have been breeched!</p>
<p>Another example, from building a web service: we start off with just our service:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388526532744_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388526532744_raw.png" /></p>
<p>We have here an application server, a database, and our user’s identity. They form the assemblage of our system. Remember, abstract concepts are objects just as code and servers are! Our identity notion is stored within the database, therefore, they’re connected.</p>
<p>Next, we decide to implement a ‘log in with Twitter’ feature.</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388533077814_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388533077814_raw.png" /></p>
<p>It’s the primary way that our users sign up and use our site. Now, Twitter’s assemblage has deterritorialized ours:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388535026314_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388535026314_raw.png" /></p>
<p>We can minimize the effects (or reterritorialize our service) by making sure to have our own concept of identity within the system, and making sure that we have our own notion of identity within the system, and just connecting Twitter’s notion of identity to our own:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388627455810_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388627455810_raw.png" /></p>
<p>Now, by containing the Twitter-assemblage entirely within our service, I don’t mean that it actually is. Obviously, the Twitter-assemblage is interconnected with a ton of other assemblages that represent other services. But from our perspective, they are now a part of our assemblage. The decisions they make affect us. While our code is separated, we’re not totally separate anymore: updates and policies of Twitter have a direct effect on us.</p>
<p>There’s also a more sublte, secondary re/de-territorialization going on here: our code from our service. These used to be isomorphic, but now our code has claimed a territory of its own, and is now just one assemblage within our system-assemblage, instead of <em>being</em> our system-assemblage.</p>
<h2>A git example</h2>
<p>The previous notion of deterritorialization largely relied on the notion of dependencies as the mechanism which we drew our diagrams. But it doesn’t have to be that way. Let’s take another example: git.</p>
<p>Every GitHub pull request is an act of deterritorialization, and every merge is one of re-territorialization. Consider this small repository, with three commits:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388550633502_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388550633502_raw.png" /></p>
<p>You do a git clone:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388551164862_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388551164862_raw.png" /></p>
<p>Make a new commit, adding a new object into your repo-assemblage:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388553101194_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388553101194_raw.png" /></p>
<p>Then you send me an email, asking me to pull from your repository:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388554578454_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388554578454_raw.png" /></p>
<p>You like the change, so you do a <code>git fetch</code>:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388557943410_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388557943410_raw.png" /></p>
<p>de-territorialized!</p>
<p>Now it makes the local copy:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388560627354_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388560627354_raw.png" /></p>
<p>and your repository has been re-territorialized. These steps happen so quickly that you probably don’t even think about them, but conceptually, this is what’s happening.</p>
<h2>A social example</h2>
<p>One last example that’s even less related to code: entering a new social group. You’re at a conference, and there are four people standing around talking:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388564936014_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388564936014_raw.png" /></p>
<p>They form an assemblage we call a ‘conversation.’ It’s in relation with an object known as “refinements” from Ruby 2.0, which means this assemblage is likely to be particularly vibrant. Heh. Anyway, you decide that your opinion matters, so you de-territorialize the assemblage, and (assuming they don’t ignore you) it re-territorializes around you:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388566884064_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388566884064_raw.png" /></p>
<p>Even the language we use here implies something similar: you ‘enter the conversation.’ Like entering a door, into a space you previously were not.</p>
<h2>Conclusion: Diagrams</h2>
<p>We can call these drawings ‘diagrams’ or ‘abstract machines.’ They can represent these kinds of conceptual relationships in a visual way, which I find really useful for understanding. Programmers call abstract machines ‘design patterns.’</p>
<p>Now that you’ve seen this process that assemblages use to relate to one another, I hope that you find this particular diagram useful. Because of its level of abstraction, it’s applicable in a wide variety of situations.</p></content:encoded>
</item>
<item>
<title>Deleuze for developers: assemblages</title>
<link>https://steveklabnik.com/writing/deleuze-for-developers-assemblages/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/deleuze-for-developers-assemblages/</guid>
<pubDate>Mon, 03 Dec 2012 00:00:00 GMT</pubDate>
<content:encoded><p>If you truly want to understand technology today, then you should at least be familiar with the philosophy of Gilles Deleuze. Unfortunately for technologists, Deleuze is rooted firmly in a philosophical tradition and a writing style that they probably find opaque. In this blog series, I plan on explaining Deleuze’s philosophy in terms that programmers can understand. This is the first in the series.</p>
<p>Enjoy.</p>
<hr />
<p>The ancient Greeks thought that atoms were it. ἄτομος, Wikipedia will tell you, comes from ‘ἀ’- meaning “not” and ‘τέμνω’, meaning ‘I cut.’ Cells are the smallest thing that there is. You can’t cut them in half. Everything is composed of a bunch of atoms.</p>
<p>Well, guess what? Modern science has found things smaller than atoms. Dammit! What we <em>do</em> know for sure, though, is that some things are made up of composites of other things. That’s for sure. We’re still not clear on what that fundamental stuff is, but composites are a pretty safe bet. Even if atoms are made up of other things, we know that other things are made up of a bunch of atoms.</p>
<p>So how can we talk about composites? Well, a programmer’s fundamental tool is abstraction. So we can black-box any particular composite and treat it as a whole entity, without worrying about what we’ve encapsulated inside. That’s good, and that’s useful, but there’s a problem: when you start thinking of a collection of things as just one thing, you miss out on a lot of important details. As we know, all abstractions are leaky, so how does a black box leak?</p>
<p>Let’s think about your body: it’s a composite, you’re made up of (among other things) a whole pile of organs, and one (skin) wrapping the rest of them up in a nice black box. If we had lived a few hundred years ago, thinking of the body as a black box would be a pretty fair metaphor: after all, if you take out someone’s heart, they die. The box needs all of its parts inside. Today, though, that’s not true: we can do heart transplants. Further, we have artificial hearts. A person with an artificial heart is not the exact same as a person without one, yet they are both still people. Thinking of a body as a totality leaks as an abstraction in modern times.</p>
<p>Let’s move away from biology for a bit and into web applications instead. Here’s an architecture diagram of a typical web service:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24382217466576_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24382217466576_raw.png" /></p>
<p>Looks pretty normal, right? Let’s modify it a bit:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24382219810338_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24382219810338_raw.png" /></p>
<p>This is the same web service, right? We’ve introduced a few database slaves and a reverse proxy, which allowed us to knock out one of our application servers. In a certain sense, this service is equivalent; in another, it’s different: It’s got another extra component, and one component has developed more complexity. We might need to hire a DBA and someone who’s good at writing Varnish configs. Different skills are needed.</p>
<p>So what do we call this thing? An ‘assemblage.’ It’s made up of smaller assemblages as well; after all, our application servers may use the Reactor pattern, the worker model of Unicorn, or the threaded model of the JVM. To quote Levi Bryant, “Assemblages are composed of heterogeneous elements or objects that enter into relations with one another.” In the body metaphor, these are organs, and in the systems diagram, they’re the application servers, databases, and job queues. Assemblages are also ‘objects’ in this context, so you can think of any of those objects as being made up of other ones. But to think of the formations of objects as a total whole or black box is incorrect, or at best, a leaky abstraction.</p>
<p>Let’s get at one other excellent example of an assemblage: the internet. Computers connect, disconnect, reconnect. They may be here one moment, gone the next, or stay for an extended period of time. The internet isn’t any less an internet when you close your laptop and remove it, nor when you add a new server onto your rack. Yet we still talk about it as though it were a whole. It’s not, exactly: it’s an assemblage.</p>
<p>Furthermore, there’s a reason that Bryant uses the generic ‘objects’ or ‘elements’ when talking about assemblages: objects can be ‘people’ or ‘ideas’ as well as physical parts. This is where the ‘systems’ idea of computer science diverges from ‘assemblage’: depending on how you want to analyze things, your users may be part of an assemblage diagram as well.</p>
<p>So what’s the point? Well, this is a basic term that can be used to think about problems. A design pattern of thought. A diagram. In future posts, I’ll actually use this pattern to address a particular problem, but you need to understand the pattern before you can understand how it’s used.</p>
<p>(and to give you a comparison of complexity of explanation, here’s the blog post by Levi Bryant on assemblages where I got that quote from. He quotes Deleuze directly and extensively: http://larvalsubjects.wordpress.com/2009/10/08/deleuze-on-assemblages/ )</p>
<hr />
<p>The next article in the series is on <a href="/deleuze-for-developers-deterritorialization">‘deterritorialization’</a>.</p></content:encoded>
</item>
<item>
<title>Draper 1.0.0.beta2 release</title>
<link>https://steveklabnik.com/writing/draper-1-0-0-beta2-release/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/draper-1-0-0-beta2-release/</guid>
<pubDate>Mon, 03 Dec 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve relased the second beta for Draper 1.0.0! Many thanks to all of you who gave the first beta a shot. We’ve fixed some issues, and I’d apprecaite you giving this one a try. This solves all known issues with Draper, including every one we’ve found in 1.0.0.beta1.</p>
<p>You can get it by installing it from Rubygems:</p>
<pre><code>$ gem install draper --pre
</code></pre>
<p>or by putting it in your Gemfile:</p>
<pre><code>gem "draper", "1.0.0.beta2"
</code></pre>
<h2>CHANGELOG</h2>
<p>These are the changes since beta1. To find out what’s new in beta1, please <a href="/posts/2012-11-30-draper-1-0-0-beta1-release">see my previous post</a>.</p>
<ul>
<li><code>has_finders</code> is now <code>decorates_finders</code>. <a href="https://github.com/haines/draper/commit/33f18aa062e0d3848443dbd81047f20d5665579f">https://github.com/haines/draper/commit/33f18aa062e0d3848443dbd81047f20d5665579f</a></li>
<li>If a finder method is used, and the source class is not set and cannot be inferred, an <code>UninferrableSourceError</code> is raised. <a href="https://github.com/haines/draper/commit/8ef5bf2f02f7033e3cd4f1f5de7397b02c984fe3">https://github.com/haines/draper/commit/8ef5bf2f02f7033e3cd4f1f5de7397b02c984fe3</a></li>
<li>Class methods are now properly delegated again. <a href="https://github.com/haines/draper/commit/731995a5feac4cd06cf9328d2892c0eca9992db6">https://github.com/haines/draper/commit/731995a5feac4cd06cf9328d2892c0eca9992db6</a></li>
<li>We no longer <code>respond_to?</code> private methods on the source. <a href="https://github.com/haines/draper/commit/18ebac81533a6413aa20a3c26f23e91d0b12b031">https://github.com/haines/draper/commit/18ebac81533a6413aa20a3c26f23e91d0b12b031</a></li>
<li>Rails versioning relaxed to support Rails 4 <a href="https://github.com/drapergem/draper/commit/8bfd393b5baa7aa1488076a5e2cb88648efaa815">https://github.com/drapergem/draper/commit/8bfd393b5baa7aa1488076a5e2cb88648efaa815</a></li>
</ul></content:encoded>
</item>
<item>
<title>Draper 1.0.0.beta1 release</title>
<link>https://steveklabnik.com/writing/draper-1-0-0-beta1-release/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/draper-1-0-0-beta1-release/</guid>
<pubDate>Fri, 30 Nov 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I’m happy to announce the release of Draper 1.0.0.beta1 today. If you use Draper, I’d appreciate you checking out the beta release in your app, kicking the tires, and letting me know what’s up.</p>
<p>You can get it by installing it from Rubygems:</p>
<pre><code>$ gem install draper --pre
</code></pre>
<p>or by putting it in your Gemfile:</p>
<pre><code>gem "draper", "1.0.0.beta1"
</code></pre>
<p>Notable changes include:</p>
<ul>
<li>Renaming <code>Draper::Base</code> to <code>Draper::Decorator</code>. This is the most significant change you’ll need to upgrade your application. <a href="https://github.com/drapergem/draper/commit/025742cb3b295d259cf0ecf3669c24817d6f2df1">https://github.com/drapergem/draper/commit/025742cb3b295d259cf0ecf3669c24817d6f2df1</a></li>
<li>Added an internal Rails application for integration tests. This won’t affect your application, but we’re now running a set of Cucumber tests inside of a Rails app in both development and production mode to help ensure that we don’t make changes that break Draper. <a href="https://github.com/drapergem/draper/commit/90a4859085cab158658d23d77cd3108b6037e36f">https://github.com/drapergem/draper/commit/90a4859085cab158658d23d77cd3108b6037e36f</a></li>
<li>Add <code>#decorated?</code> method. This gives us a free RSpec matcher, <code>is_decorated?</code>. <a href="https://github.com/drapergem/draper/commit/834a6fd1f24b5646c333a04a99fe9846a58965d6">https://github.com/drapergem/draper/commit/834a6fd1f24b5646c333a04a99fe9846a58965d6</a></li>
<li><code>#decorates</code> is no longer needed inside your models, and should be removed. Decorators automatically infer the class they decorate. <a href="https://github.com/drapergem/draper/commit/e1214d97b62f2cab45227cc650029734160dcdfe">https://github.com/drapergem/draper/commit/e1214d97b62f2cab45227cc650029734160dcdfe</a></li>
<li>Decorators do not automatically come with ‘finders’ by default. If you’d like to use <code>SomeDecorator.find(1)</code>, for example, simply add <code>#has_finders</code> to the decorator to include them. <a href="https://github.com/drapergem/draper/commit/42b6f78fda4f51845dab4d35da68880f1989d178">https://github.com/drapergem/draper/commit/42b6f78fda4f51845dab4d35da68880f1989d178</a></li>
<li>To refer to the object being decorated, <code>#source</code> is now the preferred method. <a href="https://github.com/drapergem/draper/commit/1e84fcb4a0eab0d12f5feda6886ce1caa239cb16">https://github.com/drapergem/draper/commit/1e84fcb4a0eab0d12f5feda6886ce1caa239cb16</a></li>
<li><code>ActiveModel::Serialization</code> is included in Decorators if you’ve requred <code>ActiveModel::Serializers</code>, so that decorators can be serialized. <a href="https://github.com/drapergem/draper/commit/c4b352799067506849abcbf14963ea36abda301c">https://github.com/drapergem/draper/commit/c4b352799067506849abcbf14963ea36abda301c</a></li>
<li>Properly support Test::Unit <a href="https://github.com/drapergem/draper/commit/087e134ed0885ec11325ffabe8ab2bebef77a33a">https://github.com/drapergem/draper/commit/087e134ed0885ec11325ffabe8ab2bebef77a33a</a></li>
</ul>
<p>And many small bug fixes and refactorings.</p>
<p>Before the actual release of 1.0.0, I want to improve documentation and handle a few other things, but we currently have no confirmed and one possible bug in Draper as it stands, so your feedback as I clean up these last few things would be excellent. Please file issues on the tracker if you find anything.</p>
<p>Thank you! &lt;3 &lt;3 &lt;3</p></content:encoded>
</item>
<item>
<title>Introducing the rails-api project</title>
<link>https://steveklabnik.com/writing/introducing-the-rails-api-project/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/introducing-the-rails-api-project/</guid>
<pubDate>Thu, 22 Nov 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Ruby on Rails is a great tool to build websites incredibly quickly and easily. But what about applications that aren’t websites? Rails is still a first-class choice for this use-case, but why settle for good when you could be the best?</p>
<p>That’s why I’m happy to introduce Rails API: a set of tools to build excellent APIs for both heavy Javascript applications as well as non-Web API clients.</p>
<h2>What</h2>
<p>Rails’ greatest feature is making trivial choices for you. By following conventions, you’re able to avoid bikeshedding and only focus on the actual decisions that matter for your application, while automatically building in best-practices. Rails API will do this for the API use-case.</p>
<p>Here are some initial thoughts on what Rails API will do for you:</p>
<h3>A simpler stack</h3>
<p>First, we can remove many parts of Rails that aren’t important in an API context: many middleware don’t make sense, and all of ActionView can go away.</p>
<p>To check out exactly what we mean, look at <a href="https://github.com/rails-api/rails-api#choosing-middlewares">this part</a> of the README, which explains which middlewares we include. The default stack has 22, we have 14.</p>
<p>Similarly, the structure of ActionController::Base is interesting: it includes a ton of modules that implement various features. This means that we can build an alternate ‘controller stack’ that doesn’t include all of the ones that are in Base. Check out <a href="https://github.com/rails-api/rails-api#choosing-controller-modules">this portion</a> of the README for a demonstration of the modules we do include. Base includes 29 modules, we include 11.</p>
<h3>Consistent Javascript output</h3>
<p>One of the biggest issues when building a distributed system is messages that are sent between components. In most Rails-based APIs, this means the JSON that your server side emits, and the JSON that your client consumes. By forming best practices around what JSON is sent around, we can simplify things for server-side and client-side developers.</p>
<p>The generation of this JSON should be as transparent and simple as possible on both sides. <a href="https://github.com/rails-api/active_model_serializers">ActiveModel::Serializers</a> is key here. Many new APIs are settling on HAL as a JSON variant that is important to them, and so we’ll be considering it heavily.</p>
<h3>Authentication</h3>
<p>APIs don’t need to handle cookie-based auth, and so a lot of that infrastructure isn’t important. Handling other kinds of auth in a simple way is incredibly important.</p>
<h3>JavaScript as a first-class citizen</h3>
<p>JavaScript is just as important as Ruby for these kinds of applications, and should be equally as important with respect to tooling, directory layout, and documentation.</p>
<p>This also means that it should be easy to use the various Javascript frameworks and testing tools. We have representatives from the Ember team on-board, but using other frameworks should be equally fantastic.</p>
<h3>Hypermedia</h3>
<p>Many people are interested in building Hypermedia APIs on top of Rails, and so building best-practices around this style should be taken care of, and building them should be easy.</p>
<h2>Where</h2>
<p>You can find the organization on GitHub at <a href="https://github.com/rails-api">https://github.com/rails-api</a></p>
<p>Originally, these two repositories were located at <a href="https://github.com/spastorino/rails-api">https://github.com/spastorino/rails-api</a> and <a href="https://github.com/josevalim/active_model_serializers">https://github.com/josevalim/active_model_serializers</a>.</p>
<p>We have a Google Group for discussion here: <a href="https://groups.google.com/forum/?fromgroups#!forum/rails-api-core">https://groups.google.com/forum/?fromgroups#!forum/rails-api-core</a></p>
<h2>Getting Started</h2>
<p>To get started, check out the README here: <a href="https://github.com/rails-api/rails-api#railsapi">https://github.com/rails-api/rails-api#railsapi</a>.</p>
<h2>Who</h2>
<p>Currently,</p>
<ul>
<li>Carlos Antonio</li>
<li>Rafael França</li>
<li>Santiago Pastorino</li>
<li>Steve Klabnik</li>
<li>Yehuda Katz</li>
<li>Tom Dale</li>
<li>José Valim</li>
</ul></content:encoded>
</item>
<item>
<title>Narratives</title>
<link>https://steveklabnik.com/writing/narratives/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/narratives/</guid>
<pubDate>Sun, 18 Nov 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Occasionally in this big wide world, things happen. When things happen, we like to tell each other about it. When you take a bunch of connected events and tell someone about it, you’ve formed a <em>narrative</em>. Narratives are important, because they tell the story! The details of the account that one presents can really shape how others view those events later.</p>
<p>Due to our lived experiences, we often have our own ‘internal narratives.’ You can think of them as pre-packaged ways that we tend to think about a given series of events. Human brains are wonderful pattern-matching machines, and so we see similarities in situations, and then apply the same narrative.</p>
<p>Recently, I was playing around with Unicode and Ruby, and was trying to make some funny class names. I ended up figuring some stuff out, and tweeted this, which at the time of this writing, it has 157 retweets and 48 favorites:</p>
<blockquote>
<p>This is valid Ruby:class ಠ_ಠendಠ_ಠ.new</p>
</blockquote>
<p>— Steve Klabnik (@steveklabnik) November 17, 2012</p>
<blockquote></blockquote>
<p>This tweet is factually incorrect. Yet, only two people pointed this out to me. Why?</p>
<p>Ruby is kind of a crazy language. That’s what I love about it; it’s got lots of wacky little corners, nooks, and crannies. I’ve also been doing Ruby for a long time, and tweet about it often. So here’s the internal narrative when you see this tweet:</p>
<ol>
<li>Steve knows a lot about Ruby.</li>
<li>Ruby is kinda weird.</li>
<li>That’s really funny.</li>
</ol>
<p>This particular situation fits perfectly into this internal narrative. Hence, a retweet. No checking. Just tweeting.</p>
<p>These narratives can play off of each other really easily. Now that we’ve seen one more way in which Ruby is kinda crazy, we assume that it’s even more crazy in the future. This tweet has reinforced our internal narrative about Ruby’s wackiness.</p>
<hr />
<p>So #britruby.</p>
<p>I’m sure you’re expecting me to say something specific about this situation, but I’m not going to give you what you want. What I <em>do</em> want to do is analyze the narrative of this situation, and see how it compares to actual events.</p>
<p>Josh Susser tweets this:</p>
<blockquote>
<p>Nice speaker lineup for @britruby. Except for the 100% white guys part.</p>
</blockquote>
<p>— Josh Susser (@joshsusser) November 17, 2012</p>
<blockquote></blockquote>
<p>and James Rosen tweets this:</p>
<blockquote>
<p>By what metric is @britruby “one of Europe’s… most diverse Ruby conferences”? Every speaker is a white man.</p>
</blockquote>
<p>— James A Rosen (@jamesarosen) November 17, 2012</p>
<blockquote></blockquote>
<p>That’s it. If you examine those Twitter pages, you can see some conversation; both were conducted pretty cordially. Then, BritRuby decides to cancel their conference.</p>
<p>Okay. That’s the situation. Now let’s look at two versions of the narrative. First up, <a href="http://2013.britruby.com/">the cancellation announcement</a>. I’m going to excerpt the most important parts, with my commentary afterwards.</p>
<blockquote>
<p>We started this conference to build a community within the United Kingdom. Our mission statement was to encourage Ruby developers to unite and create a community, which would allow such to network, exchange ideas and provoke innovation for the future of Ruby. We wanted to encourage jobs and allow companies the opportunity to meet the community and primarily boost the UK developer industry.</p>
</blockquote>
<p>We wanted to build a great conference that would help UK Rubyists tremendously.</p>
<blockquote>
<p>Our selection process was the content and nothing more. Not the individuals gender, race, age or nationality.</p>
</blockquote>
<p>This is trivially disproven, as everyone has biases. But the message is “We only accepted the best speakers, regardless of who they are.”</p>
<blockquote>
<p>It’s about community. It’s about helping one another to strive for success and drive budding developers to do the same. We contacted universities in a hope that we would be able to empower young minds and show them what a great language Ruby really is. They are the future.</p>
</blockquote>
<p>We are team players.</p>
<blockquote>
<p>The Ruby community has been battling with issues of race and gender equality. We at Brit Ruby were well aware of this fundamental and important issue. This was one of the reasons why we encouraged everyone to submit a speaker proposal. Sadly, BritRuby was used as the arena to air these issues on Twitter and this has fundamentally destroyed any chance we had of addressing these issues.</p>
</blockquote>
<p>BritRuby was ‘used to air these issues,’ meaning, ‘others had an agenda and took advantage of us to promote it.’</p>
<blockquote>
<p>Instead the community should have worked together and allowed us to bring these issues to light at the conference.</p>
</blockquote>
<p>These individuals are divisive, and we are inclusive.</p>
<blockquote>
<p>How can the community address these issues if every time someone tries they are shot down and accused of such awful things?</p>
</blockquote>
<blockquote>
<p>The British Ruby team although very small consisted of a diverse mix of nationality and gender.</p>
</blockquote>
<p>We cannot be tainted by accusations of monoculture, as we are diverse ourselves.</p>
<blockquote>
<p>This was a non-profit conference being run by simple developers.</p>
</blockquote>
<p>We were not even trying to make money.</p>
<blockquote>
<p>The team has been working so hard in their own time to bring a unique conference to you all. Thank you to those of you that dedicated your time, skills and to their families for putting up with the long hours.</p>
</blockquote>
<p>We, nay, you and us together all worked really hard, so you should cut us some slack.</p>
<p>Anyway, so there you go. Same events, different story: We are a collective that was trying to do our best. Outsiders took advantage of us to promote their agenda and divide all of us. Therefore, we’re taking our ball and going home.</p>
<p>Here’s the funniest thing about the cancellation: They don’t give an actual reason. The closest they get is ‘was used as an arena to air these issues’ but they don’t explain why that made them cancel their conference. This narrative attempts to shift the blame from the organizers to the critics. “We wanted to give you this beautiful thing, and we worked hard, but these other people ruined it.”</p>
<p>This particular narrative plays off of a well-known trope when discussing diversity issues: a persecution complex by the majority. Majorities fear losing their power, and often will develop internal narratives that <em>they’re</em> the ones being persecuted. Some good examples of this are the Mens Rights movement and the White Power movement. This is a milder form of that same privilege: the power of a single tweet by the ‘politically correct’ is enough to bring the most powerful of plans grinding to a halt.</p>
<hr />
<p>Let’s examine a few narratives involved in <a href="https://gist.github.com/4106776">this gist</a>:</p>
<blockquote>
<p>And here it is, brought down by careless words.</p>
</blockquote>
<p>“These people don’t actually care about us or our conference.”</p>
<blockquote>
<p>Yes, gender equality and racial equality are important. But the team’s motives were to get the best speakers who were able to make it to Manchester.</p>
</blockquote>
<p>First of all, “but” is an amazing word. You can’t even imagine the things I’ve heard said after “I’m not racist, but.” This sentence implies two things: an absolution from blame on the part of the conference because hey, they care about equality. Their second statement implies that the best speakers all happened to be white men. The converse of this is that non-whites and/or women are not the best speakers, and the overall message is that including them would degrade the quality of the conference.</p>
<blockquote>
<p>Adding a token minority speaker is offensive to that speaker, it says “You’re here because you tick a box - not because you’re skilled.”</p>
</blockquote>
<p>Anyone who seeks out women or minorities is only doing it to ‘tick boxes,’ because, as we’ve already established, they clearly are not the best speakers.</p>
<p>Oh, and you’re actually the sexist/racist one if you try to encourage them, because it’s patronizing.</p>
<blockquote>
<p>It doesn’t matter who speaks at a conference, as long as they’re capable, interesting and relevant. That’s what matters: content, not style.</p>
</blockquote>
<p>We cannot be at fault because we don’t see race or gender. Again, basic psychology trivially disproves this statement.</p>
<hr />
<p>There are a ton of other micro-narratives that occur in all of these discussions as well. Most of these are a derailing tactic, and straight-up distract from the issue. “Egalitarianism is anti-free speech,” “Feminists and their allies are irrational and hysterical,” “I don’t see race or gender,” “If they were calmer about this, it would have worked out,” “This would be better handled in private,” and many, many more. I’ve seen so many. It’s exhausting. These internal narratives get reinforced by a particular interpretation of a given situation, without examining what actually happened.</p>
<p>Yet we still have conferences that are mostly full of white dudes.</p>
<p>I lost a lot of respect for a lot of people today.</p></content:encoded>
</item>
<item>
<title>How to squash commits in a GitHub pull request</title>
<link>https://steveklabnik.com/writing/how-to-squash-commits-in-a-github-pull-request/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/how-to-squash-commits-in-a-github-pull-request/</guid>
<pubDate>Thu, 08 Nov 2012 00:00:00 GMT</pubDate>
<content:encoded><p>So you’ve contributed some code to an open source project, say, Rails. And they’d like you to squash all of the commits in your pull request. But you’re not a git wizard; how do you make this happen?</p>
<p>Normally, you’d do something like this. I’m assuming <code>upstream</code> is a git remote that is pointing at the official project repository, and that your changes are in your ‘omgpull’ branch:</p>
<pre><code>$ git fetch upstream
$ git checkout omgpull
$ git rebase -i upstream/master
&lt; choose squash for all of your commits, except the first one &gt;
&lt; Edit the commit message to make sense, and describe all your changes &gt;
$ git push origin omgpull -f
</code></pre>
<p>GitHub will then automatically update your pull request with the new commits. Super easy!</p></content:encoded>
</item>
<item>
<title>Theory and practice</title>
<link>https://steveklabnik.com/writing/theory-and-practice/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/theory-and-practice/</guid>
<pubDate>Tue, 06 Nov 2012 00:00:00 GMT</pubDate>
<content:encoded><p><img src="http://imgs.xkcd.com/comics/purity.png" alt="http://imgs.xkcd.com/comics/purity.png" /></p>
<p>purity</p>
<hr />
<p>There is always a tension between Theory and Practice. These two separate realms are connected through a process of abstraction and application. To explain this relationship by way of theory, Theory deterritorializes Practice, and Practice reterritorializes Theory: a Theory which is a becoming-Practice and a Practice which is a becoming-Theory. To explain this relationship by way of practice, Theory is abstracted Practice, and Practice is applied Theory.</p>
<p>There’s an age-old problem with this particular relationship: those who specialize in Practice often claim that those who specialize in Theory are detached from the ‘real world,’ ie, the world of Practice. Those who specialize in Theory often claim that those who specialize in Practice have no fundamental understanding of what they do, and this leads to contradictory, incongruous practices.</p>
<p>There’s a third kind of person, though: one that embodies the becoming, the abstraction/application process. These people are a conduit, fundamentally bridging the two worlds. There’s a certain art to explaining just the core of Theory in the words of someone who Practices, and there’s a certain art to combining the essences of Practices and presenting it to those who Theorize. Building this bridge is an act of creation, of building, an opening of space.</p>
<hr />
<p>Some people are great at Ivory Tower intellectual stuff, and others just don’t really care. It sucks, because those who are doing could be way better at it if they just knew some theory, and those who love to philosophize all day might be more understandable if they’d just actually <em>do</em> something sometimes.</p>
<p>The only way you can get these two camps to talk to each other is to figure out what the theory says that provides value to those who practice. Practice-ers are ruthlessly focused on value, so to get through to them, you have to speak their language. On the flip side, theorists understand that practicers don’t care too much about the theory, but love seeing their thoughts to go good use, and can appreciate when practicers stumble across flaws in their thought. So demonstrating how practicers produce contradictions in the theory can be really useful to theorists.</p>
<p>If you’re this kind of person, accept that you’re in many ways a jack of all trades, but a master of none. Of sorts, anyway. Theorists will sometimes hate you for not grokking every last detail and reference, and practical people will argue that you don’t do enough useful things. Don’t listen to either of them; you know that you’re part of neither camp, so it makes sense that they both find ‘flaws.’ You’re awesome because you know a bit of both, and can facilitate communication which makes both better. A force multiplier.</p>
<hr />
<p>You have to remember that while you’re building things, there’s an underlying set of rules that you’re implicitly following, but it’s more important to act than it is to memorize a bunch of rules, and try to analyze what you’re doing according to them. If all you do is think all day, you’ll never get anything done. Things may go wrong, but you can always fix it later. Other people can sit around and theorize about what you’re doing, leave them to it.</p>
<p>Mental masturbation is fun and all, but when all is said and done, developing a culture of shipping is one of the most important things you can do. Those who can’t do, teach. History is written by the victors.</p>
<p>On occasion, you’ll run into someone who can actually explain complicated theory stuff to you in an accessible way. If you find someone like this, make sure to hold onto them closely, as they’re really rare. But they can help provide you with some insight that will really boost your productivity, without having to invest all the time in figuring out all that wankery that the priests of theory love.</p>
<hr />
<p>“In theory, there is no difference between theory and practice. But, in practice, there is.” - Yogi Berra</p></content:encoded>
</item>
<item>
<title>I got hit by a car today</title>
<link>https://steveklabnik.com/writing/i-got-hit-by-a-car-today/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/i-got-hit-by-a-car-today/</guid>
<pubDate>Thu, 25 Oct 2012 00:00:00 GMT</pubDate>
<content:encoded><p>“Humans are really squishy, messy, complicated, and contradictory. That’s what makes life interesting.”</p>
<p>Veggie Grill is damn delicious, but it’s hard to talk and eat at the same time. I’m usually trying to shovel my Santa Fe Chickin’ sandwich into my face as hard as possible. Today, though, there was a screw-up, and I didn’t get my food for a while. So I talked instead.</p>
<p>“Do you know anything about Hegelian dialectics?”</p>
<p>My friends are the best. Most people don’t put up with rambling, half-formed ideas, but mine generally entertain them, even though they haven’t read the books I’m talking about. I’m really grateful for this, as describing things really helps me sort them out in my head. The blank look and continued chewing meant that this was once again one of those times. Whatever.</p>
<p>“Basically, dialectics are a method of finding the truth in a threefold process: First, a ‘thesis’ is presented, which makes some sort of statement about the world. Then, the ‘antithesis’ contradicts it in some fashion. Finally, a ‘synthesis’ is found which resolves the tension between the two opposing sides. It’s a really interesting way of looking at the world; I’ve found it’s way easier to find things that totally contradict each other than I thought. Synthesis is really hard, though. Think about a trial, for example: the story of the plaintiff is told, then the story of the defendant, then the jury figures out what actually happened. There’s three sides to every story.”</p>
<p>Last week, I flipped over the handlebars of a bike. I was pumping as hard as I could, trying to make the light before it turned red, and the chain couldn’t take the tension any longer. It popped off, went between the back wheel and the frame, and stopped it cold. A few seconds later my eyebrows were scraping across the pavement. Lots of blood, but no serious injuries. I’m going to have a few sweet scars, though…</p>
<p>"Anyway, we were talking about feelings and ways people act before, but it’s true of physical bodies as well. Sometimes, accidents happen, and before you can even react, you’re dead. Other times, you escape with very little damage. We’re simultaneously incredibly vulnerable and almost impervious. Some people get in a bike accident and their life is forever changed. Other times, like me last week, you get away with a few scrapes. Once I fell about 8 feet onto my knees. Landed right on some pavement. I walked away 5 minutes later, didn’t even have bruises. I still have no idea what the fuck happened there.</p>
<p>I still haven’t found the synthesis yet."</p>
<p>After lunch, it’s time to bike home. It’s about 15 miles, which I can do pretty much at the drop of a hat nowadays. I really enjoy biking up Santa Monica Boulevard, but I hate it, too. It’s nice and broad, with two lanes, a bunch of green in the middle, and some bike lanes. Of course, it isn’t all like this, but much of it is. Especially on a day like today, around noon, it’s really pretty in places.</p>
<p><img src="http://upload.wikimedia.org/wikipedia/commons/6/62/Santa_Monica_Boulevard_at_Thayer.jpg" alt="http://upload.wikimedia.org/wikipedia/commons/6/62/Santa_Monica_Boulevard_at_Thayer.jpg" /></p>
<p>So I’m about 8 miles into my trip, and the next thing I know, I’m on the ground. Here’s the basic problem:</p>
<p><img src="http://www.examiner.com/images/blog/wysiwyg/image/car-in-bikelane-small.jpg" alt="http://www.examiner.com/images/blog/wysiwyg/image/car-in-bikelane-small.jpg" /></p>
<p>Yeah. Bike lanes don’t help if someone wants to turn right. When alongside cars, it’s very, very important to keep alert for two things: Open doors from parked cars, and drivers who don’t use their turn signals. A few weeks ago, I almost got hit by someone in this same exact way, just a few blocks down the street. I was paying close attention, and actually guessed that they were going to turn before they did. Free driver psychoanalysis done here! But more seriously, this was in the middle of the block. This guy saw that there was a space at this parking garage, hit his breaks and turned right without looking.</p>
<p>“YOU STUPID MOTHERFUCKER! WATCH WHERE THE FUCK YOU’RE GOING, YOU’RE GOING TO FUCKING KILL SOMEONE! JESUS!”</p>
<p>I ran a systems diagnostic check, and I was good, but enraged. Some pedestrians were screaming, and one ran over and asked if I was okay. “I said, yes, thank you, at least until this guy fucking kills me or someone else,” got back on my bike, and drove off.</p>
<p>At least my handlebars gave him a big long scratch the whole way up his car. Fucker.</p>
<p>There’s also something else interesting in this story as well: Generally, in a car/human interaction, I am the squishy one, and the car is the tank. Yet, in this particular interaction, I basically escaped unscathed, while the car took some damage. Once again, life is surprising, producing an inversion of the expected circumstances.</p>
<p>Anyway, if you drive a car, please, please, please watch what you’re doing. Pedestrians and cyclists don’t have thousands of pounds of metal surrounding them. Cyclists, please stay safe; it’s a jungle out there.</p>
<p>Ultimately, life is made out of these little incidents. Each moment to the next, you have no idea what’s going to happen. It’s exciting, scary, and sloppy. But that’s what makes it interesting. And you also don’t know when it’s going to end.</p>
<p>I still haven’t found that synthesis…</p></content:encoded>
</item>
<item>
<title>Living with a terrible taste in music</title>
<link>https://steveklabnik.com/writing/living-with-a-terrible-taste-in-music/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/living-with-a-terrible-taste-in-music/</guid>
<pubDate>Fri, 12 Oct 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Music has always been very important to me. Throughout most of my life, I’ve listened to a lot of music, and it’s played a huge role in defining my personality at the time I listened to it. In many ways, the history of the music I’ve listened to is a history of me.</p>
<p>Here’s a look at where I’ve been, and where I’m at now.</p>
<h2>Country</h2>
<p>My parents essentially listen exclusively to country music, and so it’s basically all I listened to for the first decade of my life. And when I say country, I don’t mean old stuff, I mean proto-pop country:</p>
<p>Many, many people that I know now are quite surprised by this. Country is one of those genres that brings out a certain revulsion in people. It still comes up from time to time, and my understanding of country brings out a laugh. Once in college, I was in a friend’s hometown, the Middle of Nowhere, PA. His brother rolls up in his huge pickup truck, blasting some George Straight. My friend looks at me, and rolls his eyes. With a straight face, I start saying the words as George sings them. The look of shock and confusion on his face was priceless.</p>
<p>In many ways, that’s exactly who I was destined to be: just a country boy gettin’ down on the farm.</p>
<p>But then, I discovered skateboarding, and punk rock.</p>
<h2>Punk</h2>
<p>Of course, it wasn’t even good punk music, either. See, I had always thought skateboarding was neat, but wasn’t never had one because there was nowhere to ride it. I lived on a farm, we didn’t have any pavement anywhere. So I forgot about skating. But then I got a call from a friend, he’d just gotten a new video game:</p>
<p><img src="/img/2012-10-12/TonyHawksProSkaterPlayStation1.jpg" alt="thps 1 cover" /></p>
<p>Here’s the soundtrack list:</p>
<ul>
<li>“Police Truck” by the Dead Kennedys</li>
<li>“Here and Now” by The Ernies</li>
<li>“Vilified” by Even Rude</li>
<li>“Superman” by Goldfinger</li>
<li>“Jerry Was a Race Car Driver” by Primus</li>
<li>“Screamer”/“Nothing to Me” by Speedealer</li>
<li>“Cyco Vision” by Suicidal Tendencies</li>
<li>“New Girl” by The Suicide Machines</li>
<li>“Committed” by Unsane</li>
<li>“Euro-Barge” by The Vandals</li>
</ul>
<p>Pretty awesome. He also had this new album that came out at the same time…</p>
<p><img src="/img/2012-10-12/blink-182-enema-of-the-state-front.jpg" alt="enema of the state" /></p>
<p>From there, it was all over. All this edginess combined with my (just budding, of course) testosterone meant I was pretty much set. The funniest part about this period of my life was that I actually wasn’t really a particularly rebellious child: I maintained pretty damn good grades, save for a few semesters, I still went to church, and I was generally a good kid. I didn’t even smoke pot! Yet even the act of possessing these albums was heinously offensive. I got grounded for half a year for possessing Blink182’s “Take off your Pants and Jacket” and Limp Bizkit’s “Three Dollar Bill, Y’all$.” This constant conflict over music with my parents dominates my memories at the time. I literally printed out all the lyrics and wrote two pages explaining why I should be allowed to own a copy of the Offspring’s “Americana” even though they said ‘shit’ a few times. Punk, even third generation pop-punk, really spoke to my desire for autonomy and my attempts to be my own person.</p>
<h2>Metal</h2>
<p>Of course, slippery slopes are everywhere, and so I started searching for harder and harder music. Always the <a href="http://words.steveklabnik.com/how-do-you-find-the-time">min/maxer</a>, I looked for loopholes in my parent’s rules. The biggest hole: anything from the Christian Family Bookstore was fair game. So I scoured it for anything awesome. That’s when I found Zao:</p>
<p>Screw punk, metal was way more awesome. I grew my hair out long and learned to headbang.</p>
<p>I don’t think I need to explain to you why metal is ridiculous. I mean, come on:</p>
<p><img src="/img/2012-10-12/dethklokbandfeb2024_638.webp" alt="dethklok" /></p>
<h2>Hardcore</h2>
<p>Of course, there has to be a way to reconcile all this, right? Take two steps forward, one step back? Well, that’s hardcore.</p>
<p>My fascination with hardcore in late high school and early college was two-fold, like ‘hardcore’ itself: there’s ‘hardcore’ and then there’s ‘hardcore punk.’ Hardcore punk is a specific sub-genre of punk rock from the 80s, and hardcore is its continuation in the modern day, after that whole emo thing happened.</p>
<p>To recap: hardcore punk:</p>
<p><img src="/img/2012-10-12/a18-MinorThreat.jpg" alt="minor threat poster" /></p>
<p>hardcore:</p>
<p><img src="/img/2012-10-12/tumblr_lpqekrWMCt1qh3e7yo1_400.gif" alt="slam dancing" /></p>
<p>Uuhhh yeah. Hardcore punk also brought me to straightedge, which leads to silliness like this:</p>
<p><img src="/img/2012-10-12/KmCWp.jpg" alt="my straightedge tattoo" /></p>
<p>To this day, I still don’t regret this tattoo, and anyone that’s seen me at a Ruby conference knows that I’m not straightedge anymore. But this music and philosophy defined who I was for about 8 years, which was, at the end of it, about a third of my life. It changed my college experience. I’m (basically) permanently marked by it.</p>
<p>But no, seriously:</p>
<p><img src="/img/2012-10-12/tumblr_lpqekrWMCt1qh3e7yo1_400.gif" alt="slam dancing" /></p>
<h2>Party Rock</h2>
<p>Anyway, so nowadays, this is what I listen to:</p>
<p>&lt;iframe width="560" height="315" src="https://www.youtube.com/embed/edP0L6LQzZE?si=uTrbb2_tUchSk1FC" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen&gt;&lt;/iframe&gt;</p>
<p>&lt;iframe width="560" height="315" src="https://www.youtube.com/embed/k0BWlvnBmIE?si=m4OITv0l29x0zot5" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen&gt;&lt;/iframe&gt;</p>
<p>Here’s the thing: once I finally admitted to myself that I enjoy terrible pop music and stopped trying to apologize for my taste, I realized I was way more productive. Upbeat music with a vaguely positive vibe means I get shit done. Way better than the heavy, dreary, angry stuff I listened to in the past. I listen to hours of music a day, and I’d much rather have it be happy…</p>
<p>But that’s when I realized I’ve basically always had bad taste in music.</p>
<p>The thing is, that’s totally okay! Music has always provided an interesting backdrop for the rest of my life, and it’s fit whatever moment I was living. Writing this post was a blast, because I’d forgotten about half of this stuff, and re-living it is a good time. I can remember debating my friends about the Casualties being street punk or hardcore, if some band had really sold out, or if straightedge still matters now that Ian MacKaye smokes pot.</p>
<p>I wonder what I’ll be listening to in five years.</p></content:encoded>
</item>
<item>
<title>Run Rails with custom patches</title>
<link>https://steveklabnik.com/writing/run-rails-with-custom-patches/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/run-rails-with-custom-patches/</guid>
<pubDate>Thu, 04 Oct 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I often see comments <a href="https://github.com/rails/rails/pull/7397#issuecomment-9132009">like this</a> in the Rails bugtracker. Generally, someone is running an older version of Rails, and some bug they face has been fixed on edge. But they may be running a version that’s too old to recieve fixes, or need a fix that has yet to be included in an actual release. What to do?</p>
<p>Luckily, <a href="http://gembundler.com/">Bundler</a> exists. It makes it super easy to run your own Rails. Check it out:</p>
<h2>Step 1: Fork you!</h2>
<p>Go to GitHub, hit up <a href="http://github.com/rails/rails">rails/rails</a>, and click that fork button. My own personal fork of Rails is <a href="https://github.com/steveklabnik/rails">here</a>, for example, so in the rest of these examples, I’ll be using my own username. If you’re me, you can use it too, but seriously, stop being me. If you’re not me, substitute your own username.</p>
<h2>Step 2: apply patch</h2>
<p>Let’s get our copy going locally:</p>
<pre><code>$ git clone [email protected]:steveklabnik/rails.git
$ cd rails
</code></pre>
<p>We’ll apply the patch from the example comment above. In this case, the person wants to have this patch work on Rails 3.1.</p>
<p>You have two options here: Rails keeps a branch open for every minor version, and also tags every release. You can pick between the stable branch and a particular release. The stable branch should work just fine, but maybe you’re on a specific Rails release for a reason. For example, the latest version of Rails 3.1 is 3.1.8, but this person says they’re on 3.1.1. If they use the 3-1-stable branch, they’ll also get all the changes from 3.1.1-3.1.8, as well as unreleased changes that may exist on that branch. That’s possibly too much for you, so we’ll just work off of 3.1.1. This is probably a bad idea, since <a href="http://weblog.rubyonrails.org/2012/6/12/ann-rails-3-1-6-has-been-released/">3.1.6 included several security fixes</a>, but you’re an adult, do dumb things if you want to.</p>
<pre><code>$ git checkout v3.1.1
</code></pre>
<p>You’ll see a big message about a detached HEAD. Don’t worry about it.</p>
<p>If you want the stable branch for extra fixes,</p>
<pre><code>$ git checkout 3-1-stable
</code></pre>
<p>Now, it’s a good idea to make our own branches, so that we don’t get confused with upstream. So let’s make a new branch:</p>
<pre><code>$ git checkout -b my_patched_rails
</code></pre>
<p>Awesome. Now, we can grab the commit we wanted. We can do this with <code>cherry-pick</code>:</p>
<pre><code>$ git cherry-pick 8fc8763fde2cc685ed63fcf640cfae556252809b
</code></pre>
<p>I found this SHA1 by checking out <a href="https://github.com/rails/rails/pull/7397/commits">this page</a>, which is linked at the top of the pull request.</p>
<p>If there are multiple commits you need, you can do one of two things: either <code>cherry-pick</code> the merge commit, in which case you’ll probably need to pass the <code>-m1</code> option, or simply <code>cherry-pick</code> them all in order.</p>
<p>You may get conflicts. Yay backporting! If <code>git</code> whines at you, do what you normally do. Resolve the merge, then <code>git commit</code>.</p>
<p>Finally, push it back up to your GitHub:</p>
<pre><code>$ git push origin my_patched_rails
</code></pre>
<p>Congrats! You’ve got your own custom Rails. Now it’s time to use it.</p>
<h2>Step 3: update Gemfile</h2>
<p>Go to your Rails app, and edit this line in your Gemfile:</p>
<pre><code>gem 'rails', "3.1.1"
</code></pre>
<p>change it to this:</p>
<pre><code>gem 'rails', :git =&gt; "https://github.com/steveklabnik/rails", :branch =&gt; "my_patched_rails"
</code></pre>
<p>It’s that easy! Now bundle:</p>
<pre><code>$ bundle update
</code></pre>
<p>You should see it mention something about checking out a certain copy of Rails.</p>
<h2>Step 4: Profit!</h2>
<p>That’s it! Congrats, you’re using Rails with your patch.</p>
<h2>But I’m on 2.3, I’m not running Bundler!</h2>
<p><a href="http://gembundler.com/v0.9/rails23.html">Do this</a>.</p></content:encoded>
</item>
<item>
<title>How do you find the time?</title>
<link>https://steveklabnik.com/writing/how-do-you-find-the-time/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/how-do-you-find-the-time/</guid>
<pubDate>Thu, 27 Sep 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I’m often asked the question, “How do you find the time?” Mostly, it’s around open source, sometimes it’s about books, and occasionally, on other things. I’ve generally deflected the question with something like “I don’t know, man, I just <em>do</em> it.” But that answer isn’t very helpful. And I’ve been wondering if I can do even better. So I’ve done some reflection this week, and here’s how I find the time:</p>
<h2>Min/max</h2>
<p><img src="http://2.bp.blogspot.com/_6VPzVlB8N28/SwC5AWnireI/AAAAAAAABgA/O883tYn78As/s1600/DDplayershandbook2000.jpg" alt="http://2.bp.blogspot.com/_6VPzVlB8N28/SwC5AWnireI/AAAAAAAABgA/O883tYn78As/s1600/DDplayershandbook2000.jpg" /></p>
<p>player’s handbook</p>
<p>I have this personality type that is an asset to any competitive gamer: Dungeons and Dragons nerds call it “min/maxing”. The basic idea is this: you try to minimize the things that are bad, and maximize those that are good. This sounds super obvious, but there are many times when maximizing ‘winning’ isn’t what you’re going for. One great example is ‘pretending to be a fantasy adventurer.’ If you’re not shooting for strictly winning, then min/maxing doesn’t make any sense. But if you’re looking to compete, it’s incredibly important.</p>
<p>Speaking of competition, you should really read this book:</p>
<p><img src="http://ecx.images-amazon.com/images/I/41SZq3uDRVL._SS500_.jpg" alt="http://ecx.images-amazon.com/images/I/41SZq3uDRVL.SS500.jpg" /></p>
<p>playing to win</p>
<p><a href="http://www.sirlin.net/ptw/">“Playing to Win”</a> is an excellent book by one of the best Street Fighter players in the world. It is a must-read for anyone who wishes to compete in any way.</p>
<p>In it, he develops the theory of “Players” and “Scrubs”:</p>
<blockquote>
<p>A scrub is a player who is handicapped by self-imposed rules that the game knows nothing about.</p>
</blockquote>
<p>A great tell of someone being a scrub is when they mention ‘fairness.’ A game is an impartial, closed system. There are rules to the system that constrains the players. Simply, the more rules, the harder it is for you to achieve your win condition. If you invent extra rules, you handicap yourself. Games are not ‘fair.’ Players don’t handicap themselves in this way:</p>
<blockquote>
<p>A player should use any tournament legal move available to him that maximizes his chances of winning the game.</p>
</blockquote>
<p>If a move is legal, then it’s fair game. Good games are written in such a way that bad moves don’t exist. Use all the tools at your disposal, and use them ruthlessly.</p>
<p>Here’s the thing about the scrub mindset: it’s easy to do accidentally. Extra rules can be self-imposed, but they can also happen due to a poor analysis of what the rules actually are. For example, the biggest time I scrubbed out in my entire life was a choice that I made almost a decade ago: going to college. Every adult in my life told me that going to college was a good idea. They all said it was needed to get a job as a programmer. I didn’t bother to challenge them, and went along with it. Now I have a useless piece of paper and $70,000 in debt. College wasn’t all bad, but it certainly wasn’t worth the debt. I cherish the friendships I made, but not much else. This mistake was made because of my apathy towards the systems governing my life, and it’s one I try not to repeat.</p>
<p>Anyway, these two principles are the backdrop for everything else: properly analyze your situation, maximize the outcomes you want, and minimize the ones you don’t.</p>
<h2>Work</h2>
<p>“So what do you want out of life, Steve?”</p>
<p>“I want to teach, I want to work on open source, and I want to travel.”</p>
<p>“I think we can work something out.”</p>
<p>If you work a 9-5 job, you spend 40 hours a week at work. Let’s add a one hour commute in there too, which puts you up to 50 hours. And as Ruby will tell you…</p>
<pre><code>1.9.3-p194 :001 &gt; 24 * 7
=&gt; 168
1.9.3-p194 :002 &gt; 50 / 168.0
=&gt; 0.297619047619048
</code></pre>
<p>30% of your life. Let’s analyze this like a pro gamer, shall we? Here’s the situation we’re trying to maximize: “I want to live a full life.” In order to maximize this, we need to discover the constraints. The biggest one, of course, is your time on this Earth. For now, at least, we all will die someday. The primary resource that we all have to work with is time. You should guard your time more closely than any other thing you have. You can’t ever get it back. Every single thing you do consumes this most precious of resources.</p>
<p>When you enter into an employment agreement, you are literally selling your most precious thing in order to acquire other resources. Don’t forget that.</p>
<p>So, 9-5 job. 30% of your life. Is it worth it? Maybe! I can’t tell you. But in order to maximize your outcomes, you need proper analysis. Employment is a tricky thing, in that regard. Capitalism forces us to interact with the market in order to gain our subsistence, so we have to enter into this transaction somehow. What to do?</p>
<p>Quick, startup crowd: what’s the first rule of pricing? If you say “do it value-based, not cost-based,” then you win a <a href="http://www.kalzumeus.com/blog/">patio11</a> sticker. You’ll never get anywhere selling yourself on an hourly basis, you have to sell yourself on value. That’s another topic for another time. Point is, don’t just assume the rules: truly analyze your situation.</p>
<p>Anyway, that’s neither here nor there. On a day-to-day basis, the only reason that I can do what I do is because <a href="https://twitter.com/j3">Jeff</a> is a stand-up guy, and working with <a href="http://jumpstartlab.com/">Jumpstart Lab</a> is awesome. We do the best Ruby and Rails classes in the world, and I mean that. I don’t want to get into the details of my employment, but basically, teaching classes, contributing to open source, and speaking at conferences are all part of my job. You can see how that directly maps to my statement above.</p>
<p>So, if you appreciate</p>
<ul>
<li><a href="http://hackety.com/">My work on Hackety Hack</a></li>
<li><a href="http://contributors.rubyonrails.org/contributors/steve-klabnik/commits">My contributions to Rails</a></li>
<li><a href="https://github.com/defunkt/resque">My contributions to Resque</a></li>
<li><a href="http://vimeo.com/search?q=klabnik">My public speaking</a></li>
<li><a href="https://github.com/steveklabnik?tab=activity">All the other Open Source work that I do</a></li>
<li><a href="http://confreaks.com/videos/1089-madisonruby2012-anti-opression-101">My advocate work for feminism and social justice</a></li>
</ul>
<p>then you should consider hiring us over at Jumpstart. If I had to do normal consulting to pay my bills, I would have much less time for all of this stuff.</p>
<h3>But I don’t work at Jumpstart!</h3>
<p>So, you’re not me. You work a 9-5 job. So you’re screwed, right?</p>
<p>No way! Analyze your situation. Figure out how to maximize it. Before I did software development, I <a href="http://www.vocellipizza.com/consumer/index.php">did this</a>. About 30 hours a week while at high school, and 40 hours a week through college. I <em>still</em> managed to put in open source time. How?</p>
<p>The thing is, working on open source is something I truly love to do. For me, it’s not ‘work.’ That does not mean that it’s always roses. Sometimes it’s hard. Sometimes, it really, really sucks. Hard. However, I truly love it, so I stick it out through the hard times, and it ends up being good again. Once, I suffered a really bad break-up. My mom told me this maxim that her grandmother always told her:</p>
<blockquote>
<p>What bone you were meant to have, no dog will drag away.</p>
</blockquote>
<p>Grandmas. They’re so opaque! The point is, contributions to Open Source is something that’s a part of me, no matter what. So I’ll always find a way to do it. I don’t give up on it easily. When I first took over Shoes, it took me <strong>six months</strong> to figure out how to compile it. Six months. No compilation. That sucked. But I stuck it out.</p>
<blockquote>
<p>Eighty percent of success is showing up.Woody Allen</p>
</blockquote>
<h2>Relationships</h2>
<p>Having a full life means interacting with other people. Don’t let some productivity fetish negatively affect relationships. I mean that in a broad sense, both romantic and bromantic. ;) Make time for stuff that’s not ‘ZOMG MUST GET WORK DONE’ in your schedule too. I make sure to talk to significant others at least once or twice a day, for example. Assuming they want me to. Tonight, I’m gonna go for a bike ride on the beach. I still play StarCraft from time to time. Every Saturday, my friends and I used to get together in the computer lab and hack.</p>
<p>If you spend all of your time working, and none of your life living, you’ve failed at life. Open source is fun, but go outside, dammit. Or read a book. Or something. But if you only program computers, you’ve failed at being a human.</p>
<h2>Music</h2>
<p>Listen to awesome music while you work. I first discovered this while mowing my parent’s’ lawn: when I listened to Metallica, that grass got cut way quicker. So, while I work, I listen to shitty pop music. <a href="http://www.pandora.com/?sc=sh588283719292597139&amp;shareImp=true">Here’s my Pandora station</a>. I find that upbeat, happy music makes me upbeat and happy. I’m over being embarrassed about my love for Ke$ha. I just bob my head and write more code.</p>
<h2>Parallelize</h2>
<p>I actually cannot be productive without doing six different things at once. This is related to <a href="http://www.structuredprocrastination.com/">“Structured Procrastination”</a>. I do a little bit of work on a bunch of different things, and cycle between them quickly. One day, it’s Rails, another, it’s Resque, another is training, then back to Rails, then Hackety…</p>
<h2>lift.do</h2>
<p>I’ve been trying out <a href="http://lift.do/">Lift</a> lately, and I really like it. You can build habits with some light social interaction. You might find it useful.</p>
<h2>Cut distractions.</h2>
<p>That’s a bunch of things I do to maximize good outcomes, but what about minimizing bad ones? In the interest of minimalization, I won’t dwell on them too much, but here are some things that I’ve cut down or cut out in some way:</p>
<ul>
<li>TV</li>
<li>Reddit/HN</li>
<li>Cell phones</li>
<li>Video Games</li>
</ul>
<p>I still indulge in some of these from time to time, but I’ve cut them out of the ‘every day’ loop. My life is better off for it. Figure out what you don’t actually need, and stop it.</p>
<p>;)</p></content:encoded>
</item>
<item>
<title>Seriously: numbers: use them!</title>
<link>https://steveklabnik.com/writing/seriously--numbers--use-them-/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/seriously--numbers--use-them-/</guid>
<pubDate>Thu, 27 Sep 2012 00:00:00 GMT</pubDate>
<content:encoded><p><a href="http://vimeo.com/50340416">Turbolinks test</a> from <a href="http://vimeo.com/steveklabnik">Steve Klabnik</a> on <a href="http://vimeo.com/">Vimeo</a>.</p>
<p>If you don’t feel like watching…</p>
<p><a href="https://github.com/steveklabnik/turbolinks_test">https://github.com/steveklabnik/turbolinks_test</a></p>
<p>Results with 1000 pages:</p>
<p>Blank Rails app:</p>
<pre><code>$ rspec
user system total real
no turbolinks 11.170000 0.980000 12.460000 (138.656728)
yes turbolinks 10.800000 0.870000 11.670000 ( 80.436286)
</code></pre>
<p>With Basecamp Next’s CSS file:</p>
<pre><code>$ rspec
user system total real
no turbolinks 14.470000 1.540000 16.320000 (235.404727)
yes turbolinks 10.730000 0.870000 11.600000 ( 82.176967)
</code></pre>
<p>With Basecamp Next’s JS file:</p>
<pre><code>$ rspec
user system total real
no turbolinks 15.300000 1.700000 17.270000 (433.880904)
yes turbolinks 10.540000 0.890000 11.430000 (170.545663)
</code></pre>
<p>With Both:</p>
<pre><code>$ rspec
user system total real
no turbolinks 21.990000 2.890000 25.150000 (581.822206)
yes turbolinks 10.970000 0.910000 11.880000 (196.481247)
</code></pre>
<h2>TL;DR</h2>
<p>Turbolinks seems to speed up apps.</p></content:encoded>
</item>
<item>
<title>Resque: let's do this</title>
<link>https://steveklabnik.com/writing/resque--let-s-do-this/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/resque--let-s-do-this/</guid>
<pubDate>Sat, 22 Sep 2012 00:00:00 GMT</pubDate>
<content:encoded><p>If you’ve ever done background job stuff with Rails, you’ve probably used <a href="https://github.com/defunkt/resque">Resque</a>. Resque is “Yet Another GitHub Project They’ve Open Sourced and Then Abandoned.” But Resque is super awesome: it builds a job queue on top of Redis, which is a totally awesome piece of software.</p>
<p>Anyway, Resque has been psuedo-abandonware for a while now. In January, <a href="http://hone.heroku.com/">Terence Lee</a> from Heroku got in charge of the project, but it’s a big job. Too big of a job to properly manage alone. It’s hard to sift through <a href="https://github.com/defunkt/resque/issues?state=open">60 pulls and 120 open issues</a>, some of which have been open for a few <em>years</em>. And manage the 1.x line while working on new stuff for 2.0. And debug issues that basically boil down to “When I upgrade Resque and this random gem, everything breaks, but when I downgrade that other gem, it works again. But Resque throws the error.”</p>
<p>Even outside of heroics, the bus factor on a project that’s as important as Resque should be higher than one.</p>
<p>So Terrence gave a presentation at Frozen Rails, and in it, he outlined what needs to be done for Resque 2.0, and asked for some help getting it out the door. So myself and a few other people are gonna pitch in and help out, and we’d love to have you.</p>
<p>Here’s the deal: I will personally be looking at (and already have read) every single open issue on GitHub. Feel ignored about something involving Resque? You can come to me. I can’t guarantee that I can get it fixed, but at least I’ll help you figure out wtf is going on, and get us a bug report that we can use to build a fix.</p>
<p>Think of it as decoupling the front end from the back end: Terrence can focus on writing new code, and I can focus on paying attention to your problems, helping you solve them, and getting fixes applied to the 1.x branch.</p>
<p>If you’re interested in helping out, we’d love to have you. Helping triage issues would be awesome, fixing bugs would be awesome, writing code would be awesome, updating docs would be awesome.</p>
<p>Let’s do this.</p></content:encoded>
</item>
<item>
<title>Software and community</title>
<link>https://steveklabnik.com/writing/software-and-community/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/software-and-community/</guid>
<pubDate>Tue, 18 Sep 2012 00:00:00 GMT</pubDate>
<content:encoded><p>“Are you sure you’re not vegan? You look like a vegan.”</p>
<p>Chad Fowler made me laugh pretty hard that day. We were in Austin, at a Ruby conference, and getting some very non-vegan food at the excellent Iron Works BBQ. I had only talked to Chad a few times before, and while chowing down on some excellent ribs, we talked about something that we’d discussed in a previous conversation: “the Ruby community.”</p>
<p>I gave a talk at Lone Star Ruby last year about my involvement with Hackety Hack and Shoes, and at the end, discussed some social issues I thought Ruby has:</p>
<p>The fun starts at 27:49. But ultimately, this whole section relies on the idea that in Ruby, we have a cohesive ‘community’ of people. After I gave my talk, Chad mentioned to me that he didn’t like the idea of “the Ruby community.” At the time, I was so starstruck that I said something like “sure, okay, I’ll think about that,” not even realizing the irony. So, while trying valiantly to eat barbecue in a not completely messy way, I asked Chad more about his thoughts on this.</p>
<p>If you were wondering, if it’s good barbecue, it’s impossible to eat it neatly.</p>
<p>I don’t remember his exact words, but Chad’s point was that if we talk about Ruby like a big cohesive group, we forget about all the people who use it who don’t post on blogs, or Twitter, or contribute to GitHub. Those people are people too, but they’re effectively outside ‘the community,’ either by choice or by ignorance.</p>
<h2>Kill Your Idols</h2>
<p>It seems to me that much of software is driven by fashion, celebrity, and superstition. This shouldn’t surprise anyone who’s not a software developer, as programmers are just people, and much of the human world is driven by fashion, celebrity, and superstition. Programmers, though, regard themselves as bastions of rationality.</p>
<blockquote>
<p>I don’t argue about sports, it’s stupid. I prefer to argue about important things, like whether or not to use semicolons in JavaScript.</p>
</blockquote>
<p>— Matthew Nielsen (@xunker) September 15, 2012</p>
<blockquote></blockquote>
<p>Right? This whole idea makes us, or at least me, very uncomfortable. I’m a scientist, dammit! I measure twice, cut once. I use the right tool for the job. I swear, MongoDB, CoffeeScript, and Ember.js fits my use case perfectly!</p>
<p>Ideally, we’d make technical decisions on technical means only. However, we’re just humans. I’m not sure it’s possible to divorce our decision making from our squishy human tendencies, and after typing that sentence, I’m no longer sure that it’s a good idea, either.</p>
<h2>Leaders vs. Bosses</h2>
<blockquote>
<p>Does it follow that I reject all authority? Far from me such a thought. In the matter of boots, I refer to the authority of the bootmaker; concerning houses, canals, or railroads, I consult that of the architect or the engineer. For such or such special knowledge I apply to such or such a savant. But I allow neither the bootmaker nor the architect nor the savant to impose his authority upon me. I listen to them freely and with all the respect merited by their intelligence, their character, their knowledge, reserving always my incontestable right of criticism and censure. I do not content myself with consulting a single authority in any special branch; I consult several; I compare their opinions, and choose that which seems to me the soundest. But I recognise no infallible authority, even in special questions; consequently, whatever respect I may have for the honesty and the sincerity of such or such an individual, I have no absolute faith in any person. Such a faith would be fatal to my reason, to my liberty, and even to the success of my undertakings; it would immediately transform me into a stupid slave, an instrument of the will and interests of others.Mikhail Bakunin, “What is Authority,” 1882</p>
</blockquote>
<p>There is a difference between a leader and a boss. There’s also a difference between a leader and a celebrity. What I’m concerned with is that we occasionally turn leaders into bosses, and we turn celebrities into leaders.</p>
<p>One other Ruby person that straddles this line is Aaron “Tenderlove” Patterson. He is, for good reason, almost universally beloved by Rubyists. He’s known to all for two different reasons: as a leader, and as a celebrity. Aaron is the only person on both the core Ruby as well as core Rails teams. He is one of the most technically brilliant people I know.</p>
<p>He also posts amazing pictures of his cat on Tumblr. (The caption is “My new Yoga outfit.”)</p>
<p><img src="http://25.media.tumblr.com/tumblr_m3v8keZVhR1rt6kj7o1_1280.png" alt="http://25.media.tumblr.com/tumblr_m3v8keZVhR1rt6kj7o1_1280.png" /></p>
<p>gorbypuff</p>
<p>Another amazingly technical thing Aaron has done is <a href="http://confreaks.com/videos/427-rubyconf2010-zomg-why-is-this-code-so-slow">re-writing the AST -&gt; SQL generation stuff in ActiveRecord</a>, which sped up ActiveRecord by 5x. He deserves every single last bit of his popularity, on both sides. I happen to find the split reasons interesting.</p>
<h2>“The community” as useful abstraction</h2>
<p>While I think ‘the community’ may help drive a celebrity culture over a culture of leadership, there’s no denying that we do, in fact, have a community. If I had a nickel for every time someone in Ruby did something nice for me just because, I’d have a lot of nickels. I’m writing this post from the couch of <a href="https://twitter.com/kiiiir">a Rubyist</a> who happened to be on the same plane as me to Helsinki, and rather than stay at the airport for 10 hours overnight (checkin isn’t for another few hours yet), I crashed here. On my last trip to Germany, Konstantin and Josh from Travis not only gave me their couches to sleep on, but bought me food when I fucked up my debit card situation. One Saturday, Luis Lavena spent 8 hours on a Saturday randomly pairing with me on some Ruby on Windows stuff, and left with an “I’m sorry, but my fiance is bugging me, I told her we’d get dinner an hour ago, I need to leave.” That’s crazy, I’m some dude from another hemisphere, hang out with your fiance! Aaron helped me debug an issue in Rails the other day, even though I kept asking him stupid questions. I have tons of stories like this, and so do many other people who participate in open source.</p>
<p>In “Debt: The first 5,000 Years,” Graeber defines “everyday communism” as "any community who follows the maxim ‘from each according to their abilities, to each according to their needs." Of course, communism and community share a root for a reason, and all of these examples above are great descriptions of ’everyday communism.’ Not every exchange must go through some sort of monetary exchange. It’s often simpler to just do each other favors, and know that it will all work out in the end. That’s the basis of a gift economy.</p>
<p>In this way, I think community is a useful abstraction; we just need to make sure that we don’t get <a href="http://rubydramas.com/">too caught up in ourselves</a>. Focus on the friendships, helping each other out, and writing great software. Try not to get too trendy. Pay attention to what matters.</p>
<p>In the end, I wouldn’t give up the <strong>real</strong> friendships I’ve made via the Ruby community for the world.</p></content:encoded>
</item>
<item>
<title>Random Ruby Tricks: Class.new</title>
<link>https://steveklabnik.com/writing/random-ruby-tricks--class-new/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/random-ruby-tricks--class-new/</guid>
<pubDate>Sun, 09 Sep 2012 00:00:00 GMT</pubDate>
<content:encoded><p>If you didn’t know, classes are first-class objects in Ruby:</p>
<pre><code>1.9.3p194 :001 &gt; String.class
=&gt; Class
1.9.3p194 :002 &gt; Class.class
=&gt; Class
</code></pre>
<p>How is this useful, though?</p>
<h2>Inheritance</h2>
<p>You may create very simple classes at times. This often happens when subclassing an error of some sort:</p>
<pre><code>class MyException &lt; StandardError
end
raise MyException
</code></pre>
<p>That whole thing is awkward to me, though. Why bother with the end if you don’t need it? Turns out, it’s easy to make your own <code>Class</code> object: just use <code>Class.new</code>:</p>
<pre><code>1.9.3p194 :001 &gt; Class.new
=&gt; #&lt;Class:0x007fee209f42f8&gt;
</code></pre>
<p>But that doesn’t help us a ton. Turns out <code>Class.new</code> takes an argument: the class this new class inherits from:</p>
<pre><code>1.9.3p194 :002 &gt; Class.new(StandardError)
=&gt; #&lt;Class:0x007fee209ee2e0&gt;
</code></pre>
<p>We can then save this in our own constant:</p>
<pre><code>1.9.3p194 :003 &gt; MyException = Class.new(StandardError)
=&gt; MyException
1.9.3p194 :004 &gt; MyException.new
=&gt; #&lt;MyException: MyException&gt;
1.9.3p194 :005 &gt; MyException.ancestors
=&gt; [MyException, StandardError, Exception, Object, Kernel, BasicObject]
</code></pre>
<p>I think I like this notation a bit better than <code>class...end</code>. But many people may not be familiar with it.</p>
<h2>Passing a block</h2>
<p>Of course, you can pass a block to <code>Class.new</code> and it’ll work like you’d expect:</p>
<pre><code>1.9.3p194 :001 &gt; Foo = Class.new do
1.9.3p194 :002 &gt; def bar
1.9.3p194 :003?&gt; puts "bar"
1.9.3p194 :004?&gt; end
1.9.3p194 :005?&gt; end
=&gt; Foo
1.9.3p194 :006 &gt; Foo.new.bar
bar
=&gt; nil
</code></pre>
<p>The block gets <code>class_eval</code>’d. I haven’t found a good use for this one, exactly, but it’s good to know about!</p>
<p>EDIT: <a href="https://twitter.com/pete_higgins/status/244903490588119040">@pete_higgins has chimed in on Twitter.</a> I knew I’d done this before!</p>
<h2><code>new</code> vs <code>initialize</code></h2>
<p>Ever wonder why you call <code>Foo.new</code> but define <code>Foo.initialize</code>? It’s pretty simple. Here:</p>
<pre><code>VALUE rb_class_new_instance(int argc, VALUE *argv, VALUE klass)
{
VALUE obj;
obj = rb_obj_alloc(klass);
rb_obj_call_init(obj, argc, argv);
return obj;
}
</code></pre>
<p>Obviously! This is the source for <code>Foo.new</code>. You might not read C, but it’s pretty simple: it first allocates the space for the object using <code>rb_obj_alloc</code>, and then calls <code>initialize</code> using <code>rb_obj_call_init</code>.</p></content:encoded>
</item>
<item>
<title>Protocol and language</title>
<link>https://steveklabnik.com/writing/protocol-and-language/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/protocol-and-language/</guid>
<pubDate>Mon, 03 Sep 2012 00:00:00 GMT</pubDate>
<content:encoded><p>This is the second part of my series on protocol. The <a href="http://words.steveklabnik.com/protological-control-an-introduction">first part</a> contained a lot of background information, but now we’re ready to get into what Protocol actually <em>is</em>.</p>
<hr />
<p>I live in a pretty unique place, <a href="http://farmhouse.la/">the Farmhouse</a>. It’s a psuedo-public space in which we live, others work, and everybody has a good time. We do have <a href="http://farmhouse.la/rules">some rules</a>, though:</p>
<p><img src="http://farmhouse.s3.amazonaws.com/images/home/farmhouse-rules-chalkboard-wall-by-heather-peterson.jpg" alt="http://farmhouse.s3.amazonaws.com/images/home/farmhouse-rules-chalkboard-wall-by-heather-peterson.jpg" /></p>
<p>They’re written up on the wall in chalk. Since we have lots of visitors to the house, and these are hard and fast rules, it’s important that we communicate them in a clear way. These rules form the protocol of our house. Phrased another way, they’re the rules that govern the way that we interact with each other and the house itself while at the house.</p>
<p>Ultimately, this is what protocol is: a language. To Galloway, protocol is a very specific kind of language:</p>
<blockquote>
<p>Protocol is a language that regulates flow, directs netspace, codes relationships, and connects life-forms.</p>
</blockquote>
<p>Some of these properties are contradictory. Stated more accurately, there exists a tension between opposing forces within protocol itself. Let’s expand on these meanings.</p>
<h2>‘regulates flow’</h2>
<p>If you’ve ever read <em>1984</em>, you’re familiar with linguistic relativity, more commonly known as the Sapir-Whorf hypothesis.</p>
<blockquote>
<p>How could you have a slogan like “freedom is slavery” when the concept of freedom has been abolished?</p>
</blockquote>
<p>What’s truly interesting is that Orwell himself actually advocated for simplifying language in <a href="http://www.resort.com/~prime8/Orwell/patee.html">Politics and the English Language</a>, but specifically because he felt more complicated language was <em>obfuscatory</em>, not illuminating.</p>
<p>In any case, language enumerates the possible, and while <a href="http://en.wikipedia.org/wiki/Longest_words#German">some languages can have really long words</a>, ultimately, any language will have some things that are not expressible. <a href="http://byfat.xxx/rien-ne-tient-en-place">This often happens with translations</a>.</p>
<p>If you are the proud recipient of a computer science degree, you may have run into the <a href="http://en.wikipedia.org/wiki/Chomsky_hierarchy">Chomsky hierarchy</a> in your compilers class. If you weren’t required to take a compilers class, please call up your alma mater and demand a refund. Here is the hierarchy:</p>
<ol>
<li>unrestricted grammars</li>
<li>context-sensitive grammars</li>
<li>context-free grammars</li>
<li>regular grammars</li>
</ol>
<p>As you go up the hierarchy, each language type is enclosed by the type above it. All context-sensitive grammars are unrestricted grammars, and all context-free grammars are context-sensitive.</p>
<p>Regular expressions are one area where a language was forced to move up the hierarchy in order to express more things. Strictly speaking, a regular expression cannot match nested parenthesis. If you are interested in the math, the <a href="http://en.wikipedia.org/wiki/Pumping_lemma_for_regular_languages">Pumping Lemma</a> demonstrates why. However, this kind of thing is useful, and so extensions were developed that move them up the chain. <a href="http://perldoc.perl.org/perlfaq6.html#Can-I-use-Perl-regular-expressions-to-match-balanced-text%3f">Now you can write this</a>:</p>
<blockquote>
<p>qr/(&lt;(?:[^&lt;&gt;]++|(?1))*&gt;)/</p>
</blockquote>
<p>Maybe Orwell was on to something.</p>
<p>Others have speculated that linguistic relativity would also apply to software. Two notable examples are Kenneth E. Iverson, who created APL, and Paul Graham, Lisp enthusiast and investor extraordinaire. Iverson wrote a paper titled <a href="http://www.jsoftware.com/papers/tot.htm">“Notation as a tool of thought”</a>, which begins:</p>
<blockquote>
<p>The importance of nomenclature, notation, and language as tools of thought has long been recognized.</p>
</blockquote>
<p>Graham authored an essay called <a href="http://www.paulgraham.com/avg.html">“Beating the Averages”</a> in which describes the ‘Blub paradox’. The paradox relies on linguistic relativity:</p>
<blockquote>
<p>You can’t trust the opinions of the others, because of the Blub paradox: they’re satisfied with whatever language they happen to use, because it dictates the way they think about programs.</p>
</blockquote>
<p>I don’t mean to rain on the relativity parade, but there’s one small detail that we haven’t considered: Turing completeness. At the end of the day, most programming languages are Turing complete. This means that COBOL can do everything Lisp can, and PHP can do everything that Ruby can.</p>
<p>So are Graham and Iverson wrong?</p>
<p>Turns out there are two variations of linguistic relativity: <em>strong</em> relativity states that language <strong>determines</strong> thought, and <em>weak</em> relativity that states language <strong>influences</strong> thought. This was coined later, as it would seem that current research has found little evidence to support strong relativity, but some to support the notion of weak relativity. Graham notes this, and confines the paradox to the realm of weak:</p>
<blockquote>
<p>All languages are equally powerful in the sense of being Turing equivalent, but that’s not the sense of the word programmers care about. (No one wants to program a Turing machine.)</p>
</blockquote>
<p>To put it another way, language features create friction that encourages or discourages certain modes of thinking.</p>
<p>Python and Ruby are two very similar languages, yet they engender very different styles. One area in which this division is sharp is first-class functions. Both languages have them, but Python’s are simply expressions rather than fully-featured . <a href="http://www.artima.com/weblogs/viewpost.jsp?thread=147358">Guido van Rossum likes them this way</a>:</p>
<blockquote>
<p>But such solutions often lack “Pythonicity” – that elusive trait of a good Python feature. It’s impossible to express Pythonicity as a hard constraint. Even the Zen of Python doesn’t translate into a simple test of Pythonicity.</p>
</blockquote>
<p>Guido specifically wants to limit the scope of language in order to encourage a particular style of thinking about problems: a Pythonic one.</p>
<p>Galloway is very explicit about this regulation.</p>
<blockquote>
<p>Protocol is synonymous with possibility. From the perspective of protocol, if you can do it, it can’t be bad, because if it were bad, then it would have been outlawed years ago by protocol.</p>
</blockquote>
<h2>‘directs netspace’</h2>
<p>After that massive first part, I’m basically going to punt on the second. I don’t find it very interesting or worth elaborating on: languages and protocols direct the direction of the Internet. I don’t think anyone disputes this.</p>
<h2>‘codes relationships’</h2>
<p>Communication has an intimate connection to relationships. The act of communicating is what creates the relationship in the first place. And language is the encoding of that particular relationship.</p>
<p>In “A user’s guide to Capitalism and Schizophrenia: Deviations from Deleuze and Guattari,” Brian Massumi discusses an example of marriage vows:</p>
<blockquote>
<p>A particular man and a particular woman say “I do.” Their words undoubtedly have personal meaning for them in their heart of hearts. But their personal intention is not in itself responsible for the magical transformation that has touched their lives. What has brought them to say these words and what makes those words effectively transformative is too big to fit into a single mid. It is a complex interplay of laws, customs, social pressure, and tax law.</p>
</blockquote>
<p>Marriage is a social protocol that encodes a particular relationship. Once that relationship has been encoded, it’s made legible to others who weren’t privy to the details of that relationship. A wedding ring is only one form of relationship encoding; a couple holding hands is a similar example, whose bond is weaker both in symbol as well as in reality.</p>
<p>In the software world, HTTP encodes relationships as well. The client/server relationship is integral to the protocol’s design, as Roy Fielding <a href="http://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch_styles.htm#sec_3_4_1">describes in section 3 of his dissertation</a>:</p>
<blockquote>
<p>A client is a triggering process; a server is a reactive process. Clients make requests that trigger reactions from servers. Thus, a client initiates activity at times of its choosing; it often then delays until its request has been serviced. On the other hand, a server waits for requests to be made and then reacts to them. A server is usually a non-terminating process and often provides service to more than one client.</p>
</blockquote>
<p>He further goes on to describe why this relationship is important:</p>
<blockquote>
<p>Separation of concerns is the principle behind the client-server constraints. A proper separation of functionality should simplify the server component in order to improve scalability. This simplification usually takes the form of moving all of the user interface functionality into the client component. The separation also allows the two types of components to evolve independently, provided that the interface doesn’t change.</p>
</blockquote>
<p>In other words, the protocol encodes the relationship so that others know what a client should do, and what a server should do.</p>
<p>If you remember our good friend Saussure, you’ll remember that he made this encoding explicit, and in fact central to language itself. While Saussure’s work isn’t directly useful to linguists in the present, much of his work is foundational to the ways that we approach language today.</p>
<p>One of the most fundamental concepts in structural linguistics is the ‘sign,’ and its component parts, the ‘signifier’ and the ‘signified.’ The signified is a particular concept, and the signifier is the way the signified is expressed. A sign is a combination of a signifier and a signified. Seems pretty simple, but Saussure also posited that a sign can only gain <a href="http://en.wikipedia.org/wiki/Meaning_(semiotics)">meaning</a> and <a href="http://en.wikipedia.org/wiki/Meaning_(semiotics)">value</a> within their relationship to other signs, which means that you cannot consider an idea outside of the way in which the idea is expressed.</p>
<p>The study of signs is called <a href="http://en.wikipedia.org/wiki/Semiotics">semiotics</a>. In Semiotics, a particular set of conventions used to convey meaning are called <a href="http://en.wikipedia.org/wiki/Code_(semiotics)">codes</a>. Hence ‘encoding’ can also refer to this kind of code, as well.</p>
<p>Hence, if we combine Fielding, Galloway, and Saussure, the word ‘client’ is a signifier that holds a relationship to the signified concept ‘a triggering process’ (among other things), that forms the sign ‘client.’ This is encoded by the protocol HTTP.</p>
<h2>‘connects life-forms’</h2>
<p>From the Massumi marriage example:</p>
<blockquote>
<p>Say “I do,” and your life will never be the same. Your legal, social, and familial status instantly changes, along with your entire sexual, psychological, and financial economy. You have been pronounced man and wife. You may file a joint tax return.</p>
</blockquote>
<p>This connection is related to the previous coded relationship. In this case, ‘connects’ is an active verb rather than a passive noun. It not only <em>creates</em> a connection, but it ‘connects’. It not only relates two objects, it <em>is</em> the relation.</p>
<p>We spent most of the last section discussing connections, though, so let’s get to the interesting part: life. Galloway is very explicit about the role protocol plays with life:</p>
<blockquote>
<p>…protocol is an affective, aesthetic force that has control over “life itself.”</p>
</blockquote>
<p>In <a href="http://www.amazon.com/The-Exploit-Networks-Electronic-Mediations/dp/0816650446">the Exploit</a>, Galloway discusses the relationship between life and protocol as well:</p>
<blockquote>
<p>While the first type of network (Internet protocols) is silicon based and may use biological concepts (intelligent agents, artificial life, genetic algorithms), the second (DNA algorithms) is fully biological and yet recodes itself in computational terms (biology as computation, as opposed to evolution)</p>
</blockquote>
<p>The relationship between life and software is an interesting one.</p>
<p><img src="http://upload.wikimedia.org/wikipedia/commons/e/e5/Gospers_glider_gun.gif" alt="http://upload.wikimedia.org/wikipedia/commons/e/e5/Gospers_glider_gun.gif" /></p>
<p>In Conway’s game of life, one builds cellular automata and lets them go. There are a few simple rules that govern their reproduction, death, and life. This protocol governs all that happens within the game. Someone playing the game is an impotent god, as there is no room in the protocol for communicating with creations. Likewise, creations have no way of perceiving an ‘outside’ of the game.</p>
<p>Galloway references this quote by Frederick Cohen from <a href="http://www.amazon.com/Course-Computer-Viruses-Professional-Computing/dp/0471007684">A Short Course on Computer Viruses</a>:</p>
<blockquote>
<p>I personally believe that reproducing programs are living beings in the information environment.</p>
</blockquote>
<p>Viruses, Conway cells, <a href="http://thesims.com/en_us/home">Sims</a>, and all other kinds of ‘artificial life’ exist inside of computers. They’re native to protocol. But what about actual, real life?</p>
<p>Galloway reaches to Norbert Wiener, a mathematician who was the originator of cybernetics, for this:</p>
<blockquote>
<p>if one views the world in terms of information, then there is little instrumental difference between man and machine since both are able to affect dynamic systems via feedback loops. In this way the cybernetic system of man and machine are born. Its virtues are balance, self-regulation, circularity, and control. In a word, protocol.</p>
</blockquote>
<p>Deleuze also forms a connection between life and information in <a href="http://www.n5m.org/n5m2/media/texts/deleuze.htm">“Postscript on the Societies of Control”</a>:</p>
<blockquote>
<p>The numerical language of control is made of codes that mark access to information, or reject it. We no longer find ourselves dealing with the mass/individual pair. Individuals have become “dividuals,” and masses, samples, data, markets, or “banks.”</p>
</blockquote>
<p>A <em>dividual</em> is a play on the process of <a href="http://en.wikipedia.org/wiki/Individuation">individuation</a>. Galloway elaborates:</p>
<blockquote>
<p>Deleuze’s neologism comes from the word ‘individuate.’ Dividuation would thus be the opposite: the dissolving of the individual entity into distributed networks of information.</p>
</blockquote>
<p>Let’s use a personal example to demonstrate life submitting itself to dividuation in order to interface with protocol: me. <a href="http://steveklabnik.com/">This is my personal website</a>. In it, my public face on the Internet, I’ve split myself up into five broad categories, and link out to all of the various sub-components. Examining the last category, you can see how if you just follow me on twitter <a href="https://twitter.com/steveklabnik">;)</a>, you will not get the whole picture. You could also <a href="https://github.com/steveklabnik">check out my code on GitHub</a>, but that wouldn’t be me, either. Even the sum of these two components would not tell you everything about me. Yet I can’t possibly put <em>me</em> on the network. I must break ‘me’ into a form that conforms with protocol.</p>
<p>Finally, there’s an important relationship with Foucault’s concept of ‘biopolitics.’</p>
<blockquote>
<p>Foucault defines biopolitics as “the endeavor, begun in the eighteenth century, to rationalize the problems presented to governmental practice by the phenomena characteristic of a group of living human beings constituted as a population: health, sanitation, birthrate, longevity, race.”</p>
</blockquote>
<p>Obviously, dividuating a population into informational components would be relevant to protocol. And, in fact, life provides a path to resist protocol. Deleuze in his book <em>Foucault</em> (because that’s not confusing…):</p>
<blockquote>
<p>When power becomes bio-power resistance becomes the power of life, a vital power that cannot be confined within species, environment or the paths of a particular diagram.</p>
</blockquote>
<p>Galloway expands:</p>
<blockquote>
<p>Is life resistance a way of engaging with distributed forms of protological management? Part III of this book, “Protocol Futures,” answers yes. While the new networked technologies have forced an ever more reticent public to adapt to the control structures of global capital, there has emerged a new set of social practices that inflects or otherwise diverts these protological flows toward the goal of a utopian form of unalienated social life.</p>
</blockquote>
<h2>To conclude</h2>
<p>Protocol has an intimate and complex relationship with language. In many ways, it is a particular language, with all of the expressive power and limiting abilities that that brings.</p>
<p>The implications of this language, though, will have to wait for another time.</p></content:encoded>
</item>
<item>
<title>Random Ruby Tricks: Struct.new</title>
<link>https://steveklabnik.com/writing/random-ruby-tricks--struct-new/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/random-ruby-tricks--struct-new/</guid>
<pubDate>Sat, 01 Sep 2012 00:00:00 GMT</pubDate>
<content:encoded><p>One of my favorite classes in Ruby is <code>Struct</code>, but I feel like many Rubyists don’t know when to take advantage of it. The standard library has a lot of junk in it, but <code>Struct</code> and <code>OStruct</code> are super awesome.</p>
<h2>Struct</h2>
<p>If you haven’t used <code>Struct</code> before, here’s <a href="http://www.ruby-doc.org/core-1.9.3/Struct.html">the documentation of Struct from the Ruby standard library</a>.</p>
<p>Structs are used to create super simple classes with some instance variables and a simple constructor. Check it:</p>
<pre><code>Struct.new("Point", :x, :y) #=&gt; Struct::Point
origin = Struct::Point.new(0,0) #=&gt; #&lt;struct Struct::Point x=0, y=0&gt;
</code></pre>
<p>Nobody uses it this way, though. Here’s the way I first saw it used:</p>
<pre><code>class Point &lt; Struct.new(:x, :y)
end
origin = Point.new(0,0)
</code></pre>
<p>Wait, what? Inherit…from an instance of something? Yep!</p>
<pre><code>1.9.3p194 :001 &gt; Struct.new(:x,:y)
=&gt; #&lt;Class:0x007f8fc38da2e8&gt;
</code></pre>
<p><code>Struct.new</code> gives us a <code>Class</code>. We can inherit from this just like any other <code>Class</code>. Neat!</p>
<p>However, if you’re gonna make an empty class like this, I prefer this way:</p>
<pre><code>Point = Struct.new(:x, :y)
origin = Point(0,0)
</code></pre>
<p>Yep. Classes are just constants, so we assign a constant to that particular <code>Class</code>.</p>
<h2>OStruct</h2>
<p><code>[OStruct](http://ruby-doc.org/stdlib-1.9.3/libdoc/ostruct/rdoc/OpenStruct.html)</code>s are like <code>Struct</code> on steroids. Check it:</p>
<pre><code>require 'ostruct'
origin = OpenStruct.new
origin.x = 0
origin.y = 0
origin = OpenStruct.new(:x =&gt; 0, :y =&gt; 0)
</code></pre>
<p><code>OStruct</code>s are particularly good for configuration objects. Since any method works to set data in an <code>OStruct</code>, you don’t have to worry about enumerating every single option that you need:</p>
<pre><code>require 'ostruct'
def set_options
opts = OpenStruct.new
yield opts
opts
end
options = set_options do |o|
o.set_foo = true
o.load_path = "whatever:something"
end
options #=&gt; #&lt;OpenStruct set_foo=true, load_path="whatever:something"&gt;
</code></pre>
<p>Neat, eh?</p>
<h2>Structs for domain concepts</h2>
<p>You can use <code>Struct</code>s to help reify domain concepts into simple little classes. For example, say we have this code, which uses a date:</p>
<pre><code>class Person
attr_accessor :name, :day, :month, :year
def initialize(opts = {})
@name = opts[:name]
@day = opts[:day]
@month = opts[:month]
@year = opts[:year]
end
def birthday
"#@day/#@month/#@year"
end
end
</code></pre>
<p>and we have this spec</p>
<pre><code>$:.unshift("lib")
require 'person'
describe Person do
it "compares birthdays" do
joe = Person.new(:name =&gt; "Joe", :day =&gt; 5, :month =&gt; 6, :year =&gt; 1986)
jon = Person.new(:name =&gt; "Jon", :day =&gt; 7, :month =&gt; 6, :year =&gt; 1986)
joe.birthday.should == jon.birthday
end
end
</code></pre>
<p>It fails, of course. Like this:</p>
<pre><code>$ rspec
F
Failures:
1) Person compares birthdays
Failure/Error: joe.birthday.should == jon.birthday
expected: "7/6/1986"
got: "5/6/1986" (using ==)
# ./spec/person_spec.rb:9:in `block (2 levels) in &lt;top (required)&gt;'
Finished in 0.00053 seconds
1 example, 1 failure
Failed examples:
rspec ./spec/person_spec.rb:5 # Person compares birthdays
</code></pre>
<p>Now. We have these two birthdays. In this case, we know about why the test was failing, but imagine this failure in a real codebase. Are these month/day/year or day/month/year? You can’t tell, it could be either. If we switched our code to this:</p>
<pre><code>class Person
attr_accessor :name, :birthday
Birthday = Struct.new(:day, :month, :year)
def initialize(opts = {})
@name = opts[:name]
@birthday = Birthday.new(opts[:day], opts[:month], opts[:year])
end
end
</code></pre>
<p>We get this failure instead:</p>
<pre><code>$ rspec
F
Failures:
1) Person compares birthdays
Failure/Error: joe.birthday.should == jon.birthday
expected: #&lt;struct Person::Birthday day=7, month=6, year=1986&gt;
got: #&lt;struct Person::Birthday day=5, month=6, year=1986&gt; (using ==)
Diff:
@@ -1,2 +1,2 @@
-#&lt;struct Person::Birthday day=7, month=6, year=1986&gt;
+#&lt;struct Person::Birthday day=5, month=6, year=1986&gt;
# ./spec/person_spec.rb:9:in `block (2 levels) in &lt;top (required)&gt;'
Finished in 0.00092 seconds
1 example, 1 failure
Failed examples:
rspec ./spec/person_spec.rb:5 # Person compares birthdays
</code></pre>
<p>We have a way, way more clear failure. We can clearly see that its our days that are off.</p>
<p>Of course, there are other good reasons to package related instance variables into <code>Struct</code>s, too: it makes more conceptual sense. This code represents our intent better: a Person has a Birthday, they don’t have three unrelated numbers stored inside them somehow. If we need to add something to our concept of birthdays, we now have a place to put it.</p></content:encoded>
</item>
<item>
<title>New blog: Words</title>
<link>https://steveklabnik.com/writing/new-blog--words/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/new-blog--words/</guid>
<pubDate>Fri, 31 Aug 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve started a new blog: <a href="http://words.steveklabnik.com/">Words</a>.</p>
<p>You may have noticed that it’s a part of <a href="https://svbtle.com/">the Svbtle blogging network</a>. I’m super excited about this, but you may be confused. Let me explain.</p>
<h2>My relationship with writing</h2>
<p>I’ve been involved in Internet communities for most of my life. One of the nice things about that is that it’s forced me to express myself in writing for a very, very long time.</p>
<p>I don’t remember what site this is, but there was one that generated a PDF of your top 100 comments on Hacker News. The idea was “You’ve already written a book!” You can see this kind of thinking in many “Blog to book” books that are out there, but I found this really powerful. My top 100 comments span 19 pages; I was (at one point) number 60 on the top of Hacker News by karma. And I really was only active on HN for about a year, February 2010 to 2011. I spent much more time on /., Reddit, and a half dozen forums. I bet if you compiled a list of everything I ever wrote in a <code>&lt;textarea name="comment" id="comment"&gt;</code>, I would have written at least 3 novels by now. They’d be shit, of course, but the point is that if you’re active at all on the read/write web, then you’ve probably written a ton.</p>
<p>As a side point, I really enjoyed the anonymity that posting on the web gave me. I could be 13, yet arguing with people who were 30! And they’d actually take me seriously, since they didn’t know I wasn’t an adult! Of course, the only way they wouldn’t know is if I posted proper English… grammar nazis helped me out, in a way.</p>
<p>Anyway.</p>
<h2>Practice makes perfect</h2>
<p>Of course, this means that at first, I <em>wasn’t</em> practiced at writing, and that means that <a href="http://games.slashdot.org/comments.pl?sid=87657&amp;cid=7604346">it was any good…</a> That’s the only way that you get better, though: write and write and write and write. And get feedback on that writing. Originally, it was just people commenting back. Then upvotes. I’m pretty sure that I’ve averaged over two or three comments a day on the Internet over the past decade or so. If you include tweets, I bet the median is about the same, but the mean would go up to 6 or 8.</p>
<p>Regardless, I tend to be very off-the-cuff when writing. When I’m done writing this blog post, I will type <code>:wq&lt;enter&gt;git commit -am "add 'words'"&lt;enter&gt;git push origin,heroku&lt;enter&gt;&lt;alt-tab&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;</code> . Then tweet. I end up doing very little proofreading of the things I write, and that means that sometimes, my writing suffers. I’ve settled into a local maximum: I’m Good Enough at writing that people like what I write, and I can quickly type something out and get it up there. But I don’t think I’m getting any <strong>better</strong> at writing. And I’d like to.</p>
<h2>Gamification and Exhibitionism</h2>
<p>I’ve also come to realize something about myself over the years: I respond really well to gamification, and am somewhat of an exhibitionist. Or at the very least, an extrovert. These two things in combination are a powerful mixture.</p>
<p>When I started playing StarCraft II, I knew I wanted to get better. So I set up <a href="http://www.twitch.tv/emmagoldmansc">an account on Twitch</a>. I sometimes stream games there. I’m in Silver league, which means I’m not that good. But knowing that someone may be watching makes me want to be better at it. I sometimes <a href="https://vimeo.com/37978370">release videos of me live coding</a>. Almost everything <a href="https://github.com/steveklabnik">I code is open source</a>. Knowing that others might be watching makes me want to be at my best.</p>
<p>Gamification can help with this, too. Monotonically increasing numbers are terribly addicting, and trigger something deep down in my brain. I’ve been known to refresh my retweet count or my upvote count every so often just to watch that number get higher. Combining this with public-ness means that I can compare myself to everyone else with a number too, and I can get to the top of the leaderboard! I’m terribly competitive at gaming, and quantification is the first step towards gamification and competition. I’ve been watching the Quantified Self movement from afar specifically because I’m afraid that I’ll do nothing but ‘self-improve’ and then not use that improvement for anything worthwhile.</p>
<h2>Svbtle</h2>
<p>If you haven’t heard, <a href="http://dcurt.is/codename-svbtle">Svbtle</a> is Dustin Curtis’ blogging network. I remember reading that post when Dustin started it, and this was the part that stuck out to me:</p>
<blockquote>
<p>The goal is simple: when you see the Svbtle design, you should know that the content is guaranteed to be great.</p>
</blockquote>
<p>I’m always looking for good things to read on the Internet, so I kept this idea in the back of my head. When I saw a Svbtle blog, I raised my expectations. I can really enjoy bad things as long as I go into them expecting that they’re bad; I’m not the type of person that only enjoys high-brow stuff. And generally, blogs have low-brow expectations for me.</p>
<p>Anyway, after a couple of months, I noticed that I really did think that most posts on Svbtle were pretty damn good. At the very least, they weren’t bad in the way that many blog posts are: super flamebait, shit-talking, or half-baked. I also kept looking at my blog, and realized that many of my posts are sort of trivial, or not really up to my current standards, or have those kinds of flaws. And finally, I <em>hate</em> CSS. Like, I’m no good at it. At all. And I’m sick of messing around with my own style. But most blogging software sucks…</p>
<p>So I applied.</p>
<p>It’s sort of funny to talk about hating startups and centralized internet companies, then go from my own Jekyll blog to a hosted solution. There are a few huge wins for me with Svbtle, though:</p>
<p>Ultimately, it’s all posted on my own domain via the magic of CNAME. This means that in the future, I can re-host everything myself if I want to. I’ve already done this previously when I moved from Posterous to my own blog, and even have <a href="https://github.com/steveklabnik/blog/blob/master/redirects.rb#L32">redirects.rb</a> already set up. Secondly, Svbtle will send me an archive of all my posts if I ask, which means that I’m never going to lose the data. I will probably keep my own backups locally as well. But in the event that Svbtle goes south, I’m not locked in.</p>
<p>Getting kicked off of Svbtle for having bad content would make me really upset. If it’s a blog network that’s high quality, and my writing isn’t up to snuff, that would make me sad. Time to level up! Now that I’m there, I want to stay there. This gives me lots of motivation to write high quality things, and to improve. And Svbtle provides tools to help me become a better writer, like vetting and copy editing. That’s a pretty huge attraction to me.</p>
<p>Finally, there’s light gamification with the whole Kudos thing. I know that I’ll be trying to get as many as possible with each post, too, and that it can give me reasonable feedback in some fashion. Not as good as well-written responses, but it’s easier to give someone internet points than it is to write a full rebuttal.</p>
<h2>Two blogs</h2>
<p>I’m still going to keep this blog going; it’s gonna shift to something that’s more about code and less about politics. Words will be more politics and theory with a de-emphasis on code. This blog will be more like a Tumblr in that I’ll post more often, do less work on each post, and make the good but quick. Words will become a place where I’ll take my time, post less frequently, but really work on the polish.</p>
<p>I hope you’ll enjoy continuing to read the things I write about, here or there.</p></content:encoded>
</item>
<item>
<title>Protological Control: an introduction</title>
<link>https://steveklabnik.com/writing/protological-control-an-introduction/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/protological-control-an-introduction/</guid>
<pubDate>Mon, 13 Aug 2012 00:00:00 GMT</pubDate>
<content:encoded><blockquote>
<p>“I finished by pointing out that, like scientists, people in the web development community had to be ethically and morally aware of what they were doing. I thought this might be construed as a bit out of line by the geek side, but the people present were the ones now creating the Web, and therefore were the only ones who could be sure that what the systems produced would be appropriate to a reasonable and fair society.”Tim Berners-Lee, “Weaving the Web,” p86.</p>
</blockquote>
<p>Humans are social creatures. Regardless of your position on how human society should be, we all acknowledge that it does exist. While there may be some who live a lonely life in the wilderness, they still live within the larger social organization of a state. And even though those few people do exist, they are a tiny, tiny portion of the overall population. For the vast majority of humanity, society is a reality.</p>
<p>What do I mean when I say ‘society?’ For this, I turn to Marx:</p>
<blockquote>
<p>“Society does not consist of individuals, but expresses the sum of interrelations, the relations within which these individuals stand.”Karl Marx, Grundrisse der Kritik der Politischen Ökonomie</p>
</blockquote>
<p>Regardless of your opinions of Marx, I also don’t think this is a particularly controversial opinion: society is made up of our relationships with each other. A single person does not a society make, but our connections to each other. These connections are the foundational aspect of society; they are atomic.</p>
<p>Society is a fractal, rhizomatic object: there is not only the broader ‘society,’ but many smaller, overlapping, societies. These societies are a set of these relations within a given context. These contexts provide us with guidelines of what sorts of qualities these relations possess. For example, when amongst friends at home, a person will act differently than when around those same set of friends in a public place. Often, “social awkwardness” is a transgression of one of these boundaries; someone assumes that they exist in a different social space than those they’re interacting with.</p>
<blockquote>
<p>In an extreme view, the world can be seen as only connections, nothing else. We think of a dictionary as the repository of meaning, but it defines words only in terms of other words. I liked the idea that a piece of information is really defined only by what it’s related to, and how it’s related. There really is little else to meaning. The structure is everything. There are billions of neurons in our brains, but what are neurons? Just cells. The brain has no knowledge until connections are made between neurons. All that we know, all that we are, comes from the way our neurons are connected.Tim Berners-Lee, “Weaving the Web,” p12</p>
</blockquote>
<p>As it turns out, Tim isn’t the only person to find this line of thinking interesting. Ferdinand de Saussure was a linguist in the early 1900s who developed the idea of “structural linguistics,” and in the 1950’s and 1960’s, a French anthropologist named Claude Lévi-Strauss took his concepts and applied them to anthropology, birthing Structuralism. Others followed his lead and used this mode to develop an analysis of psychology, sociology, and more.</p>
<p>Societies exist naturally, but can also be man-made. Any opening of a new avenue for interpersonal relationships creates a new society amongst the multitude. The web is an example of this, and websites are a second, inner socius. The major difference between artificial and natural societies is not one of effect, but of cause. The end result is the same, but the initial conditions for the formation of the society determine the acceptable rules for the given relations that exist within it. Therefore, potential creators of said social enclosures should understand the power that they wield, and use said power to create the exact form of society they wish it make, with deliberate strokes.</p>
<h2>Society has a very clean mirror</h2>
<p>When people create, they end up creating something that reflects themselves and the societies that they live in. Linguistic relativity in even its weak form implies that language shapes who we are, and it is such with our creations. Many <em>ex</em> <em>nihilo</em> creation myths touch upon this property, like Adam’s receipt of God’s breath, or Atum from ancient Egypt, who not only created himself (?!?) but then other gods (first from his own shadow) and then man from his tears.</p>
<p>Here’s a fun, terrible example of this happening:</p>
<p>Amusingly, even though the tone is wrong, this YouTube user nails it:</p>
<p>(image lost to time)</p>
<p>Exactly. They didn’t test it on darker skinned folks. Of course, it may not be malicious, but this certainly meant that no dark-skinned people were involved in the production of this device, the whole way up to shipping it. If they were, it would have been caught early on before it was even beginning manufacture.</p>
<p>Because we grow up in a society, and we create new societies through technology, it stands to reason that society and our own biases influence the societies that we create. Even beyond that, if we create societies, we should be able to use the same tools that we use to analyze naturally occurring ones on artificial ones.</p>
<p>So how do we analyze societies?</p>
<h2>It’s all about power and control</h2>
<p>Remember Structuralism? Well, it has some issues, and not just the opposition that Saussure faced from Chomsky. Along comes a bunch of French philosophers, and they have beef. I won’t get into it, except to mention the name of one: Michel Foucault. He wrote this book titled “Surveiller et punir: Naissance de la Prison,” which has some awkward title translation issues, and so ends up as “Discipline and Punish: The Birth of the Prison” in English. Long time readers of my blog will remember <a href="http://blog.steveklabnik.com/posts/2011-01-22-a-few-notes-on-foucault">my notes</a> on this book, but the basics are as such:</p>
<h3>Foucault</h3>
<p>Feudal society was predicated on divine right. The body of a sovereign is literally the body of God, and since God also created everything, the sovereign is everything. Therefore, if you commit a crime, you commit it against the body of the sovereign, and therefore, he must exact punishment against your body in kind. Hence torture.</p>
<p>Eventually, though, torture became socially inconvenient, and people started challenging the very idea of divine right, and wanted democracy in some form. Therefore, the power to punish would need to take on a new form to survive in this new society. That mechanism is called “discipline.” Discipline (specifically of the body) is how control manifests itself in society. One of the great advances of Taylorism, for example, was studying how the bodies of assembly line workers operated and streamlining their motions.</p>
<p>Foucault also illustrates that control and power in modern society mirror Jeremy Bentham’s concept of the ‘Panopticon,’ which is a prison in a circular shape by which the rooms can all observe each other. The observation would help to keep them all in line, since the all-seeing tower in the middle would be able to observe everyone at all times.</p>
<p><img src="/img/2012-08-13/panopticon.png" alt="panopticon" /></p>
<p>I found this concept amusing, <a href="http://en.wikipedia.org/wiki/Litchfield_Towers">here</a> are the dorms I stayed in during my time at the University of Pittsburgh.</p>
<h3>Those in charge get to make the rules?</h3>
<p>Anyway, all of that implies this: the people who are ‘in charge’ are so because they get to define the rules. Most people would say this the other way around, but that’s wrong: You don’t get to make the rules because you’re in charge, you’re in charge because you get to make the rules. Regardless, what does it mean to make the rules?</p>
<p>In a simple sense, discipline is pretty easy to grasp: there’s a king (or boss). He makes the rules. Done. And it’s the same with our ‘creation of societies’ from above. When you make a society, you make it, so you get to make the rules! This ‘rule-making’ property is everywhere in software: operating systems are absolutely about creating a society of software programs on your computer. APIs are all about the connections between various systems, and the web, with its hyperlinks, even more explicitly so.</p>
<p>That’s why this stuff matters to programmers. We create societies all the time, and need to be careful of the kinds of societies that we create. We also participate in many, and need to be sure that we participate in the kinds of societies that we agree with.</p>
<p>One problem: we’ve already moved beyond discipline society, twice.</p>
<h2>Deleuze and Guattari</h2>
<p>Oh these two! What a pair! If you picked up a random copy of either volume of “Capitalism and Schizophrenia” without knowing what it was, you would be oh so confused. I mean, I vaguely knew what I was getting into, and I was still confused.</p>
<p>Deleuze specifically wrote an essay called “Postscript on the Societies of Control”. You can find it online <a href="http://www.n5m.org/n5m2/media/texts/deleuze.htm">here</a>.</p>
<blockquote>
<p>These are the societies of control, which are in the process of replacing disciplinary societies. “Control” is the name Burroughs proposes as a term for the new monster, one that Foucault recognizes as our immediate future.</p>
</blockquote>
<p>He also discusses it in <a href="http://www.amazon.com/Negotiations-1972-1990-Gilles-Deleuze/dp/0231075812">Negotiations</a>.</p>
<p>Control societies are different from disciplinary societies because discipline is all about confinement. Control is a slippery beast compared to discipline:</p>
<blockquote>
<p>Enclosures are molds, distinct castings, but controls are a modulation, like a self-deforming cast that will continuously change from one moment to the other, or like a sieve whose mesh will transmute from point to point.</p>
</blockquote>
<p>Discipline is direct, while control is indirect. I won’t get into control too much more here, because we’re past that too; control’s time on this earth was fleeting. Now we live in the era of protocol.</p>
<h2>Galloway</h2>
<p>Alexander Galloway is an Associate Professor of Media, Culture, and Communication at NYU, and every programmer should read him. Specifically, his book <a href="http://www.amazon.com/Protocol-Control-Exists-Decentralization-Leonardo/dp/0262072475">Protocol: How control exists after decentralization</a>. Galloway takes Deleuze’s work and applies it to our modern computer systems, and terms that ‘protocol.’</p>
<blockquote>
<p>Protocol is a system of distributed management.Protocol facilitates peer-to-peer relationships between autonomous entities.Protocol is anti-hierarchy and anti-authority.Protocol engenders localized decision making, not centralized.Protocol is robust, flexible, and universal.Protocol can accommodate massive contingency.Protocol is the outcome (not the antecedent) of distributed power.Galloway, “Protocol”, p82</p>
</blockquote>
<p>Sounds awesome, right? Sounds like the web.</p>
<blockquote>
<p>I saw one [advocate of informational organization systems] after the next shot down because the developers were forcing them to reorganize their work to fit the system. I would have to create a system with common rules that would be acceptable to everyone. This meant as close as possible to no rules at all.This notion seemed impossible until I realized that the diversity of different computer systems could be a rich resource– something to be represented, not a problem to be eradicated. The model I chose for my minimalist system was hypertext.Tim Berners-Lee, “Weaving the Web,” p15.</p>
</blockquote>
<p>Tim recognized that centralization was the root of exploitation:</p>
<blockquote>
<p>It also shows how a technical decision to make a single point of reliance can be exploited politically for power and commercially for profit, breaking the technology’s independence from those things, and weakening the web as a universal space.Tim Berners-Lee, “Weaving the Web,” p129.</p>
</blockquote>
<p>Tim saw that control was an issue:</p>
<blockquote>
<p>Whether inspired by free-market desires or humanistic ideals, we all felt that control was the wrong perspective. … Technically, if there was any centralized point of control, it would rapidly become a bottleneck that restricted the web’s growth, and the web would never scale up. Its being ‘out of control’ was very important.Tim Berners-Lee, “Weaving the Web,” p99</p>
</blockquote>
<p>Galloway, however, contends something else, and this is the crux of it:</p>
<blockquote>
<p>Thus it is an oversight for theorists like Lawrence Lessig (despite his strengths) to suggest that the origin of Internet communication was one of total freedom and lack of control. Instead, it is clear to me that the exact opposite of freedom – that is, control – has been the outcome of the last forty years of developments in networked communications. The founding principle of the Net is control, not freedom. Control has existed from the beginning.Perhaps it is a different type of control than we are used to seeing. It is control borne from high degrees of technical organization (protocol) not this or that limitation on individual freedom or decision making (fascism).To put it another way, in order for protocol to enable radically distributed communications between autonomous entities, it must employ a strategy of universalization, and of homogeneity. It must be anti-diversity. It must promote standardization in order to enable openness. It must organize peer groups into bureaucracies like the IETF in order to create free technologies.In short, control in distributed networks is not monolithic. It proceeds in multiple, parallel, contradictory, and often unpredictable ways. It is a complex of interrelated currents and counter-currents.Galloway, “Protocol”, p141-143.</p>
</blockquote>
<p>This is the kind of control that startups exert on the rest of the world. This is the kind of control that’s hard to see coming, because it’s inexact. This is the kind of control that lets you <em>feel</em> like you have freedom, even when you don’t. This is the kind of control that <em>likes</em> open standards, and uses them against you.</p>
<p>Discipline is Microsoft. Protocol is Google. Protocol is Facebook. Protocol is Twitter.</p>
<p>It’s in this tradition that I critique App.net and GitHub.</p></content:encoded>
</item>
<item>
<title>DayZ</title>
<link>https://steveklabnik.com/writing/dayz/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/dayz/</guid>
<pubDate>Wed, 25 Jul 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve recently come across one of the most addicting, interesting experiences I’ve ever had online: <a href="http://dayzmod.com/">DayZ</a>. I hesitate to call it a ‘video game,’ because while it is a mod for a first person shooter game, it basically ignores all common wisdom about what makes for a good game. I think that the broader implications are also interesting across other domains: ‘common wisdom’ isn’t always correct. Sometimes communities reach local maximums, and something drastic needs to be done to break out of the rut. So what is DayZ, anyway?</p>
<p>Essentially, DayZ is a simulation of a zombie apocalypse.</p>
<p>You wake up on a beach with essentially nothing in your pockets. This means that you have no way to defend yourself, and no supplies. Check out the bottom right of that screenshot: See that canteen and knife and fork? Those are water and food indicators, and if you don’t eat or drink often enough, you’ll faint from fatigue, and even lose blood (which is a proxy for health, and is the middle indicator). So you have to sneak around until you can find a hatchet, food, and water. See those two things on the upper right? That’s how much audio and visual ‘noise’ you give off. Right now, this player is basically invisible, so they’re safe.</p>
<p>The game takes realism very seriously. There’s the food and water bit, but this also isn’t some Left 4 Dead run and gun bullshit: if you take damage, you’ll start to bleed, and will need to bandage yourself or bleed out. If you get shot, you’re basically dead. You <em>might</em> be able to take two bullets from small arms. I already mentioned the food and water. You can break bones, which seriously limits mobility, and will probably cause the shakes. Better take some painkillers! There are a few vehicles, but they’re all in disrepair, so find some parts, fix them up, and then stop and put some gas in them! Voice chat only works for a 40 yard radius around where you stand, so you can’t talk to people that are far away. You don’t have a map at first, so you won’t know where the hell you are until you find one. Of course, you can read the street signs, so obviously, you’ll know when you’re in черногорск. ;). (That’s ‘Cherno’ as it’s affectionately called by players, because ‘Chernogorsk’ is too long to say.) Oh, and when you die, you die. You lose all your stuff. This also adds to the crazy tenseness of the game. There’s actually 24 hours on each server, so if you play on one with your GMT offset, it’ll be light when it’s light and dark when it’s dark outside.</p>
<p>The biggest thing, though, is that this is a true sandbox: there are no ‘levels’ or ‘experience points’ or ‘missions’ or ‘objectives.’ You just play. Average survival time is about 45 minutes, and as you live longer, the tension goes way, way up. Because there are no missions, you develop a real attachment to the ‘emergent stories’ that come out of the game, which is the big, big draw for me. Here’s an example:</p>
<p>I spawned just outside of Электрозаводск (‘Electro’), and one of my roommates had been playing for about twenty minutes, and was in Пута (“Puta”), just north. He had found a barn with two hatchets in it, and obviously, as a new player, that’s a big deal. Электро and Пута are about <a href="http://dayzdb.com/map#4.099.124">two kilometers away from each other</a>, so he decided to wait for me. I ran up over that hill, and as I got closer, I didn’t pay close enough attention, and a zombie started chasing me. I decided to sprint to the barn, so I’d at least be able to defend myself, and Kyle decided to brace himself: my running would obviously draw more zombies, and the barn had three doors: we couldn’t cover them all. If we weren’t careful, this could be the end.</p>
<p>As I sprint through the door, Kyle steps into it, and swings his hatchet. “Holy shit dude, more coming, hurry up!” I quickly located the other one, as Kyle took out the second undead trying to eat us. I stepped up too, and covered the other door. Thirty seconds of frantic hatchet-ing later, we were safe. But Kyle was starting to get thirsty, and Электро is really dangerous. We searched the rest of the settlement, but no luck. No water to be found. We decided that rather than risk the big costal town, it’d be best to <a href="http://dayzdb.com/map#4.085.109">make the two klick journey north to Могилевка</a>. While trying to locate the road, Kyle attracted the attention of another zombie, and after dispatching it, found an empty watter bottle! Luckily, Пута has a public well, so he could fill it and drink up. Problem solved.</p>
<p>Now, if this scenario had been a scripted part of some zombie RPG, I wouldn’t have cared about it at all. I’ve now told this story three times in person, and it doesn’t get old. This tense five minutes emerged naturally out of putting players in the world and letting them go. As a result, since they’re <em>your</em> stories, you care about them a lot more. People end up sharing these stories, too: <a href="http://www.youtube.com/watch?v=0oMdOPF8a8U">Check out this Jeep theft on YouTube</a>. He spends the first 3 minutes or so telling the story.</p>
<h2>Ignoring video game tropes</h2>
<p>So what really makes DayZ special? There are lots of sandbox games, and there are lots of zombie games. Well, for me, it’s that DayZ ignores much about what makes for conventional games.</p>
<p>See, I’ve been playing video games for most of my life. I started with an NES long ago, and spent many, many, many hours with various games across genres. I play fewer games these days, and actually took a pretty big break from them over the last two years or so. I would go to GameStop, look at what was offered, shrug, and walk out. It’s not that I’m growing up <a href="http://www.theesa.com/facts/pdfs/ESA_EF_2011.pdf">(the average gamer is 37)</a>, it’s that I’ve played all these games before. I spent over a hundred hours with Diablo II, so why bother with Diablo III?</p>
<p>When there’s a game that breaks out of these standard genres and franchises, I get excited. I really got into music games until Activision drove Guitar Hero into the ground, Minecraft was incredibly addicting, and now DayZ with its emergent gameplay, focus on realism, and tense atmosphere.</p>
<h3>Games need to be accessible</h3>
<p>DayZ does not have mass appeal. Its uncompromising approach to realism means that it will alienate many people. You will die often, and it will feel random much of the time. You will get killed by other players. You’ll break a bone miles away from the nearest hospital with no morphine, and have to crawl for an hour or die.</p>
<h3>Games should be polished</h3>
<p>Frankly, <a href="http://www.dayzwiki.com/wiki/index.php?title=Known_Bugs">DayZ is buggy as shit</a>. And not trivial ones, either:</p>
<blockquote>
<p>Dropping Items can cause them to disappear</p>
</blockquote>
<p>Honestly, in many ways, it makes the game even more tense. Bad things can happen to you at any time. The world is scary.</p>
<p>The interface sucks. This is more of an ARMAII issue, but one of my roommates was trying to get something out of another roommate’s backpack, and since you both click to manipulate the UI and click to shoot, accidentally shot him in the stomach. This totally could happen in a zombie apocalypse: fumble around with the safety off… the interface can be confusing, seemingly arbitrary, and complicated. Still totally worth it.</p>
<p>In many ways, this is the <a href="http://www.jwz.org/doc/worse-is-better.html">worse is better</a> principle.</p>
<h3>More is better</h3>
<p>Many games are ‘just like this other game, but with more stuff.’ DayZ has a really pleasant minimalism in many ways: the HUD is basically nonexistent, the core gameplay is pretty simple, and while there are a lot of details, most of them are not too complicated: break a leg, use some morphine. Hungry? Eat some food.</p>
<p>But these simple components come together in a way that gives the game a large amount of depth. But that depth is not from some big crazy feature that took weeks to build.</p>
<p>In fact, the author of the mod has <em>removed</em> entire features. There was a ‘humanity’ system at one time, and ‘bandits,’ which were players that killed other players and had their skin changed. That’s been taken out. I didn’t play the game with that particular feature in, but the author isn’t afraid to cut stuff out if it isn’t a good idea, even if he’s spent lots of time building the feature.</p>
<h3>A good story matters</h3>
<p>Most simulation games have some sort of goal. Even many ‘sandbox’ games, like Grand Theft Auto, have some sort of points, goals, or missions, and you’re free to ignore them, but they’re there. Minecraft is a great example of a true sandbox game, and I’m not suggesting that DayZ has a monopoly here. But it’s a genre that doesn’t have that many games in it, so another entry is very welcome; the genre isn’t as tired as many others.</p>
<p>Story is often used as a reason for progression; as you go through the story, you get bigger, better, and generally more powerful. DayZ manages to have progression without a story; there’s sort of five broad stages to the game: pre-hatchet, hatchet, gun, tent, PvP. This is hard to do! A story provides nice justification. But DayZ doesn’t need it.</p>
<p>Also, as I mentioned earlier, you sort of invent stories as you play. They emerge. And therefore, you care about <em>your</em> DayZ story more than the one that would have been concocted if this game was not a sandbox.</p>
<h3>A player should be quite powerful</h3>
<p>Often, the player is a total badass. Video games are often dripping in machismo, and so they are <strong>THE ULTIMATE DESTROYER OF WORLDS</strong>. Maybe, after killing hundreds of enemy soldiers, getting worn down over time, one gets a lucky shot and you go down.</p>
<p>DayZ is different. You are weak. The environment is dangerous. Even though you can kill zombies, often, the noise attracts others, and they get dangerous in packs. And you can always run out of ammo…</p>
<p>Also, other players are just as desperate and dangerous as you are. Sometimes, you’ll be friendly, but more often, you’re gonna end up getting shot. After all, if you’re not friendly, <em>I</em> could get shot, so maybe I’d better take you out first…</p>
<h2>DayZ is awesome</h2>
<p>Seriously, you should give it a try. ARMAII goes on sale on Steam often, I was able to pick it up for $17 the other day. If you’re bored with the mainstream of gaming, it just might be enough of a shake-up to be interesting to you.</p></content:encoded>
</item>
<item>
<title>Sparrow</title>
<link>https://steveklabnik.com/writing/sparrow/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/sparrow/</guid>
<pubDate>Sat, 21 Jul 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I purchased Sparrow back when it was $9, and didn’t find it particularly amazing. I did like it, but ended up using the web interface more. I certainly got my $9 of value out of it, though, even though I used it for a short time.</p>
<p>However.</p>
<p>There’s been a lot of complaining about <a href="http://sprw.me/">Sparrow’s acquisition</a>, and a lot of complaining about complaining. Most of the complaining about the complaining argues that those who complain should feel like me: You bought it, you probably got a ton of value out of it. You can’t (and shouldn’t) complain.</p>
<p>But.</p>
<p>Theoretically, a market works through feedback mechanisms. That’s the whole ‘free market’ argument, right? People that participate in the market come together, the magic happens, and everything gets allocated correctly. Vote with your dollars. Shop somewhere else, and poof! A store that’s too expensive won’t be able to keep its doors open. A triumph for ‘free commerce.’</p>
<p>So why does everyone get all huffy when certain people provide the market with feedback about the way that business has been conducted?</p></content:encoded>
</item>
<item>
<title>Why I don't like factory_girl</title>
<link>https://steveklabnik.com/writing/why-i-don-t-like-factory_girl/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/why-i-don-t-like-factory_girl/</guid>
<pubDate>Sat, 14 Jul 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Once upon a time, I was building my First Serious Rails App. I was drawn to Rails in the first place because of automated testing and ActiveRecord; I felt the pain of not using an ORM and spending about a week on every deploy making sure that things were still okay in production. So of course, I tried to write a pretty reasonable suite of tests for the app.</p>
<p>To gloss over some details to protect the innocent, this app was a marketplace: some people owned Things, and some people wanted to place Orders. Only certain Things could fulfill an Order, so of course, there was also a ThingType table that handled different types of Things. Of course, some Types came in multiple Sizes, so there also needed to be a Size Table and a ThingTypeSize table so that a User could own a Thing of a certain Type and a certain Size.</p>
<p>Stating that creating my objects for tests was difficult would be an understatement.</p>
<p>Then I read a blog post about FactoryGirl. Holy crap! This would basically save me. With one simple <code>Factory(:thing)</code> I could get it to automatically build a valid list of all that other crap that I needed!</p>
<p>So of course, I had to write my spec for a thing:</p>
<pre><code>describe Order do
it "generates quotes only from Things that are of the right size" do
order = Factory(:order)
thing = Factory(:thing, :size =&gt; order.size)
thing = Factory(:thing)
order.quote!
order.quote.thing.should == thing
end
end
</code></pre>
<p>This test worked. It also generated around 15 objects, saved them in the database, and queried them back out. I don’t have the code running anymore, but it was like 30-40 queries, and took a second or two to run.</p>
<p>That was one test. I was trying to test a lot, even though I wasn’t good at test first yet, so my suite got to be pretty big. Also, sometimes my factories weren’t the best, so I’d spend a day wondering why certain things would start failing. Turns out I’d defined them slightly wrong, validations started to fail, etc.</p>
<h2>How did we get here?</h2>
<p>This story is one of Ruby groupthink gone awry, basically. Of course, we know that fixtures get complicated. They get complicated because we have these crazy ActiveRecord models, don’t use plain Ruby classes when appropriate, and validations make us make extra objects just to get tests to pass. Then fixtures get out of date. So let’s introduce a pattern!</p>
<p>Of course, since we know that Factories are really useful when things get complicated, let’s make sure to use them from the start, so we don’t have to worry about them later. Everyone started doing this. Here’s how new Rails apps get started:</p>
<pre><code>steve at thoth in ~/tmp
$ rails new my_app
create
create README.rdoc
&lt;snip&gt;
create vendor/plugins/.gitkeep
run bundle install
Fetching gem metadata from https://rubygems.org/.........
&lt;snip&gt;
Using uglifier (1.2.6)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
steve at thoth in ~/tmp
$ cd my_app
steve at thoth in ~/tmp/my_app
$ cat &gt;&gt; Gemfile
gem "rspec-rails"
gem "factory_girl_rails"
^D
steve at thoth in ~/tmp/my_app
$ bundle
Fetching gem metadata from https://rubygems.org/........
&lt;snip&gt;
steve at thoth in ~/tmp/my_app
steve at thoth in ~/tmp/my_app
$ rails g rspec:install
create .rspec
exist spec
create spec/spec_helper.rb
$ rails g resource foo
invoke active_record
create db/migrate/20120714180554_create_foos.rb
create app/models/foo.rb
invoke rspec
create spec/models/foo_spec.rb
invoke factory_girl
create spec/factories/foos.rb
invoke controller
create app/controllers/foos_controller.rb
invoke erb
create app/views/foos
invoke rspec
create spec/controllers/foos_controller_spec.rb
invoke helper
create app/helpers/foos_helper.rb
invoke rspec
create spec/helpers/foos_helper_spec.rb
invoke assets
invoke coffee
create app/assets/javascripts/foos.js.coffee
invoke scss
create app/assets/stylesheets/foos.css.scss
invoke resource_route
route resources :foos
steve at thoth in ~/tmp/my_app
$ rake db:migrate
== CreateFoos: migrating =====================================================
-- create_table(:foos)
-&gt; 0.0065s
== CreateFoos: migrated (0.0066s) ============================================
$ cat &gt; spec/models/foo_spec.rb
require 'spec_helper'
describe Foo do
it "does something" do
foo = Factory(:foo)
foo.something!
end
end
^D
$ bundle exec rake spec
/Users/steve/.rvm/rubies/ruby-1.9.3-p194/bin/ruby -S rspec ./spec/controllers/foos_controller_spec.rb ./spec/helpers/foos_helper_spec.rb ./spec/models/foo_spec.rb
*DEPRECATION WARNING: Factory(:name) is deprecated; use FactoryGirl.create(:name) instead. (called from block (2 levels) in &lt;top (required)&gt; at /Users/steve/tmp/my_app/spec/models/foo_spec.rb:5)
F
Pending:
FoosHelper add some examples to (or delete) /Users/steve/tmp/my_app/spec/helpers/foos_helper_spec.rb
# No reason given
# ./spec/helpers/foos_helper_spec.rb:14
Failures:
1) Foo does something
Failure/Error: foo.something!
NoMethodError:
undefined method `something!' for #&lt;Foo:0x007f82c70c07a0&gt;
# ./spec/models/foo_spec.rb:6:in `block (2 levels) in &lt;top (required)&gt;'
Finished in 0.01879 seconds
2 examples, 1 failure, 1 pending
Failed examples:
rspec ./spec/models/foo_spec.rb:4 # Foo does something
Randomized with seed 27300
rake aborted!
/Users/steve/.rvm/rubies/ruby-1.9.3-p194/bin/ruby -S rspec ./spec/controllers/foos_controller_spec.rb ./spec/helpers/foos_helper_spec.rb ./spec/models/foo_spec.rb failed
Tasks: TOP =&gt; spec
(See full trace by running task with --trace)
</code></pre>
<p>Woo! Failing test! Super easy. But what about that test time?</p>
<pre><code>Finished in 0.01879 seconds
</code></pre>
<p>Now, test time isn’t everything, but a hundredth of a second. Once we hit a hundred tests, we’ll be taking almost two full seconds to run our tests.</p>
<p>What if we just <code>Foo.new.something!</code>?</p>
<pre><code>Finished in 0.00862 seconds
</code></pre>
<p>A whole hundredth of a second faster. A hundred tests now take one second rather than two.</p>
<p>Of course, once you add more complicated stuff to your factories, your test time goes up. Add a validation that requires an associated model? Now that test runs twice as slow. You didn’t change the test at all! But it got more expensive.</p>
<p>Now, a few years in, we have these massive, gross, huge, long-running test suites.</p>
<h2>What to do?</h2>
<p>Now, I don’t think that test times are the end-all-be-all of everything. I really enjoyed <a href="http://gmoeck.github.com/2012/07/09/dont-make-your-code-more-testable.html">this post</a> that floated through the web the other day. I think that the ‘fast tests movement’ or whatever (which I am/was a part of) was a branding mistake. The real point wasn’t about fast tests. Fast tests are really nice! But that’s not a strong enough argument alone.</p>
<p>The point is that we forgot what testing is supposed to help in the first place.</p>
<h2>Back to basics: TDD</h2>
<p>A big feature of tests is to give you feedback on your code. Tests and code have a symbiotic relationship. Your tests inform your code. If a test is complicated, your code is complicated. Ultimately, because tests are a client of your code, you can see how easy or hard your code’s interfaces are.</p>
<p>So, we have a pain in our interface: our objects need several other ones to exist properly. How do we fix that pain?</p>
<p>The answer that the Rails community has taken for the last few years is ‘sweep it under the rug with factories!’ And that’s why we’re in the state we’re in. One of the reasons that this happened was that FactoryGirl is a pretty damn good implementation of Factories. I do use FactoryGirl (or sometimes Fabrication) in my request specs; once you’re spinning up the whole stack, factories can be really useful. But they’re not useful for actual unit tests. Which I guess is another way to state what I’m saying: we have abandoned unit tests, and now we’re paying the price.</p>
<p>So that’s what it really boils down to: the convenience of factories has set Rails testing strategies and software design back two years.</p>
<p>You live and learn.</p></content:encoded>
</item>
<item>
<title>How can I contribute to Ruby on Rails?</title>
<link>https://steveklabnik.com/writing/how-can-i-contribute-to-ruby-on-rails-/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/how-can-i-contribute-to-ruby-on-rails-/</guid>
<pubDate>Thu, 05 Jul 2012 00:00:00 GMT</pubDate>
<content:encoded><p>After RailsConf this year, I joined the Rails Issue Team. This means that I help triage issues that people have filed, try to reproduce errors, and point core team members at ones that are most important. Since doing that, a few people have asked me how to get started, so I decided to draw up my thoughts here.</p>
<p>Note that there is also an <a href="http://edgeguides.rubyonrails.org/contributing_to_ruby_on_rails.html">official Rails Guide on contribution</a> too.</p>
<h2>Who helps with Rails?</h2>
<p>First up is the Rails Core team: <a href="http://rubyonrails.org/core/">http://rubyonrails.org/core/</a> Core team members essentially have the authority to merge code into the main Rails codebase. Someone from core reviews every patch that goes into Rails.</p>
<p>Issues team: This group of people has the authority to tag, close, and edit tickets, but can’t actually merge code.</p>
<p>Documentation team: A few awesome people contribute by writing documentation rather than just code. Docs are super important, so all these people are also awesome.</p>
<p>Everyone else!: Rails is open source after all, and 1224 people have contributed at least one patch.</p>
<p>You can find everyone’s commits and count here: <a href="http://contributors.rubyonrails.org/">http://contributors.rubyonrails.org/</a></p>
<h3>About payment</h3>
<p>Nobody is paid to just work on Rails itself. Everyone is a volunteer. Rails runs on a ‘scratch your own itch’ policy. This has two big effects:</p>
<ol>
<li>Pull requests can sit sometimes. Pull requests must be merged by a Core member, so after your patch is submitted, you’re basically waiting on someone from a group of about 12 to decide ‘hey, I should merge some pull requests today’ and get on it. There are currently 168 pull requests open on Rails, the oldest of which is three months since its last comment.</li>
<li>Feature requests may simply fall on deaf ears. If you have a suggestion for Rails, and you don’t want to implement it, you need to convince someone to write the code for you. But it’s not as if the core team has a list of requested features that it goes over when working. Of course, they have a list of the features that they want to add, but it’s not like it’s comprehensive from every feature requested. That said, it’s possible that your idea may spark the attention of someone, so sharing a feature request is valuable. But don’t feel entitled here.</li>
</ol>
<h2>I found a bug, what should I do?</h2>
<p>File a ticket on the <a href="https://github.com/rails/rails/issues">Rails issue tracker</a> and give us as much information as possible.</p>
<p>Note that only 3.2 is getting bug fixes. 3.1 and earlier are security fix only.</p>
<p>Good: Clear bug report, listed versions of Rails and/or Ruby and backtrace if applicable</p>
<p>Better: Steps to reproduce, maybe with a sample app.</p>
<p>Best: All of the above with a patch that fixes the problem.</p>
<h2>I’ve found a security vulnerability, what should I do?</h2>
<p>Don’t submit something to the tracker. Instead, check out <a href="http://rubyonrails.org/security">this page</a> which outlines the security policy. Basically, send an email to <a href="mailto:[email protected]">[email protected]</a>.</p>
<h2>I have a feature request, what should I do?</h2>
<p>Please don’t file an Issue; it’s a bad medium for discussing these kinds of things. It also makes it harder to keep track of what’s a bug and what’s a wish for future changes. Please post to the <a href="https://groups.google.com/forum/?fromgroups#!forum/rubyonrails-core">rails-core mailing list</a> instead. Your idea is much more likely to be seen and discussed there.</p>
<p>You may get no response from the list. People tend to say nothing if they’re not interested. It happens.</p>
<h2>I want feedback on an idea before I work up a patch</h2>
<p>Same as a feature request, please ping the <a href="https://groups.google.com/forum/?fromgroups#!forum/rubyonrails-core">rails-core mailing list</a>.</p>
<h2>I’ve written a patch, now what?</h2>
<p>Submit a pull request with your patch. Someone from core will review it, and may ask you to modify what you’ve done. Eventually it will either get merged into Rails or rejected.</p>
<h2>I want to write docs, now what?</h2>
<p>Don’t sumbit them as issues to the main Rails repo, instead, check out <a href="http://weblog.rubyonrails.org/2012/3/7/what-is-docrails/">docrails</a>.</p>
<h2>I want to help, but don’t know what to work on!</h2>
<p>We try to tag each issue with the part of Rails it affects. If you don’t know where to get started, pick your favorite part of Rails and sort the issues by that part. For example <a href="https://github.com/rails/rails/issues?labels=asset+pipeline&amp;page=1&amp;sort=updated&amp;state=open">here are all the issues relating to the asset pipeline</a>.</p>
<p>Try to reproduce the bug locally, write a test that fails, write a patch, boom!</p>
<h2>How do I set up my local machine to work on Rails</h2>
<p>Check the Rails Guide <a href="http://edgeguides.rubyonrails.org/contributing_to_ruby_on_rails.html#running-the-test-suite">here</a>.</p></content:encoded>
</item>
<item>
<title>Introducing issue2pr</title>
<link>https://steveklabnik.com/writing/introducing-issue2pr/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/introducing-issue2pr/</guid>
<pubDate>Fri, 29 Jun 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Ever had an issue on GitHub, and then you fixed the bug? Opened a brand new pull request, which becomes a new issue, and then you have to remember to link the two via a comment, or manually go back and close the issue as well?</p>
<p>Turns out that GitHub has the power to turn Issues into Pull Requests in the API, but not in the web UI. So I wrote a tiny web app that lets you use the API call yourself: <a href="http://issue2pr.herokuapp.com/">http://issue2pr.herokuapp.com/</a></p>
<p>Of course, the code is up on GitHub, here: <a href="https://github.com/steveklabnik/issue2pr">https://github.com/steveklabnik/issue2pr</a></p>
<p>Issues (and pull requests) welcome!</p></content:encoded>
</item>
<item>
<title>Show solidarity to all who resist</title>
<link>https://steveklabnik.com/writing/show-solidarity-to-all-who-resist/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/show-solidarity-to-all-who-resist/</guid>
<pubDate>Thu, 31 May 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I found this on a flyer in my neighborhood.</p>
<hr />
<p>On may 19th, three people associated with the Occupy movement were arrested in Chicago, just before the anti-NATO protest. Two infiltrators had been working with them for months, trying to persuade them to escalate their tactics – just as in the recent Cleveland bomb scare. Whether the infiltrators would have succeeded in pushing them further than they wanted to go, we’ll never know. Desperate to make an arrest that would intimidate the NATO protesters, divide the Occupy movement from its most radical members, and create an impressive spectacle, the police attacked. Their most substantial evidence is beer-making supplies at the house the NATO 3 were arrested in, which they ridiculously claim are bomb materials.</p>
<p>While we are confident that the NATO 3 will beat these ludicrous charges, they are being held in solitary confinement on 1.5 million dollars bond each. It often comforts prisoners to know that people on the outside love them and are thinking about them, so a solidarity demonstration was called for May 24th, in St Louis, MO. It was noissy, vibrant, nighttime march, enjoyed by the people it passed. “Solidarity with all who resist!” was painted on a business window, and, of course, a businessman got angry, and, of course, there was an altercation; the cops showed up, hit people, were allegedly hit in return, and so on. By the end of the night, ten people were in jail – one charged with five felony counts of assault, property damage and riot, as well as two misdemeanors. Others are charged with felony resisting, interference, and misdemeanor riot. Upon their release, we learned that while being interrogated one prisoner had a knife held to their throat, that they were choked and lifted off the ground and punched in the chest for non-compliance. One comrade, who is facing the most serious charges, was brought to the interrogation room where a detective repeatedly struck him in the face, breaking his nose while he was shackled to the floor. They continued to make threats of violence if he continued to “not comply.”</p>
<p>Fuck.</p>
<p>This.</p>
<p>Shit.</p>
<p>As a wise woman once said, “If you hit me and I hit back, we still ain’t even.” There is no way in society as it exists today to have a fair fight with the police; they have vast, systemic power, and will always be better equipped and better financed than we are. Nor can we let them hurt anyone, particularly those close to us, with complete impunity, without a response from us. We must fight, but we must fight asymmetrically. If we must respond, we are forced into reaction instead of attack, let it be waves of reaction – if strange and inadequate, funny and off putting – that build off each other to create an ocean of solidarity. Solidarity actions in response to the repression of solidarity actions with people we’ve never met accused of crimes that never occurred – this is utterly bizarre, but this is our life.</p>
<p>You should care about this. You should care that the government is making tactical strikes against its enemies. Maybe you don’t think of yourself as an enemy of the government or capitalism; maybe you would never consider it publicly, in a quiet, accepted way or otherwise. Still, think about your life: do you feel safer, happier, more fulfilled because of your job, the police, the taxes you pay? Even if you feel those things on a surface level, do you feel like something is missing, that you could be living in a way that is more glorious and beautiful? Do you see people around you who suffer, or read books about terrible things in America’s past, or hear of people in far-off lands whose lives seem impossibly hard, and wish that you could do something about it – something more meaningful than giving to a charity or voting in the election? We suggest a first step:</p>
<p>Show solidarity to all who resist.</p></content:encoded>
</item>
<item>
<title>Finale</title>
<link>https://steveklabnik.com/writing/finale/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/finale/</guid>
<pubDate>Wed, 30 May 2012 00:00:00 GMT</pubDate>
<content:encoded><p>This morning, my mother woke up and found that my father had passed away in the night.</p>
<p>Thanks for all the love and support you all have shown me over the last two months. It’s been hard, but you all have made it easier.</p>
<p>Don’t be worried if you don’t see me around a lot for the next few days. Just trying to take some time off from the hustle to help my family heal. And to just play video games and be a bit mindless.</p>
<p>No, seriously. Thank you.</p></content:encoded>
</item>
<item>
<title>Shoes 4 work is starting: get involved!</title>
<link>https://steveklabnik.com/writing/shoes-4-work-is-starting--get-involved/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/shoes-4-work-is-starting--get-involved/</guid>
<pubDate>Fri, 25 May 2012 00:00:00 GMT</pubDate>
<content:encoded><p>As you know, I’ve been doing a lot of work over the last few years to keep _why’s <a href="http://shoesrb.com/">Shoes</a> project going. A few other intrepid individuals have been doing a lot as well. We’re starting to work on Shoes 4, and we’d love your help.</p>
<h2>A little about Shoes</h2>
<p>If you didn’t know, Shoes is a GUI toolkit for Ruby. It uses blocks heavily:</p>
<pre><code>Shoes.app do
para "Push the button"
button "Me! Me!" do
alert "Good job."
end
end
</code></pre>
<p>Super simple. You get native widgets on all platforms.</p>
<h2>Shoes 4</h2>
<p>Shoes 4 is a total re-write of Shoes. It’s being done in JRuby. There are a few related projects:</p>
<ul>
<li>ShoesSpec: An executable specification for Shoes implementations, so that others can co-exist. A few other Shoes projects exist already, and a unified interface will be great.</li>
<li>Shoes-mocks: Sort of the opposite, a dummy set of Shoes that you can use to test applications written in Shoes.</li>
<li>The website: A new website is being made as well. Help here would be just as awesome as work on Shoes itself.</li>
</ul>
<h2>What we need</h2>
<p>Of course, pull requests are always welcome. But if you use Shoes or want to, please get involved in <a href="http://github.com/shoes/shoes4/issues">the discussions</a> about what Shoes 4 will look like on the bug tracker. Feel free to open an Issue with any topic for discussion. Thoughts from users, current and potential, are always awesome!</p>
<p>Let’s all have fun developing Shoes 4!</p></content:encoded>
</item>
<item>
<title>Mixins: a refactoring anti-pattern</title>
<link>https://steveklabnik.com/writing/mixins--a-refactoring-anti-pattern/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/mixins--a-refactoring-anti-pattern/</guid>
<pubDate>Mon, 07 May 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I spend an unusually large amount of time thinking about interactions between what I call ‘past me’ and ‘future me.’ It seems that my life changes significantly every few years, and I like to ground myself by imagining how odd it would be if ‘current me’ could tell ‘past me’ things like ‘<a href="http://www.oscon.com/oscon2012/public/schedule/detail/24042">Someday, you’ll be speaking at OSCON</a>.’</p>
<p>It’s not always general life stuff, though, it’s often a trick or technique with code. How many times have you learned something new, and then had that terrible urge to go back and re-write all of that terrible, terrible code you’ve written in the past? This happens to me constantly.</p>
<p>So here, let me show you something I realized that I used to do wrong all the time.</p>
<p>Let’s do it via a quiz. Of course, because I’m framing it this way, you’ll get the answer, but seriously, just play along.</p>
<h2>Two factorizations</h2>
<p>Which code is better factored?</p>
<pre><code># app/models/post.rb
class Post
include Behaviors::PostBehavior
end
# app/models/behaviors/post_behavior.rb
module Behaviors
module PostBehavior
attr_accessor :blog, :title, :body
def initialize(attrs={})
attrs.each do |k,v| send("#{k}=",v) end
end
def publish
blog.add_entry(self)
end
# ... twenty more methods go here
end
end
</code></pre>
<p>or</p>
<pre><code>class Post &lt; ActiveRecord::Base
def publish
blog.add_entry(self)
end
# ... twenty more methods go here
end
</code></pre>
<p>One line of reasoning asserts that the first example is better. Here’s why: all of the behavior is in one place, and the persistence is in another. A clear win.</p>
<p>Another line of reasoning asserts that the second is better. Here’s why: it’s significantly simpler. A clear win.</p>
<p>So which is right?</p>
<h2>They’re both wrong</h2>
<p>Both of these justifications are wrong. As far as I’m concerned, these two classes are equally complex, but because there is a lot more ceremony in version one, version two is preferable. Assertion one is wrong because both are still in the same place, they’re just in two different files. Assertion two is wrong because it’s not simpler than version one, it’s equivalently complex.</p>
<h2>Measures of complexity</h2>
<p>Whenever we refactor, we have to consider what we’re using to evaluate that our refactoring has been successful. For me, the default is complexity. That is, any refactoring I’m doing is trying to reduce complexity. So how do I measure complexity?</p>
<p>One good way that I think about complexity on an individual object level comes from security practices: the ‘attack surface.’ We call this ‘encapsulation’ in object oriented software design. Consider these two objects:</p>
<pre><code>class Foo
def bar
end
def baz
end
end
class Bar
def quxx
end
end
</code></pre>
<p>Bar has one less way into the object than Foo does. Hence the idea of a ‘surface’ that an attacker would have access to. A bigger surface needs more defense, more attention, and is harder to lock down.</p>
<p>Note that I said ‘one less.’ If you said “Foo has a surface of two, and Bar has a surface of one,” you’d be wrong. Luckily, Ruby provides us with a simple way to calculate attack surface. Just use this monkey patch: (I’m not suggesting that you actually use this monkey patch.)</p>
<pre><code>class BasicObject
def attack_surface
methods.count
end
end
</code></pre>
<p>This gives us</p>
<pre><code>1.9.2p318 &gt; Foo.new.attack_surface
=&gt; 59
1.9.2p318 &gt; Bar.new.attack_surface
=&gt; 58
</code></pre>
<p>Oooooh yeah. <code>Object.new.methods.count</code> is 57… we forgot about those. This is why mixins do not really reduce the complexity of your objects. Both versions of Post above have the same number of methods. How many, you ask?</p>
<pre><code>&gt; Post.new.methods.count
=&gt; 337
&gt; Post.new.methods.count - Object.new.methods.count
=&gt; 236
</code></pre>
<p>In this case, my <code>Post</code> is an object that inherits from <code>ActiveRecord::Base</code> and defines nothing else. ActiveRecord adds in 236 methods to our object. That is a pretty huge increase.</p>
<h2>Reducing complexity through encapsulation</h2>
<p>Several people have been dancing around this issue for a while by saying something like ‘Consider ActiveRecord a private interface, and never use it from outside of your class.’ They feel this exact pain, but enforce it through social means rather than reducing the complexity of objects. Now, there are good reasons to do that, so I don’t think it’s categorically wrong, but it certainly is a compromise.</p>
<p>An implementation of DataMapper would be able to solve this problem,</p>
<pre><code>class Post
def initialize(title, body)
@title = title
@body = body
end
end
class PostMapper
def save(post)
@database.insert(:posts, [post.title, post.body])
end
end
PostMapper.new.save(Post.new("Hello DataMapper", "DM rocks"))
</code></pre>
<p>or the repository pattern:</p>
<pre><code>class PostRepository
def first
title, body = @database.select(:posts, "LIMIT 1")
Post.new(title, body)
end
end
</code></pre>
<p>All incredibly theoretical and pretend. But note that the <code>Post</code> in both instances has no methods defined on it that deal with persistence whatsoever. We’ve used encapsulation to hide all of the details of the database in the database, the details of the post logic in the <code>Post</code>, and we use the Mapper or Repository to mediate between the two.</p>
<h2>Using Mixins the right way</h2>
<p>Mixins are awesome, though, and can be used to reduce duplication. They’re for cross-cutting concerns:</p>
<pre><code>class Post
def summary
"#{author}: #{body[0,200]}"
end
end
class Comment
def summary
"#{author}: #{body[0,200]}"
end
end
</code></pre>
<p>moves to</p>
<pre><code>class Post
include Summarizer
end
class Comment
include Summarizer
end
module Summarizer
def summary
"#{author}: #{body[0,200]}"
end
end
</code></pre>
<p>This example isn’t fantastic, but whatever. We’ve eliminated some duplication, even if we didn’t reduce our method surface. This is a win along a different axis, we just shouldn’t fool ourselves that we’ve made our classes ‘simpler.’</p>
<h2>This isn’t the only metric</h2>
<p>Now, I don’t mean to imply that method count is the be-all, end-all metric of complexity. Real software development is too complicated to wrap up in one tiny number. However, I do think there’s a strong correlation between small, focused classes and easy to understand software systems. Aggregates can be used to make it easier to understand the use of a cluster of smaller objects. But that’s another post.</p></content:encoded>
</item>
<item>
<title>On the eve of #m1gs</title>
<link>https://steveklabnik.com/writing/on-the-eve-of--m1gs/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/on-the-eve-of--m1gs/</guid>
<pubDate>Mon, 30 Apr 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Walking down the street, a woman cries for help. She’s blind, and the curb is large, and she’s unsure of herself. Knowing it’s not safe, she seeks the help of a stranger to guide her on her way. Three or four people pass by, but then someone stops, gives her their arm, and guides her safely to her destination.</p>
<p><strong>HOOOONK!!!</strong> A large truck is blocked from going down the street. A Lexus with out-of-state-plates is in the way, and the driver is standing outside of the car, looking agitated. A passer-by stops to ask if they can help, and the man frantically tries to explain. His English is poor, but not that poor, but due to the stress of the situation, he’s losing his words. Luckily, the stranger speaks a little of his native tongue. The astute observer would notice that this tiny bit of familiarity causes a massive change in demeanor in this chaotic moment. 311 is called, a translator is found, and the man will be able to move his car soon.</p>
<p>These moments, these tiny acts of selflessness and mutual aid, are what revolutions are truly made of.</p></content:encoded>
</item>
<item>
<title>Cards for dad - follow-up</title>
<link>https://steveklabnik.com/writing/cards-for-dad--follow-up/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/cards-for-dad--follow-up/</guid>
<pubDate>Sun, 22 Apr 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Hey everyone! It’s been a few weeks, and I wanted to update you guys on the card situation.</p>
<p>First of all, thank you so much. Every single one of you who sent cards, thank you! It was truly amazing to see so much support. I was even more surprised that I basically put my address on the internet and nothing particularly bad happened. Zero trolls. Restored some of my faith in humanity.</p>
<p>We ended up with about 500 cards. My dad is well enough to stay at home, but the chemo is pretty rough, so he has good days and bad days. I go home to visit about once a week, but we don’t really know how well and if it’s really working yet, so no actual understanding of how long he has. Regardless, it all made a big impact, so thank you again.</p>
<p>Here’s some pictures:</p>
<p><img src="http://i.imgur.com/mcQSG.jpg" alt="http://i.imgur.com/mcQSG.jpg" /></p>
<p><img src="http://i.imgur.com/GmmhJ.jpg" alt="http://i.imgur.com/GmmhJ.jpg" /></p>
<p><img src="http://i.imgur.com/RV4k8.jpg" alt="http://i.imgur.com/RV4k8.jpg" /></p>
<p><img src="http://i.imgur.com/QVTJK.jpg" alt="http://i.imgur.com/QVTJK.jpg" /></p>
<p><img src="http://i.imgur.com/YxXqj.jpg" alt="http://i.imgur.com/YxXqj.jpg" /></p>
<p>Oh, and I didn’t read very many of them personally, since they were for my dad, not for me. My parents read and cherished all of them, though. They said that you all said too many nice things about me, so thank you for all of that as well.</p></content:encoded>
</item>
<item>
<title>Building StoreEngine</title>
<link>https://steveklabnik.com/writing/building-storeengine/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/building-storeengine/</guid>
<pubDate>Thu, 05 Apr 2012 00:00:00 GMT</pubDate>
<content:encoded><p>So I said I was gonna build antisocialne.ws. But I’m changing gears instead: I’d rather do this project, which has a better spec and needs a reference implementation.</p>
<p><a href="http://vimeo.com/39850969">Building StoreEngine</a> from <a href="http://vimeo.com/steveklabnik">Steve Klabnik</a> on <a href="http://vimeo.com/">Vimeo</a>.</p></content:encoded>
</item>
<item>
<title>About antisocialne.ws</title>
<link>https://steveklabnik.com/writing/about-antisocialne-ws/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/about-antisocialne-ws/</guid>
<pubDate>Tue, 03 Apr 2012 00:00:00 GMT</pubDate>
<content:encoded><p><a href="http://vimeo.com/39718634">About Antisocialne.ws</a> from <a href="http://vimeo.com/steveklabnik">Steve Klabnik</a> on <a href="http://vimeo.com/">Vimeo</a>.</p></content:encoded>
</item>
<item>
<title>Anti-capitalism 101</title>
<link>https://steveklabnik.com/writing/anti-capitalism-101/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/anti-capitalism-101/</guid>
<pubDate>Sat, 31 Mar 2012 00:00:00 GMT</pubDate>
<content:encoded><p>People are often surprised to find out that I identify as an anti-capitalist. At least in America, many people see capitalism as inevitable, or even as the way things always were. Unfortunately, there’s so much rhetoric and confusion around this topic that it always takes a long time to break it down. I’d been meaning to write something about this topic, but then I found this really great work called <a href="http://libcom.org/files/Capitalism_ex-dummies_web.pdf">Capitalism, Class, and Class Struggle for (ex) Dummies</a>. I’ve adapted it with my own thoughts and edits below. It in turn is adapted from <a href="http://libcom.org/library/capitalism-introduction">this essay</a> and <a href="http://libcom.org/library/class-class-struggle-introduction">this essay</a> from LibCom.org, so if you’d like to see the history, you can read those versions as well.</p>
<p>I plan on refining this as things go along, as it’s still a bit jargony in places. Any suggestions are very welcome!</p>
<hr />
<h2>Capitalism: an introduction</h2>
<p>At its root, capitalism is an economic system based on three things: wage labour (working for a wage), private ownership of the means of production (things like factories, machinery, farms, and offices), and production for exchange and profit.</p>
<p>While some people own means of production, or capital, most of us don’t and so to survive we need to sell our ability to work in return for a wage, or else scrape by on benefits. This first group of people is the capitalist class or “bourgeoisie” in Marxist jargon, and the second group is the working class or “proletariat”. This jargon is only included so that you can make connections to other literature, we’ll stick away from such things later in this text.</p>
<p>Capitalism is based on a simple process—money is invested to generate more money. When money functions like this, it functions as capital. For instance, when a company uses its profits to hire more staff or open new premises, and so make more profit, the money here is functioning as capital. As capital increases (or the economy expands), this is called ‘capital accumulation’, and it’s the driving force of the economy.</p>
<p>Those accumulating capital do so better when they can shift costs onto others. If companies can cut costs by not protecting the environment, or by paying sweatshop wages, they will. So catastrophic climate change and widespread poverty are signs of the normal functioning of the system. Furthermore, for money to make more money, more and more things have to be exchangeable for money. Thus the tendency is for everything from everyday items to DNA sequences to carbon dioxide emissions—and, crucially, our ability to work—to become commodified.</p>
<p>And it is this last point—the commodification of our creative and productive capacities, our ability to work—which holds the secret to capital accumulation. Money does not turn into more money by magic, but by the work we do every day.</p>
<p>In a world where everything is for sale, we all need something to sell in order to buy the things we need. Those of us with nothing to sell except our ability to work have to sell this ability to those who own the factories, offices, etc. And of course, the things we produce at work aren’t ours, they belong to our bosses.</p>
<p>Furthermore, because of long hours, productivity improvements etc, we produce much more than necessary to keep us going as workers. The wages we get roughly match the cost of the products necessary to keep us alive and able to work each day (which is why, at the end of each month, our bank balance rarely looks that different to the month before). The difference between the wages we are paid and the value we create is how capital is accumulated, or profit is made.</p>
<p>This difference between the wages we are paid and the value we create is called “surplus value”. The extraction of surplus value by employers is the reason we view capitalism as a system based on exploitation—the exploitation of the working class. This process is essentially the same for all wage labour, not just that in private companies. Public sector workers also face constant attacks on their wages and conditions in order to reduce costs and maximise profits across the economy as a whole.</p>
<h3>Unwaged labour</h3>
<p>The accumulation of capital also relies on unwaged work, such as housework or domestic labour. This includes the reproduction of labour power in the form of producing and raising children—the next generation of workers—and servicing the current workforce—physically, emotionally, and sexually. This unpaid labour is predominantely carried out by women.</p>
<p>Servicing men and children at home serves capital: by making housework and reproduction a women’s ‘natural and feminine’ process rather than work, capitalism benefits in the form of free labour. When capital pays husbands they get two workers, not one. Denying domestic labour a wage makes this work invisible, and divides the working class into waged and unwaged at the expense of both.</p>
<h3>Competition</h3>
<p>In order to accumulate capital, our boss must compete in the market with bosses ofother companies. They cannot afford to ignore market forces, or they will lose ground to their rivals, lose money, go bust, get taken over, and ultimately cease to be our boss. Therefore even the bosses aren’t really in control of capitalism, capital itself is. It’s because of this that we can talk about capital as if it has agency or interests of its own, and so often talking about ‘capital’ is more precise than talking about bosses.</p>
<p>Both bosses and workers, therefore, are alienated by this process, but in different ways. While from the workers’ perspective, our alienation is experienced through being controlled by our boss, the boss experiences it through impersonal market forces and competition with other bosses.</p>
<p>Because of this, bosses and politicians are powerless in the face of ‘market forces,’ each needing to act in a way conducive to continued accumulation (and in any case they do quite well out of it!). They cannot act in our interests, since any concessions they grant us will help their competitors on a national or international level.</p>
<p>So, for example, if a manufacturer develops new technology for making cars which doubles productivity it can lay off half its workers, increase its profits and reduce the price of its cars in order to undercut its competition. If another company wants to be nice to its employees and not sack people, eventually it will be driven out of business or taken over by its more ruthless competitor—so it will also have to bring in the new machinery and make the layoffs to stay competitive.</p>
<p>Of course, if businesses were given a completely free hand to do as they please, monopolies would soon develop and stifle competition which would lead to the system grinding to a halt. The state intervenes, therefore to act on behalf of the long-term interests of capital as a whole. We observed this happen in America back in the Robber Baron days.</p>
<h3>The State</h3>
<p>The primary function of the state in capitalist society is to maintain the capitalist system and aid the accumulation of capital. As such, the state uses repressive laws and violence against the working class when we try to further our interests against capital. For example, bringing in anti-strike laws, or sending in police or soldiers to break up strikes and demonstrations.</p>
<p>The “ideal” type of state under capitalism at the present time is liberal democratic, however in order to continue capital accumulation at times different political systems are used by capital to do this. State capitalism in the USSR, and fascism in Italy and Germany are two such models, which were necessary for the authorities at the time in order to co-opt and crush powerful working-class movements which threatened the very continuation of capitalism.</p>
<p>When the excesses of bosses cause workers to fight back, alongside repression the state occasionally intervenes to make sure business as usual resumes without disruption. For this reason national and international laws protecting workers’ rights and the environment exist. Generally the strength and enforcement of these laws ebbs and flows in relation to the balance of power between employers and employees in any given time and place. For example, in France where workers are more well-organised and militant, there is a maximum working week of 35 hours. In the UK, where workers are less militant the maximum is 48 hours, and in the US where workers are even less likely to strike there is no maximum at all.</p>
<h3>History</h3>
<p>Capitalism is presented as a ‘natural’ system, formed a bit like mountains or land masses by forces beyond human control, that it is an economic system ultimately resulting from human nature. However it was not established by ‘natural forces’ but by intense and massive violence across the globe. First in the ‘advanced’ countries, enclosures drove self-sufficient peasants from communal land into the cities to work in factories. Any resistance was crushed. People who resisted the imposition of wage labour were subjected to vagabond laws and imprisonment, torture, deportation or execution. In England under the reign of Henry VIII alone 72,000 people were executed for vagabondage.</p>
<p>Later capitalism was spread by invasion and conquest by Western imperialist powers around the globe. Whole civilisations were brutally destroyed with communities driven from their land into waged work. The only countries that avoided conquest were those—like Japan—which adopted capitalism on their own in order to compete with the other imperial powers. Everywhere capitalism developed, peasants and early workers resisted, but were eventually overcome by mass terror and violence.</p>
<p>Capitalism did not arise by a set of natural laws which stem from human nature: it was spread by the organised violence of the elite. The concept of private property of land and means of production might seem now like the natural state of things, however we should remember it is a man-made concept enforced by conquest. Similarly, the existence of a class of people with nothing to sell but their labour power is not something which has always been the case—common land shared by all was seized by force, and the dispossessed forced to work for a wage under the threat of starvation or even execution. As capital expanded, it created a global working class consisting of the majority of the world’s population whom it exploits but also depends on.</p>
<h3>The Future</h3>
<p>Capitalism has only existed as the dominant economic system on the planet for a little over 200 years. Compared to the half a million years of human existence it is a momentary blip, and therefore it would be naive to assume that it will last for ever. It is entirely reliant on us, the working class, and our labour which it must exploit, and so it will only survive as long as we let it.</p>
<h2>Class &amp; Class Struggle: an introduction</h2>
<p>The first thing to say is that there are various ways of referring to class. Often, when people talk about class, they talk in terms of cultural/sociological labels. For example, middleclass people like foreign films, working class people like football, upper-class people like top hats and so on.</p>
<p>Another way to talk about class, however, is based on classes’ economic positions. We talk about class like this because we see it as essential for understanding how capitalist society works, and consequently how we can change it.</p>
<p>It is important to stress that our definition of class is not for classifying individuals or putting them in boxes, but in order to understand the forces which shape our world, why our bosses and politicians act the way they do, and how we can act to improve our conditions.</p>
<h3>Class and Capitalism</h3>
<p>The economic system which dominates the world at present is called capitalism. As mentioned earlier, capitalism is essentially a system based on the self-expansion of capital—commodities and money making more commodities and more money.</p>
<p>This doesn’t happen by magic, but by human labour. For the work we do, we’re paid for only a fraction of what we produce. The difference between the value we produce and the amount we’re paid in wages is the “surplus value” we’ve produced. This is kept by our boss as profit and either reinvested to make more money or used to buy swimming pools or fur coats or whatever.</p>
<p>In order for this to take place, a class of people must be created who don’t own anything they can use to make money i.e. offices, factories, farmland or other means of production. This class must then sell their ability to work in order to purchase essential goods and services in order to survive. This class is the working class.</p>
<p>So at one end of the spectrum is this class, with nothing to sell but their ability to work. At the other, those who do own capital to hire workers to expand their capital. Individuals in society will fall at some point between these two poles, but what is important from a political point of view is not the positions of individuals but the social relationship between classes.</p>
<h3>The Working Class</h3>
<p>The working class then, or ‘proletariat’ as it is sometimes called, the class who is forced to work for wages, or claim benefits if we cannot find work or are too sick or elderly to work, to survive. We sell our time and energy to a boss for their benefit.</p>
<p>Our work is the basis of this society. And it is the fact that this society relies on the work we do, while at the same time always squeezing us to maximise profit, that makes it vulnerable.</p>
<h3>Class Struggle</h3>
<p>When we are at work, our time and activity is not our own. We have to obey the alarm clock, the time card, the managers, the deadlines and the targets.</p>
<p>Work takes up the majority of our lives. We may see our managers more than we see our friends and partners. Even if we enjoy parts of our job we experience it as something alien to us, over which we have very little control. This is true whether we’re talking about the nuts and bolts of the actual work itself or the amount of hours, breaks, time off etc.</p>
<p>Work being forced on us like this compels us to resist.</p>
<p>Employers and bosses want to get the maximum amount of work from us, from the longest hours, for the least pay. We, on the other hand, want to be able to enjoy our lives: we don’t want to be over-worked, and we want shorter hours and more pay.</p>
<p>This antagonism is central to capitalism. Between these two sides is a push and pull: employers cut pay, increase hours, speed up the pace of work. But we attempt to resist: either covertly and individually by taking it easy, grabbing moments to take a break and chat to colleagues, calling in sick, leaving early. Or we can resist overtly and collectively with strikes, slow-downs, occupations etc.</p>
<p>This is class struggle. The conflict between those of us who have to work for a wage and our employers and governments, who are the capitalist class, or ‘bourgeoisie’. By resisting the imposition of work, we say that our lives are more important than our boss’s profits. This attacks the very nature of capitalism, where profit is the most important reason for doing anything, and points to the possibility of a world without classes and privately owned means of production. We are the working class resisting our own existence. We are the working class struggling against work and class.</p>
<h3>Beyond the Workplace</h3>
<p>Class struggle does not only take place in the workplace. Class conflict reveals itself in many aspects of life. For example, affordable housing is something that concerns all working class people. However, affordable for us means unprofitable for them. In a capitalist economy, it often makes more sense to build luxury apartment blocks, even while tens of thousands are homeless, than to build housing which we can afford to live in. So struggles to defend social housing, or occupying empty properties to live in are part of the class struggle.</p>
<p>Similarly, healthcare provision can be a site of class conflict. Governments or companies attempt to reduce spending on healthcare by cutting budgets and introducing charges for services to shift the burden of costs onto the working class, whereas we want the best healthcare possible for as little cost as possible.</p>
<h3>The ‘Middle Class’</h3>
<p>While the economic interests of capitalists are directly opposed to those of workers, a minority of the working class will be better off than others, or have some level of power over others. When talking about history and social change it can be useful to refer to this part of the proletariat as a “middle class”, despite the fact that it is not a distinct economic class, in order to understand the behaviour of different groups.</p>
<p>Class struggle can sometimes be derailed by allowing the creation or expansion of the middle class—Margaret Thatcher encouraged home ownership by cheaply selling off social housing in the UK during the big struggles of the 1980s, knowing that workers are less likely to strike if they have a mortgage, and allowing some workers to become better off on individual levels, rather than as a collective. And in South Africa the creation of a black middle class helped derail workers’ struggles when apartheid was overturned, by allowing limited social mobility and giving some black workers a stake in the system.</p>
<p>Bosses try to find all sorts of ways to materially and psychologically divide the working class, including by salary differentials, professional status, race and by gender. It should be pointed out again that we use these class definitions in order to understand social forces at work, and not to label individuals or determine how individuals will act in given situations.</p>
<h3>Conclusion</h3>
<p>Talking about class in a political sense is not about which accent you have but the basic conflict which defines capitalism—those of us who must work for a living vs. those who profit from the work that we do. By fighting for our own interests and needs against the dictates of capital and the market we lay the basis for a new type of society—a society without money, class, or capitalism, organised for the direct fulfilment of our needs: a Libertarian Communist society.</p></content:encoded>
</item>
<item>
<title>Birthday cards for Dad</title>
<link>https://steveklabnik.com/writing/birthday-cards-for-dad/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/birthday-cards-for-dad/</guid>
<pubDate>Mon, 26 Mar 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Hey everyone. If you follow me on Twitter, you’ll know that I’ve been a bit absent lately. Basically, my dad has <a href="http://en.wikipedia.org/wiki/Angiosarcoma">angiosarcoma</a>. It’s a really rare cancer of the blood vessels. A city the size of Pittsburgh sees two cases every ten years, roughly. It’s certainly terminal, while he had some chemotherapy a few days ago, we won’t know if it’s helped at all for another week or so, and if it works, it’ll give him another few weeks to months at best. It came on really fast: he felt some shortness of breath the first week of February, went into the hospital in week three, by week four he almost died from a blood vessel bursting due to weakness from the disease.</p>
<p>Anyway, you all have sent me tons of well-wishes and such over the last week, and it’s truly appreciated. It’s nice to know that I have support, as I try to support my family. I’m the oldest son, so that carries a lot of responsibilities, and you guys have helped me a lot.</p>
<p>However, if I can ask you for one more thing…</p>
<p>My dad turns 54 on Sunday. My aunt told my mom that for his birthday, she’s going to email everyone she knows, ask them to send him a birthday card, and she’s hoping to get 100. I think that’s a pretty awesome idea, but I also happen to know that while I’ve mentioned to my dad that ‘people on the internet I know send well-wishes,’ I think it’d be super awesome to blow my aunt out of the water and give my dad a zillion cards for his last birthday.</p>
<p>So yeah, if you have a few minutes this week, and you’re thinking of my father, send a birthday card to my house. Here’s the address:</p>
<p><em>note</em>: Five years later, I’ve removed the address. I haven’t lived there in years, please don’t see this now and send mail to some random person kthx.</p>
<p>Oh yeah: My dad is also Steve. I’m not a junior because he has no middle name: his mother said they were too poor to give him one.</p>
<p>Additionally, I mean April 1. That’s his birthday. However, if you live far away and it’ll be a bit late, that’s perfectly fine, still send one. I think that ‘oh, this card came from so far away that it took some time’ is pretty awesome in its own way.</p>
<p>I really think this would be an awesome last gift to my dad. One of the worst things about being in the hospital is that you feel cut off from everything outside of that room, and while I’ve spent almost every minute of the last 8 days in there with him, keeping your spirits up helps the healing process quite a bit.</p>
<p>Thanks. &lt;3 &lt;3 &lt;3</p></content:encoded>
</item>
<item>
<title>Transmuting Philosophy into Machinery</title>
<link>https://steveklabnik.com/writing/transmuting-philosophy-into-machinery/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/transmuting-philosophy-into-machinery/</guid>
<pubDate>Thu, 08 Mar 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I’m so very close to releasing the beta of <a href="http://designinghypermediaapis.com/">Get Some REST</a>. However, I’m about to get on a plane to Poland, and that’s a bad time to launch things. ;) <a href="http://wrocloverb.com/">wroc_love.rb</a>, here I come!</p>
<p>Anyway, I figured I’d give you a preview of some of the content that I’ve been working on. This particular article is short, but it’s also a public response to something, and so, to whet your appetite, I figured I’d give you a little taste of what’s to come. It’s out of context, so I’m a little light on explaining why you’d do each step of the process; there is much more on that elsewhere in the book! Rather than explaing all the <em>why</em>, I just actually do it. This is one of the last articles, and it’s a fairly simple example once you’ve grokked all the fundamentals, so it’s a bit small, lengthwise.</p>
<p>If all goes well, I’ll be releasing at wroc_love.rb. So keep your eyes peeled.</p>
<p>Without further ado, the article.</p>
<hr />
<blockquote>
<p>Kessler’s idea was, that besides the law of mutual struggle there is in nature the law of mutual aid, which, for the success of the struggle for life, and especially for the progressive evolution of the species, is far more important than the law of mutual contest. This suggestion - which was, in reality, nothing but a further development of the ideas expressed by Darwin himself in The Descent of Man, seemed to me so correct and of so great an importance, that since I became aquainted with it I began to collect materials for further developing the idea, which Kessler had only cursorily sketched in his lecture, but had not lived to develop. He died in 1881.Peter Kropotkin, “Mutual Aid: A Factor of Evolution”, p21</p>
</blockquote>
<p>Rob Conery is a pretty cool guy. While I always enjoy reading <a href="http://blog.wekeroad.com/">his blog</a>, he’s been at battle with the Hypermedia crowd recently. It’s always good natured, though, and he means well.</p>
<p>He recently posed <a href="http://wekeroad.com/2012/03/03/moving-the-philosophy-into-machinery/">an interesting question to his blog</a>:</p>
<blockquote>
<p>I would like to invite the good people who have engaged with me over the last few days to jump in and write me up an API – and by way of explanation – show how their ideas can be translated into reality.</p>
</blockquote>
<p>Great! After all, social exchange is one of the building blocks of society:</p>
<blockquote>
<p>In so far as the process of exchange transfers commodities from hands in which they are non-use-values to hands in which they are use-values, it is a process of social metabolism.Karl Marx, “Capital, Volume 1”, p198</p>
</blockquote>
<p>Let’s apply what we’ve learned about the basics of designing hypermedia APIs. Here are his requirements:</p>
<h2>Use Cases</h2>
<p>This is step one: simple authentication and then consumption of basic data. The client will be HTML, JS, and Mobile.</p>
<h3>Logging In</h3>
<p>Customer comes to the app and logs in with email and password. A token is returned by the server upon successful authentication and a message is also received (like “thanks for logging in”).</p>
<h3>Productions</h3>
<p>Joe User is logged in and wants to see what he can watch. He chooses to browse all productions and can see in[sic] the app which ones he is allowed to watch and which ones he isn’t. He then chooses to narrow his selection by category: Microsoft, Ruby, Javascript, Mobile. Once a production is selected, a list of Episodes is displayed with summary information. Joe wants to view Episode 2 of Real World ASP.NET MVC3 – so he selects it. The video starts.</p>
<h3>Episodes.</h3>
<p>Kelly User watches our stuff on her way to work every day, and when she gets on the train [sic] will check and see if we’ve pushed any new episodes recently. A list of 5 episodes comes up – she chooses one, and watches it on her commute.</p>
<h2>The design process</h2>
<h3>Step 1: Evaluate Process</h3>
<p>Fortunately, this has been done for us, in the Use Cases above. Sweet!</p>
<h3>Step 2: Create state machine</h3>
<p>Taking all of this into account, I drew out this state machine:</p>
<p><img src="/img/2012-03-08/state-machine.png" alt="state machine" /></p>
<p>Basically, you start at a root. Two options: see the newest list of productions, or see them all. You can filter all of them by a category. Eventually, you end up picking one. This workflow should be enough to support all of our use cases.</p>
<h3>Step 3: Evaluate Media Type</h3>
<p>Okay, so, he mentions this in use cases:</p>
<blockquote>
<p>The client will be HTML, JS, and Mobile.</p>
</blockquote>
<p>I’m not 100% sure what he means here: I think it’s that we’ll be building a site on this API (html), it’ll have heavy JS usage (js), and probably a mobile version or possibly a native client (mobile).</p>
<p>Given this information, I’m going to choose JSON as a base format. Besides, our developers tend to like it. ;)</p>
<p>After that choice is made, we also need these things:</p>
<ul>
<li>Filtering things means a templated query of some kind, so we’ll need some kind of templating syntax.</li>
<li>We need lists of things as well as singular things. I like to simply this by representing singular things as a list of one item. So, lists and individual items.</li>
<li>We also have a few attributes we need to infer from these loose requirements. No biggie. :)</li>
</ul>
<h3>Step 4: Create Media Types</h3>
<p>Based on this, I’ve made up <a href="/tekpub-productions.html">the application/vnd.tekpub.productions+json media type</a>. Key features, based on our evaluation:</p>
<ul>
<li>Each transition in our state machine has a relation attribute</li>
<li>Each transition that needs to be parameterized has some sort of template syntax</li>
<li>Each attribute that we need someone to know about has a definition</li>
<li>Everything is always a list. It may contain just one item. Our client’s interface can detect this special case and display something different if it wants.</li>
</ul>
<h3>Step 5: Implementation!</h3>
<p>That’s for Rob! ahaha!</p>
<p>However, you might want a sample implementation. I’m willing to make one if there’s confusion about it, but I figured I’d put the article out and see if that’s interesting to people before I went through the effort.</p>
<h2>What about auth?</h2>
<p>Oh, I didn’t handle the auth case. That’s because auth happens at the HTTP level, not at the application level. HTTP BASIC + SSL or Digest should be just fine.</p>
<h2>But, but, but… I didn’t get any verbs! Or URLS!</h2>
<p>I know. Fielding:</p>
<blockquote>
<p>A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state, or in defining extended relation names and/or hypertext-enabled mark-up for existing standard media types. Any effort spent describing what methods to use on what URIs of interest should be entirely defined within the scope of the processing rules for a media type (and, in most cases, already defined by existing media types). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]</p>
</blockquote>
<p>As well as</p>
<blockquote>
<p>A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations. [Failure here implies that clients are assuming a resource structure due to out-of band information, such as a domain-specific standard, which is the data-oriented equivalent to RPC’s functional coupling].</p>
</blockquote>
<p>And</p>
<blockquote>
<p>A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]</p>
</blockquote>
<p>Soooooooooo yeah.</p>
<h2>Further Exploration</h2>
<h3>Sources Cited</h3>
<ul>
<li><a href="http://www.amazon.com/Mutual-Aid-Evolution-Peter-Kropotkin/dp/0875580246">“Mutual Aid: A Factor of Evolution”, Kropotkin</a></li>
<li><a href="http://www.amazon.com/Capital-Critique-Political-Economy-Classics/dp/0140445684/ref=sr_1_1?s=books&amp;ie=UTF8&amp;qid=1331214700&amp;sr=1-1">“Capital, Volume I”, Karl Marx</a></li>
<li><a href="http://wekeroad.com/2012/02/28/someone-save-us-from-rest/">“Someone save us from REST”</a></li>
<li><a href="http://wekeroad.com/2012/03/03/moving-the-philosophy-into-machinery/">“Moving the Philosophy into Machinery”</a></li>
<li><a href="http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven">“REST APIs Must be Hypertext Driven”</a></li>
<li><a href="/nodes/the-design-process-an-overview">The Design Process: An Overview</a></li>
</ul>
<h3>Next Articles</h3>
<p>For more on why media types are awesome, check <a>Programming the Media Type</a>.</p>
<h3>Terms Used</h3>
<p>None really at the moment! This article stands alone.</p></content:encoded>
</item>
<item>
<title>Hypermedia API reading list</title>
<link>https://steveklabnik.com/writing/hypermedia-api-reading-list/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/hypermedia-api-reading-list/</guid>
<pubDate>Mon, 27 Feb 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Originally, this post was titled “A RESTful Reading List,” but please note that <a href="/posts/2012-02-23-rest-is-over">REST is over. Hypermedia API is the new nomenclature.</a></p>
<p>I’ve been doing an intense amount of research on Hypermedia APIs over the last few months, and while I didn’t save every resource I found, I’ve made a list here of the most important.</p>
<p>I’ll be updating this post as I get new resources, so check back!</p>
<h2>The book list</h2>
<p>If you want to go from ‘nothing to everything,’ you can do it by reading just a few books, actually. I’m going to make all of these links affiliate. I purchase a <em>lot</em> of books myself, maybe suggesting things to you can help defray the cost of my massive backlog. All are easily searchable on Amazon if you’re not comfortable with that.</p>
<p>Start off with <a href="http://www.amazon.com/gp/product/0596529260/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0596529260">Restful Web Services</a> by Leonard Richardson and Sam Ruby. This book is fantastic from getting you from zero knowledge to “I know how Rails does REST by default,” which is how most people do REST. But as you know, that’s flawed. However, understanding this stuff is crucial, not only for when you interact with REST services, but also for understanding how Hypermedia APIs work differently. This baseline of knowledge is really important.It also comes in <a href="http://www.amazon.com/gp/product/0596801688/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0596801688">cookbook form</a>. You really only need one or the other; pick whichever format you like.</p>
<p>Next up, read <a href="http://www.amazon.com/gp/product/0596805829/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0596805829">REST in Practice: Hypermedia and Systems Architecture</a>. This book serves as a great <em>bridge</em> to understanding Hypermedia APIs from the RESTful world. Chapters one through four read like Richardson &amp; Ruby; yet they start slipping in the better hypermedia terminology. Chapter five really starts to dig into how Hypermedia APIs work, and is a great introduction. Chapter six covers scaling, chapter seven is an introduction to using ATOM for more than an RSS replacement, nine is about security, and eleven is a great comparison of how Hypermedia and WS-* APIs differ. All in all, a great intermediate book.</p>
<p>To really start to truly think in Hypermedia, though, you <em>must</em> read <a href="http://www.amazon.com/gp/product/1449306578/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=1449306578">Building Hypermedia APIs with HTML5 and Node</a>. Don’t let the title fool you, as Mike says in the introduction:</p>
<blockquote>
<p>[HTML5, Node.js, and CouchDB] are used as tools illustrating points about hypermedia design and implementation.</p>
</blockquote>
<p>This is not a Node.js book. I find Node slightly distasteful, but all the examples were easy to follow, even without more than a cursory working knowledge.</p>
<p>Anyway, the book: Mike says something else that’s really important in the intro:</p>
<blockquote>
<p>While the subject of the REST architectural style comes up occasionally, this book does not explore the topic at all. It is true that REST identifies hypermedia as an important aspect of the style, but this is not the case for the inverse. Increasing attention to hypermedia designs can improve the quality and functionality of many styles of distributed network architecture including REST.</p>
</blockquote>
<p>And, in the afterward:</p>
<blockquote>
<p>However, the aim of this book was not to create a definitive work on designing hypermedia APIs. Instead, it was to identify helpful concepts, suggest useful methodologies, and provide pertinent examples that encourage architects, designers, and developers to see the value and utility of hypermedia in their own implementations.</p>
</blockquote>
<p>I think these two statements, taken together, describe the book perfectly. The title is “Building Hypermedia APIs,” not “Designing.” So why recommend it on an API design list? Because understanding media types, and specifically hypermedia-enabled media types, is the key to understanding Hypermedia APIs. Hence the name.</p>
<p>Mike is a great guy who’s personally helped me learn much of the things that I know about REST, and I’m very thankful to him for that. I can’t recommend this book highly enough.</p>
<p>However, that may leave you wondering: Where’s the definitive work on how to actually build and design a Hypermedia API? Did I mention, totally unrelated of course, that <a href="http://designinghypermediaapis.com/">I’m writing a book</a>? ;)</p>
<p>Yes, it still has REST in the title. Think about that for a while, I’m sure you can see towards my plans. I’m planning on a beta release as soon as I’m recovered from some surgery this week, but I’m not sure how long that will take, exactly. So keep your eyes peeled.</p>
<h3>Books I don’t recommend</h3>
<p><a href="http://www.amazon.com/gp/product/1449310508/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=1449310508">REST API Design Rulebook</a>, while I haven’t actually read it, seems quite terrible. Let me copy an <a href="http://www.amazon.com/review/R2F4STDF7XS7U3/ref=cm_cr_dp_perm?ie=UTF8&amp;ASIN=1449310508&amp;nodeID=283155&amp;tag=&amp;linkCode=">Amazon review</a>:</p>
<blockquote>
<p>The first chapters give a good feel for the vocabulary, and some good techniques for implementing REST. A lot of the ‘rules’, especially those related to basic CRUD operations, are clean and simple with useful examples.Unfortunately, the later chapters get more and more focused on specifying something called ‘WRML’, which is a concept/language newly introduced in this book as far as I can tell.Personally I would recommend ignoring the sections dealing with WRML (or keep them in mind as a detailed example of one possible way of handling some of the REST issues).As to WRML itself: yuck. It appears to be an attempt to drag in some of the unnecessary complexity of SOAP with little added benefit. Not recommended.</p>
</blockquote>
<p>Looking up information about WRML, I can agree, 100%. Ugh. So nasty. So this gets a big fat downvote from me.</p>
<h3>Books I want to read</h3>
<p>There aren’t any in this category. Should there be? You tell me!</p>
<h2>Web resources</h2>
<p>There are so many, this will just be a partial list for now.</p>
<p>Of course, <a href="http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm">Fielding’s dissertation</a> is essential.</p>
<p>Roy has also written <a href="http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven">REST APIs Must be Hypertext Driven</a>. This post is central for understanding why “Hypermedia API” is a much better name than REST, and why hypermedia in general is so essential.</p>
<p>I’ve written <a href="http://timelessrepo.com/haters-gonna-hateoas">this post about HATEOAS</a>. It’s a pretty basic, simple introduction to the topic.</p>
<p>I gave a talk called <a href="http://vimeo.com/30764565">Everything you know about REST is wrong</a>.</p>
<p><a href="http://vimeo.com/20781278">This talk by Jon Moore</a> was instrumental in giving me a mental breakthrough about HATEAOS. He was kind enough to <a href="https://gist.github.com/1445773">share some code with me</a> that he used in the presentation as well. This is also an earlier example of the usage of “Hypermedia APIs.”</p>
<p>A classic: <a href="http://tomayko.com/writings/rest-to-my-wife">How I explained REST to my wife</a>. A great story, simple and easy to explain.</p>
<p>Another classic is <a href="http://www.infoq.com/articles/webber-rest-workflow">How to GET a cup of coffee</a>. It does a great job of explaining how to model your business processes as state machines, and then convert them to HTTP.</p>
<p>In which Mike Mayo has a realization that HATEOAS is not simply academic: <a href="http://mikemayo.org/2012/how-i-learned-to-stop-worrying-and-love-rest">http://mikemayo.org/2012/how-i-learned-to-stop-worrying-and-love-rest</a></p>
<p>A recent resource that’s popped up is <a href="http://publish.luisrei.com/rest.html">Designing a RESTful Web API</a>. It’s a nice, basic overview of lots of things.</p>
<h2>Related resources</h2>
<p><a href="http://www.amazon.com/gp/product/1449308929/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=1449308929">APIs: A Strategy Guide</a> seems really interesting. This isn’t about REST or Hypermedia APIs specifically, but more making a case for why you’d want an API in the first place. Which is a related topic for all of us API enthusiasts, for sure. I haven’t read it yet, but it’s on the related list.</p>
<p><a href="http://www.amazon.com/gp/product/0262572338/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0262572338">Protocol: How Control Exists after Decentralization</a> is one of my favorite books ever. This book manages to be both a hard computer science book as well as referencing a broad range of philosophy, history, and other fields as well.</p>
<p>If sentences like</p>
<blockquote>
<p>A perfect exmaple of a distributed network is the rhizome described in Deleuze and Guattari’s A Thousand Plateaus. Reacting specifically to what they see as the totalitarianism inherent in centralized and even decentralized networks, Deleuze and Guattari instead describe the rhizome, a horizontal meshwork derived from botany. The rhizome links many autonomous nodes together in a manner that is neither linear nor hierarchical. Rhizomes are heterogeneous and connective, that is to say, “Any point of a rhizome can be connected to anything other.”</p>
</blockquote>
<p>immediately followed by a hardcore, low-level diagram of the four levels of networking: application layer, transport layer, internet layer, and link layer. With full (and accurate) descriptions of TCP, IP, DNS, the SYN-ACK/SYN-ACK handshake, and HTTP following gets you all hot and bothered, you <em>need</em> to read this book. Hell, if you don’t like literature stuff, the politics in this book are amazing. This book draws the connections between <em>why</em> the APIs we’re building matter, and for that matter, the systems that we programmers create. Seriously, I can’t recommend this book enough.</p>
<p><a href="http://www.amazon.com/gp/product/0801882575/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0801882575">Hypertext 3.0: Critical Theory and New Media in an Era of Globalization</a> is related, and absolutely interesting. Here’s the eight main sections:</p>
<ol>
<li>Hypertext: An introduction</li>
<li>Hypertext and Critical Theory</li>
<li>Reconfiguring the Text</li>
<li>Reconfiguring the Author</li>
<li>Reconfiguring Writing</li>
<li>Reconfiguring Narrative</li>
<li>Reconfiguring Literary Education</li>
<li>The Politics of Hypertext: Who Controls the Text?</li>
</ol>
<p>While not <em>directly</em> useful for those designing APIs, those of us who like to draw broad connections between disciplines will find that this book has lots of interesting parallels. Especially around the politics angle, as well as reconfiguring narrative.</p></content:encoded>
</item>
<item>
<title>REST is over</title>
<link>https://steveklabnik.com/writing/rest-is-over/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rest-is-over/</guid>
<pubDate>Thu, 23 Feb 2012 00:00:00 GMT</pubDate>
<content:encoded><h1>REST is</h1>
<p><img src="/img/2012-02-23/restisover.png" alt="Rest is OVER" /></p>
<hr />
<p>Yep. Sorry to have to inform you. REST is totally over. The cool kids are moving on. We’re building “Hypermedia APIs” now. Such is life.</p>
<h2>A lesson from the anti-globalization movement</h2>
<p>Way back in the day, <a href="http://en.wikipedia.org/wiki/COINTELPRO">COINTELPRO</a> was at the forefront of America’s fight against “subersive” organizations and individuals. One goal of COINTELPRO was to create tension and division amongst radical groups, in order to disrupt their operations. Techniques such as Concern Trolling are really effective at this kind of thing.</p>
<p>In 2008, for the Republican National Convention in St. Paul, a document was passed around:</p>
<blockquote>
<p>At an anti-RNC conference held over the weekend of February 9th and 10th, a broad spectrum of groups revealed what are being called the “St. Paul Principles” of unity for resisting the 2008 Republican National Convention (RNC).</p>
</blockquote>
<p>This is a departure from the sectarian squabbles that have plagued past years’ anti-convention organizing. Pitting groups of differing political beliefs against each other has been a frequent tactic of state repression since the days of COINTELPRO.</p>
<p>By drafting the principles together, the co-signing organizations are taking historic steps to actively extinguish divisiveness from their respective groups. The principles will ensure respect for the soon-to-be-permitted march on September 1 by people planning non-permitted activities, and in turn, participants in the September 1 march will adhere to the principles and do nothing to sow division among the many activists coming to the Twin Cities to protest the RNC.</p>
<p>The principles are:</p>
<ol>
<li>Our solidarity will be based on respect for a diversity of tactics and the plans of other groups.</li>
<li>The actions and tactics used will be organized to maintain a separation of time or space.</li>
<li>Any debates or criticisms will stay internal to the movement, avoiding any public or media denunciations of fellow activists and events.</li>
<li>We oppose any state repression of dissent, including surveillance, infiltration, disruption and violence. We agree not to assist law enforcement actions against activists and others.</li>
</ol>
<blockquote></blockquote>
<p>Please draw your attention to the third principle. The reasons behind this rule are interesting:</p>
<ol>
<li>‘Solidarity’ kinda goes out the window when you’re busy arguing about each other’s drama.</li>
<li>Every second you have in the media is precious. Why waste it talking about each other when you could be talking about your issue?</li>
<li>Media will jump on any kind of debate as a weakness. The only way to make these sort of self-reflective discussions productive is to keep them internal.</li>
</ol>
<p>So, keeping that all in mind, why are we arguing about what REST means all the time? Yes, it’s annoying that the common usage of REST is not actually REST. Yes, it’s really hard when someone is wrong on the Internet. Yes, words do matter. However, it’s a question of what’s most productive with our time. Every moment we waste arguing over what REST means could have been spent discussing how to properly build APIs instead. But why bother being productive when we can be critical?</p>
<h2>‘Hypermedia API’ is more clear</h2>
<p>The real problem is that REST is just bad branding. This isn’t Roy’s fault, he wasn’t thinking about such things when trying to write his thesis. But really, from an outside perspective, a ‘real RESTful API’ does two things:</p>
<ol>
<li>Uses HTTP correctly.</li>
<li>Serves hypermedia responses.</li>
</ol>
<p>HTTP does most of the heavy lifting in terms of bringing us into REST compliance. So why are we talking about how to transfer representations of state?</p>
<p>The phrase “Hypermedia API” is much more direct: it’s an API where hypermedia is at the center. Those RESTish APIs could never be called ‘Hypermedia APIs,’ as it’s quite obvious that they don’t use hypermedia. It is not quite clear that they don’t transfer state and representations, though. ;)</p>
<h2>We just really shouldn’t fight</h2>
<p>Ultimately, it really comes back to the first point, though. Arguing about this is just wasting everyone’s time. It’s time to take a deep breath, step back, and just let REST go. Language changes. It happens. It might be a little bit sad, but life will move on. Let’s build fantastic APIs instead. &lt;3 &lt;3 &lt;3</p>
<p>Oh, and I didn’t actually kick this off, credit for that goes to Mike Amundsen and O’Reilly with <a href="http://www.amazon.com/Building-Hypermedia-APIs-HTML5-Node/dp/1449306578/ref=sr_1_1?ie=UTF8&amp;qid=1330039178&amp;sr=8-1">Building Hypermedia APIs with HTML5 and Node</a>. Once something has an O’Reilly book about it, it’s legit. ;) Additionally, the term has been bandied about in the past, in various presentations and talks, but I feel that now’s the time to really step forward and start calling a spade a spade.</p></content:encoded>
</item>
<item>
<title>An API ontology</title>
<link>https://steveklabnik.com/writing/an-api-ontology/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/an-api-ontology/</guid>
<pubDate>Mon, 13 Feb 2012 00:00:00 GMT</pubDate>
<content:encoded><p><strong>NOTE</strong>: The alpha of my book on APIs is out! Check it out at <a href="http://designinghypermediaapis.com/">http://designinghypermediaapis.com</a>.</p>
<p>As I’ve done research on APIs for <a href="http://designinghypermediaapis.com/">Designing Hypermedia APIs</a>, I’ve become increasingly interested in different styles of API. I currently see most real-world deployed APIs fit into a few different categories. All have their pros and cons, and it’s important to see how they relate to one other.</p>
<p>You may find this amusing if you’ve <a href="http://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch_styles.htm">read some of the literature</a> on the topic, but I’ve created this list in a top-down way: APIs as black boxes, rather than coming up with different aspects of an API and categorizing them based on that. I also decided to look at actually deployed APIs, rather than theoretical software architectures.</p>
<p>If you have an API that doesn’t fit into one of these categories, I’d love to hear about it. I’d also like to further expand these descriptions, if you have suggestions in that regard, please drop me a line, too.</p>
<h2>HTTP GET/POST</h2>
<h3>Synopsis:</h3>
<p>Provide simple data through a simple GET/POST request.</p>
<h3>Examples:</h3>
<ul>
<li><a href="http://placekitten.com/">http://placekitten.com/</a></li>
<li><a href="http://code.google.com/apis/maps/documentation/staticmaps/">http://code.google.com/apis/maps/documentation/staticmaps/</a></li>
<li><a href="http://loripsum.net/api">http://loripsum.net/api</a></li>
</ul>
<h3>Description:</h3>
<p>Simple data is made available via an HTTP GET or POST request. The vast majority of these services seem to return images, but data is possible as well.</p>
<p>These API are technically a sub-type of *-RPC, but I feel that their lack of business process makes them feel different. It’s basically just one specific remote procedure, available over HTTP.</p>
<h2>*-RPC</h2>
<h3>Synopsis:</h3>
<p>Remote procedure call; call a function over the web.</p>
<h3>Examples:</h3>
<ul>
<li><a href="http://codex.wordpress.org/XML-RPC_Support">http://codex.wordpress.org/XML-RPC_Support</a></li>
<li><a href="http://www.flickr.com/services/api/request.rest.html">http://www.flickr.com/services/api/request.rest.html</a></li>
<li><a href="http://services.sunlightlabs.com/docs/Sunlight_Congress_API/">http://services.sunlightlabs.com/docs/Sunlight_Congress_API/</a></li>
</ul>
<h3>Description:</h3>
<p>Similiar to how structured programming is built around functions, so is RPC. Rather than call functions from your own programs, RPC is a way to call functions over the Internet.</p>
<p>All calls are made through some sort of API endpoint, and usually sent over HTTP POST.</p>
<p>Major flavors include XML-RPC and JSON-RPC, depending on what format data is returned in.</p>
<p>Note that while Flickr’s API says REST, it is very clearly RPC. Yay terminology!</p>
<h2>WS-* (or SOAP)</h2>
<h3>Synopsis:</h3>
<p>Serialize and send objects over the wire.</p>
<h3>Examples:</h3>
<ul>
<li><a href="http://www.flickr.com/services/api/request.soap.html">http://www.flickr.com/services/api/request.soap.html</a></li>
<li><a href="https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&amp;content_ID=developer/e_howto_api_soap_PayPalSOAPAPIArchitecture">https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&amp;content_ID=developer/e_howto_api_soap_PayPalSOAPAPIArchitecture</a></li>
<li><a href="http://www.salesforce.com/us/developer/docs/api/Content/sforce_api_quickstart_intro.htm">http://www.salesforce.com/us/developer/docs/api/Content/sforce_api_quickstart_intro.htm</a></li>
</ul>
<h3>Description:</h3>
<p>SOAP stands for “Simple Object Access Protocol,” and that describes it pretty well. The idea behind these APIs is to somehow serialize your objects and then send them over the wire to someone else.</p>
<p>This is usually accomplished by downloading a WSDL file, which your IDE can then use to generate a whole ton of objects. You can then treat these as local, and the library will know how to make the remote magic happen.</p>
<p>These are much more common in the .NET world, and have fallen out of favor in startup land. Many larger businesses still use SOAP, though, due to tight integration with the IDE.</p>
<h2>“REST”</h2>
<h3>Synopsis:</h3>
<p>Ruby on Rails brought respect for HTTP into the developer world. A blending of RPC, SOAP, and hypermedia API types.</p>
<h3>Examples:</h3>
<ul>
<li><a href="http://developer.github.com/">http://developer.github.com/</a></li>
<li><a href="https://dev.twitter.com/">https://dev.twitter.com/</a></li>
<li><a href="http://developers.facebook.com/docs/reference/api/">http://developers.facebook.com/docs/reference/api/</a></li>
</ul>
<h3>Description:</h3>
<p>Originally, REST was synonymous with what is now called “Hypermedia APIs.” However, after large amounts of misunderstanding, REST advocates are rebranding REST to “Hypermedia APIs” and leaving REST to the RESTish folks. See ’<a href="/posts/2012-02-23-rest-is-over">REST is over’</a> for more.</p>
<p>REST is basically “RPC and/or SOAP that respects HTTP.” A large problem with RPC and SOAP APIs is that they tunnel everything through one endpoint, which means that they can’t take advantage of many features of HTTP, like auth and caching. RESTful APIs mitigate this disadvantage by adding lots of endpoints; one for each ‘resource.’ The SOAPish ones basically allow you to CRUD objects over HTTP by using tooling like ActiveResource, and the RPC ones let you perform more complicated actions, but always with different endpoints.</p>
<h2>Hypermedia</h2>
<h3>Synopsis:</h3>
<p>Hypermedia is used to drive clients through various business processes. The least understood and deployed API type, with one exception: the World Wide Web.</p>
<h3>Examples:</h3>
<ul>
<li><a href="http://www.twilio.com/docs/api/rest">http://www.twilio.com/docs/api/rest</a></li>
<li><a href="http://www.spire.io/docs/tutorials/rest-api.html">http://www.spire.io/docs/tutorials/rest-api.html</a></li>
<li><a href="http://kenai.com/projects/suncloudapis/pages/Home">http://kenai.com/projects/suncloudapis/pages/Home</a></li>
</ul>
<h3>Description:</h3>
<p>Originally called REST, Hypermedia APIs take full advantage of HTTP. They use hypermedia formats to drive business processes, providing the ultimate decoupling of clients and servers.</p>
<p>You can tell an API is Hypermedia by providing only one API endpoint, but which accepts requests at other endpoints that are provided by discovery. You navigate through the API by letting the server’s responses guide you.</p></content:encoded>
</item>
<item>
<title>Introducing metadown</title>
<link>https://steveklabnik.com/writing/introducing-metadown/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/introducing-metadown/</guid>
<pubDate>Mon, 23 Jan 2012 00:00:00 GMT</pubDate>
<content:encoded><p>Because I don’t have enough gems made already, I made another one last night: <a href="https://rubygems.org/gems/metadown">metadown</a>.</p>
<h2>What’s Metadown do?</h2>
<p>This blog originally used Jekyll. When I moved it to my own personal blog implementation, I noticed something: Markdown doesn’t actually have support for adding YAML at the top of your files, like Jekyll does. I always <em>knew</em> this, I just didn’t think about it before. And I love using Markdown, so I ended up extracting my own version of this trick into Metadown.</p>
<p>Basically, sometimes you need metadata about your markup file. YAML is a nice format for writing key/value pairs, lists, and other things… so I’ve let you smash the two of them together in one file by adding some <code>---</code>s at the top.</p>
<h2>Gimme an Example</h2>
<pre><code>require 'metadown'
data = Metadown.render("hello world")
data.output #=&gt; "&lt;p&gt;hello, world&lt;/p&gt;"
data.metadata #=&gt; "{}"
text = &lt;&lt;-MARKDOWN
---
key: "value"
---
hello world
MARKDOWN
data = Metadown.render(text)
data.output #=&gt; "&lt;p&gt;hello, world&lt;/p&gt;\n"
data.metadata #=&gt; {"key" =&gt; "value"}
</code></pre>
<h2>Where’s the code?</h2>
<p>It’s implemented using a custom parser for Redcarpet, has a test suite, and works on every Ruby that’s not JRuby. You can <a href="https://github.com/steveklabnik/metadown">check out the source here</a>. Issues and Pull Requests welcome. There’s at least one thing that I know I’ll need to add to it in the near future.</p></content:encoded>
</item>
<item>
<title>Moving from Sinatra to Rails</title>
<link>https://steveklabnik.com/writing/moving-from-sinatra-to-rails/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/moving-from-sinatra-to-rails/</guid>
<pubDate>Tue, 17 Jan 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I love both Sinatra and Rails, for different reasons. I’ve heard a few different heuristics for which framework would be better for your application, but I’m not sure the answer is all that simple, really. Regardless of which is correct for your application, I haven’t seen a lot of people discussing how to move between the two frameworks.</p>
<h2>Step One: Evaluate your test coverage</h2>
<p>In <a href="http://www.amazon.com/gp/product/0131177052/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0131177052">Working Effectively with Legacy Code</a> (this is an affiliate link. Does this bother you?), Michael Feathers introduces a fantastic technique called “Lean on the Compiler.” It’s on page 315, for those of you playing along at home.</p>
<blockquote>
<p>The primary purpose of a compiler is to translate source code into some other form, but in statically typed languages, you can do much more with a compiler. You can take advantage of its type checking and use it to identify which changes you need to make. I call this practice Leaning on the Compiler.Lean on the Compiler is a powerful technique, but you have to know what its limits are; if you don’t, you can end up making some serious mistakes.</p>
</blockquote>
<p>Now, in a language like Ruby, we don’t have type checking. We do have something that double checks our code correctness: tests. Just like static type checks, tests cannot <em>ensure</em> that you’ve done your transformations, but they sure as hell can help. This becomes step one in the effort to move from one framework to another: evaluate your test coverage.</p>
<h3>Acceptance Tests are most important</h3>
<p>When looking at your tests, first check out your acceptance tests. These are the most important, for two reasons: they’re abstracted away from the framework itself, and their purpose in life is to make sure that major functionality is working. Do you have your happy paths covered? Is there any major functionality that’s <em>not</em> covered by acceptance tests?</p>
<p>While the happy path is a big deal, when moving to a new framework, we’re going to introduce a high chance of people encountering issues, so the sad path is also important. I personally tend to not write very many sad path acceptance tests and leave that for tests at the lower level. This is a good time to take stock of the worst of your worst paths: is there certain functionality that’s <em>absolutely</em> important to be handled 100% correctly? Then write some new acceptance tests. As with any kind of engineering project, there’s a tradeoff here: you can’t get coverage of every possible situation. Tests are supposed to give you confidence in your code, so liberally add coverage for any situation that makes you nervous.</p>
<h3>Integration tests are important too</h3>
<p>These tests are a bit more tricky, as they are tied into the framework that you’re using. I’m not sure that anyone writes many of these kinds of tests with Sinatra, outside of maybe model to model integration tests.</p>
<h3>Model and View tests: not a big deal</h3>
<p>These kinds of tests are much more of a sanity check than anything else for the purposes of this kind of move. It’s good that they pass, but really, they shouldn’t be too reliant on the framework you’re using. They’re better for making sure you’ve put things in the correct directories and haven’t forgotten anything in the move.</p>
<p>I’m not even sure what a ‘view test’ would be in Sinatra, and they tend to be not in favor with Rails projects anyway, so that’s more of a general ‘framework to framework’ bit of advice than anything else.</p>
<h3>Controllers don’t exist in Sinatra…</h3>
<p>… so you don’t really write those kinds of tests. Controller tests don’t seem to be that popular in Rails-land these days either.</p>
<h2>Step Two: Git Can help</h2>
<p>I’ll show you how I managed transitioning files over. However, I’m really interested in the idea of using a <a href="http://help.github.com/subtree-merge/">subtree merge</a> to keep being able to update the original Sinatra project while working on the new Rails project.</p>
<p>First step is, of course, make a new branch. This transition will probably take some time, and you may want to push hotfixes into production during that period. Using master is not advised.</p>
<p>Next, make a copy of everything and shove it in a temp directory. Then delete everything in the git repo (except the .git directory, of course) and commit that blank slate. Back up one directory above your project directory, run <code>rails new myproject</code> with your project’s name, and you’ll get a blank rails app. <code>cd myproject</code> and <code>mkdir sinatra</code>, then copy the backup you made from the temp directory into the sinatra directory. Finally, commit this.</p>
<p>Now you’ve got a new blank Rails app with all your old code in a directory, and you can start moving things over.</p>
<h2>Step Three: Set up your test harness</h2>
<p>Since we’re going to allow our tests to guide our move, it pays to get tests up and running first! Depending on what testing framework you use, get it going with Rails. In our case, we were using minitest to test our code. This took a little bit of effort to get working with Rails, but there weren’t a ton of problems.</p>
<p>As always: red, green, refactor. I’d make a simple test that doesn’t test anything, <code>assert true</code>. Make sure that your <code>rake test</code> or <code>rspec .</code> or whatever you’ll use to run the tests works and then remove the dummy test.</p>
<p>There are two strategies for moving tests over: you can move a chunk at a time or move one at a time. Chunks are easier, but then you get commits where the build is broken. It’s really up to you and your team’s tastes: a huge test suite with a number of failures can show you how close you are to being done with the first 90% of the work. And since you’re not planning on releasing in this half-finished state, having a broken build is not <em>that</em> terrible… As always, you’re the professional: make the call.</p>
<h2>Step Four: Move your models</h2>
<p>Since models are the simplest to move, I like to do them first. You <em>should</em> just be able to copy over each test file and model file, though because you were using Sinatra, your models may be all in one file. This step should be largely painless, though: you’re not really relying on any framework-specific things.</p>
<p>Model tests can still suss out problems with your environment, though, like incorrect database settings, certain environment variables…</p>
<p>The idea to begin with the easiest thing comes from <a href="http://www.daveramsey.com/home/">Dave Ramsey</a>, oddly enough. My aunt works at a bank, and when I went to college, she bought me a few of his books to help me learn about personal finance. Dave is a big fan of the ‘get rid of all debt’ school of thought, and so a large portion of his work is strategies for getting out of debt. Dave contends that paying off loans with the highest interest, while mathematically the fastest way to pay off your obligations, is not actually the best strategy. People like to see progress and are heartened by seeing it. By paying off the smallest loan first, one is much more likely to feel more positive about the headway one is making, and so it’s a much better strategy in the end.</p>
<p>The same logic applies with this move: why start with the hard stuff? Knock out some quick wins to keep your spirits high.</p>
<h2>Step Five: Convert your controllers</h2>
<p>Next up: get your controllers created. This is a pretty manual process:</p>
<pre><code>get "foos" do
@foos = Foo.all
render :"foos/index"
end
</code></pre>
<p>becomes</p>
<pre><code>class FooController &lt; ApplicationController
def index
@foos = Foo.all
end
end
</code></pre>
<p>The biggest issue with moving all of this stuff is that Sinatra and Rails use <code>redirect</code> and <code>redirect_to</code>, respectively. So you’ll have to convert that stuff. However, I wouldn’t recommend changing things like <code>redirect "/foos/#{id}"</code> to <code>redirect_to foo_path(foo)</code> just yet. When dealing with legacy code, you want to change as little as possible with each step so that you know when you have introduced an error. If you try to convert things to a Rails style as well, you run the risk of introducing errors. Therefore, in all of these moves, leave the hell enough alone as much as possible. Once your code is up and running, you can gladly refactor. Just don’t do it now.</p>
<p>Don’t forget to generate your routes, too. Sinatra’s DSL is like a combination of Rails’ routing and controllers. Set those up in this step as well.</p>
<p>Since we don’t have tests, this part is very error-prone. Luckily, our acceptance tests will catch these issues in step seven. So give it your best shot, but don’t worry about being 100% perfect. Focus on getting the basic structural changes in place.</p>
<p>Having tests is so nice. :/</p>
<h2>Step Six: Move your views</h2>
<p>This should be as simple as the models: put your views in the correct directory, and things should be golden. If you were using inline views with Sinatra, well, you shouldn’t have that many of them, so breaking them out should be pretty easy.</p>
<h2>Step Seven: Listen to your tests</h2>
<p>Okay! Those last moves were really awkward since we didn’t have tests to check our work. This is where the acceptance tests come in. You can move these over in batches or one at a time, but the acceptance tests will tell you where you forgot a route, if your view is missing, or if you left some little bit of Sinatra-ness in your Rails somewhere.</p>
<p>You’re almost home! Once you get all your acceptance tests working, you should pat yourself on the back! You’re not done yet, but you’ve made a lot of progress.</p>
<h2>Step Eight: Track your exceptions!</h2>
<p>You should have been doing this anyway, but if you weren’t, set up some sort of mechanism to catch and handle exceptions. <a href="http://airbrakeapp.com/pages/home">Airbrake</a> is one such service, but even just recording them yourself would be fine. You <em>need</em> to have this set up in some form, as you’re likely to generate errors, and examining your exceptions is the best way to track down actual problems.</p>
<h2>Step Nine: Plan out deployment strategies</h2>
<p>Of course, now that you’re done developing, it’s time to get the app out to users. This’ll require a comprehensive plan for actually making the jump. A little bit of foresight can go a long way here.</p>
<p>My strategy was to roll out the new Rails app under a new domain: http://beta.myapp.com/. I then pinged my users with a message along the lines of “We’ve made some infrastructure updates, if you’d like to help us test them out, visit the beta site.”</p>
<p>This approach does have some sticky points, however. The first one is the database. In my case, I had a high level of confidence in my code, and this application wasn’t for anything that was considered mission-critical. We also had a decent backup strategy in place. Therefore, I connected the beta site up to the production data store. This meant there were no migration issues later. However, this may make you uncomfortable, but there are other options, too. You can treat the beta as a sandbox, and tell people that the data will not be persisted after the beta, or you can migrate people’s data back to the production data store afterwards.</p>
<p>Another approach is to automatically migrate a portion of your users over to the new code base. Ideally, nobody notices: your code base shouldn’t have changed in looks or functionality.</p>
<h2>Step Ten: Push to production!</h2>
<p>Congrats! Execute on the plan you created in step nine, and get ready to answer tons of emails about why your app is broken! Just kidding, I hope! I really recommend the beta strategy for this reason. No plan survives first contact with the enemy. Your users <em>will</em> find things you didn’t, no matter how much manual testing you did along the way.</p>
<h2>Another strategy: mounting</h2>
<p>I haven’t actually tried this myself, but I’ve been thinking about another way to make this happen: mount your Sinatra app inside of a Rails app, and use that to move things over slowly. <a href="http://stackoverflow.com/a/6972706">Here’s an example</a> of how to make this happen technically. If you had a really big Sinatra application, I could see how this might get you deploying faster; just carve out one vertical chunk of your app, move it over, and keep the rest of your app running in Sinatra.</p></content:encoded>
</item>
<item>
<title>Implementing HATEOS with presenters</title>
<link>https://steveklabnik.com/writing/implementing-hateoas-with-presenters/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/implementing-hateoas-with-presenters/</guid>
<pubDate>Fri, 06 Jan 2012 00:00:00 GMT</pubDate>
<content:encoded><p>I’m a big fan of using the presenter pattern to help separate logic from presentation. There’s a great gem named <a href="https://github.com/jcasimir/draper">Draper</a> that can help facilitate this pattern in your Rails apps. When doing research for <a href="http://designinghypermediaapis.com/">my book about REST</a>, I realized that the presenter pattern is a great way to create responses that comply with the hypermedia constraint, a.k.a. HATEOAS. I wanted to share with you a little bit about how to do this.</p>
<p>Please note that ’<a href="/posts/2012-02-23-rest-is-over">REST is over’</a>.</p>
<p>Note: We’ll be creating HTML5 responses in this example, as HTML is a hypermedia format, and is therefore conducive to HATEOAS. JSON and XML don’t cut it.</p>
<h2>First, some setup</h2>
<p>I fired up a brand new Rails app by doing this:</p>
<pre><code>$ rails new hateoas_example
$ cd hateoas_example
$ cat &gt;&gt; Gemfile
gem "draper"
^D
$ bundle
$ rails g resource post title:string body:text
$ rake db:migrate
$ rails g draper:decorator Post
</code></pre>
<p>Okay, now we should be all set up. We’ve got a Rails app, it’s got draper in the Gemfile, we have a Post resource, and our PostDecorator.</p>
<h2>The View</h2>
<p>I like to do the view first, to drive our what we need elsewhere. Here it is:</p>
<pre><code>&lt;h2&gt;Title&lt;/h2&gt;
&lt;p&gt;&lt;%= @post.title %&gt;&lt;/p&gt;
&lt;h2&gt;Body&lt;/h2&gt;
&lt;p&gt;&lt;%= @post.body %&gt;&lt;/p&gt;
&lt;h2&gt;Links&lt;/h2&gt;
&lt;ul&gt;
&lt;% @post.links.each do |link| %&gt;
&lt;li&gt;&lt;%= link_to link.text, link.href, :rel =&gt; link.rel %&gt;&lt;/li&gt;
&lt;% end %&gt;
&lt;/ul&gt;
</code></pre>
<p>We’re displaying our title and body, but we also want to spit out some links. These links should have a few attributes we need. I might even (shhhhhhh) extract this link thing out into a helper to add the rel stuff every time. It just depends. For this example, I didn’t feel like it.</p>
<h2>The Controller</h2>
<p>Well, we know we’re gonna need a <code>@post</code> variable set, so let’s get that going in our controller:</p>
<pre><code>class PostsController &lt; ApplicationController
def show
@post = PostDecorator.find(params[:id])
end
end
</code></pre>
<p>Super simple. Yay Draper!</p>
<h2>The Presenter</h2>
<p>We know we need a <code>links</code> method that returns some links, and those links need to have rel, href, and text attributes. No problem!</p>
<pre><code>class Link &lt; Struct.new(:rel, :href, :text)
end
class PostDecorator &lt; ApplicationDecorator
decorates :post
def links
[self_link, all_posts_link]
end
def all_posts_link
Link.new("index", h.posts_url, "All posts")
end
def self_link
Link.new("self", h.post_url(post), "This post")
end
end
</code></pre>
<p>Now, we could have just returned an array of three-element arrays, but I really like to use the Struct.new trick to give us an actual class. It makes error messages quite a bit better, and reminds us that we don’t happen to have an array, we have a Link.</p>
<p>We construct those links by taking advantage of the ‘index’ and ‘self’ rel attributes that are <a href="http://www.iana.org/assignments/link-relations/link-relations.xml">defined in the registry</a>.</p>
<h2>The output</h2>
<p>That gives us this:</p>
<pre><code>&lt;!DOCTYPE html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;title&gt;HateoasSample&lt;/title&gt;
&lt;link href="/assets/application.css?body=1" media="screen" rel="stylesheet" type="text/css" /&gt;
&lt;link href="/assets/posts.css?body=1" media="screen" rel="stylesheet" type="text/css" /&gt;
&lt;script src="/assets/jquery.js?body=1" type="text/javascript"&gt;&lt;/script&gt;
&lt;script src="/assets/jquery_ujs.js?body=1" type="text/javascript"&gt;&lt;/script&gt;
&lt;script src="/assets/posts.js?body=1" type="text/javascript"&gt;&lt;/script&gt;
&lt;script src="/assets/application.js?body=1" type="text/javascript"&gt;&lt;/script&gt;
&lt;meta content="authenticity_token" name="csrf-param" /&gt;
&lt;meta content="0k+SQVv6yr0d12tGWYx7KNXUWaf6f+wgUUNITsAOnHI=" name="csrf-token" /&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;h2&gt;Title&lt;/h2&gt;
&lt;p&gt;A post, woo hoo!&lt;/p&gt;
&lt;h2&gt;Body&lt;/h2&gt;
&lt;p&gt;this is some text that's the body of this post&lt;/p&gt;
&lt;h2&gt;Links&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="http://localhost:3000/posts/1" rel="self"&gt;This post&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="http://localhost:3000/posts" rel="index"&gt;All posts&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/body&gt;
&lt;/html&gt;
</code></pre>
<p>You’ll probably want to make a layout that ignores all of the JS stuff, but for this example, I just left it as-is. It’s just that easy. Happy linking!</p></content:encoded>
</item>
<item>
<title>ActiveRecord considered harmful</title>
<link>https://steveklabnik.com/writing/active-record-considered-harmful/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/active-record-considered-harmful/</guid>
<pubDate>Fri, 30 Dec 2011 00:00:00 GMT</pubDate>
<content:encoded><blockquote>
<p>It is practically impossible to teach OO design to students that have had a prior exposure to Rails: as potential programmers they are mentally mutilated beyond hope of regeneration.Edsger W. Dijkstra (paraphrased)</p>
</blockquote>
<p>I love ActiveRecord. It was the first ORM I’d ever interacted with. My first technical employer had commissioned a DBA, and so of course, we wrote all our own queries. Which was fine; I know my way around a JOIN or two. The problem came when it was time to make a new class; time to write “SELECT * FROM ‘tableName’ WHERE ‘id’=‘%’”… for each class. “Just copy one of the other small classes, hack out everything, and change the table names,” were my instructions. Fine. Whatever. But I knew there had to be a better way…</p>
<p>Along comes Rails. Holy crap, ActiveRecord is <em>awesome</em>! It writes the <em>exact</em> SQL I would have written myself! I don’t need to do <em>anything</em>. This is the best thing since sliced bread. But years later, not everything is rainbows and sunshine. I’ve written a <em>lot</em> of crappy code, and most of it is due to following Rails ‘best practices.’ Which is totally fine! Rails is kinda getting old these days. It’s no longer the new and shiny. But while Rails developers have gotten really good at busting out little CRUD apps, we haven’t moved web application design <em>forward</em> in a really, really long time.</p>
<p>And that’s what Rails did, really. That fifteen minute blog video was shocking. I know several people that threw away two or three week old projects and re-built their stuff in Rails within moments of watching it. And Rails has continued to lead the way in improving how we build rich applications on the web; the (not so but kinda whatever nobody but me cares) RESTful routing was great. The asset pipeline, though it has some bugs, has been great. The obsession with DRY has been great. The test obsession has been great. But Rails has also not been great in aiding us in writing maintainable software. Many of Rails’ design decisions make it difficult to write <em>great</em> software. It helps us write good software, but that isn’t enough anymore.</p>
<p>The real problem is that to truly move forward, Rails will have to re-invent itself, and I’m not sure that it can.</p>
<h2>ActiveRecord is the problem</h2>
<p>I’m not going to talk about all of the problems Rails has today, but I’d like to show you the biggest, most central one: ActiveRecord.</p>
<p>ActiveRecord’s greatest strength is also its problem: Tying class names to table names. This means that it’s impossible to de-couple your persistence mechanism from your domain logic. You can manage it through a combination of <code>set_table_name</code>, making a bunch of <code>Repository</code> classes, and careful coding… but then you might as well be using DataMapper. Except the Ruby library of the same name doesn’t really implement the DataMapper pattern all that well either, having the same issue of tying it all together.</p>
<p>This has another interesting effect: it led directly to the ‘fat model’ recommendation. While ‘get your stuff out of the controller’ is correct, it’s lead Rails developers to build huge, monolithic models that are hard to test, and violate SRP. It took me two and a half years to realize that Ruby classes in the models folder don’t have to inherit from <code>ActiveRecord::Base</code>. That is a problem.</p>
<p>We’ve gotten into a situation with a local minimum: our quest for great software has lead us to DRY ourselves into a corner. Now we have code that’s incredibly tightly coupled. Our ‘single responsibility’ is ‘anything and everything that tangentially relates to a Post.’</p>
<h2>ActionController is the problem</h2>
<p>ActionController relies on instance variables to pass information from the controller to the view. Have you ever seen a 200 line long controller method? I have. Good luck teasing out which instance variables actually get set over the course of all those nested <code>if</code>s.</p>
<p>The whole idea is kinda crazy: Yeah, it looks nice, but we literally just say ‘increase the scope of variables to pass data around.’ If I wrote a post saying “Don’t pass arguments to methods, just promote your data to a global” I’d be crucified. Yet we do the same thing (albeit on a smaller scale) every time we write a Rails application.</p>
<h2>ActionView is the problem</h2>
<p>The whole idea of logic in templates leads to all kinds of problems. They’re hard to test, they’re hard to read, and it’s not just a slippery slope, but a steep one. Things go downhill <em>rapidly</em>.</p>
<p>What I’d really like to see is Rails adopting a ‘ViewModel + templates’ system, with logic-less templates and presenter-esque models that represent the views. The differences between Django’s idea of ‘views’ and Rails’ idea of ‘views’ are interesting here.</p>
<h2>MVC is the problem</h2>
<p>If you’ll notice, I basically have said that models are a problem. Controllers are a problem. Views are a problem. MVC has served the web well, even if it isn’t the GUI style MVC that named the pattern. But I think we’re reaching its limits; the impedance mismatch between HTTP and MVC, for example, is pretty huge. There are other ways to build web applications; I’m particularly excited about <a href="http://rubyconf-webmachine.heroku.com/">WebMachine</a>. I don’t have a constructive alternative to offer here, I just know there’s a problem. I’m still mulling this one over.</p>
<h2>It’s still good, even with problems</h2>
<p>I love Rails. I build software with it daily. Even with its flaws, it’s been a massive success. But because I love Rails, I feel like I can give it straightforward criticism: it’s easier to trash something you love. The real issue is that changing these things would require some really serious changes. It’d involve re-architecting large portions of things that people classically identify with Rails, and I’m not sure that Rails wants to or can do that.</p>
<p>This post is light on examples. I want this to be the <em>starting point</em> of a discussion, not the end of it. Expect more, in detail, from me in the future. What do you think? Are these problems pain points for you? Are they worth fixing? Are they actually problems?</p></content:encoded>
</item>
<item>
<title>Devise: ActionController::RoutingError (No Route Matches [GET] /users/sign_out)</title>
<link>https://steveklabnik.com/writing/devise-actioncontroller-routingerror-no-route-matches-get-slash-users-slash-sign-out/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/devise-actioncontroller-routingerror-no-route-matches-get-slash-users-slash-sign-out/</guid>
<pubDate>Thu, 22 Dec 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Just a quick note about Devise, and its RESTful implications. I ran across this error today, and thought I’d share.</p>
<p>I was trying to log out, so I hit the normal route for such things with my browser. Here’s the error:</p>
<pre><code>Devise: ActionController::RoutingError (No Route Matches [GET] /users/sign_out)
</code></pre>
<p>Uhhhhh what? I run rake routes…</p>
<pre><code>$ rake routes | grep destroy_user
destroy_user_session DELETE /users/sign_out(.:format) {:action=&gt;"destroy", :controller=&gt;"devise/sessions"}
</code></pre>
<p>So, wtf? Well, there is that pesky <code>DELETE</code>… Googling for this error led me to <a href="https://github.com/plataformatec/devise/commit/f3385e96abf50e80d2ae282e1fb9bdad87a83d3c">this commit</a>. Looks like they changed some behavior, you can’t just go to that page anymore and log out. Bummer. But why?</p>
<h2>HTTP verbs: transfer semantics, not state</h2>
<p>As I’ve been doing my research for <a href="http://designinghypermediaapis.com/">my book on REST</a>, I’ve been doing a lot of reading about HTTP. And I kept coming across these kinds of curious comments in the spec. For example, <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6">sec 9.6: PUT</a>:</p>
<blockquote>
<p>HTTP/1.1 does not define how a PUT method affects the state of an origin server.</p>
</blockquote>
<p>Uhhh… what? Don’t we all know that PUT means that we should be updating a resource? And that we have to send the whole representation?</p>
<p>When trying to get to the bottom of this, I came across <a href="http://www.imc.org/atom-protocol/mail-archive/msg05425.html">this comment from Fielding</a>:</p>
<blockquote>
<p>FWIW, PUT does not mean store. I must have repeated that a million times in webdav and related lists. HTTP defines the intended semantics of the communication – the expectations of each party. The protocol does not define how either side fulfills those expectations, and it makes damn sure it doesn’t prevent a server from having absolute authority over its own resources. Also, resources are known to change over time, so if a server accepts an invalid Atom entry via PUT one second and then immediately thereafter decides to change it to a valid entry for later GETs, life is grand.</p>
</blockquote>
<p>Soooooooo wtf?</p>
<p>Let’s take a look again at what <code>PUT</code> does:</p>
<blockquote>
<p>The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.</p>
</blockquote>
<p>It says “store,” Roy says “I don’t mean store.” Uhhhh…</p>
<p>Here’s my ‘translated for the laymen’ version of that quote:</p>
<blockquote>
<p>PUT means ‘I’d like to later GET something at this URI.’ If something is already there, update it. If there isn’t anything there, then create it.</p>
</blockquote>
<p>That’s it. It’s talking about the semantics of what goes on: create or update. It doesn’t actually say anything about how this is implemented. But if you PUT something to a URI, a GET needs to 200 afterwards. So what’s the difference between PUT and POST? <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5">HTTP sec 9.5: POST</a>:</p>
<blockquote>
<p>The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line.The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database.The action performed by the POST method might not result in a resource that can be identified by a URI.</p>
</blockquote>
<p>Again, it’s really vague about what it <em>does</em>. With POST, it basically says “You have no idea what a POST does.” What you <em>do</em> know is the semantics of the action, POST ‘requests a new subordinate, but it might not create something.’</p>
<p>The <em>main</em> difference is actually mentioned <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.2">here</a>:</p>
<blockquote>
<p>Methods can also have the property of “idempotence” in that (aside from error or expiration issues) the side-effects of N &gt; 0 identical requests is the same as for a single request. The methods GET, HEAD, PUT and DELETE share this property.</p>
</blockquote>
<p>Semantics again. PUT is <em>idempotent</em>, and POST is not. They could both be used for creation, they could both be used for updating. With POST, you don’t need a URI, and PUT specifies a specfic one. That’s it. Nowhere in those two sentences states ‘store in the database,’ nowhere does it says ‘full representation,’ nowhere does it say ‘POST is create and PUT is update.’ However you fulfill these semantics are up to you, but the semantics are what’s important.</p>
<h2>So wtf does this have to do with Devise?</h2>
<p>The issue is that Devise’s old semantics were wrong. A GET to <code>/users/sign_out</code> shouldn’t modify state: <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.1">HTTP sec 9.1.1: Safe Methods</a>:</p>
<blockquote>
<p>In particular, the convention has been established that the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered “safe”.</p>
</blockquote>
<p>When Devise used GETs to log you out, that was a violation of the semantics of GET. Here’s the interesting part, though: Since POST is a super generic ‘unsafe action’ method, you could also use POST to represent a logging out. POST also has unsafe, non-idempotent semantics. DELETE specifically says delete, and POST says ‘any action,’ and ‘delete’ is a subset of ‘any action.’ So DELETE is <em>better</em>, but POST is not <em>wrong</em>, categorically.</p>
<h2>Fixing the ‘bug’</h2>
<p>So how do we take care of this? Personally, I did the pragmatic thing. In <code>/config/initializers/devise.rb</code></p>
<pre><code>config.sign_out_via = :delete
config.sign_out_via = :get if Rails.env.test?
</code></pre>
<p>Now, during normal operations, we have our usual DELETE semantics, but in our test environment, we can just hit it with GET. This way we don’t have to hit a page, use Javascript to make a link with DELETE, and hit the page. This keeps my test times down, means I can run my tests with rack-test and not Selenium, and still gives me a high level of confidence that my tests work properly, even though it’s technically not the exact same thing as production.</p>
<h2>In conclusion</h2>
<p>The HTTP spec defines <em>semantics</em> but not implementation details. Semantics should be obeyed. But in testing, obeying them 100% may not be worth it.</p></content:encoded>
</item>
<item>
<title>Redcarpet is awesome</title>
<link>https://steveklabnik.com/writing/redcarpet-is-awesome/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/redcarpet-is-awesome/</guid>
<pubDate>Wed, 21 Dec 2011 00:00:00 GMT</pubDate>
<content:encoded><p>It’s true.</p>
<p>If you haven’t used it yet, <a href="https://github.com/tanoku/redcarpet">Redcarpet</a> is the Markdown parser that <a href="https://github.com/blog/832-rolling-out-the-redcarpet">GitHub uses</a> to work all that magic on their site. So of course, it’s awesome.</p>
<h2>You can use it and abuse it</h2>
<p>What makes it <em>really</em> awesome is the custom renderers feature. Here’s the one from the documentation:</p>
<pre><code># create a custom renderer that allows highlighting of code blocks
class HTMLwithAlbino &lt; Redcarpet::Render::HTML
def block_code(code, language)
Albino.safe_colorize(code, language)
end
end
markdown = Redcarpet::Markdown.new(HTMLwithAlbino)
</code></pre>
<p>You can guess what that does: uses some library called Albino to render code blocks. There are a whole bunch of hooks you can use to make a custom renderer. They make it super easy to do something neat with markdown.</p>
<p>So I did.</p>
<h2>Check it: outline generation</h2>
<p>I’m working on… this project. And it needs to render a bunch of articles that are Markdown files. There will be a sidebar. I want to have links to each section. But I’m a good programmer, which means I’m lazy. So why bother making my own sidebar? Especially when I can abuse Redcarpet to do it.</p>
<p>Oh, and disclaimer: normally, I’m all about super clean code. Last night, that wasn’t the mood I was in. This code is probably terrible. That’s part of the fun! Please feel free to {clean up,obfuscate,golf,unit test} this code, <a href="mailto:[email protected]">email me</a>, and tell me how awesome you are.</p>
<p>With that disclaimer out of the way, round one:</p>
<pre><code>class OutlineRenderer &lt; Redcarpet::Render::HTML
attr_accessor :outline
def initialize
@outline = []
super
end
def header(text, header_level)
text_slug = text.gsub(/\W/, "_").downcase
self.outline &lt;&lt; [header_level, "&lt;a href='##{text_slug}'&gt;#{text}&lt;/a&gt;"]
"&lt;h#{header_level} id='#{text_slug}'&gt;#{text}&lt;/h#{header_level}&gt;"
end
end
</code></pre>
<p>Every time we hit a header, Redcarpet gives us the text and the header level. We grab the text, turn it into a slug, and then append a two-element array to our outline. It keeps track of the level of this header, and makes a link from the slug. Then, we spit out a header tag, adding on an <code>id</code> element that we linked to in our link.</p>
<p>Next up, rendering:</p>
<pre><code>renderer = OutlineRenderer.new
r = Redcarpet::Markdown.new(renderer)
content = r.render(some_content)
outline = renderer.outline
</code></pre>
<p>This instantiates our Renderer, creates our Markdown parser, and renders the content. We also grab the outline we made. Sweet.</p>
<p>Finally, rendering the outline:</p>
<pre><code>def format_outline(outline)
prev_level = 2
"&lt;ul&gt;" + outline.inject("") do |html, data|
level, link = data
if prev_level &lt; level
html += "&lt;ul&gt;"
elsif prev_level &gt; level
html += "&lt;/ul&gt;"
end
prev_level = level
html += "&lt;li&gt;#{link}&lt;/li&gt;"
html
end + "&lt;/ul&gt;"
end
</code></pre>
<p>This… is amazing. And terrible. Don’t drink scotch and code, kids. Or do, whatever. This takes our <code>Array</code> of <code>Array</code>s that we made with our renderer and runs <code>inject</code> over it. If our previous level is less than the new level, we indent with a <code>&lt;ul&gt;</code>, and if it’s greater, we outdent with a <code>&lt;/ul&gt;</code>. Then we render our link, and wrap the whole thing in its own set of <code>&lt;ul&gt;</code>s. Done!</p>
<p>We can call this helper method in our layout, and bam! We get automatic outlines generated for our Markdown files. Like I said, this is quick and dirty, but for a first pass, I don’t feel too bad about it. Get it done, and then get it done right.</p></content:encoded>
</item>
<item>
<title>Write better cukes with the rel attribute</title>
<link>https://steveklabnik.com/writing/write-better-cukes-with-the-rel-attribute/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/write-better-cukes-with-the-rel-attribute/</guid>
<pubDate>Tue, 20 Dec 2011 00:00:00 GMT</pubDate>
<content:encoded><p>The other day, I was working on some Cucumber features for a project, and I discovered a neat technique that helps you to write better Cucumber steps.</p>
<p>Nobody wants to be <a href="http://elabs.se/blog/15-you-re-cuking-it-wrong">cuking it wrong</a>, but what does that really mean? Here’s Jonas’ prescription:</p>
<blockquote>
<p>A step description should never contain regexen, CSS or XPath selectors, any kind of code or data structure. It should be easily understood just by reading the description.</p>
</blockquote>
<p>Great. Let’s <a href="http://www.theregister.co.uk/2007/06/25/thoughtworks_req_manage/">pop the why stack</a> a few times, shall we?</p>
<p><strong>Q1</strong>: Why do we want to have descriptions not use regexen, CSS selectors, or code? <strong>A1</strong>: To give it a simpler language.</p>
<p><strong>Q2</strong>: Why do we want it to be in a simpler language? <strong>A2</strong>: So that it’s easily understandable for stakeholders.</p>
<p><strong>Q3</strong>: Why do we want it to be easily understandable for stakeholders? <strong>A3</strong>: Because then we can share a <a href="http://domaindrivendesign.org/node/132">common language</a>.</p>
<p><strong>Q4</strong>: Why is a common language important? <strong>A4</strong>: A shared language assists in making sure our model matches the desires of our stakeholders.</p>
<p><strong>Q5</strong>: Why do we want to match the desires of our stakeholders? <strong>A5</strong>: That’s the whole reason we’re on this project in the first place!</p>
<p>Anyway, that’s what it’s really all about: developing that common language. Cukes should be written in that common language so that we can make sure we’re on track. So fine: common language. Awesome. Let’s do this.</p>
<h2>Write some cukes in common language</h2>
<p>Time to write a cuke:</p>
<pre><code>Scenario: Editing the home page
Given I'm logged in as an administrator
When I go to the home page
And I choose to edit the article
And I fill in some content
And I save it
Then I should see that content
</code></pre>
<p>Basic CMS style stuff. I’m not going to argue that this is the best cuke in the world, but it’s pretty good. What I want to do is examine some of these steps in more detail. How would you implement these steps?</p>
<pre><code>When /^I choose to edit the article$/ do
pending
end
When /^I fill in some content$/ do
pending
end
When /^I save it$/ do
pending
end
Then /^I should see that content$/ do
pending
end
</code></pre>
<p>Go ahead. Write them down somewhere. I’ll wait.</p>
<p>… done yet?</p>
<h2>Implementing a step</h2>
<p>Done? Okay! Before I show you my implementation, let’s talk about this step:</p>
<pre><code>When /^I choose to edit the article$/
</code></pre>
<p>When writing this step, I realized something. When trying to write steps like these, there’s a danger in tying them too closely to your specific HTML. It’s why many people don’t write view tests: they’re brittle. I actually like view tests, but that’s another blog post. Point is this: we know we’re going to follow a link, and we know that we want that link to go somewhere that will let us edit the article. We don’t really care <em>where</em> it is in the DOM, just that somewhere, we’ve got an ‘edit article’ link. How to pull this off?</p>
<h3>First idea: id attribute</h3>
<p>You might be thinking “I’ll give it an id attribute!” Here’s the problem with that: ids have to be unique, per page. With article editing, that might not be a problem, but it’s certainly not a general solution. So that’s out.</p>
<h3>Second time: class attribute</h3>
<p>“Okay, then just use a class. Your blog sucks.” Well, let’s check out what <a href="http://www.w3.org/TR/html5/elements.html#classes">the HTML5 spec says about classes</a>.</p>
<blockquote>
<p>Basically nothing about semantics.</p>
</blockquote>
<p>Okay, so that’s paraphrased. But still, the spec basically says some stuff about the details of implementing classes, but absolutely nothing about the semantics of a class. In practice, classes are largely used for styling purposes. We don’t want to conflate our styling with our data, so overloading class for this purpose might work, but feels kinda wrong.</p>
<h3>What about the text?</h3>
<p>We could match on the text of the link. After all, that’s what people use to determine what links to click on. The link with the text “Edit this article” lets us know that that link will let us edit a article.</p>
<p>Matching on the text is brittle, though. What happens when marketing comes through and changes the text to read “Edit my article”? Our tests break. Ugh.</p>
<p>There’s got to be a better way. Otherwise, I wouldn’t be writing this blog post.</p>
<h3>The best way: the rel attribute</h3>
<p>When doing research for <a href="http://designinghypermediaapis.com/">my book on REST</a>, I’ve been doing a lot of digging into various standards documents. And one of the most important attributes from a REST perspective is one that nobody ever talks about or uses: the <code>rel</code> attribute. From <a href="http://www.w3.org/TR/html5/links.html#attr-hyperlink-rel">the HTML5 spec</a>:</p>
<blockquote>
<p>The rel attribute on a and area elements controls what kinds of links the elements create. The attribue’s value must be a set of space-separated tokens. The allowed keywords and their meanings are defined below.</p>
</blockquote>
<p>Below? That’s <a href="http://www.w3.org/TR/html5/links.html#linkTypes">here</a>:</p>
<blockquote>
<p>The following table summarizes the link types that are defined by this specification. This table is non-normative; the actual definitions for the link types are given in the next few sections.alternate: Gives alternate representations of the current document. author: Gives a link to the current document’s author. bookmark: Gives the permalink for the nearest ancestor section.</p>
</blockquote>
<p>Hey now! Seems like we’re on to something. There’s also <a href="http://tools.ietf.org/html/rfc5988">RFC 5988</a>, “Web Linking”. <a href="http://tools.ietf.org/html/rfc5988#section-4">Section four</a> talks about Link Relation Types:</p>
<blockquote>
<p>In the simplest case, a link relation type identifies the semantics of a link. For example, a link with the relation type “copyright” indicates that the resource identified by the target IRI is a statement of the copyright terms applying to the current context IRI.Link relation types can also be used to indicate that the target resource has particular attributes, or exhibits particular behaviours; for example, a “service” link implies that the identified resource is part of a defined protocol (in this case, a service description).</p>
</blockquote>
<p>Bam! Awesome! This is exactly what we want!</p>
<h2>So how do we use rel attributes?</h2>
<p>I’ll be going into more depth about these kinds of topics in <a href="http://designinghypermediaapis.com/">my book</a>, but here’s the TL;DR:</p>
<ol>
<li>There are a set of official types. Try to use those if they’re applicable, but they’re quite general, so that’s often not the case.</li>
<li>You can put whatever else you want. Space delineated. *</li>
<li>The best way is to use a URI and then make a resource at that URI that documents the relation’s semantics.</li>
</ol>
<p>We’ll go with option two for now, for simplicity. In a real application, make it a URI.</p>
<p>(*) Technically, this isn’t true. Extension relations are <em>required</em> to be URIs, or something that can be serialized to a URI. Again, details are outside of the scope of this post.</p>
<h2>Making our link, with semantics.</h2>
<p>Here’s what a link with our newly minted relation looks like:</p>
<pre><code>&lt;a href="/articles/1/edit" rel="edit-article"&gt;Edit this article&lt;/a&gt;
</code></pre>
<p>Super simple. Just that one little attribute. Now we can write a step to match:</p>
<pre><code>When /^I choose to edit the article$/ do
find("//a[@rel='edit-article']").click
end
</code></pre>
<p>This code matches what we’d do as a person really well. “Find the link that edits an article, and click on it.” We’ve not only made the title of our step match our idea of what a person would do, but the code has followed suit. Awesome. We can move this link anywhere on the page, our test doesn’t break. We can change the text of the link, and our test doesn’t break. So cool.</p>
<h2>What about stuff that’s not links? What about data attributes?</h2>
<p>That’s what <a href="http://designinghypermediaapis.com/">my book</a> is going to talk about, sorry. These kinds of practical examples are one of the reasons I decided to write it in the first place, and I don’t want to publish all the content on my blog…</p>
<h2>Better tests through web standards</h2>
<p>Turns out that diving around in standards has some practical benefits after all, eh? Think about the relationship between your cukes, your tests, and your API clients: Cucumber, through Selenium, is an automated agent that interacts with your web service. API clients are automated agents that interact with your web service. Hmmmm…</p>
<p>If you want to know more about this, that’s what <a href="http://designinghypermediaapis.com/">my book</a> is for. I’ll be covering topics like this in depth, and explaining standards in simple language.</p>
<p>Seriously. Did you sign up for <a href="http://designinghypermediaapis.com/">my book</a> yet? ;)</p></content:encoded>
</item>
<item>
<title>Marx, anarchism, and web standards</title>
<link>https://steveklabnik.com/writing/marx-anarchism-and-web-standards/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/marx-anarchism-and-web-standards/</guid>
<pubDate>Thu, 15 Dec 2011 00:00:00 GMT</pubDate>
<content:encoded><h3>An aside for software devs</h3>
<p>You might not care about anarchism, and I can almost guarantee that you don’t care about Marx, but please bear with me. I think my point is best made by putting the web stuff at the end, so please, just read it. ;) It’ll be good for you, I swear. I’ll explain better at the end.</p>
<h2>Domain Specific Languages and information density</h2>
<p>I’ve been noticing an effect lately that’s certainly not new; it’s just come up frequently. When working within a domain that’s new to them, most people tend to not respect the density of the information being given to them, and it causes them to draw incorrect inferences.</p>
<p>For example, this tweet earlier:</p>
<blockquote>
<p>Does anybody else’s head explode when they read an “unless” statement? What good is readability if comprehension goes out the window?</p>
</blockquote>
<p>— Christopher Deutsch (@cdeutsch) December 15, 2011</p>
<blockquote></blockquote>
<p>And this response:</p>
<blockquote>
<p>@cdeutsch There are weirder idioms. @users.collect{|u| u.email} vs @users.collect(&amp;:email) :) That should weird you out more.</p>
</blockquote>
<p>— Brian P. Hogan (@bphogan) December 15, 2011</p>
<blockquote></blockquote>
<p>Now, I’m not saying that Christopher is in the wrong here, in fact, he agrees with what I’m about to say:</p>
<blockquote>
<p>@philcrissman @bphogan I’m sure I’ll get used to the “unless” pattern someday. But from a Ruby outsiders perspective it’s hard to comprehend</p>
</blockquote>
<p>— Christopher Deutsch (@cdeutsch) December 15, 2011</p>
<blockquote></blockquote>
<p>From someone new to Ruby, <code>Symbol#to_proc</code> or <code>unless</code> are hard to understand initially, and that’s because they’ve increased the density of information being conveyed. <code>unless</code> is the same as <code>if not</code> and <code>&amp;:foo</code> is the same as <code>{|a| a.foo }</code>. Both of these constructs condense something more complicated into something that is simpler, but denser.</p>
<p>You’ll note that I said ‘simpler,’ but by a certain measure, <code>&amp;:foo</code> is actually more complex. When I say <code>&amp;:foo</code> is simpler, I’m meaning for someone who’s well versed in Ruby, functional programming, or first-class functions. I have this exact background, and so for me, <code>collection.map &amp;:foo</code> is simpler and more readable than <code>collection.map {|a| a.foo }</code>. When I read the first example, I say in my head “Map foo over the collection.” You have to grok what map really is to get that sentence or the corresponding code. Whereas what (I imagine) someone who does not have this kind of background thinks when they see the second example is “Okay, so map is like an each, and for each thing in the collection, we’re going to call foo on it.” This is a totally valid interpretation, but notice how much longer it is, and how much more involved in the details it is. That’s cool, but to someone who groks map, it has a much larger amount of mental overhead, in the same way that my concise explanation causes much more thinking for someone who doesn’t grok it.</p>
<p>This happens often in education. DHH made <a href="http://news.ycombinator.com/item?id=3328427">a comment</a> about this recently that illustrates this principle, and he couches it in terms of “learnability” vs. “readability”:</p>
<blockquote>
<p>If you optimize a framework for beginners, you’re optimizing for learnability. That’s great for the first few days or even weeks of learning. But once you’re past that, it’s not so great. What you care more about is the usability of the thing once you know it. There are plenty of cases where learnability and usability are in conflict. Letting learnability win is a short-term relief.If you on the other hand optimize for usability, for making things simple and easy to use for someone who understands the basics, you’ll often end up with something that has great learnability as well. Maybe not as much as could be achieved as if that was your only goal, but plenty still.</p>
</blockquote>
<p>I think that ‘learnability’ is a pretty good shortening for ‘light density’ and ‘usability’ is decent for ‘heavy density.’ It’s the same effect, though. For the rest of this essay, I’ll be using ‘learnable’ and ‘usable’ to mean this particular usage.</p>
<p>Any time you encounter experts in a particular domain, they’ll often have fairly specific language that corresponds to that domain. This language is usually designed to be usable, not learnable. This is because they’ve already done the learning; learnability holds no utility for them. However, usability is incredibly… useful. To them. They’re working at a higher level of abstraction, and don’t want to get bogged down in details they already know well. Using learnable language would cause them to take twice as long to say things; to return to the density analogy, dense information is transferred from one person to another more quickly. If you can read a sentence a second, but that sentence is dense, you acquire much more information per second than if it was lighter.</p>
<p>This tendency means that most language that’s specific to a domain will generally trend towards the usable at the expense of the learnable. The impact this has on individuals new to the domain, however, is that of a wall. An impediment. Overcoming this obstacle requires a bit of good faith on the part of the beginner; to cross quickly over the chasm between beginner and expert, they must recognize and respect this aspect of the conversations they will invariably become a part of. When faced with a term that is used in a strange way, beginners should ask for clarification, and not start arguments over semantics they don’t yet even understand. Experts will recognize these arguments as coming from a place where concepts are not yet fully understood, and while they may recognize the need to help educate, if the newbie is being belligerent, they may just ignore them instead. Nobody wins; the signal/noise ratio has been decreased, the beginner doesn’t learn, and everyone’s time is wasted.</p>
<p>Here’s three other situations where I’ve seen this happen lately:</p>
<h2>Marx and the labor theory of value</h2>
<p>Philosophy writing is generally a great example of text that is very much usable, and not at all learnable. Some writers can still be learnable, but most are not, in my experience. One of the reasons this happens is that they introduce concepts early in a text and then utilize them later without referring back to the earlier definition. This isn’t a problem for anyone who’s thinking about taking off the training wheels or anyone who reads the entire text. The danger comes in when someone <em>not</em> versed in the entire text attempts to take portions of it out of its context.</p>
<p>Consider Marx, and <em>Capital</em>. The meaning of ‘value’ is a central concern of his writing, and indeed, entire critique of the political economy. It’s so important that the first few chapters are devoted to an (excruciatingly, frankly) detailed explanation of his thoughts on the true meaning of value. The rest of <em>Capital</em> is built on top of this: at least in my understanding of Marx, it all boils back down to that one question. And when having discussions between people who are devotees of Marx and those who come from other schools of economics, this kind of language gets in the way.</p>
<p>It also causes ideas to be misrepresented: the first time I was ever introduced to the labor theory of value, it was by a close friend who’s very libertarian. This was a few years ago, so it’s an obvious paraphrase, but he summarized it thusly: “Yeah, I mean, the labor theory of value basically says that people should be paid for however much work they put into things, so if I take six hours to make a widget and you take four, the price of a widget from you should be six yet mine should only be four, and ideas like ‘market value’ should be disregarded. It’s totally irrelevant if I suck at making widgets, I should get paid more than you anyway.” Which, to put it lightly, is a misrepresentation. While explaining the labor theory of value is outside of the scope of this post, what I will say is that to Marx, ‘value’ is something intrinsic to an object; it’s the ‘socially necessary abstract labor’ inherent to it. Talk about dense language! What a capitalist would call ‘value,’ a Marxist would call ‘price.’</p>
<p>As you can see, even just one little word, ‘value,’ can be quite dense! Can you imagine a discussion intended to be ‘learnable’ to outsiders about what’s meant? Imagine the expansion: ‘value’ -&gt; ‘socially necessary abstract labor’ -&gt; … Marx is already long enough; Capital would be thousands of pages! Yet to a beginner who flips to Chapter 12, they’ll read a sentence that contains ‘value’ and draw poor conclusions! They wouldn’t even realize they’re making a mistake, I mean, how could five letters be misinterpreted?</p>
<p>Furthermore, people who haven’t read Marx don’t generally draw distinctions between his critique of capitalism and his solution: communism. This is annoying when trying to explain to people that I love his critique, but am critical of his answers to its problems; they perceive this weakness in his answer as a weakness in his description of the problem. Furthermore, they then say “but I thought you call yourself a communist?” and I respond with “sure; the issues I have are with the dictatorship of the proletariat, not with the general idea of communism” and then their eyes glaze over and they change the subject. Information density claims another victim…</p>
<p>Oh, and a great example of Marxist economics in a usable form <a href="http://people.wku.edu/jan.garrett/303/marxecon.htm">is here</a>.</p>
<h2>Anarchism and ‘anarcho’-capitalists</h2>
<p>Arguments often boil down to these kinds of questions of definition, but one place where I see it happen almost <em>constantly</em> is amongst anarchists. I mean, from the outset, anarchists have to battle against the general definition of ‘chaos and disorder’ versus the domain-specific ‘without rulers.’ Within that, ‘rulers’ in anarchism is fraught with the same sort of questions that ‘value’ has for Marx. The prime example of this are the terribly misguided ‘anarcho’-capitalists, better described as ‘voluntaryists.’</p>
<p>Here’s the deal: ancaps lack an understanding of the vast majority of historical anarchist thought, and so try to appropriate the term ‘anarchism’ to describe their philosophy which is decidedly not anarchist. The ones who do have started using ‘voluntaryist’ to describe themselves, which is a great example of using information density to mislead, but that’s a whole separate rant. Here’s the 411, from <a href="http://infoshop.org/page/AnarchistFAQSectionF1">the Anarchist FAQ</a>, which has its own things to say about density when it comes to the language specific to political theory discussions:</p>
<blockquote>
<p>“Anarcho”-capitalists claim to be anarchists because they say that they oppose government. As noted in the last section, they use a dictionary definition of anarchism. However, this fails to appreciate that anarchism is a political theory. As dictionaries are rarely politically sophisticated things, this means that they fail to recognise that anarchism is more than just opposition to government, it is also marked a opposition to capitalism (i.e. exploitation and private property). Thus, opposition to government is a necessary but not sufficient condition for being an anarchist – you also need to be opposed to exploitation and capitalist private property. As “anarcho”-capitalists do not consider interest, rent and profits (i.e. capitalism) to be exploitative nor oppose capitalist property rights, they are not anarchists.Part of the problem is that Marxists, like many academics, also tend to assert that anarchists are simply against the state. It is significant that both Marxists and “anarcho”-capitalists tend to define anarchism as purely opposition to government. This is no co-incidence, as both seek to exclude anarchism from its place in the wider socialist movement. This makes perfect sense from the Marxist perspective as it allows them to present their ideology as the only serious anti-capitalist one around (not to mention associating anarchism with “anarcho”-capitalism is an excellent way of discrediting our ideas in the wider radical movement). It should go without saying that this is an obvious and serious misrepresentation of the anarchist position as even a superficial glance at anarchist theory and history shows that no anarchist limited their critique of society simply at the state. So while academics and Marxists seem aware of the anarchist opposition to the state, they usually fail to grasp the anarchist critique applies to all other authoritarian social institutions and how it fits into the overall anarchist analysis and struggle. They seem to think the anarchist condemnation of capitalist private property, patriarchy and so forth are somehow superfluous additions rather than a logical position which reflects the core of anarchism.</p>
</blockquote>
<p>Part of the problem with the second half of this quote is that I’m such an ‘expert’ on this kind of language that I don’t even know if it’ll make sense to you; without the kind of background reading in socialist political philosophies, it might just be gibberish. At least, the first paragraph should be pretty straightforward, and you can take the second as an example of this kind of language.</p>
<p>Giving you a short explanation of why anarchists are against Capitalism is a great example in and of itself of domain specific language and density. Here:</p>
<blockquote>
<p>Property is theft.</p>
</blockquote>
<p>This is a quote by <a href="http://en.wikipedia.org/wiki/Pierre-Joseph_Proudhon">Proudhon</a>, the first person to call himself an anarchist. Let’s unpack the first few layers of this statement:</p>
<p>Property. Here’s <a href="http://en.wikipedia.org/wiki/Private_property">Wikipedia’s explanation</a>:</p>
<blockquote>
<p>Private property is the right of persons and firms to obtain, own, control, employ, dispose of, and bequeath land, capital, and other forms of property. Private property is distinguishable from public property, which refers to assets owned by a state, community or government rather than by individuals or a business entity. Private property emerged as the dominant form of property in the means of production and land during the Industrial Revolution in the early 18th century, displacing feudal property, guilds, cottage industry and craft production, which were based on ownership of the tools for production by individual laborers or guilds of craftspeople.Marxists and socialists distinguish between “private property” and “personal property”, defining the former as the means of production in reference to private enterprise based on socialized production and wage labor; and the latter as consumer goods or goods produced by an individual.</p>
</blockquote>
<p>Whew! There’s a few things to note here: Captialists like to pretend that capitalism is synonymous with ‘trade,’ and not something that started in the 1800s. Likewise, that private property rights are something that has always existed. However, as this alludes to, there are many different kinds of property rights that have existed at different places and times.</p>
<p>So in ‘property is theft,’ Proudhon is referring to private ownership of the ‘means of production.’ Let’s expand that. Again, <a href="http://en.wikipedia.org/wiki/Means_of_production">Wikipedia</a>:</p>
<blockquote>
<p>Means of production refers to physical, non-human inputs used in production—the factories, machines, and tools used to produce wealth — along with both infrastructural capital and natural capital. This includes the classical factors of production minus financial capital and minus human capital. They include two broad categories of objects: instruments of labour (tools, factories, infrastructure, etc.) and subjects of labour (natural resources and raw materials). People operate on the subjects of labour, using the instruments of labour, to create a product; or, stated another way, labour acting on the means of production creates a product. When used in the broad sense, the “means of production” includes the “means of distribution” which includes stores, banks, and railroads. The term can be simply and picturesquely described in an agrarian society as the soil and the shovel; in an industrial society, the mines and the factories.</p>
</blockquote>
<p>We could continue to discuss how to distinguish between this ‘private property’ and ‘possessions,’ which anarchists are <em>not</em> against, but I’m just trying to demonstrate that this word ‘property’ is incredibly complex.</p>
<p>Okay, so Proudhon claims that ‘owning the physical inputs used in factories used to produce wealth is theft.’ I could expand on ‘theft,’ but really, I think my point about density is made. For more on this, see <a href="http://anarchism.pageabode.com/afaq/secB3.html">Why are anarchists against private property?</a>.</p>
<h2>Web standards</h2>
<p>Web standards are another great example of a domain that has a lot of very specific language. And one that people often think they can grab random chunks out of and quote without fully understanding the rest of the context.</p>
<p>This is going on right now <a href="https://github.com/rails/rails/pull/505">with Rails</a>. There’s a discussion about if the <code>PATCH</code> HTTP verb should get support, and if it should be the verb that matches to the <code>update</code> action or not. It’s ended up in a discussion about the semantics of <code>PUT</code>, which has resulted in a lot of random quoting of standards documents by myself and others. Here’s some running commentary on some of the comments. It’s impossible to do this without it becoming semi-personal, so let me just say upfront that I think everyone is participating honestly in this discussion, but I think it’s a great example of people who aren’t familiar with a domain jumping in and drawing incorrect conclusions.</p>
<p>First up, benatkin comments:</p>
<blockquote>
<p>I googled for HTTP verbs and clicked the first result and PATCH isn’t listed.http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.htmlWhere is it?</p>
</blockquote>
<p>Anyone participating in a discussion about <code>PATCH</code> should be reasonably familiar with <code>PATCH</code>. Learning where <code>PATCH</code> is defined is as simple as <a href="https://www.google.com/#hl=en&amp;q=http+patch">Googling HTTP PATCH</a>, which shows it being defined in <a href="http://tools.ietf.org/html/rfc5789">RFC 5879</a>. With that said, this is a good example of asking for clarification, and not immediately progressing into an argumentatitive “It’s not in HTTP 1.1, so it’s bullshit!” style of learning where <code>PATCH</code> is being defined.</p>
<p>Of course, the thread starts to dissolve later, when @stevegraham mentions:</p>
<blockquote>
<p>i must be the only person in the world that disagrees with “PUT requires a complete replacement”, as per RFC2616 “HTTP/1.1 does not define how a PUT method affects the state of an origin server”</p>
</blockquote>
<p>Now, Steve is an awesome guy, but he’s a bit misguided in this case. This is a great example of drawing an incorrect conclusion based on one sentence out of context. He’s not <em>wrong</em> in a strict sense, <a href="http://www.ietf.org/rfc/rfc2616.txt">RFC2616</a> does contain that line. However, this is because <a href="http://www.imc.org/atom-protocol/mail-archive/msg05425.html">HTTP defines the semantics of communication</a>, and the semantics are ‘idempotent creation or replacement.’ The fact that HTTP does not define how PUT affects state is irrelevant, an entire representation is needed for idempotency reasons. <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6">PUT’s definition</a> also says pretty clearly:</p>
<blockquote>
<p>The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.</p>
</blockquote>
<p>“The enclosed entity be stored” is pretty straightforward: it needs an entity, not a portion of an entity. Furthermore, how is a server going to create a resource without a complete representation if the resource didn’t already exist?</p>
<p>In this case, the standard’s organization also doesn’t help: if you just read the section titled <code>PUT</code>, you wouldn’t get the full understanding, since the fact that it’s safe and idempotent is mentioned above in the section regarding those two things. I’m not sure why those aspects aren’t in each definition, and are in a different section above, but the point is that you need to consider the full document in its entirety to understand the semantics of <code>PUT</code>. Steve is only reading one section and then extrapolating from there.</p>
<p>There’s a lot more in that pull request, but one more example: Konstantin points out that Rails supports a lot of things that aren’t standards:</p>
<blockquote>
<p>You mean Rails should not support proposed standards like, say, Cookies?</p>
</blockquote>
<p>Yep. Did you know that? Cookies aren’t actually a standard, just a proposed one.</p>
<p>Anyway, I also want to say this: I am not a perfect interpreter of these things either. I often change my opinions after learning more things, and I think this is a good thing. There’s nothing the matter with being wrong; it’s how you handle it that matters. The discussion in this thread continues. RFCs are also not perfect, and do have wiggle-room; but it’s important that agreements are followed. Amongst people who discuss REST and HTTP, the fact that PUT requires a full representation is not controversial; it’s simply understood as true.</p>
<h2>It’s good for you!</h2>
<p>I don’t want to turn anyone off from learning new things; exploring new domains is a great path towards personal growth. I’ve said a few times that I think more programmers should read Marx, and it’s because I think this experience of jumping into the deep end of a new set of language that you don’t fully understand is a tremendous educational experience. But to truly learn, an open mind and open ears are crucial. Making arguments on semantics doesn’t work if you don’t understand the context and semantics of words, as words change significantly when placed in different surroundings.</p></content:encoded>
</item>
<item>
<title>Fast Rails tests with CanCan</title>
<link>https://steveklabnik.com/writing/fast-rails-tests-with-cancan/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/fast-rails-tests-with-cancan/</guid>
<pubDate>Mon, 12 Dec 2011 00:00:00 GMT</pubDate>
<content:encoded><p>If you haven’t used it, <a href="https://github.com/ryanb/cancan">CanCan</a> is a great library for Rails that handles authorization for you. Its calling card is simplicity; just do this:</p>
<pre><code>class Ability
include CanCan::Ability
def initialize(user)
user ||= User.new
if user.is? :paying_customer
can :show, Article
else
can :show, Article, :free =&gt; true
cannot :show, Article, :free =&gt; false
end
end
end
</code></pre>
<p>And then in a controller:</p>
<pre><code>def show
@article = Article.find(params[:id])
authorize! :read, @article
end
</code></pre>
<p>Super simple! However, as simple as CanCan is, if you want to keep your test times super low, there’s a few things that you should know about.</p>
<h2>Incidental coupling: the silent killer</h2>
<p>The biggest problem with CanCan from the Fast Tests perspective is that we’ve got coupling with class names. In order to test our Ability class, we also have to load User and Article. Let’s try it without concern for how isolated our tests are:</p>
<pre><code>require "cancan/matchers"
require "spec_helper"
describe Ability do
let(:subject) { Ability.new(user) }
let(:free_article) { Article.new(:free =&gt; true) }
let(:paid_article) { Article.new(:free =&gt; false) }
context "random people" do
let(:user) { nil }
it "can see free Articles" do
subject.can?(:show, free_article).should be_true
end
it "cannot see non-free Articles" do
subject.can?(:show, paid_article).should be_false
end
end
context "paying users" do
let(:user) do
User.new.tap do |u|
u.roles &lt;&lt; :paying_customer
end
end
it "can see free Articles" do
subject.can?(:show, free_article).should be_true
end
it "can see non-free Articles" do
subject.can?(:show, paid_article).should be_true
end
end
end
</code></pre>
<p>A few notes about testing with CanCan: I like to organize them with contexts for the different kinds of users, and then what their abilities are below. It gives you a really nice way of setting up those <code>let</code> blocks, and our tests read real nicely.</p>
<p>Anyway, so yeah. This loads up all of our models, connects to the DB, etc. This is unfortunate. It’d be nice if we didn’t have to load up <code>User</code> and <code>Article</code>. Unfortunately, there’s no real way around it in our <code>Ability</code>, I mean, if you want them to read all Articles, you have to pass in <code>Article</code>… hmm. What about this?</p>
<pre><code>let(:free_article) { double(:class =&gt; Article, :free =&gt; true) }
let(:paid_article) { double(:class =&gt; Article, :free =&gt; false) }
</code></pre>
<p>Turns out this works just as well. CanCan reflects on the class that you pass in, so if we just give our double the right class, it’s all gravy. Now we’re not loading <code>Article</code>, but what about <code>User</code>?</p>
<p>The key is in the <code>||=</code>. If we pass in a <code>User</code>, we won’t call <code>User.new</code>. So let’s stub that:</p>
<pre><code>let(:user) { double(:is? =&gt; false) }
let(:user) do
double.tap do |u| #tap not strictly needed in this example, but if you want multiple roles...
u.stub(:is?).with(:paying_customer).and_return(true)
end
end
</code></pre>
<p>Sweet! The first one is a random user: all their <code>is?</code> calls should be false. The second is our user who’s paid up; we need to stub out their role properly, but that’s about it.</p>
<h2>It’s that easy</h2>
<p>With some dillegence, isolating your tests isn’t hard, and works really well. I have a few more tests in the example I took this from, and 14 examples still take about a third of a second. Not too shabby!</p>
<p>Happy authorizing!</p>
<p>Oh, and the final spec, for reference:</p>
<pre><code>require "cancan/matchers"
require "app/models/ability"
describe Ability do
let(:subject) { Ability.new(user) }
let(:free_article) { double(:class =&gt; Article, :free =&gt; true ) }
let(:paid_article) { double(:class =&gt; Article, :free =&gt; false) }
context "random people" do
let(:user) { double(:is? =&gt; false) }
it "can see free Articles" do
subject.can?(:show, free_article).should be_true
end
it "cannot see non-free Articles" do
subject.can?(:show, paid_article).should be_false
end
end
context "users" do
let(:user) do
double.tap do |u|
u.stub(:is?).with(:user).and_return(true)
end
end
it "can see free Articles" do
subject.can?(:show, free_article).should be_true
end
it "can see non-free Articles" do
subject.can?(:show, paid_article).should be_true
end
end
end
</code></pre></content:encoded>
</item>
<item>
<title>Book review: New Programmer's Survival Manual</title>
<link>https://steveklabnik.com/writing/book-review-new-programmers-survival-manual/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/book-review-new-programmers-survival-manual/</guid>
<pubDate>Mon, 28 Nov 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Disclaimer: I do have a small credit in this book. A long time ago, the author mentioned that he was looking for people to help out, so I stepped up. It was only participating in discussions like “What do you think about version control’s use in industry?”; I didn’t help with any actual writing, and yesterday was my first time reading virtually all the final content. No monetary interest, of course.</p>
<h3>Lots of new books lately</h3>
<p>I’ve been doing a lot of reading over the last few weeks, some of it technical, some of it non. I’ve been meaning to type up a few book reviews, and yesterday I finished reading my copy of Josh Carter’s <a href="http://pragprog.com/book/jcdeg/new-programmer-s-survival-manual">“New Programmer’s Survival Manual,”</a> published by PragProg. Since that’s a bit lighter material than Spinoza, I figured I’d write it up first.</p>
<p>The <em>Survival Manual</em> is intended to be the book you should buy after you graduate from college, but before you get a job. It’s a bunch of tips, arranged by topic, that give you a taste of what ‘real world’ programming is like, as well as what working life is about. I think that it accomplishes this goal wonderfully, but if you’d like to hear specifics, read on.</p>
<h2>The Good</h2>
<p>One of my roommates is not particularly technical. She picked the book up, and read the introduction and the first tip. She then said something that, after reading all of it, I agree with 100%. “I like this guy’s writing style. It’s very… readable.” Josh’s prose feels… comfortable. Like you’re talking to a down-to-earth friend, or co-worker. Which is great, given the topic. I really enjoyed the actual <em>reading</em> of this book. Especially in technical books, this is a great quality to have, but I guess this book isn’t exactly technical; it’s much more about soft skills than hard.</p>
<p>I also really enjoyed the cookbook-style organization. Basically, the book is one big ball of tips that you can read individually. They’re grouped together with three or four others that are on the same topic, but each stands alone, and they cross-reference each other when appropriate. While I sat down and read it front to back, I’d imagine it’s pretty easy to cherry-pick certain things and read it in a more non-linear fashion.</p>
<p>I also feel that ‘pragmatic’ is a good description of most of the content. In particular, as much as we all love DVCS, the book frankly states that most places still use centralized version control, which is totally true. He makes sure to carefully not be particularly dogmatic about the specifics of anything, focusing on strategy more than tactics. For example, “Get good with your editor, make sure it can work effectively without using the mouse, for speed,” not “zomg emacs r teh suck, use vim.”</p>
<p>Finally, Josh links to copious other works, and provides references for all of it. Some stuff is just general internet culture, but the bibliography alone would assist any programmer in finding some useful primary materials to read. Here’s the first six: <em>SICP</em>, <em>Getting Things Done</em>, <em>Extreme Programming Explained</em>, <em>TDD By Example</em>, <em>The Mythical Man-Month</em>, and <em>Foundations of Object Oriented Languages: Types and Semantics</em>. Oh, and just because there’s some XP/Agile stuff in those particular books, don’t think it’s slanted towards that: waterfall is explained as well. Just so happens those books were first.</p>
<h2>The Bad</h2>
<p>There’s not that much bad that I can say about this book, really. There are a few small things, though:</p>
<p>There’s a ‘white belt brown belt black belt’ metaphor that’s supposed to indicate a difficulty of each tip. Given that there’s only two black belt tips, I didn’t feel that three levels were really needed. I also just thought that there wasn’t a huge difference between white and brown, either, so I’m not sure that it really adds any utility.</p>
<p>Because this book is fairly high-level, it might be hard to actually <em>apply</em> these lessons. This might be more of a problem with readers than with the actual book, but I can’t count how many times I (and others) have said “Of course, write descriptive variable names!!1” and then typed <code>x = 5</code>. It’s just the way things are. And since this book is essentially a collection of advice at that high level, I can see someone reading this book, and then just simply not following the advice. I’m not sure if this is something Josh could have fixed, or if it’s inherent in this kind of book. Also, it won’t affect a diligent reader. It is something to think about, though.</p>
<h2>Summary: If you just graduated, buy this book</h2>
<p>This book is totally worth it. The style reminds me of the seminal <em>Pragmatic Programmers</em>. If you’re a freshly minted graduate, put this book at the top of your reading list, then also grab a copy of all the books in the bibliography to read later.</p></content:encoded>
</item>
<item>
<title>Burnout</title>
<link>https://steveklabnik.com/writing/burnout/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/burnout/</guid>
<pubDate>Thu, 17 Nov 2011 00:00:00 GMT</pubDate>
<content:encoded><p>I have a confession to make: Other than one little stint at RubyC, I haven’t really written code in almost a month. And I haven’t really written a ‘real’ blog post in almost six weeks.</p>
<p>It happens. I was hitting it really hard there for a while, and the combination of stresses from travel, speaking, writing, conference parties, and everything else was really getting to me. I don’t think that I was burned out <em>yet</em>, but I decided to explicitly take the last two weeks off from doing almost anything to prevent myself from getting to that point.</p>
<p>This doesn’t mean that I’m going to <em>stop</em> doing any of these things. I’d be lying if I tried to say that I’d be reintroducing long-term balance into my life. That balance never really existed. I’ve always been one to throw myself at things 100%. Balance is something I don’t do particularly well.</p>
<p>One thing I am going to be working on, though, is this: Hackety and Shoes stress me out. Like a lot. This causes me to avoid working on them, which makes them not improve so quickly, which makes me stress about it worse. I’m not really sure how to fix that one… but at some point, I’m going to have to confront that.</p>
<p>Anyway, hopefully I’ll be back at it soon. Those of you waiting on some of my projects, my bad. Things will still be slow for a bit. I’m going to D.C. this weekend to teach kids with Hackety and Lego Mindstorms, so maybe that’ll get me out of this funk. Or maybe it’ll delay it another couple of days.</p>
<p>But I’ll be around again soon.</p></content:encoded>
</item>
<item>
<title>Your startup is not a platform</title>
<link>https://steveklabnik.com/writing/your-startup-is-not-a-platform/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/your-startup-is-not-a-platform/</guid>
<pubDate>Tue, 25 Oct 2011 00:00:00 GMT</pubDate>
<content:encoded><p>No, seriously. This is a public service announcment to all MBAs who want to do startups: ‘platform’ does not mean ‘web site.’</p>
<p>I know that ‘platform’ is a hot thing right now. But seriously, the term has become basically useless. It’s almost to the point that I just pretend you don’t even say it.</p>
<p>A platform is primarily a B2B play. It can be B2C too, maybe, but largely, to have a platform, you need to enable other people to build something. Usually, that’s another business. But if your website does not enable others to build something, or you’re not white labeling it in some way, you’re not a platform.</p>
<p>A platform basically needs an API. APIs are the primary way that two pieces of software connect to one another. If your web service doesn’t have an API, it’s not a platform. White labeling being an exception.</p>
<p>I’d make this rant longer, but I really need a good platform that addresses my market segment. Ideally, it’ll enable some win-win synergies with other channel partners and grow the pie for both of us. Until then, I really won’t find a good solution for this real pain I feel as a content producer.</p></content:encoded>
</item>
<item>
<title>For science: I'm killing my cell phone</title>
<link>https://steveklabnik.com/writing/for-science-im-killing-my-cell-phone/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/for-science-im-killing-my-cell-phone/</guid>
<pubDate>Wed, 19 Oct 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Yep. Doin’ it. This morning I swung by the Sprint store and picked up a 4G hotspot. Gonna transfer my number to Google Voice this evening.</p>
<h2>lolwhut?</h2>
<p>Yep. I’m terribly addicted to my phone right now. This is fine, but I’m always interested in improving myself. I have a few friends who’ve done similar things, and really enjoyed it. So I’m giving it a solid 6 months. Sometimes, experiments work, and sometimes they don’t. I’m doing this for fun more than anything, and so it might not work out. Google Maps is damn handy.</p>
<h2>So I can’t call you?</h2>
<p>Naw, it just means there might be a tad more latency. I’m on my computer all the effing time anyway, so I doubt that you’ll really notice. If someone calls when I’m not at my computer, Google will just take a recording and email me the transcript. No biggie.</p>
<h2>Yeah, well, have fun with that!</h2>
<p>I will. And I’ll keep you all posted on how it goes. I plan to do a retrospective monthly.</p>
<p>EDIT: I had to change my number because Google Voice wouldn’t transfer my old one. So get a hold of me some other way, and I’ll give you the new one.</p></content:encoded>
</item>
<item>
<title>Matz is nice so we are nice</title>
<link>https://steveklabnik.com/writing/matz-is-nice-so-we-are-nice/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/matz-is-nice-so-we-are-nice/</guid>
<pubDate>Wed, 19 Oct 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Rubyists have a saying, or at least, we used to. “Matz is nice so we are nice.” This has sort of been lost over the years… some people who have been around Ruby for a long time still say this, but it’s something that’s gone in and out of popularity.</p>
<h2>Nice should be the default</h2>
<p>Most of being nice, in a software context, is about being considerate. Not unnecessarily trash talking other people’s stuff. Not going out of your way to be a jerk. Try to be helpful. Treat others how you want to be treated.</p>
<h2>Sometimes, the truth hurts</h2>
<p>Being considerate of others, however, can only go so far. While a lot of what we do is subjective, a lot of is objective, too. If you’re building a new project, not saying “this is why we’re awesome” is silly. Old code is often bad. Other people’s code is often bad. The key in “not unnecessarily trash talking’ is the <em>neccesary</em> part. There’s a difference between”My library is better" and “My library is better and their library sucks.”</p>
<h2>The last thing I’ll say about RVM Drama</h2>
<p>After talking to a few people, I guess I committed a bit of the sin I was trying to rail against: I got some feedback that it was thought that I was villainizing Sam, and a little easy on Wayne. This is natural, because I’m friends with Wayne and I don’t really know Sam, but in the interest of being nice myself, I just want to say this: I think that all that drama last week was largely a perfect storm of a few different factors, and I don’t think that Sam set out to hurt anyone specifically. If and when I ever have an issue with RVM, I’m totally going to try out rbenv. I like having options. I like there being multiple options. TMTOWTDI. And Sam builds a ton of other software that I really enjoy, like Sprockets, and pow. There’s a big difference between “that guy is an asshole” and “I think that that situation could have been handled better.”</p>
<p>Let’s all build lots of cool software together.</p></content:encoded>
</item>
<item>
<title>Rubinius is awesome</title>
<link>https://steveklabnik.com/writing/rubinius-is-awesome/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/rubinius-is-awesome/</guid>
<pubDate>Tue, 04 Oct 2011 00:00:00 GMT</pubDate>
<content:encoded><p>You walk into work tomorrow morning, and your boss says this:</p>
<blockquote>
<p>Boss: Hey, we’re gonna need to go ahead and have you implement require_relative in rubinius. We have some new servers coming in, and we’ll be running it in production, but we use some 1.9.2 features of MRI that haven’t been implemented yet. So if you could just go ahead and get to implementing that, that would be terrific, OK?</p>
</blockquote>
<p>Wat do?</p>
<p>(Disregard that your boss would never say this. I kinda wish some of mine would…)</p>
<blockquote>
<p>You: Excuse me, I believe you have my stapler…</p>
</blockquote>
<p>Err, sorry.</p>
<blockquote>
<p>You: No problem, boss. Rubinius is mostly just Ruby, and it has a really helpful core team. I’m sure it won’t be a big deal.</p>
</blockquote>
<p>Would you say this? Well, you should. I’m not gonna lie and say that <em>every</em> feature is super simple to implement, or rbx-2.0 would be out already, but for a language interpreter, Rubinius is one of the easiest I’ve ever tried to get into. I actually did implement require_relative last night, and it took me about an hour and a half.</p>
<p>This is my story.</p>
<p>(note: dramatazation. Actual wording made unneccesarily silly to protect the innocent. please do try this at home)</p>
<pre><code>$ ssh stevessecretserver
$ tmux attach
&lt;brings up irssi, I'm already in #rubinius on Freenode&gt;
#rubinius: steveklabnik: hey, I need require\_relative to run my app. How hard
would you think that is to implement?
#rubinius: evan: Naw, man, it'd take like fifteen minutes. Go check out
http://rubygems.org/gems/rbx-require-relative, it's alredy basically got what
you want
#rubinius: steveklabnik: sweet.
#rubinius: brixen: just grep for 'def require'
#rubinius: brixen: and run bin/mspec -tx19 core/kernel/require\_relative to
run the RubySpecs for it.
</code></pre>
<p>Armed with this knowledge, I found where require is defined in rbx:</p>
<pre><code>def require(name)
Rubinius::CodeLoader.require name
end
module_function :require
</code></pre>
<p>Of course, I had to look at CodeLoader:</p>
<pre><code># Searches for and loads Ruby source files and shared library extension
# files. See CodeLoader.require for the rest of Kernel#require
# functionality.
def require
Rubinius.synchronize(self) do
return false unless resolve_require_path
return false if CodeLoader.loading? @load_path
if @type == :ruby
CodeLoader.loading @load_path
end
end
if @type == :ruby
load_file
else
load_library
end
return @type
end
</code></pre>
<p>Hmm, see <code>CodeLoader.require</code>. There are codeloader18 and codeloader19 files…</p>
<pre><code>module Rubinius
class CodeLoader
# Searches $LOAD_PATH for a file named +name+. Does not append any file
# extension to +name+ while searching. Used by #load to resolve the name
# to a full path to load. Also used by #require when the file extension is
# provided.
def search_load_path(name, loading)
$LOAD_PATH.each do |dir|
path = "#{dir}/#{name}"
return path if loadable? path
end
return name if loading and loadable? "./#{name}"
return nil
end
end
end
</code></pre>
<p>Okay. So we define specific stuff into the 19 file. There’s also a kernel19. So I worked this up:</p>
<pre><code># Attempt to load the given file, returning true if successful. Works just
# like Kernel#require, except that it searches relative to the current
# directory.
#
def require_relative(name)
Rubinius::CodeLoader.require_relative(name)
end
module_function :require_relative
</code></pre>
<pre><code># requires files relative to the current directory. We do one interesting
# check to make sure it's not called inside of an eval.
def self.require_relative(name)
scope = Rubinius::StaticScope.of_sender
script = scope.current_script
if script
require File.join(File.dirname(script.data_path), name)
else
raise LoadError.new "Something is wrong in trying to get relative path"
end
end
</code></pre>
<p>But then… running the specs gives me bunches of errors. What gives?</p>
<pre><code>&lt;back in irc&gt;
#rubinius: steveklabnik: yo, my specs are failing, and I'm a n00b. Halp
pl0x?
#rubinius: brixen: oh, yeah, well, the StaticScope is the current scope,
and you're down another method, so that's _totally_ gonna screw up
your paths. Just get it above and pass it down. Oh, and File.join is
dumb, File.expand_path that shit!
#rubinius: steveklabnik: words.
</code></pre>
<p>So then I made this one little change:</p>
<pre><code># Attempt to load the given file, returning true if successful. Works just
# like Kernel#require, except that it searches relative to the current
# directory.
#
def require_relative(name)
scope = Rubinius::StaticScope.of_sender
Rubinius::CodeLoader.require_relative(name, scope)
end
module_function :require_relative
</code></pre>
<pre><code># requires files relative to the current directory. We do one interesting
# check to make sure it's not called inside of an eval.
def self.require_relative(name, scope)
script = scope.current_script
if script
require File.expand_path(name, File.dirname(script.data_path))
else
raise LoadError.new "Something is wrong in trying to get relative path"
end
end
</code></pre>
<pre><code>$ bin/mspec -tx19 core/kernel/require_relativerubinius 2.0.0dev (1.9.2 cade3517 yyyy-mm-dd JI) [x86_64-apple-darwin10.8.0].....................Finished in 0.191995 seconds1 file, 20 examples, 47 expectations, 0 failures, 0 errors
</code></pre>
<p>One more pull request, and I’ve got a patch in on rbx. I had one a few months ago, but this one was way cooler.</p>
<h2>In conclusion</h2>
<p>Even if I didn’t have some code to look at, because Rubinius is just Ruby, it was really easy to dive in and check out the code, because it’s all Ruby. Seriously. Rubinius is just Ruby. Just Ruby. Ruby.</p>
<p>So dive in and add some features today! If having an awesome alternate Ruby implementation isn’t enough for you, the rbx team will bribe you with special stickers for committers, and sweet black-on-black tshirts for people with 10 commits or more!</p></content:encoded>
</item>
<item>
<title>GitHub is anarchy for programmers</title>
<link>https://steveklabnik.com/writing/github-is-anarchy-for-programmers/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/github-is-anarchy-for-programmers/</guid>
<pubDate>Sat, 01 Oct 2011 00:00:00 GMT</pubDate>
<content:encoded><p>I finally got to see Zach Holman give his talk, <a href="http://speakerdeck.com/u/holman/p/how-github-uses-github-to-build-github">How GitHub Uses GitHub to Build GitHub</a>. It was awesome. Zach is a great speaker, and the slides don’t do it justice. I highly recommend catching it on video or in person sometime.</p>
<p>Anyway, there’s something that I found really interesting about GitHub’s ‘process’: it’s basically anarchism, applied to software development.</p>
<p>If you’re not familliar with anarchism, check out <a href="http://truth-reason-liberty.blogspot.com/p/what-i-believe-in.html">“What I believe in”</a>. It’s a pretty decent introduction to a complicated topic. Ultimately, anarchism is fairly simple, but it requires dismantling a <em>lot</em> of beliefs that Americans in particular hold about society, power, and control. We’re quite happy in this country to be slaves to the rich, but that’s another blog post. ;)</p>
<p>Another excellent post on that blog is <a href="http://truth-reason-liberty.blogspot.com/2010/11/digression-on-what-it-means-to-be.html">"A Digression On What It Means to Be An Anarchist</a>. I’m going to use it as a framework to point out my comparison to GitHub. The first sentence is stolen from TFA, but the second sentence is my GitHub comparison.</p>
<h2>What anarchists oppose</h2>
<ul>
<li>Hierarchy – anarchists oppose domination of one person or group of people by another as detrimental to human society. GitHub has built a very flat process, where “some days the CEO ships more code than I do.”</li>
<li>Authority – all forms of authority must bear a heavy burden of proof in order to demonstrate their legitimacy and necessity. GitHub has hired people that have built awesome things, and have shown themselves to be a trusted authority in whatever kind of software that they build. Nobody needs to question if anyone is in the position to deploy or review some code.</li>
<li>The state – centralised rule of a set geographical area (country) or people (nation) by a government of elites is inherently illegitimate. Okay, this has nothing to do with GitHub all together. All abstractions are leaky here. :) The same goes for Capitalism and State Socialism too.</li>
<li>Nationalism and fascism – these are but the worst forms of the state, gaining the loyalty of the people with strong, often brutal discipline and by developing an almost religious, fevered love of the state and the rulers in the form of patriotism. GitHub actively uses multiple programming languages in production, and while it came out of the Ruby world, they don’t really run around pitching “Ruby is awesome and the only thing you should ever use.”</li>
<li>Discrimination – nobody should be excluded or discriminated against based on nothing more than their gender, ethnicity, sexuality, background, or beliefs. I don’t know everyone at GitHub, but it’s a pretty diverse crew overall. This is hard for any tech company, but they’re doing pretty well on the race/sex thing.</li>
</ul>
<h2>What anarchists stand for</h2>
<ul>
<li>Liberty – all people should be free to live their life as they see fit, without rules and laws passed from above that serve no purpose other than control and domination, as long as they are not infringing the right of anybody else to the same. See Homan’s talk. That’s pretty much how they work.</li>
<li>Equality – as stated above, nobody should face discrimination because of their gender, ethnicity, sexuality, background, or beliefs. See above comment about GitHub’s varied devs.</li>
<li>Community – human civilisation evolved, from its primitive roots, through the priciple of Mutual Aid. GitHub has a lot of elements of mutal aid in their process, where they mention some people in some pull requests, work together on features, and generally high five until the code is running in production.</li>
<li>Solidarity – humanity is divided only between the rulers and the ruled. Other divisions, those which bring about sexism, racism, heterosexism, and other bigotries, are promoted by the ruling class in order to divide their subjects and keep them under control. GitHub holds tons of events, buys everyone lots of beer, and just generally encourages lots of friendly interactions between everyone. CodeConf was a great group of diverse people hanging out and having a good time.</li>
</ul>
<h2>How anarchy would work</h2>
<ul>
<li>Self-management – groups, such as workforces or local communities, would be free to operate and govern themselves free of any higher authority. GitHub has no deadlines, no prioritization meetings, and 100% “do what you want” time.</li>
<li>Free association – all individuals would be free to live where they wanted and associate with who they chose. Zach mentioned in his talk that different people work on different parts of the site, and it’s really based on what people are interested in and what they’ve done in the past.</li>
<li>Voluntary federation – instead of the state, where indivudal communities and groups of people are bound together by the coercive force of a central authority, local communities and workers collectives can choose for themselves which other communities or collectives to associate with. Sort of like the free association comment, people get together and work on what they want to, and these groups are organic.</li>
<li>Direct democracy – unlike in parliamentary democracy, these spokespeople would be just that, elected not to a position of authority but to voice decisions that remain in the hands of the people, as in trade union and workers council structures. Anybody can comment on a pull request, add some commits, put in their two cents, and be heard.</li>
<li>Mutual Aid – in participatory communities and workers collectives, Mutual Aid is a central principle. Easily summed up with the phrase “from each according to his ability, to each according to his need,” this boils down to voluntary cooperation, fair distribution of resources, and help and support to those who need it within a local community. As mentioned before, everybody helps each other out at GitHub to get things done.</li>
<li>Free, fair trade – for the sharing of resources between different communities and individuals who opt out of Mutual Aid, anarchy would see the emergence of a truly free market. This doesn’t really fit with software, I think. Same with ‘individual liberty.’</li>
<li>Collective defence – this is not to say that anarchist society will contain “perfect people,” and there will certainly be acts of aggression, oppression, and violence – albeit on a lesser scale than is commonplace in today’s world. I’d liken this to things like “the site is down.” Everyone at GitHub cares about GitHub being up, and so they don’t need someone who’s responsible for ‘defense,’ as if shit hits the fan, everyone chips in and takes care of it.</li>
<li>Justice, not vengeance – courts would be elected for each individual case, rather than appointed and given unnecessary authority, with the aim to establish guilt or innocence, negotiate reparations, and organise rehabilitation rather than to support the oppressive prison systems which only make matters worse by serving as little more than universities of crime. I’m not sure this fits well here.</li>
</ul>
<h2>All abstractions are leaky</h2>
<p>Just like when ‘lean’ was a concept that applied manufacturing techniques to software development, I don’t think that ‘anarchism’ is a 100% fit with GitHub’s process. I do think that it’s close enough to hold a lot of value, however, even to the point that Zach had to write a blog post about <a href="http://zachholman.com/posts/scaling-github-employees/">scaling</a>, which is something that <em>always</em> gets leveled at people talking about anarchism. I see GitHub in the same way that I see <a href="http://en.wikipedia.org/wiki/Anarchism_in_Spain">anarchist Catalonia</a> or the <a href="http://en.wikipedia.org/wiki/Free_Territory">Free Territory</a> in the Ukraine. It’s a place where ideals are actually working in practice. Let’s just hope that the <a href="http://en.wikipedia.org/wiki/Anarchist_Catalonia#Clashes_with_the_Communists">communists don’t betray them when the fascists show up</a>.</p>
<p>Damn leaky abstractions.</p></content:encoded>
</item>
<item>
<title>Real, modern Ruby development</title>
<link>https://steveklabnik.com/writing/real-modern-ruby-development/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/real-modern-ruby-development/</guid>
<pubDate>Wed, 28 Sep 2011 00:00:00 GMT</pubDate>
<content:encoded><p>I came across a blog post the other day titled <a href="http://ascarter.net/2011/09/25/modern-ruby-development.html">Modern Ruby Development</a>. While it’s a perfectly fine blog post (other than the digs at rvm…) it really should have been titled something more along the lines of “My default tooling to build a Rails application.” I thought of this yesterday, as I was invited to show up at <a href="http://www.vertigo.com/">Vertigo</a> and show them how we do things in the Open Source world. Vertigo builds lots of cool things, but they tend to be very .NET centric, and it was neat for both of us to see how the other half lives.</p>
<p>Shortly before my ‘presentation,’ we decided that I’d basicaly just build a little web app in front of them. Live coding to the max! We decided for maximum awesome, and since I was in town for the Twilio conference, that I’d make a Twilio app that they could text and it’d text you ’hello world: you said #{msg}" back. I thought I’d share that process with you, too. I ended up throwing away the code, though, because I committed my account credentials into the git repo to simplify things, so I’ll just be describing it to you. Check my other posts for code stuff. ;)</p>
<p>Oh, and one more thing: lots of this stuff is about getting ‘iteration zero’ out of the way: it’s setup work that you have to do to set the tone for everything else. In its own way, it’s also a tooling overview, I’ve just focused on the tools that help the process rather than the language stuff. I’ve already written a more ‘dev proccess’ style post over at <a href="http://timelessrepo.com/bdd-with-rspec-and-steak">Timeless</a>, about BDD.</p>
<h2>rvm –rvmrc –create 1.9.2@hello_text</h2>
<p>First up: decide what Ruby you’re using. I talked breifly about MRI, Rubinius, and JRuby, and some pros and cons for each. I still tend to use MRI for my production code at the moment, but soon, rbx will become my default.</p>
<p>I use rvm because it does exactly what I want.</p>
<h2>mvim README.md</h2>
<p>I really enjoy <a href="http://tom.preston-werner.com/2010/08/23/readme-driven-development.html">README driven development</a>. I didn’t actually demo this due to time constraints, but I generally write at least a minimal README.md before doing anything else. During Windy City Rails, Tom shared something that he likes doing: <code>mv README.md SPEC.md</code>. That way, you can keep your actual README in sync with what the code does right now, but have the final one that you want to work towards too.</p>
<h2>git push origin</h2>
<p>Now that we have a file, we can push it up to GitHub. A few clicks on the web, a <code>git add remote</code> and <code>git push -u origin master</code> and we’re off to the races.</p>
<h2>mvim Gemfile &amp;&amp; bundle</h2>
<p>Next up we talked about Bundler, Gemfiles, and managing gems. I talked about the ability to run your own private gem server for internal gems, and how easy it is to build and publish a gem.</p>
<h2>mkdir spec</h2>
<p>I then generally type this. I would be lying to you if I told you that I do TDD 100% of the time. Sometimes, I get lazy. But I try to. And since testing is such a huge part of Ruby culture, and we have world-class testing tools, this is where I started.</p>
<p>I wrote specs for a model class called SmsGateway. This class would handle sending a message to a phone via Twilio. I was able to show off rspec, talk about DSLs, mocking, and TDD.</p>
<h2>mvim something.rb</h2>
<p>In this case it’s sms_gateway.rb, but you know what I mean. Write some code, <code>rspec spec/sms_gateway_spec.rb</code>, repeat. Red, Green, Refactor. Wheee!</p>
<h2>mvim .travis.yml</h2>
<p>I wanted to show Travis off, so in my demo I did this before implementing the gateway class, but normally I’d do this after the first spec passes. Continuous Integration and even Continuous Deployment are awesome. Travis makes it super easy.</p>
<h2>Sign up for external services</h2>
<p>I went over to Twilio’s site, registered a new account, and got my credentials. I used their docs to figure out where to put them, and how to get it all going inside my gateway class (I just mocked it out before), and talked about how needing to Google for docs is a weakness of Ruby.</p>
<h2>heroku create &amp;&amp; git push heroku master</h2>
<p>Then I showed them how to get going with Heroku, and how easy it was to start giving them hundreds of dollars per month when you need to scale. .NET people are used to paying for everything. ;)</p>
<p>I also showed off one of my latest projects, which is an IRC bot that lets me deploy rstat.us into production. I have some serious Hubot envy. :D</p>
<h2>… repeat!</h2>
<p>I’d be lying if I said it worked the first time, I accidentally made a loop where the app texted itself. Whoops! But that was cool, and showing off my debugging process (I’m a <code>puts/throw params</code> debugger) was fun, as well. After realizing what I did wrong, we had the app working.</p>
<h2>It’s not quite perfect</h2>
<p>This dev process isn’t perfect yet. Nothing ever is. I’m always trying to improve upon it, but I’m pretty happy with how quickly I’m able to turn out new projects with a reasonable amount of quality. What’s your workflow? How does mine suck? <a href="mailto:[email protected]">Email me</a>, I’d love to hear about it.</p></content:encoded>
</item>
<item>
<title>I'm deleting my Facebook tonight</title>
<link>https://steveklabnik.com/writing/im-deleting-my-facebook-tonight/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/im-deleting-my-facebook-tonight/</guid>
<pubDate>Mon, 26 Sep 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Well, <a href="http://adrianshort.co.uk/2011/09/25/its-the-end-of-the-web-as-we-know-it/">it’s the end of the web as we know it</a>. I’d already been thinking about this all weekend, and last night, I decided. I just can’t deal with Facebook’s crazy privacy stuff anymore. I mean, I already basically <a href="http://rstat.us/">forked Twitter</a> over their ToS, and Facebook has been a lot more evil for a lot longer. Frictionless sharing is pretty much the last straw.</p>
<p>I’ll be using this as an excuse to keep helping out with the federated web. There’s a lot of cool stuff in this area, but it needs help. If you’re a developer, consider helping out with the <a href="https://github.com/LockerProject/Locker">Locker Project</a> or <a href="https://github.com/homesteading">the Homesteading Project</a>. The only way that we can prevent people from becoming digial sharecroppers is by helping them to ‘own their own land.’</p>
<p>If we’re Facebook friends, you can always find out how to reach me at <a href="http://steveklabnik.com/">http://steveklabnik.com</a>. Please be careful, and consider if Facebook’s privacy settings are really worth it to you. They may be, and they may not. I’m under no illusions that the vast majority of people will give up Facebook, but I figured I’d warn you anyways.</p></content:encoded>
</item>
<item>
<title>More rstat.us refactoring</title>
<link>https://steveklabnik.com/writing/more-rstat-dot-us-refactoring/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/more-rstat-dot-us-refactoring/</guid>
<pubDate>Fri, 23 Sep 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Hey everyone! I just wanted to share One More Thing with you about this rstat.us refactoring.</p>
<p>The main thrust of the last article I posted was to show you a technique for extracting a class, getting it under some kind of test, and then refactoring it a bit. Refactoring is always an iterative process, and Ryan Bates from the always awesome <a href="http://railscasts.com/">Railscasts</a> asked me why I made the Salmon class into a module, especially given my recent <a href="/2011/09/06/the-secret-to-rails-oo-design.html">rant</a> against modules. The answer was, ‘because it’s simpler, and the first thing I thought of.’ He shared with me an alternate implementation that I like too, and I wanted to share with you. Check it:</p>
<pre><code>class SalmonNotifier
def initialize(user, feed)
@user = user
@feed = feed
end
def mention(update)
deliver OStatus::Salmon.new(update.to_atom("http://#{@user.author.domain}/"))
end
def follow
deliver OStatus::Salmon.from_follow(@user.author.to_atom, @feed.author.to_atom)
end
def unfollow
deliver OStatus::Salmon.from_unfollow(@user.author.to_atom, @feed.author.to_atom)
end
protected
def deliver(salmon)
send_envelope_to_salmon_endpoint salmon.to_xml(@user.to_rsa_keypair)
end
def send_envelope_to_salmon_endpoint(envelope)
uri = URI.parse(@feed.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<p>This follows a more OO style, and is a bit more well-factored. Here’s his description of what he did:</p>
<blockquote>
<ol>
<li>Switched to a class and gave it a name that is a noun (maybe name should be different though).</li>
</ol>
</blockquote>
<ol>
<li>
<p>Moved common method params into instance variables.</p>
</li>
<li>
<p>This simplified the mention/follow/unfollow methods enough to be on one line.</p>
</li>
<li>
<p>Renamed "<code>send_to_salmon_endpoint</code>" to "<code>deliver</code>" because it feels nicer, the "salmon endpoint" can be assumed due to the name of the class. I generally don't like to put the class name in the method name.</p>
</li>
<li>
<p>Extracted commented out into its own method with the same name. I don't know if this is really necessary though (same reason as above).</p>
</li>
</ol>
<p>The only thing that bothers me now is the constant access of <code>@user.author</code> and <code>@feed.author</code>. This makes me think it should be interacting directly with authors. However you still need access to <code>@user.to_rsa_keypair</code> but maybe that method could be moved elsewhere more accessible.</p>
<blockquote></blockquote>
<p>All of these changes are pretty good. Ryan’s suggestions for moving forward are pretty good, as well, but I’d add one more thing: we’re starting to collect a bunch of methods all related to delivering more generic HTTP requests in the protected section. This also might be worth pulling out, too. Protected/private in general is a minor smell that indicates there’s code being used multiple times that’s not part of the main objective of the class. It might not be worth it yet, but then again, it might. Speaking of protected methods, you might wonder why I ended up mocking out my protected method in the last post, as well. There are no more tests that actually test the method is working properly. This is true, and it’s because I was leading into thinking of something like this. Mocked tests are almost always unit tests, and two methods are two separate units, so I had just ended up doing it out of habit. This further shows that maybe an extraction of another class is worthwhile.</p></content:encoded>
</item>
<item>
<title>Extracting domain models: a practical example</title>
<link>https://steveklabnik.com/writing/extracting-domain-models-a-practical-example/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/extracting-domain-models-a-practical-example/</guid>
<pubDate>Thu, 22 Sep 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Hey everyone! We’ve been doing a lot of refactoring on rstat.us lately, and I wanted to share with you a refactoring that I did. It’s a real-world example of doing the domain models concept that I’ve been talking about lately.</p>
<h2>Step one: check the tests</h2>
<blockquote>
<p>I don’t know how much more emphasized step 1 of refactoring could be: don’t touch anything that doesn’t have coverage. Otherwise, you’re not refactoring; you’re just changing shit. - Hamlet D’Arcy</p>
</blockquote>
<p>One of the best reasons to extract extra domain models is that they’re often much easier to test. They have less dependencies, less code, and are much simpler than the Mega Models that happen if you have a 1-1 model to table ratio.</p>
<p>Let’s check out the code. It’s in our <a href="https://github.com/hotsh/rstat.us/blob/362cb38031/app/models/user.rb#L209">User model</a>:</p>
<pre><code># Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
f = Feed.first :id =&gt; to_feed_id
u = Update.first :id =&gt; update_id
base_uri = "http://#{author.domain}/"
salmon = OStatus::Salmon.new(u.to_atom(base_uri))
envelope = salmon.to_xml self.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(f.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
</code></pre>
<p>You’re probably saying “this is pretty reasonable.” Prepare to get shocked…</p>
<p>This code has very little to do with the user, other than using some of the User’s attributes. This really could be factored out into a domain object whose job it is to push Salmon notifications. But first, let’s see if we can get some tests in place so we know <em>everything</em> isn’t broken. Examining this method, we need two things: an <code>update_id</code> and a <code>feed_id</code>. Here’s my first stab at an integration test:</p>
<pre><code>describe "salmon update" do
it "integration regression test" do
feed = Feed.create
update = Update.create
salmon = double(:to_xml =&gt; "&lt;xml&gt;&lt;/xml&gt;")
uri = double(:host =&gt; "localhost", :port =&gt; "9001", :path =&gt; "/")
Ostatus::Salmon.should_receive(:new).and_return(salmon)
Uri.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(mock(:post =&gt; true))
user = User.create
user.send_mention_notification(update.id, feed.id)
end
end
</code></pre>
<p>This is an integration test, we’re obviously testing stuff with a ton of models. After running <code>ruby test/models/notify_via_salmon.rb</code> about a zillion times, I ended up with this running test:</p>
<pre><code>require 'minitest/autorun'
require 'rspec/mocks'
require 'mongo_mapper'
require 'whatlanguage'
MongoMapper.connection = Mongo::Connection.new("localhost")
MongoMapper.database = 'rstatus-test'
require_relative '../../app/models/feed'
require_relative '../../app/models/update'
require_relative '../../app/models/author'
require_relative '../../app/models/authorization'
$:.unshift("lib")
require_relative '../../app/models/user'
require 'ostatus'
describe "salmon update" do
before :each do
User.all.each(&amp;:destroy)
RSpec::Mocks::setup(self)
end
it "integration regression test" do
author = Author.create
user = User.create!(:username =&gt; "steve", :author =&gt; author)
feed = user.feed
update = Update.create!(:feed_id =&gt; feed.id, :author_id =&gt; author.id, :text =&gt; "hello world")
salmon = double(:to_xml =&gt; "&lt;xml&gt;&lt;/xml&gt;")
uri = double(:host =&gt; "localhost", :port =&gt; "9001", :path =&gt; "/")
OStatus::Salmon.should_receive(:new).and_return(salmon)
URI.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(mock(:post =&gt; true))
user.send_mention_notification(update.id, feed.id)
end
end
</code></pre>
<p>Holy. Fuck. Seriously. Let me just quote <a href="http://avdi.org/devblog/2011/04/07/rspec-is-for-the-literate/">Avdi Grimm</a>:</p>
<blockquote>
<p>Mocks, and to some degree RSpec, are like the Hydrogen Peroxide of programming: they fizz up where they encounter subtle technical debt.</p>
</blockquote>
<p>This test class is insane. It runs faster than many people’s integration tests, clocking in at about 0.4 seconds. But that’s still an order of magnitude slower than I’d like. A suite with 200 tests would still take over a minute. Also, look at all this junk that we have to do for setup.</p>
<p>All of this pain and effort doesn’t mean that testing sucks: it means that our implementation is terrible. So let’s use that to guide us. We’ll try to fix this code by eliminating all of this junk.</p>
<h2>Step two: simple extraction</h2>
<p>First, let’s extract this out to a domain model. Here’s the new code:</p>
<pre><code># Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
NotifyViaSalmon.mention(update_id, to_feed_id)
end
</code></pre>
<pre><code>module NotifyViaSalmon
extend self
def mention(update_id, to_feed_id)
f = Feed.first :id =&gt; to_feed_id
u = Update.first :id =&gt; update_id
base_uri = "http://#{author.domain}/"
salmon = OStatus::Salmon.new(u.to_atom(base_uri))
envelope = salmon.to_xml self.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(f.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<p>Now, when we run the test (via <code>ruby test/models/notify_via_salmon_test.rb</code>) we see an error:</p>
<pre><code>1) Error:
test_0001_integration_regression_test(SalmonUpdateSpec):
NameError: undefined local variable or method `author' for NotifyViaSalmon:Module
/Users/steveklabnik/src/rstat.us/app/models/notify_via_salmon.rb:8:in `mention'
</code></pre>
<p>This is a form of Lean On The Compiler. This test is now failing because it’s relying on stuff that used to be internal to the User, and we don’t have that stuff now. After doing this a few times, we’re left with this:</p>
<pre><code># Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
NotifyViaSalmon.mention(self, update_id, to_feed_id)
end
</code></pre>
<pre><code>module NotifyViaSalmon
extend self
def mention(user, update_id, to_feed_id)
f = Feed.first :id =&gt; to_feed_id
u = Update.first :id =&gt; update_id
base_uri = "http://#{user.author.domain}/"
salmon = OStatus::Salmon.new(u.to_atom(base_uri))
envelope = salmon.to_xml user.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(f.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<p>Okay. Now we’re cooking. We have a test that’s isolated, yet still does what we thought it did before.</p>
<h2>Step three: break dependencies</h2>
<p>Next step: let’s break the hard dependency this method has on both Feed and Update. We can do this by moving the finds up into the User method instead of keeping them in the mention method:</p>
<pre><code># Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
feed = Feed.first :id =&gt; to_feed_id
update = Update.first :id =&gt; update_id
NotifyViaSalmon.mention(self, update, feed)
end
</code></pre>
<pre><code>module NotifyViaSalmon
extend self
def mention(user, update, feed)
base_uri = "http://#{user.author.domain}/"
salmon = OStatus::Salmon.new(update.to_atom(base_uri))
envelope = salmon.to_xml user.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(feed.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<p>Okay. Tests passing. Sweet. Now we can try testing <em>just</em> the mention method. Let’s try it by killing most of that crap that was in our test:</p>
<pre><code>require 'minitest/autorun'
require 'rspec/mocks'
require_relative '../../app/models/notify_via_salmon'
describe "salmon update" do
before :each do
RSpec::Mocks::setup(self)
end
it "integration regression test" do
NotifyViaSalmon.mention(stub, stub, stub)
end
end
</code></pre>
<p>Let’s try running this and seeing what happens:</p>
<pre><code>$ ruby test/models/notify_via_salmon_test.rb
Loaded suite test/models/notify_via_salmon_test
Started
E
Finished in 0.000992 seconds.
1) Error:
test_0001_integration_regression_test(SalmonUpdateSpec):
RSpec::Mocks::MockExpectationError: Stub received unexpected message :author with (no args)
</code></pre>
<p>First of all, daaaaamn. 0.001 seconds. Nice. Secondly, okay, we are getting some messages sent to our stubs. Let’s flesh them out to make things pass:</p>
<pre><code>require 'minitest/autorun'
require 'rspec/mocks'
require 'ostatus'
require_relative '../../app/models/notify_via_salmon'
describe "salmon update" do
before :each do
RSpec::Mocks::setup(self)
end
it "integration regression test" do
user = stub(:author =&gt; stub(:domain =&gt; "foo"), :to_rsa_keypair =&gt; stub)
update = stub(:to_atom =&gt; "")
feed = stub(:author =&gt; stub(:salmon_url =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:new).and_return(salmon)
uri = stub(:host =&gt; "", :port =&gt; "", :path =&gt; "")
URI.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(stub(:post =&gt; ""))
NotifyViaSalmon.mention(user, update, feed)
end
end
</code></pre>
<p>Okay! Note how similar this looks to the previous test. We’re still mocking a bunch of ‘external’ stuff. But other than that, our test is pretty simple: we make three stubs, and we call our mention method.</p>
<h3>Step four: repeat</h3>
<p>We could do some more work on this method. There are a few things that are a bit smelly: the nested stub of User is not great. The fact that we’re stubbing out three external dependencies isn’t great. But before we get to that, let’s check out another method that looks similar: <code>send_{follow,unfollow}_notification</code>. Here’s <a href="https://github.com/hotsh/rstat.us/blob/362cb38031/app/models/user.rb#L167">some code</a>:</p>
<pre><code># Send Salmon notification so that the remote user
# knows this user is following them
def send_follow_notification to_feed_id
f = Feed.first :id =&gt; to_feed_id
salmon = OStatus::Salmon.from_follow(author.to_atom, f.author.to_atom)
envelope = salmon.to_xml self.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(f.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
# Send Salmon notification so that the remote user
# knows this user has stopped following them
def send_unfollow_notification to_feed_id
f = Feed.first :id =&gt; to_feed_id
salmon = OStatus::Salmon.from_unfollow(author.to_atom, f.author.to_atom)
envelope = salmon.to_xml self.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(f.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
</code></pre>
<p>Look familliar? Yep! 90% the same code! We’ll do the same process to these methods, just like the other one. Check out this code and test:</p>
<pre><code># Send Salmon notification so that the remote user
# knows this user is following them
def send_follow_notification to_feed_id
feed = Feed.first :id =&gt; to_feed_id
NotifyViaSalmon.follow(self, feed)
end
# Send Salmon notification so that the remote user
# knows this user has stopped following them
def send_unfollow_notification to_feed_id
feed = Feed.first :id =&gt; to_feed_id
NotifyViaSalmon.unfollow(self, feed)
end
# Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
feed = Feed.first :id =&gt; to_feed_id
update = Update.first :id =&gt; update_id
NotifyViaSalmon.mention(self, update, feed)
end
</code></pre>
<pre><code>module NotifyViaSalmon
extend self
def mention(user, update, feed)
base_uri = "http://#{user.author.domain}/"
salmon = OStatus::Salmon.new(update.to_atom(base_uri))
envelope = salmon.to_xml user.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(feed.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
def follow(user, feed)
salmon = OStatus::Salmon.from_follow(user.author.to_atom, feed.author.to_atom)
envelope = salmon.to_xml user.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(feed.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
def unfollow(user, feed)
salmon = OStatus::Salmon.from_unfollow(user.author.to_atom, feed.author.to_atom)
envelope = salmon.to_xml user.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(feed.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<pre><code>require 'minitest/autorun'
require 'rspec/mocks'
require 'ostatus'
require_relative '../../app/models/notify_via_salmon'
describe NotifyViaSalmon do
before :each do
RSpec::Mocks::setup(self)
end
it ".mention" do
user = stub(:author =&gt; stub(:domain =&gt; "foo"), :to_rsa_keypair =&gt; stub)
update = stub(:to_atom =&gt; "")
feed = stub(:author =&gt; stub(:salmon_url =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:new).and_return(salmon)
uri = stub(:host =&gt; "", :port =&gt; "", :path =&gt; "")
URI.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(stub(:post =&gt; ""))
NotifyViaSalmon.mention(user, update, feed)
end
it ".follow" do
user = stub(:author =&gt; stub(:domain =&gt; "foo", :to_atom =&gt; ""), :to_rsa_keypair =&gt; stub)
feed = stub(:author =&gt; stub(:salmon_url =&gt; "", :to_atom =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:from_follow).and_return(salmon)
uri = stub(:host =&gt; "", :port =&gt; "", :path =&gt; "")
URI.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(stub(:post =&gt; ""))
NotifyViaSalmon.follow(user, feed)
end
it ".unfollow" do
user = stub(:author =&gt; stub(:domain =&gt; "foo", :to_atom =&gt; ""), :to_rsa_keypair =&gt; stub)
feed = stub(:author =&gt; stub(:salmon_url =&gt; "", :to_atom =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:from_unfollow).and_return(salmon)
uri = stub(:host =&gt; "", :port =&gt; "", :path =&gt; "")
URI.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(stub(:post =&gt; ""))
NotifyViaSalmon.unfollow(user, feed)
end
end
</code></pre>
<p>Now we can see some of these patterns start to emerge, eh? All of this stuff is starting to come together. Let’s get rid of the URI and Net::HTTP stuff, as it’s just straight up identical in all three. Pretty basic Extract Method:</p>
<pre><code>module NotifyViaSalmon
extend self
def mention(user, update, feed)
base_uri = "http://#{user.author.domain}/"
salmon = OStatus::Salmon.new(update.to_atom(base_uri))
send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)
end
def follow(user, feed)
salmon = OStatus::Salmon.from_follow(user.author.to_atom, feed.author.to_atom)
send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)
end
def unfollow(user, feed)
salmon = OStatus::Salmon.from_unfollow(user.author.to_atom, feed.author.to_atom)
send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)
end
protected
def send_to_salmon_endpoint(salmon, keypair, uri)
envelope = salmon.to_xml keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(uri)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<p>We run our tests, it still all works. Now we can just mock out the endpoint method, and all of our tests on the individual methods become much, much simpler:</p>
<pre><code>require 'minitest/autorun'
require 'rspec/mocks'
require 'ostatus'
require_relative '../../app/models/notify_via_salmon'
describe NotifyViaSalmon do
before :each do
RSpec::Mocks::setup(self)
end
it ".mention" do
user = stub(:author =&gt; stub(:domain =&gt; "foo"), :to_rsa_keypair =&gt; stub)
update = stub(:to_atom =&gt; "")
feed = stub(:author =&gt; stub(:salmon_url =&gt; ""))
NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)
NotifyViaSalmon.mention(user, update, feed)
end
it ".follow" do
user = stub(:author =&gt; stub(:domain =&gt; "foo", :to_atom =&gt; ""), :to_rsa_keypair =&gt; stub)
feed = stub(:author =&gt; stub(:salmon_url =&gt; "", :to_atom =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:from_follow).and_return(salmon)
NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)
NotifyViaSalmon.follow(user, feed)
end
it ".unfollow" do
user = stub(:author =&gt; stub(:domain =&gt; "foo", :to_atom =&gt; ""), :to_rsa_keypair =&gt; stub)
feed = stub(:author =&gt; stub(:salmon_url =&gt; "", :to_atom =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:from_unfollow).and_return(salmon)
NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)
NotifyViaSalmon.unfollow(user, feed)
end
end
</code></pre>
<p>We’ve managed to get rid of all of the mocking of the URI stuff. It’s all details now. We also have great information about what is actually needed for this test: we know exactly what our objects expect the caller to pass. We’re only depending on ourselves and some external libraries. Our tests are fast. Overall, I’m feeling much better about this code than I was before.</p>
<h2>In conclusion</h2>
<p>I hope you’ll take a few things away from this post.</p>
<p><strong>Start with tests first!</strong> If you don’t do this, it’s very easy to introduce simple errors. Don’t be that guy. You know, the one that has to make <a href="https://github.com/hotsh/rstat.us/pull/398/files">pull requests like this…</a> I suck.</p>
<p><strong>Don’t be afraid to change the tests!</strong> As soon as you’ve verified that you’ve transcribed the code correctly, don’t be afraid to just nuke things and start again. Especially if you have integration level tests that confirm that your features actually <em>work</em>, your unit tests are expendable. If they’re not useful, kill them!</p>
<p><strong>Mock like a motherfucker.</strong> When done correctly, mocks allow you to help design your code better, make your tests fast, and truly isolate the unit under test. It’s pretty much all upside, unless you suck at them.</p>
<p><strong>Extract, extract, extract!</strong> Adele Goldberg once said, “In Smalltalk, everything happens somewhere else.” Make tons of little methods. They help to provide seams that are useful for testing. Smaller methods are also easier to understand.</p>
<p><strong>Refactoring is a marathon.</strong> Even with all this work, this code isn’t the best. The tests could even use a little TLC. But it’s still better than it was before. Those who ship, win. Make things a little better, <code>git commit &amp;&amp; git push</code>, repeat.</p></content:encoded>
</item>
<item>
<title>We forget that open source is made of people</title>
<link>https://steveklabnik.com/writing/we-forget-that-open-source-is-made-of-people/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/we-forget-that-open-source-is-made-of-people/</guid>
<pubDate>Mon, 12 Sep 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Programmers like to think that software is the purest form of meritocracy. We like to consider ourselves scientists; what we do is entirely impersonal. I’m at Lone Star Ruby right now, and the first two talks are entirely about software as a science.</p>
<p>But this is a fallacy. There’s a human element to open source that’s often totally forgotten, and it drives people away.</p>
<h2>Software Celebrity and Hero Worship</h2>
<p>We sort of acknowledge this, in a way. There are most certainly software celebrities, even if their celebrity is localized to the community in which they operate. I heard someone say this today:</p>
<blockquote>
<p>If I can go to one Ruby conference and not hear anyone mention Zed Shaw or _why, I can die happy.</p>
</blockquote>
<p>If I say “jnicklas” you say “Capybara.” If I say “DHH” you say “Rails.” If I say “matz” you say “Ruby.” We have our heroes, villains, and foils, just like anybody else. But we forget that in every celebrity, there is a real person in there. Actual people, who have real thoughts, feelings, ideas, emotions, and that spend all of their time working for free just to make your lives better out of the goodness of their heart. Often <em>after</em> working at day jobs, often <em>at the expense</em> of a personal life, often <em>straining relationships</em> with people they care about.</p>
<h2>Competition Means Somebody Loses</h2>
<p>I used to think that Git was great because it made forks trivial. It changed what we mean by ‘fork.’ This is largely a good thing. But ‘the eff word’ used to be a last resort. The nuclear option. You’re fed up, can’t take it anymore, and so this is what they’ve driven you to do. Forking was Serious Business.</p>
<p>Now, GitHub has made it so easy to fork a project and distribute it to tons of people, people just fork things willy-nilly. And, from the outside, competition is good: in theory, it means that the good ideas work and the bad ideas don’t.</p>
<p>(A small aside: forking in git is different than ‘forking’ in svn. This overloading of terminology is unfortunate. The fork button is actually one of my favorite features of GitHub, because it makes contribution easier. But as it’s easier to publish new software, it’s easier to ‘fork’ in the svn sense. It used to be a pain to start a new project, and GitHub’s made that so easy that it happens more often than I think it should.)</p>
<p>But guess what: competition sucks. Cooperation is much better. One of the biggest problems with competition is that it means that somebody has to lose. For every winner, someone is a loser. That person is going to be hurting. In the case of open source, someone has spent (possibly) years of their life working on something, and now, all of that is gone. That shit stings.</p>
<p>Sometimes, forks are necessary. Sometimes, there are approaches that are fundamentally different. Forking is still an option, even if it’s a last resort. I’m also not saying that feelings should totally trump technical merit. All I’m saying is this: consider the blood, sweat, and tears of others before you act. Just in general.</p>
<p>It’s also not a zero-sum game, either. The pie can be grown. But that also doesn’t mean that it’s grown in a way that makes anyone feel good about it. There’s a difference between “We represent two sides of some coin” and “Those fuckers make shitty software.” Both approaches can grow the pie. One is far better than the other.</p>
<p>What’s the human cost of rapid competition between software projects? Is this cost worth it?</p>
<h2>Open Source: All take, no give</h2>
<p>The Pareto Principle: 80% of the effects come from 20% of the causes. This is absolutely true in OSS, and it might be more imbalanced. A few people contribute, and lots of people use. Sometimes, this is a good thing: it’s actually what draws me to open source. I can help people in a scalable way. My code is currently being used by tens of thousands of people. This is awesome. Some documentation that I’ve written is being read by millions of people. This is way more awesome.</p>
<p>But I rarely get any help with my projects. It’s really easy to let that get to a person.</p>
<p>Do you know how many people have told me that what I do is important? I got given a Ruby Hero award for my work on Hackety and Shoes. Do you know how many commits I have in on them? Probably less than 200. Do you know why? I’ve spent weeks of my life trying to track down bugs, but some of them are too hard for me, at my current stage of programming-fu. And it gets depressing. And everyone that’s ever offered to help has flaked on me. And all the recognition for something that was largely done by someone else gets to me, really deeply. And I get tons of people screaming about Lion support, and ‘this doesn’t work,’ and all sorts of things… and no patches. It’s why the ‘patches accepted’ saying was coined: this shit grinds on you after a while.</p>
<h2>All of my Heroes are Dying</h2>
<p>Today, this situation with rbenv and rvm made me realize something: everyone that I truly, deeply respect in open source has either burned out or left. I’ve said once before that there are four people that I truly respect in software: Zed Shaw, _why, Yehuda Katz, and (more recently) Wayne Seguin (and also Aaron Patterson. Dammit. This list is getting long). All of these developers are brilliant, work hard, and are tirelessly supporting their communities. That is, until they were driven away. This isn’t totally applicable to Yehuda, but it happened to Zed. It happened to _why. And I can see it happening, right now, to Aaron and Wayne. And Yehuda isn’t really doing Ruby…</p>
<p>But seeing this happen makes me deeply sad. For them personally, for my own selfish reasons, and that that’s the state of our communities.</p>
<h2>Where Does This Leave Us?</h2>
<p>I don’t have any answers. I only have a bunch of questions:</p>
<ul>
<li>Why do we drive people away from Open Source?</li>
<li>Can we learn to work together, instead of competing?</li>
<li>How can we better recognize this human element?</li>
</ul>
<p>If you don’t contribute to Open Source: please consider thanking someone that does open source, in a project that you love. Consider helping out, in even the smallest of ways.</p>
<p>Thanks.</p>
<p><a href="/2011/08/19/matz-is-nice-so-we-are-nice.html">My next post is a bit of a follow-up to this</a>.</p></content:encoded>
</item>
<item>
<title>Better Ruby Presenters</title>
<link>https://steveklabnik.com/writing/better-ruby-presenters/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/better-ruby-presenters/</guid>
<pubDate>Fri, 09 Sep 2011 00:00:00 GMT</pubDate>
<content:encoded><p>My <a href="/2011/09/06/the-secret-to-rails-oo-design.html">last blog post</a> caused a bit of a stir in some circles. I got a bunch of emails. Apparently, I need to expand on a few things. So here we go. Let’s rap about the Presenter pattern, shall we?</p>
<h2>No seriously, helpers suck</h2>
<blockquote>
<p>In Ruby, everything is an object. Every bit of information and code can be given their own properties and actions. - ruby-lang.org/about</p>
</blockquote>
<p>… except helpers. Why is it in Ruby that everything is an object, even integers, yet as soon as we need to format a date, we all turn into Dijkstra and bust out structured programming? Actually, come to think of it, Dijkstra wouldn’t even write any code, because it’s beneath him, but you get the idea. (if you don’t get this, it’s a joke. About proofs, telescopes, and CS…) Helpers are like a compliment that you don’t want to give, but feel obligated to: “Yeah, I mean, well, you tried, and that’s what counts, right?”</p>
<p>This is the topic of a future post, but when programming in a language, you want to work with its primary methods of abstraction. In Ruby, that’s objects, and there’s a good reason for that: functions don’t provide a sufficient amount of power to tackle hard problems. I don’t want to get into this either, but there’s a reason that objects exist, and that nobody’s making procedural languages anymore. No biggie. C has a special place in my heart.</p>
<p>But I digress: objects &gt; functions. At least in the context of getting stuff done in Ruby. This pretty much applies to most of the rest of the points in this post, so just keep that in the back of your brain.</p>
<h2>Why not a class method?</h2>
<p>Well, first of all, <a href="http://nicksda.apotomo.de/2011/07/are-class-methods-evil/">class methods also suck</a>. Here’s the issue: Can you tell me the difference between these two methods?</p>
<pre><code>def foo
"hello"
end
class Bar
def self.foo
"hello"
end
end
foo
Bar.foo
</code></pre>
<p>Yep, that’s right. You can see it clearly from the last two lines: class methods are functions that happen to be namespaced. That means they’re slightly better, but really, <code>DictionaryPresenter.as_dictionary(something)</code> might as well just be a helper. At least <code>as_dictionary(something)</code> is shorter.</p>
<p>Here’s another reason: I didn’t make that method be a class method, even though there was no state, because a really real presenter (I mean, that was a real world example, but…) generally works a little different. Usually, I make presenters that actually stand in for the objects they’re presenting. Check this out. Here’s the presenter I showed you:</p>
<pre><code>class DictionaryPresenter
def initialize(collection)
@collection = collection
end
def as_dictionary
dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
@collection.each do |p|
dictionary[p.title[0]] &lt;&lt; p
end
dictionary
end
end
</code></pre>
<p>The real-world presenter that I used this for looked like this:</p>
<pre><code>class DictionaryPresenter
include Enumerable
def initialize(collection)
@dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
collection.each do |p|
@dictionary[p.title[0]] &lt;&lt; p
end
end
def each &amp;blk
@dictionary.each &amp;blk
end
end
</code></pre>
<p>… or close to this. There was an ‘other’ category, and a few other things… but you get the idea. Now, instead of this:</p>
<pre><code>@posts = DictionaryPresenter.new(Post.all).as_dictionary
</code></pre>
<p>You do this:</p>
<pre><code>@posts = DictionaryPresenter.new(Post.all)
</code></pre>
<p>And the presenter actually stands in for the hash. A subtle but important difference. This gives you more options, because you have a real, live object instead of just some random processing. With this simplistic presenter, you might not see a huge benefit, but often, you’ll want something a bit more complicated. And since this was what I had in my head, I didn’t make <code>as_dictionary</code> be a static method.</p>
<p>Oh, and presenters that decorate a single object rather than a collection often implement <code>method_missing</code>, so that you only <em>add</em> methods rather than hide most of them. Some don’t. It depends.</p>
<h2>Blocks vs. Policy</h2>
<p>I also showed off a Ruby version of a design pattern known as the Policy pattern, or sometimes the Strategy pattern. Turns out that Policy is often a bit heavyweight for Ruby, especially in an example like this, so I wanted to show you an alternate version of it, taking advantage of a Ruby feature: blocks.</p>
<p>For reference, the old code, put into the new code:</p>
<pre><code>class DictionaryPresenter
include Enumerable
def initialize(policy, collection)
@dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
collection.each do |p|
@dictionary[policy.category_for(p)] &lt;&lt; p
end
end
def each &amp;blk
@dictionary.each &amp;blk
end
end
class UserCategorizationPolicy
def self.category_for(user)
user.username[0]
end
end
</code></pre>
<p>We could use blocks instead:</p>
<pre><code>class DictionaryPresenter
include Enumerable
def initialize(collection, &amp;blk)
@dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
collection.each do |p|
@dictionary[blk.call(p)] &lt;&lt; p
end
end
def each &amp;blk
@dictionary.each &amp;blk
end
end
DictionaryPresenter.new(Post.all) do |item|
item.title[0]
end
</code></pre>
<p>While this is shorter, and a bit more Rubyish, it also means that we lose the <a href="http://en.wikipedia.org/wiki/Reification_(computer_science)">reification</a> of our concept of a policy. While this is <em>shorter</em>, I find it more confusing. Why am I giving this one line block to the presenter? It’s not really clear to me. And I can’t give a better name to item… and if I want to change the policy, it has to be done in every place we use the presenter…</p>
<p>This does get to the root of another thing that will end up being a follow-up: What’s the difference between closures and objects? If you don’t know, maybe this will get you thinking:</p>
<pre><code>#ugh, this is slightly odd, thanks Ruby!
def apply(proc=nil, &amp;blk)
if block_given?
blk.call
else
proc.call
end
end
proc = Proc.new do
puts "a proc!"
end
class MyProc
def call
puts "an object!"
end
end
my_proc = MyProc.new
apply do
puts "a block!"
end
apply proc
apply my_proc
</code></pre>
<p>Chew on that a while.</p>
<h2>This is too verbose</h2>
<p>Absolutely. <code>DictionaryPresenter.new(PostCategorizationPolicy, Post.all).as_dictionary</code> is too long. 100% there. Good thing that I’d <em>actually</em> write this:</p>
<pre><code>Dictionary.new(Post.all, ByTitle)
</code></pre>
<p>I switched the arguments around, because it reads better. When writing blog posts, you have to balance writing code samples that explain what you’re trying to say, and ones that are ‘real.’ I tend to be a bit more verbose with things like this, because it’s easier for someone who just jumps in to remember what all the parts are… then again, by <em>not</em> showing the ‘real code,’ people might end up implementing the crappy, verbose version. It’s always a trade-off, and I may or may not make the right choice. It happens.</p>
<p>And yes, even earlier in this blog post: I said <code>DictionaryPresenter</code> and I’d really just say <code>Dictionary</code>.</p>
<h2>Where does all of this go?</h2>
<p>You’ve got two options, and only one of them is good: You should make a directory in app for your presenters. In this case, I’d be running <code>touch app/presenters/dictionary.rb</code> pretty quick. The other case is to put things in lib. This whole debacle uncovers something that @avdi went on a rant about over the last few days: models don’t have to inherit from anything. Here’s <a href="http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller#Concepts">what Wikipedia says about models</a>:</p>
<blockquote>
<p>The model manages the behaviour and data of the application domain, responds to requests for information about its state (usually from the view), and responds to instructions to change state (usually from the controller). In event-driven systems, the model notifies observers (usually views) when the information changes so that they can react.</p>
</blockquote>
<p>See anything about “saves stuff to the database?” Okay, ‘manages… data’ could be, but the point is, it’s not <em>required</em>. Models are the reification of a concept that exists in your application domain. Persistance is not required for every concept.</p>
<p>Therefore, if it relates to your problem domain, it goes in app. If it doesn’t, it goes in lib. Something like <code>Dictionary</code>? Depends on how important you think it is. It’s probably pretty important; that’s why you made it a class.</p>
<h2>Design Patterns? Are you serious?</h2>
<p>Yes. Design patterns don’t suck, Design patterns <em>in Java</em> suck. I’m not even going to start this one here, expect more in this space soon.</p>
<p>Design patterns are great, and they’re totally useful in Ruby.</p>
<h2>In conclusion: A shout-out</h2>
<p>I hope you’ve enjoyed this small digression on Presenters. If you’re gonna use presenters in your application, I encourage you to write simple ones as simple Ruby classes, but if they get slightly more complicated, you need to check out <a href="https://github.com/jcasimir/draper">Draper</a> by my buddy Jeff Casimir. It lets you do stuff like this:</p>
<pre><code>class ArticleDecorator &lt; ApplicationDecorator
decorates :article
def author_name
model.author.first_name + " " + model.author.last_name
end
end
article = ArticleDecorator.find(1)
article.author_name
</code></pre>
<p>There’s a lot of features to it, so check it out on GitHub. Totally worthwhile.</p></content:encoded>
</item>
<item>
<title>The secret to Rails OO design</title>
<link>https://steveklabnik.com/writing/the-secret-to-rails-oo-design/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-secret-to-rails-oo-design/</guid>
<pubDate>Tue, 06 Sep 2011 00:00:00 GMT</pubDate>
<content:encoded><p>UPDATE: I also have a follow-up <a href="/2011/09/09/better-ruby-presenters.html">here</a>.</p>
<p>I often tell people that I learned Ruby via Rails. This is pretty much the worst way to do it, but I’d learned so many programming languages by then that it didn’t hinder me too much. The one thing that it did do, however, was give me a slightly twisted sense of how to properly design the classes needed in a Rails app. Luckily, I obsessively read other people’s code, and I’ve noticed that there’s one big thing that is common in most of the code that’s written by people whose design chops I respect.</p>
<p>This particular thing is also seemingly unique to those people. It’s not something that people who don’t write good code attempt, but do badly. It’s like a flag, or signal. Now, when I see someone employ this, I instantly think “they get it.” Maybe I’m giving it too much credit, but this advanced design technique offers a ton of interconnected benefits throughout your Rails app, is easy to imploy, and speeds up your tests by an order of magnitude or more. Unfortunately, to many beginning Rails devs, it’s non-obvious, but I want <em>you</em> to write better code, and so I’m here to ‘break the secret,’ if you will, and share this awesome, powerful technique with you.</p>
<p>It’s called the ‘Plain Old Ruby Domain Object.’</p>
<p>Yep, that’s right. A Ruby class that inherets from nothing. It’s so simple that it hides in plain sight. Loved by those who’ve mastered Rails, Plain Old Ruby Objects, or “POROs” as some like to call them, are a hidden weapon against complexity. Here’s what I mean. Examine this ‘simple’ model:</p>
<pre><code>class Post &lt; ActiveRecord::Base
def self.as_dictionary
dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
Post.all.each do |p|
dictionary[p.title[0]] &lt;&lt; p
end
dictionary
end
end
</code></pre>
<p>We want to display an index page of all our posts, and do it by first letter. So we build up a dictionary, and then put our posts in it. I’m assuming we’re not paginating this, so don’t get caught up in querying for all Posts. The important thing is the idea: we can now display our posts by title:</p>
<pre><code>- Post.as_dictionary do |letter, list|
%p= letter
%ul
- list.each do |post|
%li= link_to post
</code></pre>
<p>Sure. And in one way, this code isn’t <em>bad</em>. It’s also not good: We’ve mixed a presentational concern into our model, which is supposed to represent buisness logic. So let’s fix that, via a Presenter:</p>
<pre><code>class DictionaryPresenter
def initialize(collection)
@collection = collection
end
def as_dictionary
dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
@collection.each do |p|
dictionary[p.title[0]] &lt;&lt; p
end
dictionary
end
end
</code></pre>
<p>We can use it via <code>DictionaryPresenter.new(Post.all).as_dictionary</code>. This has tons of benefits: we’ve moved presentation logic out of the model. We’ve <em>already</em> added a new feature: any collection can now be displayed as a dictionary. We can easily write isolated tests for this presenter, and they will be <em>fast</em>.</p>
<p>This post isn’t about the Presenter pattern, though, as much as I love it. This sort of concept appears in other places, too, “this domain concept deserves its own class.” Before we move to a different example, let’s expand on this further: if we want to sort our Posts by title, this class will work, but if we want to display, say, a User, it won’t, because Users don’t have titles. Furthermore, we’ll end up with a lot of Posts under “A,” because the word “a” is pretty common at the beginning of Posts, so we really want to take the second word in that case. We can make two kinds of presenters, but now we lose that generality, and the concept of ‘display by dictionary’ has two representations in our system again. You guessed it: POROs to the rescue!</p>
<p>Let’s change our presenter slightly, to also accept an organizational policy object:</p>
<pre><code>class DictionaryPresenter
def initialize(policy, collection)
@policy = policy
@collection = collection
end
def as_dictionary
dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
@collection.each do |p|
dictionary[@policy.category_for(p)] &lt;&lt; p
end
dictionary
end
end
</code></pre>
<p>Now, we can inject a policy, and have them be different:</p>
<pre><code>class UserCategorizationPolicy
def self.category_for(user)
user.username[0]
end
end
class PostCategorizationPolicy
def self.category_for(post)
if post.starts_with?("A ")
post.title.split[1][0]
else
post.title[0]
end
end
end
</code></pre>
<p>Bam!</p>
<pre><code>DictionaryPresenter.new(PostCategorizationPolicy, Post.all).as_dictionary
</code></pre>
<p>Yeah, so that’s getting a bit long. It happens. :) You can see that now each concept has one representation in our system. The presenter doesn’t care how things are organized, and the policies only dictate how things are organized. In fact, my names sorta suck, maybe it should be “UsernamePolicy” and “TitlePolicy”, actually. We don’t even care what class they are!</p>
<p>It goes further than that in other directions, too. Combining the flexibility of Ruby with one of my favorite patterns from “Working Effectively with Legacy Code,” we can take complex computations and turn them into objects. Look at this code:</p>
<pre><code>class Quote &lt; ActiveRecord::Base
#&lt;snip&gt;
def pretty_turnaround
return "" if turnaround.nil?
if purchased_at
offset = purchased_at
days_from_today = ((Time.now - purchased_at.to_time) / 60 / 60 / 24).floor + 1
else
offset = Time.now
days_from_today = turnaround + 1
end
time = offset + (turnaround * 60 * 60 * 24)
if(time.strftime("%a") == "Sat")
time += 2 * 60 * 60 * 24
elsif(time.strftime("%a") == "Sun")
time += 1 * 60 * 60 * 24
end
"#{time.strftime("%A %d %B")} (#{days_from_today} business days from today)"
end
end
</code></pre>
<p>Yikes! This method prints a turnaround time, but as you can see, it’s a complex calculation. We’d be able to understand this much more easily of we used Extract Method a few times to break it up, but then we risk polluting our Quote class with more stuff that’s only relevant to pretty turnaround calculation. Also, please ignore that this is also presentation on the model; we just care that it’s a complex bit of code for this example.</p>
<p>Okay, so here’s the first step of this refactoring, which Feathers calls “Break Out Method Object.” You can open your copy of “Working Effectively With Legacy Code” and turn to page 330 to read more. If you don’t have a copy, get one. Anyway, I digress. Here’s the plan of attack:</p>
<ol>
<li>Create a new class for the computation</li>
<li>Define a method on that class to do the new work.</li>
<li>Copy the body of the old method over, and change variable references to instance variables.</li>
<li>Give it an initialize method that takes arguments to set the instance variables used in step 3.</li>
<li>Make the old method delegate to the new class and method.</li>
</ol>
<p>I’ve changed this slightly for Ruby, since we can’t Lean On The Compiler, and a few of Feathers’ steps are about doing this. Anyway, let’s try this on that code. Step 1:</p>
<pre><code>class Quote &lt; ActiveRecord::Base
def pretty_turnaround
#snip
end
class TurnaroundCalculator
end
end
</code></pre>
<p>Two:</p>
<pre><code>class TurnaroundCalculator
def calculate
end
end
</code></pre>
<p>Three:</p>
<pre><code>class TurnaroundCalculator
def calculate
return "" if @turnaround.nil?
if @purchased_at
offset = @purchased_at
days_from_today = ((Time.now - purchased_at.to_time) / 60 / 60 / 24).floor + 1
else
offset = Time.now
days_from_today = @turnaround + 1
end
time = offset + (@turnaround * 60 * 60 * 24)
if(time.strftime("%a") == "Sat")
time += 2 * 60 * 60 * 24
elsif(time.strftime("%a") == "Sun")
time += 1 * 60 * 60 * 24
end
"#{time.strftime("%A %d %B")} (#{days_from_today} business days from today)"
end
end
</code></pre>
<p>I like to give it a generic name at first, and then give it a better one in step 5, after we see what it really does. often our code will inform us of a good name.</p>
<p>Four:</p>
<pre><code>class TurnaroundCalculator
def initialize(purchased_at, turnaround)
@purchased_at = purchased_at
@turnaround = turnaround
end
def calculate
#snip
end
end
</code></pre>
<p>Five:</p>
<pre><code>class Quote &lt; ActiveRecord::Base
def pretty_turnaround
TurnaroundCalculator.new(purchased_at, turnaround).calculate
end
end
</code></pre>
<p>Done! We should be able to run our tests and see them pass. Even if ‘run our tests’ consists of manually checking it out…</p>
<p>So what’s the advantage here? Well, we now can start the refactoring process, but we’re in our own little clean room. We can extract methods into our TurnaroundCalcuator class without polluting Quote, we can write speedy tests for just the Calculator, and we’ve split out the idea of calculation into one place, where it can easily be changed later. Here’s our class, a few refactorings later:</p>
<pre><code>class TurnaroundCalculator
def calculate
return "" if @turnaround.nil?
"#{arrival_date} (#{days_from_today} business days from today)"
end
protected
def arrival_date
real_turnaround_time.strftime("%A %d %B")
end
def real_turnaround_time
adjust_time_for_weekends(start_time + turnaround_in_seconds)
end
def adjust_time_for_weekends(time)
if saturday?(time)
time + 2 * 60 * 60 * 24
elsif sunday?(time)
time + 1 * 60 * 60 * 24
else
time
end
end
def saturday?(time)
time.strftime("%a") == "Sat"
end
def sunday?(time)
time.strftime("%a") == "Sun"
end
def turnaround_in_seconds
@turnaround * 60 * 60 * 24
end
def start_time
@purchased_at or Time.now
end
def days_from_today
if @purchased_at
((Time.now - @purchased_at.to_time) / 60 / 60 / 24).floor + 1
else
@turnaround + 1
end
end
end
</code></pre>
<p>Wow. This code I wrote three years ago isn’t perfect, but it’s almost understandable now. And each of the bits makes sense. This is after two or three waves of refactoring, which maybe I’ll cover in a separate post, becuase this was more illustrative than I thought… anyway, you get the idea. This is what I mean when I say that I shoot for roughly five-line methods in Ruby; if your code is well-factored, you can often get there.</p>
<p>This idea of extracting domain objects that are pure Ruby is even in Rails itself. Check out this route:</p>
<pre><code>root :to =&gt; 'dashboard#index', :constraints =&gt; LoggedInConstraint
</code></pre>
<p>Huh? LoggedInConstraint?</p>
<pre><code>class LoggedInConstraint
def self.matches?(request)
current_user
end
end
</code></pre>
<p>Whoah. Yep. A domain object that describes our routing policy. Awesome. Also, validations, blatantly stolen from <a href="http://omgbloglol.com/post/392895742/improved-validations-in-rails-3">omgbloglol</a>:</p>
<pre><code>def SomeClass &lt; ActiveRecord::Base
validate :category_id, :proper_category =&gt; true
end
class ProperCategoryValidator &lt; ActiveModel::EachValidator
def validate_each(record, attribute, value)
unless record.user.category_ids.include?(value)
record.errors.add attribute, 'has bad category.'
end
end
end
</code></pre>
<p>This isn’t a plain Ruby class, but you get the idea.</p>
<p>Now, you might be thinking, “Steve: This isn’t just for Rails! You’ve lied!” Why yes, actually, you’ve caught me: this isn’t the secret to Rails OO, it’s more of a general OO design guideline. But there’s something special about Rails which seems to lure you into the trap of never breaking classes down. Maybe it’s that <code>lib/</code> feels like such a junk drawer. Maybe it’s that the fifteen minute examples only ever include ActiveRecord models. Maybe it’s that more Rails apps than not are (WARNING: UNSUBSTANTIATED CLAIM ALERT) closed source than open, so we don’t have as many good examples to draw upon. (I have this hunch since Rails is often used to build sites for companies. Gems? Sure? My web app? Not so much. I have no stats to back this up, though.)</p>
<p>In summary: Extracting domain objects is good. They keep your tests fast, your code small, and make it easier to change things later. I have some more to say about this, specifically the “keeps test fast” part, but I’m already pushing it for length here. Until next time!</p></content:encoded>
</item>
<item>
<title>The self improvement pomodoro</title>
<link>https://steveklabnik.com/writing/the-self-improvement-pomodoro/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-self-improvement-pomodoro/</guid>
<pubDate>Sun, 04 Sep 2011 00:00:00 GMT</pubDate>
<content:encoded><p>By now, the <a href="http://www.pomodorotechnique.com/">pomodoro technique</a> is pretty well-known amongst programmers. I forget where I heard of it first, but if you haven’t, here it is: Break your day up into 30 minute chunks. Work for 25 minutes, break for 5. Repeat. It’s easy to stay concentrated when you know a break is coming up soon, the fact that you can fit two in an hour fits with my natural sense of order in the world, and once you get in the rhythm, you Get Shit Done. Heck, I liked the idea so much that I <a href="http://rubygems.org/gems/pomodoro">made a Ruby gem</a>.</p>
<p>That said, I hadn’t been following the technique for a while, because it’s easy to lose discipline. Happens to the best of us. Recently, though, it’s crept back into my workflow, largely thanks to Jeff Casimir. I was even lazy enough to not use my gem, and try something else. So I grabbed the software he uses from the App Store, and fired it up. It presented me with an interesting option to give my pomodoro a name before starting it. Pretty much without thinking, I typed ‘rstat.us rails3 port,’ since that’s what I was working on, and away I went.</p>
<p>The more I think about it, however, the more interesting the naming aspect becomes. I can see how long and how often I’m working on various things, and over the past few weeks, it’s been neat to examine. When doing some reading this morning, my groggy brain combined a few different things together:</p>
<ul>
<li>Programmers often don’t have the time to practice their craft</li>
<li>I work on a lot of random software projects</li>
<li>These projects are often of varying quality, especially regarding things like documentation and tests</li>
<li>Almost none of the gems that have been released are at a 1.0 release (can anyone find this blog post for me? I can’t. It’s something about “Just Release It”…)</li>
</ul>
<p>So, with an aim of combining all these things, I’m going to give something a try, and you should too: I’m going to name at least one pomodoro a day “self-improvement,” and use that time to go back over my random projects on GitHub and clean them up. I’m going to use that time to start programming in a totally new language. I’m going to use that time to do that refactoring I’ve been avoiding. I’m going to use that time to read a book.</p>
<p>Basically, I’m going to force myself to invest in myself. Make sure to find time to invest in you, too.</p></content:encoded>
</item>
<item>
<title>I'm making it dead simple to contribute to Ruby's documentation</title>
<link>https://steveklabnik.com/writing/im-making-it-dead-simple-to-contribute-to-ruby-s-documentation/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/im-making-it-dead-simple-to-contribute-to-ruby-s-documentation/</guid>
<pubDate>Mon, 22 Aug 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Okay! So, if you’d read <a href="/2011/05/10/contributing-to-ruby-s-documentation.html">my previous article on this</a>, you’d know how easy it is to contribute to Ruby’s Documentaiton.</p>
<blockquote>
<p>But Steve, I’m still kinda scared.</p>
</blockquote>
<p>Okay, so here we go: I’m making it even easier on you.</p>
<p>Send me what you want changed, and how, and I’ll make a patch and submit it on your behalf.</p>
<p>No, seriously, I will. I already did once. <a href="http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/38873">This guy</a> posted something about how it was too hard, <a href="http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/38875">so I made a patch for him</a>. And now Ruby is better because of it.</p>
<h2>Patches don’t have to be patches</h2>
<p>Seriously, I don’t even mean diff outputs. I just got this email:</p>
<pre><code>&gt; I'm eager to contribute docs but I don't know where to start: I don't
&gt; mean pulling down the repo and all that, I mean, I see this link
&gt; here:http://segment7.net/projects/ruby/documentation\_coverage.txt but
&gt; all I see is a ton of "# is documented" comments. Where does a
&gt; relative n00b begin? Poking around in the source code is, well,
&gt; intimidating. I'm completely new to this but I take direction well.
&gt; Just point me in the right direction. Just give me something to
&gt; document, please!
</code></pre>
<p>I sent this back:</p>
<pre><code>&gt; No code diving needed! That's part of it. Let's do this: Find the documentation for one of your favorite methods. Here, I'll pick one: http://ruby-doc.org/core/classes/Array.html#M000278 okay, that looks okay, but look at compact!:
&gt;
&gt; &gt; Removes nil elements from the array. Returns nil if no changes were made, otherwise returns &lt;/i&gt;ary&lt;/i&gt;.
&gt;
&gt; Why's that &lt;/i&gt; there? Total screw-up. For example. So, send me this email:
&gt;
&gt; "Hey Steve-
&gt;
&gt; Check out the docs for Array#compact!, here: http://ruby-doc.org/core/classes/Array.html#M000279 . There's an extra &lt;/i&gt; that's screwing things up.
&gt;
&gt; -Jonathan"
&gt;
&gt; Done! I'll go from there. How about this one:
&gt;
&gt; "Hey Steve-
&gt;
&gt; I checked out the docs for Time._load, here: http://www.ruby-doc.org/core/classes/Time.html#M000394
&gt;
&gt; These docs kind of suck. What if I don't know what Marshal is? There should at _least_ be a link to Marshall, here: http://ruby-doc.org/core/classes/Marshal.html And it should probably say something like "You can get a dumped Time object by using _dump: http://www.ruby-doc.org/core/classes/Time.html#M000393
&gt;
&gt; - Jonathan"
&gt;
&gt; I'm sure you can find something that's formatted wrong, or worded incorrectly, or anything else.
&gt;
&gt; -Steve
</code></pre>
<p>Now, the <em>closer</em> it is to an actual patch, the faster I’ll be able to do it. A vague “this documentation is confusing, but I’m not sure why” is helpful, but will take longer. I’d rather get that email than not. If you’re not sure, hit send.</p>
<h2>Just Do It</h2>
<p>And so I’ll do it again. Scared to deal with doc patches? I’ll make them up. I’m serious. I’ll do it. Send them to me.</p>
<p>That’s all.</p>
<p>Send me patches. <a href="mailto:[email protected]">[email protected]</a>. Do it.</p></content:encoded>
</item>
<item>
<title>Some people understand REST and HTTP</title>
<link>https://steveklabnik.com/writing/some-people-understand-rest-and-http/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/some-people-understand-rest-and-http/</guid>
<pubDate>Sun, 07 Aug 2011 00:00:00 GMT</pubDate>
<content:encoded><p>This is a follow-up post to my post <a href="/2011/07/03/nobody-understands-rest-or-http.html">here</a>. You probably want to read that first.</p>
<p>UPDATE: Please note that ‘<a href="/posts/2012-02-23-rest-is-over">REST is over’</a>. ’Hypermedia API’ is the proper term now.</p>
<h2>A few words on standards versus pragmatism</h2>
<p>When I wrote my first post on this topic, I tried to take a stance that would be somewhat soft, yet forceful. Engineering is the art of making the proper trade-offs, and there are times when following specifications is simply not the correct decision. With that said, my motivation for both of these posts is to eradicate some of the ignorance that some developers have about certain areas of the HTTP spec and Fielding’s REST paper. If you understand the correct way, yet choose to do something else for an informed reason, that’s absolutely, 100% okay. There’s no use throwing out the baby with the bathwater. But ignorance is never a good thing, and most developers are ignorant when it comes to the details of REST.</p>
<p>Secondly, while I think that REST is the best way to develop APIs, there are other valid architectural patterns, too. Yet calling non-REST APIs ‘RESTful’ continues to confuse developers as to what “RESTful” means. I’m not sure what exactly we should call “RESTish” APIs (hey, there we go, hmmm…) but I’m also not under the illusion that I personally will be able to make a huge dent in this. Hopefully you, humble reader, will remember this when dealing with APIs in the future, and I’ll have made a tiny dent, though.</p>
<h2>So who <em>does</em> understand REST?</h2>
<p>As it turns out, there are two companies that you’ve probably heard of who have APIs that are much more RESTful than many others: <a href="http://www.twilio.com/docs/api/rest/">Twilio</a> and <a href="http://developer.github.com/">GitHub</a>. Let’s take a look at GitHub first.</p>
<h3>GitHub: logically awesome</h3>
<p>GitHub’s developer resources are not only beautiful, but thorough. In addition, they make use of lots more of REST.</p>
<h3>The good</h3>
<p>GitHub uses <a href="http://developer.github.com/v3/mime/">custom MIME</a> types for all of their responses. They’re using the vendor extensions that I talked about in my post, too. For example:</p>
<p>application/vnd.github-issue.text+json</p>
<p>Super cool.</p>
<p>Their <a href="http://developer.github.com/v3/#authentication">authentication</a> works in three ways: HTTP Basic, OAuth via an Authentication Header, or via a parameter. This allows for a maximum amount of compatibility across user agents, and gives the user some amount of choice.</p>
<p>Their <a href="http://developer.github.com/v3/#pagination">Pagination</a> uses a header I didn’t discuss in part I: the Link header. <a href="http://tools.ietf.org/html/rfc5988">Here</a>’s a link to the reference. Basically, Link headers enable HATEOAS for media types which aren’t hypertext. This is important, especially regarding JSON, since JSON isn’t hypermedia. More on this at the end of the post. Anyway, so pagination on GitHub:</p>
<pre><code>$ curl -I "https://api.github.com/users/steveklabnik/gists"
HTTP/1.1 200 OK
Server: nginx/1.0.4
Date: Sun, 07 Aug 2011 16:34:48 GMT
Content-Type: application/json
Connection: keep-alive
Status: 200 OK
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4994
Link: &lt;https://api.github.com/users/steveklabnik/gists?page=2&gt;; rel="next", &lt;https://api.github.com/users/steveklabnik/gists?page=33333&gt;; rel="last"
Content-Length: 29841
</code></pre>
<p>The Link header there shows you how to get to the next page of results. You don’t need to know how to construct the URL, you just have to parse the header and follow it. This, for example, is a great way to connect a resource that’s not text-based, such as a PNG, to other resources.</p>
<h3>The bad</h3>
<p>There’s really only one place that GitHub doesn’t knock it out of the park with their new API, and that’s HATEOAS. GitHub’s API isn’t discoverable, because there’s no information at the root:</p>
<pre><code>$ curl -I https://api.github.com/
HTTP/1.1 302 Found
Server: nginx/1.0.4
Date: Sun, 07 Aug 2011 16:44:02 GMT
Content-Type: text/html;charset=utf-8
Connection: keep-alive
Status: 302 Found
X-RateLimit-Limit: 5000
Location: http://developer.github.com
X-RateLimit-Remaining: 4993
Content-Length: 0
</code></pre>
<p>Well, at least, this is how they present it. If you ask for JSON:</p>
<pre><code>$ curl -I https://api.github.com/ -H "Accept: application/json"
HTTP/1.1 204 No Content
Server: nginx/1.0.4
Date: Sun, 07 Aug 2011 16:45:32 GMT
Connection: keep-alive
Status: 204 No Content
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4991
Link: &lt;users/{user}&gt;; rel="user", &lt;repos/{user}/{repo}&gt;; rel="repo", &lt;gists&gt;; rel="gists"
</code></pre>
<p>You do get Links, but you have to construct things yourself. As a user, you get the same thing. It doesn’t change the links to point to your repos, it doesn’t give you links to anything else that you can do with the API.</p>
<p>Instead, the root should give you a link to the particular resources that you can actually view. Maybe something like this:</p>
<pre><code>$ curl -I https://api.github.com/ -H "Accept: application/json" -u "username:password"
HTTP/1.1 204 No Content
Server: nginx/1.0.4
Date: Sun, 07 Aug 2011 16:45:32 GMT
Connection: keep-alive
Status: 204 No Content
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4991
Link: &lt;/gists/public&gt;; rel="public_gists", &lt;/user/repos&gt;; rel="repos", &lt;gists&gt;; rel="gists"
</code></pre>
<p>And a bunch more, for all of the other resources that are available. This would make the API truly discoverable, and you wouldn’t be forced to read their gorgeous documentation. :)</p>
<h3>Twilio</h3>
<p>I’ve always really enjoyed Twilio. Their API is incredibly simple to use. I once hooked up a little “Text me when someone orders something from my site” script, and it took me about fifteen minutes. Good stuff.</p>
<h3>The good</h3>
<p>Twilio has got the HATEOAS thing down. Check it out, their home page says that the base URL is “https://api.twilio.com/2010-04-01”. Without looking at any of the rest of their docs, (I glanced at a page or two, but I didn’t really read them fully yet), I did this:</p>
<pre><code>$ curl https://api.twilio.com/2010-04-01
&lt;?xml version="1.0"?&gt;
&lt;TwilioResponse&gt;
&lt;Version&gt;
&lt;Name&gt;2010-04-01&lt;/Name&gt;
&lt;Uri&gt;/2010-04-01&lt;/Uri&gt;
&lt;SubresourceUris&gt;
&lt;Accounts&gt;/2010-04-01/Accounts&lt;/Accounts&gt;
&lt;/SubresourceUris&gt;
&lt;/Version&gt;
&lt;/TwilioResponse&gt;
</code></pre>
<p>I introduced some formatting. Hmm, okay, Accounts. Let’s check this out:</p>
<pre><code>$ curl https://api.twilio.com/2010-04-01/Accounts&lt;?xml version="1.0"?&gt;
&lt;TwilioResponse&gt;&lt;RestException&gt;&lt;Status&gt;401&lt;/Status&gt;&lt;Message&gt;Authenticate&lt;/Message&gt;&lt;Code&gt;20003&lt;/Code&gt;&lt;MoreInfo&gt;http://www.twilio.com/docs/errors/20003&lt;/MoreInfo&gt;&lt;/RestException&gt;&lt;/TwilioResponse&gt;
</code></pre>
<p>Okay, so I have to be authenticated. If I was, I’d get something like this:</p>
<pre><code>&lt;TwilioResponse&gt;
&lt;Account&gt;
&lt;Sid&gt;ACba8bc05eacf94afdae398e642c9cc32d&lt;/Sid&gt;
&lt;FriendlyName&gt;Do you like my friendly name?&lt;/FriendlyName&gt;
&lt;Type&gt;Full&lt;/Type&gt;
&lt;Status&gt;active&lt;/Status&gt;
&lt;DateCreated&gt;Wed, 04 Aug 2010 21:37:41 +0000&lt;/DateCreated&gt;
&lt;DateUpdated&gt;Fri, 06 Aug 2010 01:15:02 +0000&lt;/DateUpdated&gt;
&lt;AuthToken&gt;redacted&lt;/AuthToken&gt;
&lt;Uri&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d&lt;/Uri&gt;
&lt;SubresourceUris&gt;
&lt;AvailablePhoneNumbers&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/AvailablePhoneNumbers&lt;/AvailablePhoneNumbers&gt;
&lt;Calls&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Calls&lt;/Calls&gt;
&lt;Conferences&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Conferences&lt;/Conferences&gt;
&lt;IncomingPhoneNumbers&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/IncomingPhoneNumbers&lt;/IncomingPhoneNumbers&gt;
&lt;Notifications&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Notifications&lt;/Notifications&gt;
&lt;OutgoingCallerIds&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/OutgoingCallerIds&lt;/OutgoingCallerIds&gt;
&lt;Recordings&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Recordings&lt;/Recordings&gt;
&lt;Sandbox&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Sandbox&lt;/Sandbox&gt;
&lt;SMSMessages&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/SMS/Messages&lt;/SMSMessages&gt;
&lt;Transcriptions&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Transcriptions&lt;/Transcriptions&gt;
&lt;/SubresourceUris&gt;
&lt;/Account&gt;
&lt;/TwilioResponse&gt;
</code></pre>
<p>Awesome. I can see my all of the other resources that I can interact with. Other than knowing how to authenticate, I can follow the links from the endpoint, and discover their entire API. Rock. This is the way things are supposed to be.</p>
<h3>The bad</h3>
<p>This:</p>
<pre><code>$ curl https://api.twilio.com/2010-04-01/Accounts -H "Accept: application/json"
&lt;?xml version="1.0"?&gt;
&lt;TwilioResponse&gt;&lt;RestException&gt;&lt;Status&gt;401&lt;/Status&gt;&lt;Message&gt;Authenticate&lt;/Message&gt;&lt;Code&gt;20003&lt;/Code&gt;&lt;MoreInfo&gt;http://www.twilio.com/docs/errors/20003&lt;/MoreInfo&gt;&lt;/RestException&gt;&lt;/TwilioResponse&gt;
</code></pre>
<p>Versus this:</p>
<pre><code>$ curl https://api.twilio.com/2010-04-01/Accounts.json
{"status":401,"message":"Authenticate","code":20003,"more_info":"http:\/\/www.twilio.com\/docs\/errors\/20003"}
</code></pre>
<p>:/ Returning JSON when your resource ends with ‘.json’ isn’t bad, but not respecting the Accept header, even when you return the right MIME type, is just unfortunate.</p>
<h2>… and a little Announcement</h2>
<p>It seems that this is a topic that people are really interested in. Part I of this article was pretty well-received, and I got lots of great email and feedback from people. It was also made pretty clear by <a href="http://twitter.com/#!/wayneeseguin/status/97733413611638784">a few</a> people that they want more content from me on this topic.</p>
<p>So I decided to write a book about it. You can check out the site for “<a href="http://designinghypermediaapis.com/">Get Some REST</a>”, and put in your email address. Then you’ll get updated when I start pre-release sales.</p>
<p>So what’s in “Get Some REST”? It’s going to be a full description of how to build RESTful web applications, from the ground up. Designing your resources, laying out an API, all the details. I’m going to try to keep most of the content language-agnostic, but provide code samples in Rails 3.1, as well.</p>
<p>I plan on writing a bunch of content, and then releasing the book at half-price in beta. Early adopters will be able to get their two cents in, and I’ll cover things they still have questions on. It’ll be available under a liberal license, in PDF, ePub, all that good stuff.</p>
<p>I’ve also set up a Twitter account at <a href="http://twitter.com/hypermediaapis">@hypermediaapis</a>. I’ll be tweeting updates about the book, and also other good content related to RESTful design.</p></content:encoded>
</item>
<item>
<title>Twitter is too important to be owned by Twitter</title>
<link>https://steveklabnik.com/writing/twitter-is-to-important-to-be-owned-by-twitter/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/twitter-is-to-important-to-be-owned-by-twitter/</guid>
<pubDate>Sun, 24 Jul 2011 00:00:00 GMT</pubDate>
<content:encoded><p>When I spoke on a panel at <a href="http://therealtimereport.com/">RTLM NY ’11</a>, I got a little rise out of the crowd when I said “Twitter is too important to be owned by Twitter.” This is still absolutely true, and yesterday was a great example of that: the #fuckyouwashington debacle.</p>
<p>If you haven’t heard, <a href="http://twitter.com/#!/jeffjarvis">Jeff Jarvis</a> started something on Twitter last night: <code>#fuckyouwashington</code>.</p>
<p>As of right this second, when I <a href="http://twitter.com/#!/search/realtime/%23fuckyouwashington">search for #fuckyouwashington on Twitter</a>, they’re coming in at two or three a second. It’s crazy. And it’s really interesting to read what people have to say, pro and con, picking their own issues, all of that stuff. Yet, it’s been removed from the trending topics.</p>
<p>Now, maybe this particular trend was simply blocked because of the profanity. Sure, whatever. I don’t like that, but some might. And Twitter certainly allows all kinds of trends to stick around that would also be ‘offensive,’ for example, #babymakingsong, #turnoff, and #itsalwaystheuglypeople are currently trending. Don’t even get me started on trends like #whatblackgirlslike #whatwhitegirls like and the other racially charged topics. Sure, fine, it’s their site, they can do whatever they want.</p>
<p>That, however, is the root of the issue: they can do whatever they want. And we’re all beholden to them. Nothing we can do. Maybe I’m not sensitive to four letter words, and I <em>do</em> want to hear about things like this. Too bad, Twitter says no. Maybe I don’t want <a href="http://venturebeat.com/2011/07/08/twitter-adding-promoted-tweets-to-your-stream-this-summer/">promoted tweets added to my stream</a>. Too bad, Twitter says yes.</p>
<p>And that’s why I’m feeling worse and worse about Twitter every day. It sucks, because I really like Twiter, the website. But Twitter, the company, really bums me out. They promised all of us nerds long ago that they wouldn’t do this kind of thing, and they’re not keeping that promise. Which means I’m not really getting emotionally invested in Twitter any more. It’s still valuable enough to me to keep using, but I can’t care about it the way I used to.</p>
<p>This is why we’re still hacking away at <a href="http://rstat.us/">rstat.us</a>. If you haven’t heard about it before, you can <a href="http://blog.steveklabnik.com/2011/03/23/announcing-rstat-us.html">read my release announcement</a>. If you’re lazier than that, here’s the summary: rstat.us is a distributed, open-source Twitter. You can use our public node at the website, or get your own copy of the code and run your own, make whatever customizations you’d like, and you’re still on equal footing with everyone else in the network. We’re on the cusp of a 1.0 announcement, and I’d love to have been able to make that in this post, but we’re still working out some last kinks. If you’ve seen it before, you won’t find much differences right now, but we have 100% standards compliance, a complete visual refresh, and some other fun goodies waiting in the wings, ready to come out shortly.</p>
<p>Just remember: if you use a service like Twitter that’s controlled by a company like Twitter, you have to play by their rules. And sometimes, they’ll change the rules, and it’ll be time to move on.</p></content:encoded>
</item>
<item>
<title>Nobody understands REST or HTTP</title>
<link>https://steveklabnik.com/writing/nobody-understands-rest-or-http/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/nobody-understands-rest-or-http/</guid>
<pubDate>Sun, 03 Jul 2011 00:00:00 GMT</pubDate>
<content:encoded><h2><strong>HI <a href="http://news.ycombinator.com/item?id=3635085">HN</a>, PLEASE READ THIS!!!</strong></h2>
<p>Since I’ve posted this, I’ve refined a few of my positions on things. Everyone learns and grows, and while I still stand by most of what I said, I specifically don’t agree that versioning the media type is how to properly version APIs. Hypermedia APIs should not actually use explicit versioning, but I’d rather see a version in the URI with HATEOAS than no HATEOAS and versioned media types. I’ve been meaning to update this post and write more, but alas, my work on <a href="http://designinghypermediaapis.com/">Get some REST</a> has taken priority. I don’t have a HN account, so feel free to <a href="mailto:[email protected]">email me</a> with any thoughts or questions!</p>
<p>Furthermore, everything in engineering is ALWAYS a trade-off. I primarily wish that more people understood the tools that HTTP provides them with, and made an informed choice, rather than cargo-culting what they’ve seen others do.</p>
<p>Update: Part II of this post is <a href="/2011/08/07/some-people-understand-rest-and-http.html">here</a>. Check it out, and there’s an announcement at the end!</p>
<p>Update: Please note that <a href="/posts/2012-02-23-rest-is-over">REST is over. Hypermedia API is the new nomenclature.</a></p>
<p>The more that I’ve learned about web development, the more that I’ve come to appreciate the thoroughness and thoughtfulness of the authors of the HTTP RFC and Roy Fielding’s dissertation. It seems like the answers to most problems come down to “There’s a section of the spec for that.” Now, obviously, they’re not infallible, and I’m not saying that there’s zero room for improvement. But it really disappoints me when people don’t understand the way that a given issue is supposed to be solved, and so they make up a partial solution that solves their given case but doesn’t jive well with the way that everything else works. There are valid criticisms of the specs, but they have to come from an informed place about what the spec says in the first place.</p>
<p>Let’s talk about a few cases where either REST or HTTP (which is clearly RESTful in its design) solves a common web development problem.</p>
<h3>I need to design my API</h3>
<p>This one is a bit more general, but the others build off of it, so bear with me.</p>
<p>The core idea of REST is right there in the name: “Representational State Transfer” It’s about transferring representations of the state… of resources. Okay, so one part isn’t in the name. But still, let’s break this down.</p>
<h3>Resources</h3>
<p>From <a href="http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_2_1_1">Fielding’s dissertation</a>:</p>
<blockquote>
<p>The key abstraction of information in REST is a resource. Any information that can be named can be a resource: a document or image, a temporal service (e.g. “today’s weather in Los Angeles”), a collection of other resources, a non-virtual object (e.g. a person), and so on. In other words, any concept that might be the target of an author’s hypertext reference must fit within the definition of a resource. A resource is a conceptual mapping to a set of entities, not the entity that corresponds to the mapping at any particular point in time.</p>
</blockquote>
<p>When we interact with a RESTful system, we’re interacting with a set of resources. Clients request resources from the server in a variety of ways. But the key thing here is that resources are <em>nouns</em>. So a RESTful API consists of a set of URIs that map entities in your system to endpoints, and then you use HTTP itself for the verbs. If your URLs have action words in them, you’re doing it wrong. Let’s look at an example of this, from the early days of Rails. When Rails first started messing around with REST, the URLs looked like this:</p>
<pre><code>/posts/show/1
</code></pre>
<p>If you use Rails today, you’ll note that the corresponding URL is this:</p>
<pre><code>/posts/1
</code></pre>
<p>Why? Well, it’s because the ‘show’ is unnecessary; you’re performing a GET request, and that demonstrates that you want to show that resource. It doesn’t need to be in the URL.</p>
<h3>A digression about actions</h3>
<p>Sometimes, you need to perform some sort of action, though. Verbs are useful. So how’s this fit in? Let’s consider the example of transferring money from one Account to another. You might decided to build a URI like this:</p>
<pre><code>POST /accounts/1/transfer/500.00/to/2
</code></pre>
<p>to transfer $500 from Account 1 to Account 2. But this is wrong! What you really need to do is consider the nouns. You’re not transferring money, you’re creating a Transaction resource:</p>
<pre><code>POST /transactions HTTP/1.1
Host: &lt;snip, and all other headers&gt;
from=1&amp;to=2&amp;amount=500.00
</code></pre>
<p>Got it? So then, it returns the URI for your new Transaction:</p>
<pre><code>HTTP/1.1 201 OK
Date: Sun, 3 Jul 2011 23:59:59 GMT
Content-Type: application/json
Content-Length: 12345
Location: http://foo.com/transactions/1
{"transaction":{"id":1,"uri":"/transactions/1","type":"transfer"}}
</code></pre>
<p>Whoah, <a href="http://timelessrepo.com/haters-gonna-hateoas">HATEOS</a>! Also, it may or may not be a good idea to return this JSON as the body; the important thing is that we have the Location header which tells us where our new resource is. If we give a client the ID, they might try to construct their own URL, and the URI is a little redundant, since we have one in the Location. Regardless, I’m leaving that JSON there, because that’s the way I typed it first. I’d love to <a href="mailto:[email protected]">hear your thoughts on this</a> if you feel strongly one way or the other.</p>
<p>EDIT: I’ve since decided that yes, including the URI is a bad idea. The Location header makes much more sense. More on this in Part ii, yet to come.</p>
<p>Anyway, so now we can GET our Transaction:</p>
<pre><code>GET /transactions/1 HTTP/1.1
Accept: application/json
</code></pre>
<p>and the response:</p>
<pre><code>HTTP/1.1 blah blah blah
{"id":1,"type":"transfer","status":"in-progress"}
</code></pre>
<p>So we know it’s working. We can continue to poll the URI and see when our transaction is finished, or if it failed, or whatever. Easy! But it’s about manipulating those nouns.</p>
<h3>Representations</h3>
<p>You’ll notice a pair of headers in the above HTTP requests and responses: Accept and Content-Type. These describe the different ‘representation’ of any given resource. From <a href="http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_2_1_2">Fielding</a>:</p>
<blockquote>
<p>REST components perform actions on a resource by using a representation to capture the current or intended state of that resource and transferring that representation between components. A representation is a sequence of bytes, plus representation metadata to describe those bytes. Other commonly used but less precise names for a representation include: document, file, and HTTP message entity, instance, or variant.A representation consists of data, metadata describing the data, and, on occasion, metadata to describe the metadata (usually for the purpose of verifying message integrity).</p>
</blockquote>
<p>So <code>/accounts/1</code> represents a resource. But it doesn’t include the form that the resource takes. That’s what these two headers are for.</p>
<p>This is also why adding <code>.html</code> to the end of your URLs is kinda silly. If I request <code>/accounts/1.html</code> with an <code>Accept</code> header of <code>application/json</code>, then I’ll get JSON. The <code>Content-Type</code> header is the server telling us what kind of representation it’s sending back as well. The important thing, though, is that a given resource can have many different representations. Ideally, there should be one unambiguous source of information in a system, and you can get different representations using <code>Accept</code>.</p>
<h3>State and Transfer</h3>
<p>This is more about the way HTTP is designed, so I’ll just keep this short: Requests are designed to be stateless, and the server holds all of the state for its resources. This is important for caching and a few other things, but it’s sort of out of the scope of this post.</p>
<p>Okay. With all of that out of the way, let’s talk about some more specific problems that REST/HTTP solve.</p>
<h3>I want my API to be versioned</h3>
<p>The first thing that people do when they want a versioned API is to shove a /v1/ in the URL. <em>THIS IS BAD!!!!!1</em>. <code>Accept</code> solves this problem. What you’re really asking for is “I’d like the version two representation of this resource.” So use accept!</p>
<p>Here’s an example:</p>
<pre><code>GET /accounts/1 HTTP/1.1
Accept: application/vnd.steveklabnik-v2+json
</code></pre>
<p>You’ll notice a few things: we have a + in our MIME type, and before it is a bunch of junk that wasn’t there before. It breaks down into three things: <code>vnd</code>, my name, and <code>v2</code>. You can guess what v2 means, but what about <code>vnd</code>. It’s a <a href="http://tools.ietf.org/html/rfc4288#section-3.2">Vendor MIME Type</a>. After all, we don’t really want just any old JSON, we want my specific form of JSON. This lets us still have our one URL to represent our resource, yet version everything appropriately.</p>
<p>I got a comment from <a href="http://avdi.org/">Avdi Grimm</a> about this, too:</p>
<blockquote>
<p>Here’s an article you might find interesting: http://www.informit.com/articles/article.aspx?p=1566460The author points out that MIMETypes can have parameters, which means you can actually have a mimetype that looks like this:vnd.example-com.foo+json; version=1.0Sadly, Rails does not (yet) understand this format.</p>
</blockquote>
<h3>I’d like my content to be displayed in multiple languages</h3>
<p>This is related, but a little different. What about pages in different languages? Again, we have a question of representation, not one of content. /en/whatever is not appropriate here. Turns out, <a href="http://tools.ietf.org/html/rfc2616#section-14.4">there’s a header for that: Accept-Language</a>. Respect the headers, and everything works out.</p>
<p>Oh, and I should say this, too: this doesn’t solve the problem of “I’d like to read this article in Spanish, even though I usually browse in English.” Giving your users the option to view your content in different ways is a good thing. Personally, I’d consider this to fall out in two ways:</p>
<ul>
<li>It’s temporary. Stick this option in the session, and if they have the option set, it trumps the header. You’re still respecting their usual preferences, but allowing them to override it.</li>
<li>It’s more permanent. Make it some aspect of their account, and it trumps a specific header. Same deal.</li>
</ul>
<h3>I’d like my content to have a mobile view</h3>
<p>Sounds like I’m beating a dead horse, but again: it’s a representation question. In this case, you’d like to vary the response by the User-Agent: give one that’s mobile-friendly. There’s a whole list of <a href="http://www.w3.org/TR/mobile-bp/">mobile best practices</a> that the w3c recommends, but the short of it is this: the User-Agent should let you know that you’re dealing with a mobile device. For example, here’s the first iPhone UA:</p>
<pre><code>Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A543a Safari/419.3
</code></pre>
<p>Then, once detecting you have a mobile User-Agent, you’d give back a mobile version of the site. Hosting it on a subdomain is a minor sin, but really, like I said above, this is really a question of representation, and so having two URLs that point to the same resource is kinda awkward.</p>
<p>Whatever you do, for the love of users, please don’t detect these headers, then redirect your users to m.whatever.com, at the root. One of my local news websites does this, and it means that every time I try to follow a link from Twitter in my mobile browser, I don’t see their article, I see their homepage. It’s infuriating.</p>
<h3>I’d like to hide some of my content</h3>
<p>Every once in a while, you see a story like this: <a href="http://www.boingboing.net/2010/10/25/local-newspaper-boas.html">Local paper boasts ultimate passive-agressive paywall policy</a>. Now, I find paywalls distasteful, but this is not the way to do it. There are technological means to limit content on the web: making users be logged-in to read things, for example.</p>
<p>When this was discussed on Hacker News, <a href="http://news.ycombinator.com/item?id=1834075">here’s</a> what I had to say:</p>
<p>nkurz:</p>
<blockquote>
<p>I presume if I had an unattended roadside vegetable stand with a cash-box, that I’d be able to prosecute someone who took vegetables without paying, certainly if they also made off with the cash-box. Why is this different on the web? And if a written prohibition has no legal standing, why do so many companies pay lawyers to write click-through “terms of service” agreements?</p>
</blockquote>
<p>me:</p>
<blockquote>
<p>Why is this different on the web?Let’s go through what happens when I visit a web site. I type a URL in my bar, and hit enter. My web browser makes a request via http to a server, and the server inspects the request, determines if I should see the content or not, and returns either a 200 if I am allowed, and a 403 if I’m not. So, by viewing their pages, I’m literally asking permission, and being allowed.It sounds to me like a misconfiguration of their server; it’s not doing what they want it to.</p>
</blockquote>
<h3>I’d like to do some crazy ajax, yet have permalinks</h3>
<p>This is an example of where the spec is obviously deficient, and so something had to be done.</p>
<p>As the web grew, AJAXy ‘web applications’ started to become more and more the norm. And so applications wanted to provide deep-linking capabilities to users, but there’s a problem: they couldn’t manipulate the URL with Javascript without causing a redirect. They <em>could</em> manipulate the anchor, though. You know, that part after the #. So, Google came up with a convention: <a href="http://code.google.com/web/ajaxcrawling/docs/getting-started.html">Ajax Fragments</a>. This fixed the problem in the short term, but then the spec got fixed in the long term: <a href="http://dev.w3.org/html5/spec-author-view/history.html">pushState</a>. This lets you still provide a nice deep URL to your users, but not have that awkward #!.</p>
<p>In this case, there was a legitimate technical issue with the spec, and so it’s valid to invent something. But then the standard improved, and so people should stop using #! as HTML5 gains browser support.</p>
<h3>In conclusion</h3>
<p>Seriously, most of the problems that you’re solving are social, not technical. The web is decades old at this point, most people have considered these kinds of problems in the past. That doesn’t mean that they always have the right answer, but they usually do have an answer, and it’d behoove you to know what it is before you invent something on your own.</p></content:encoded>
</item>
<item>
<title>Single text, many masters</title>
<link>https://steveklabnik.com/writing/single-text-many-masters/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/single-text-many-masters/</guid>
<pubDate>Sat, 21 May 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Hey everyone. Here’s a draft of an essay I’ve been working on. I’d love to hear your feedback.</p>
<p>The word ‘engineering’ has a deep connection to the word ‘trade-offs’ in my mind. Most engineering decisions come down to evaluating a few differing alternatives, and often multiple factors end up being negatively correlated. You can make something stronger, but then it will be heavier. It can be made faster, but then it’s significantly more expensive. A good engineer is able to take all of these factors into account, and design a system such that it maximizes its effectiveness across the sum of all of the relevant constraints. No matter if you consider the act of writing software an art, science, or engineering, its indisputable that designing complex software systems is identical in this respect. There are dozens of different metrics that system architects take into consideration while crafting a plan of attack, but but there’s a deeper question of balance here that’s significantly different than these more traditional engineering issues.</p>
<p>Text, in the form of source code, presents unique challenges of composition. These difficulties all stem from the same root: source code is a singular text, but must be intelligible to multiple, simultaneous audiences. More traditional forms of authorship still take audience into consideration, of course, but the decision to engage a diverse group of people is the choice of the writer. While mass appeal may be something that certain authors strive to attain, it’s not an inherent property of their chosen form of expression. Source code, while text, inhabits a multiplicity of forms, and software developers are confronted with this inherent multi-faceted complexity when composing any particular software work. Some of these forms suit certain audiences better than others, and so it falls to the programmer to manage which form they are currently working in, consider which audience they are attempting to write for, and arrange all of these forms amongst one another in such a way that any particular audience is able to navigate and extract the proper information from the source without confusion.</p>
<p>In this post, I’ll expand on the concept of audiences for code, and in a future post, I’ll explore the simultaneous forms that code takes.</p>
<h2>The multitude of audiences</h2>
<h3>The default audience: the computer</h3>
<blockquote>
<p>Science is what we understand well enough to explain to a computer. Art is everything else we do.Don Knuth</p>
</blockquote>
<p>This may seem obvious, but the when considering the question of “Who are programs written for?”, many would say “The computer. Duh.” In many ways, a computer is the primary reader of a particular piece of source code. The code that’s given to a computer will be executed billions of times per second, repeated, recalculated, and re-interpreted over and over and over again.</p>
<p>The computer’s native understanding of software comes from machine code. Machine code are the binary numbers that the CPU loads into memory and processes directly. For example, here’s a line of machine code for an x86 machine that puts the value ‘97’ into the AL register, graciously stolen <a href="http://en.wikipedia.org/wiki/Assembly_language#Assembly_language">from Wikipedia</a>:</p>
<pre><code>10110000 01100001
</code></pre>
<p>While most would consider this unintelligible, computers were actually programmed this way at one time. My uncle actually did this by flipping switches to set the binary and pushed a button to store it in memory. Unfortunately, what’s good for the computer isn’t good for the human programmer. This is why assembly language was created. Assembly language has a 1 to 1 mapping to machine code, but is much easier for humans to understand. Here’s that same line in assembly:</p>
<pre><code>MOV AL, 61h ; Load AL with 97 decimal (61 hex)
</code></pre>
<p>The <code>MOV</code> corresponds with <code>10110</code>, <code>AL</code> maps to <code>000</code>, and 61 hex is <code>01100001</code>. <code>MOV</code> is short for ‘move,’ though, and this mnemonic is just a bit easier to understand than <code>10110</code>. This is the most basic example of a compositional trade-off. It’s not a true trade-off, because they map perfectly to one another, but it illustrates the difference between composing in a language that the computer understands and one that’s more natural for the programmer. Another important concept comes into play, that of <em>compilation</em>. Virtually every work of composition created in software is automatically translated to another form before it is executed. We’ll address this concept more fully when we discuss form later.</p>
<p>If that’s where assembly stopped, it would remain a 1 to 1 mapping. However, virtually every assembly language also offers macros, and this moves the code further away from the machine and destroys the synchrony of the two forms. Here’s an example:</p>
<pre><code>MOV EAX, [EBX]
</code></pre>
<p>The <code>[]</code> characters change the meaning of <code>EBX</code>, rather than be the value stored in that particular register, they imply that the value is a memory address, and we want to move the contents of that address to <code>EAX</code>. The generated machine code could now be processed into multiple valid assembly forms, and so the transformation is only perfect in one direction, even if it’s possible to ‘decompile’ it into one of those possible encodings. This is considered to be an acceptable trade-off for human readability; we very rarely want to turn machine code back into assembly.</p>
<p>There’s also a jump between higher level languages, as well. Here’s the assembly statements that adds 2 and 3 together:</p>
<pre><code>MOV EAX, 2
ADD EAX, 3
</code></pre>
<p><code>ADD</code>, of course, is the statement that adds a number to the register that’s given. Now <code>EAX</code> has the value 5. Here’s the same code, but in C:</p>
<pre><code>int x = 2;
x = x + 3;
</code></pre>
<p>Pretty simple. You can see how the C is much easier to understand; we say what type <code>x</code> is (an integer), and it’s a bit more explicit. <code>x</code> is equal to <code>x</code> + three. However, since the C is divorced from the machine, and is written for the person, we can change our compiler to make assembly code that works on a different kind of processor architecture. If we were to compile the above C for x86_64, a 64 bit version of x86, we might get some assembly that’d look like this:</p>
<pre><code>MOVL RAX, 2
ADDL RAX, 3
</code></pre>
<p>While this code looks similar to the above, it is quite different. This uses the native 64 bit types, rather than the 32 bit types above. The other important thing is that by writing code that’s divorced from the machine, and written for people, we’re able to translate it into the code for multiple machines. If we had written the assembly above, when moving to another architecture, it would have required a total re-write. And while this particular sample looks very similar, a more complex piece of code will be significantly divergent, but I don’t want to go into the details of two kinds of assembly code. Because we can define the languages for humans, and the language of computers is somewhat beholden to the physical machine itself, it’s significantly easier to do the translation from C to the two kinds of machines, rather than trying to translate from one machine to another. What we lose in this kind of translation, though, is efficiency. Code that was hand-crafted for each machine would be more performant, and better represent each individual platform.</p>
<p>Even though we may choose to use a language that’s more understandable to people, it still has to be understood by the computer in some form. This translation will introduce some amount of penalty, and so it’s important that this gets taken into consideration. Sometimes, code must be written in a way that’s not easy for a person to read, because it’s easier for the computer to be efficient with a more opaque implementation.</p>
<h3>The reflexive audience: the programmer himself</h3>
<blockquote>
<p>Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.Brian Kernighan</p>
</blockquote>
<p>Everyone who writes code has experienced this at some time or another. You write a whole bunch of code, and then a few months goes by, and you take a look at it, and it’s absolutely unintelligible. This happens because at the time of inception, the author of a particular piece of code has an incredibly close relationship to it. As it was just written, the code is obvious to the author. They’re in the proper mindset to understand the intention that was drawn upon to necessitate bringing those lines into the world, and so no extra explanation is necessary. As time goes on, however, the author becomes more close to the third audience, other programmers. It’s important for coders to recognize this fact, and take preventative steps to ameliorate this confusion.</p>
<p>Even though the author will approach the position of the other audience eventually, this audience is distinct because there is a certain level of explanation that sits between undocumented, inexpressive code and code that’s well explained, and this is the position most code is in. An explanation that’s helpful to those who understand the code, but not to those who don’t is better than nothing. This sort of code may be overly contextual, and could use some added information to improve its clarity.</p>
<h3>The other audience: colleagues and coworkers</h3>
<blockquote>
<p>Always code as if the guy who ends up maintaining your code is a violent psychopath who knows where you live.Martin Golding</p>
</blockquote>
<p>As I touched on earlier, there’s a similarity between the ‘other’ audience and the reflexive. The primary distinction is drawn around the proximity to the source. The other does not have the advantage of having authored the code, and therefore doesn’t have the native’s understanding of the underlying logical organization of the source. This disadvantage can be overcome via composing in such a manner that the meaning is emergent from the design. Even if it’s too complex to be obvious, good documentation can address this particular deficiency.</p>
<p>Ultimately, much of software design is about modeling. Software that solves a particular problem should emulate the nature of the challenge it’s attempting to address. If this can be achieved, it’s significantly easier for those who understand the problem to figure out how the software works. Therefore, good design can help improve the effectiveness of a given piece of source to communicate its intent. Along a similar vector, if the design is similar to code that solves a particular issue, it’s easier to understand. As an example, a friend recently asked for feedback about an interface that he’d designed. It loaded a save game file for StarCraft 2. This is what he came up with:</p>
<pre><code>replay_file = File.new("spec/fixtures/1v1-game1.sc2replay")
@replay = SC2Refinery::Parser.parse(replay_file)
</code></pre>
<p>However, Ruby already has several kinds of code in its standard library that loads some information from disk and parses it into some kind of data structure that you can use in your program. The JSON, YAML, and Marshall classes already use a set of methods to import and export data, and they’re named <code>load</code> and <code>dump</code>, and they’re part of the class directly. Also, in this case, the user of the code shouldn’t need to deal with the creation of a file, since it’s unreasonable to assume that a game replay would come from any other source. Therefore, after some discussion, he adopted the following interface instead:</p>
<pre><code>@replay = SC2Refinery.load("spec/fixtures/1v1-game1.sc2replay")
</code></pre>
<p>This is much nicer to use, and is much simpler. While it may not seem like a whole lot, when rules like this are applied across an entire codebase, they can significantly increase understanding. Multiple reductions of mental overhead add up quickly.</p>
<p>My new favorite trick for adding a little bit of modeling that significantly reduces overhead for the user is the Presenter Pattern. Jeff Casimir demonstrated this very clearly in his presentation at RailsConf 2011, “<a href="http://dl.dropbox.com/u/69001/Fat%20Models%20Aren%27t%20Enough%20-%20RailsConf.pdf">Fat Models Aren’t Enough</a>”. Here’s a slightly modified example. Imagine that we have a system that manages students, and we’d like to display a report card for them. We might start with some code that looks like this:</p>
<pre><code>student = Student.find(options[:student_id])
term = Term.find(options[:term_id])
report_type = ReportType.find(options[:report_type])
puts "#{student.last_name}, #{student.first_name}"
puts "#{report_type.report_title} for #{term.start_date} to #{term.end_date}"
student.courses.each do |course|
course_report = student.report_data_for_course(course)
puts course_report.to_s
end
</code></pre>
<p>Honestly, until this past week, this is the exact code that I would have written. But it turns out that we can do better. Basically, we’re displaying some information that comes from a combination of three different objects. If we think about it some more, we’re really trying to display a report card. So let’s make an object that represents the card, and delegates to its sub-objects. It will then know how to display itself.</p>
<pre><code>class ReportCard
delegate :start_date, :end_date, :to =&gt; :term
delegate :first_name, :last_name, :courses, :report_data_for_course, :to =&gt; :student
delegate :report_title, :to =&gt; :report_type
def initialize(params)
@student = Student.find params[:student_id]
@term = Term.find params[:term_id]
@report_type = ReportType.find params[:report_type_id]
end
def student_name
[last_name, first_name].join(", ")
end
def title
"#{report_title} for #{start_date} to #{end_date}"
end
def course_reports
out = ""
courses.each do |course|
out += report_data_for_course(course)
end
out
end
end
</code></pre>
<p>Now, this is a lot of code. However, as you can see, it’s all focused on composing the information and exposing an interface that makes sense for a report card. Using it is super easy:</p>
<pre><code>report = ReportCard.new(options)
puts report.student_name
puts report.title
puts report.course_reports
</code></pre>
<p>Bam! It’s incredibly obvious. This code is much more clear than before. We’ll see if I’m still as hot on this pattern as I am now in a few months, but I feel the extra object adds a significant amount of clarity.</p>
<p>If the model is too hard to create, or if additional clarity is needed, documentation in the form of comments can also help to improve the understanding of the ‘other’ audience. Comments can be a difficult form of prose to write, because they need to be written at the correct level of abstraction. If they simply repeat what the code does, they’re useless, and if they’re too high-level, certain details and semantics may not be made clear.</p>
<p>Individual bits of code can also be made more clear by developing a narrative within any particular method that’s being written. Telling a story with code may not be something you’ve considered before, but it’s really about maintaining a proper flow in the actions your code is taking. For example, if there’s a bunch of error handling strewn about inside of a method, it’s less clear than bunching all of the error handling near the end. Most code should be an act in three parts: input, processing, and output. If these three parts are mixed together, it can appear much more complicated.</p>
<h3>The forgotten audience: end-users</h3>
<blockquote>
<p>If I asked my customers what they wanted, they’d have told me, “A faster horse.”Henry Ford</p>
</blockquote>
<p>In the end, all software is used by someone. Use-value is the driving force of virtually all code. Code that doesn’t do anything may be making some kind of important philosophical statement, but isn’t really the sort that I’m talking about.</p>
<p>The introduction of a user imposes significant restrictions upon the way that code is composed. End-users do not need to understand the code itself, but they do need to be able to understand its external interfaces. These needs place an imposition on the way that the code needs to be written, because it <em>must</em> address this issue of interface. Sometimes, interface requirements can create a burden on the internal implementation. Needing to support certain behaviors and forms can create complexity for an implementor.</p>
<p>Documentation created for end users must be completely different than that which is created for those inspecting the code itself. Most end-users will not be literate in the arts of software development, and so approach the software object in an entirely different way than those who write code do. Yet, the same semantics must be passed on to them, but at a higher level. There’s a strong movement within the community to start designing software with this kind of end-user documentation in mind, called <a href="http://tom.preston-werner.com/2010/08/23/readme-driven-development.html">README driven development</a>. There are advantages to thinking on this level when beginning, but a nice effect of doing it first is that you can ensure it gets done. A surprising amount of software has poor documentation for its users, because it’s created after the software is finished, and at that time there’s intense pressure to ship it out the door. Writing down information for the end user first ensures that it’s done properly, that all development works in accordance with the documentation, and that all of the use-cases for an end-user have been thought of and are being addressed.</p></content:encoded>
</item>
<item>
<title>Contributing to Ruby's documentation</title>
<link>https://steveklabnik.com/writing/contributing-to-ruby-s-documentation/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/contributing-to-ruby-s-documentation/</guid>
<pubDate>Fri, 20 May 2011 00:00:00 GMT</pubDate>
<content:encoded><h3>Update!</h3>
<p>I’m now making it even easier than this: <a href="/2011/08/22/im-making-it-dead-simple-to-contribute-to-ruby-s-documentation.html">Read my new post about how I’ll do this for you</a>. That said, if you want to do it yourself, the following will tell you how.</p>
<h3>Original Article</h3>
<p>Ruby 1.9.3 is coming out soon! <a href="http://blog.segment7.net/2011/05/09/ruby-1-9-3-documentation-challenge">drbrain has challenged the Ruby community to improve its documentation</a>, but some people were asking about how to do so. So I made a video!</p>
<p>Some small errata: drbrain has informed me that he should edit the Changelog, not me. So don’t do that. :)</p>
<p><a href="http://vimeo.com/23522731">How to contribute to Ruby’s documentation.</a> from <a href="http://vimeo.com/steveklabnik">Steve Klabnik</a> on <a href="http://vimeo.com/">Vimeo</a>.</p>
<p>If you don’t want to watch me talk about it, here’s the same info, in text:</p>
<p>Getting the Ruby source is pretty easy. You can find it on GitHub, here: <a href="http://github.com/ruby/ruby">http://github.com/ruby/ruby</a> . Click the “fork” button and clone down your own fork:</p>
<pre><code>$ git clone [email protected]:YOURUSERNAME/ruby.git
</code></pre>
<p>After that’s done, type <code>cd ruby</code> and add the main project as an upstream. This will let you keep up-to-date with the latest changes:</p>
<pre><code>$ git remote add upstream https://github.com/ruby/ruby.git
$ git fetch upstream
</code></pre>
<p>Okay! Now that you’re all set up, poke around and find something that needs documented. I like to just look through the source, but you can also look <a href="http://segment7.net/projects/ruby/documentation_coverage.txt">here</a> for a list of things that have no docs. Documentation is written in rdoc, and I’d check the recent commits that drbrain has been making to guide you in style. <a href="https://github.com/ruby/ruby/commit/071a678a156dde974d8e470b659c89cb02b07b3b">This commit</a>, for example, is a pretty good template. You can also check out the formatting guides <a href="http://rdoc.rubyforge.org/RDoc/Markup.html">here</a>. There’s also <a href="http://rdoc.rubyforge.org/RDoc/Parser/Ruby.html">this</a> which explains some directives for .rb files and <a href="http://rdoc.rubyforge.org/RDoc/Parser/C.html">this</a> which handles directives for .c files.</p>
<p>Now that you’ve made a change to the documentation, you can regenerate the docs by using rdoc. First, grab the latest version from rubygems:</p>
<p>$ gem install rdoc</p>
<p>Always best to have the latest tools. Now do this to generate the docs:</p>
<pre><code>$ rdoc --o tmpdoc lib/rss*
</code></pre>
<p>I’m passing it in an output directory with op, since the doc directory is not an rdoc directory. rdoc will complain and refuse to overwrite those files, which is a good thing. I’m also passing in a pattern of what to compile documentation for, compiling all of it takes a few minutes! In this case, I chose to document the rss library.</p>
<p>Now you have a website in rdocout. Open up its index.html, and poke around for what you’ve changed. If it all looks good, you’re ready to make a patch!</p>
<pre><code>$ rm -r tmpdoc
$ git add .
$ git commit -m "adding documentation for $SOMETHING"
</code></pre>
<p>Now, you have two options here. One is to simply push the change up to GitHub, and make a pull request.</p>
<pre><code>$ git push origin
</code></pre>
<p>… aaand pull request. The core Ruby development doesn’t really happen on GitHub though, and so your patch may take a while to get included. If you really want to do it right, submit a patch to RedMine. We’ll use git to make this patch:</p>
<pre><code>$ git format-patch HEAD~1
</code></pre>
<p>This says “make a patch out of the last commit.” It’ll tell you a file name, it should start with 000.</p>
<p>Now, sign up for the Ruby RedMine <a href="http://bugs.ruby-lang.org/account/register">here</a>. Once you’ve clicked the confirmation email, <a href="http://bugs.ruby-lang.org/projects/ruby-trunk/issues/new">open a new ticket</a>, and assign it to Eric Hodel, category DOC, and give it your Ruby version, even though it’s not a big deal here. Click ‘choose file’ and pick your patch, then ‘create and continue’ and BAM! You’re done!</p>
<p>Let’s all pitch in and make this the best documented Ruby release ever! In writing documentation, you might even find some things that you’d like to help improve. ;)</p></content:encoded>
</item>
<item>
<title>The next iteration of my blog</title>
<link>https://steveklabnik.com/writing/the-next-iteration-of-my-blog/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-next-iteration-of-my-blog/</guid>
<pubDate>Thu, 12 May 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Well well well, some things never change. Long ago, I hosted my own blog, and here we are again. I’ve moved everything over to Jekyll, if that means anything to you. You can now <a href="https://github.com/steveklabnik/blog">look at my blog on GitHub</a>. Big thanks to <a href="http://www.jstorimer.com/2009/12/29/jekyll-on-heroku.html">this article</a> for lots of help and pointers with final deployment stuff, and <a href="http://mikeferrier.ca/2011/04/29/blogging-with-jekyll-haml-sass-and-jammit/">this one</a> for haml and sass and stuff. Yay!</p>
<h2>New-ish beginnings</h2>
<p>So why’d I do this? Well, this is just part of moving more towards hosting all of my own services, and not relying on services that others provide. This isn’t some sort of argument over individualism vs. collectivism, it’s more about <a href="http://nomoresharecropping.org/">No More Sharecropping</a>.</p>
<p>I’m still relying on people for hosting, of course. Nobody is 100% removed, but with my own backup of everything, I’m in total control here. It’s really minimal work to keep this going, and only a few hours of setup to move everything over.</p>
<h2>A small change in focus, too</h2>
<p>I’m also going to stop bitching about politics so often, and try to be more constructive in general. I have a lot to say about this new humanities angle that my life has taken as of late. Hence, I’ve changed the title of my blog. And you can expect more posts along those lines. More about books, more about art and code, less whining. One may think I’m maturing or something…</p></content:encoded>
</item>
<item>
<title>The first week of rstat.us: users, press, and scaling, oh my!</title>
<link>https://steveklabnik.com/writing/the-first-week-of-rstat-us-users-press-scaling-oh-my/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-first-week-of-rstat-us-users-press-scaling-oh-my/</guid>
<pubDate>Wed, 30 Mar 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Hey everyone. A lot of people have been asking me about <a href="http://rstat.us/">rstat.us</a> lately, so I figured I’d tell a little bit of the story as it’s gone down so far.</p>
<h2>Stats</h2>
<p>First, here’s some numbers:</p>
<ul>
<li>Users: 4553</li>
<li>Uniques: 25,000</li>
<li>Pageviews: 119,385</li>
<li>Pages/visit: 4.77</li>
<li>Statuses posted: 9387</li>
</ul>
<p>Here’s some fun info: stats on statuses by language:</p>
<ul>
<li>german 118</li>
<li>russian 1355</li>
<li>none 97 &lt;- ha!</li>
<li>english 4836</li>
<li>spanish 1412</li>
<li>dutch 98</li>
<li>french 1155</li>
<li>portuguese 272</li>
<li>farsi 66</li>
<li>pinyin 1</li>
</ul>
<h2>Code</h2>
<p>Lines, by committer:</p>
<ul>
<li>28 AndrewVos</li>
<li>124 BRIMIL01</li>
<li>6 Blaine Cook</li>
<li>107 Brendan Taylor</li>
<li>924 Brian Miller</li>
<li>45 Caley Woods</li>
<li>162 Carol Nichols</li>
<li>20 Claudio Perez Gamayo</li>
<li>1347 Dominic Dagradi</li>
<li>4 James Larkby-Lahet</li>
<li>99 Jorge H. Cuadrado</li>
<li>258 Kat Hagan</li>
<li>10 Lauren Voswinkel</li>
<li>143 LindseyB</li>
<li>16 MenTaLguY</li>
<li>3 Michael Stevens</li>
<li>3 Murilo Santana</li>
<li>10 Nate Good</li>
<li>1 Peter Aronoff</li>
<li>24 Shebanian</li>
<li>44 Stephen Paul Weber</li>
<li>1409 Steve Klabnik</li>
<li>8 Tony Arcieri</li>
<li>478 Zachary Scott</li>
<li>104 burningTyger</li>
<li>28 james cook</li>
<li>56 kat</li>
<li>200 wilkie</li>
<li>10 wolfwood</li>
</ul>
<p>Also, wilkie has a lot more code in the 3 gems that we build off of, for our ostatus implementation.</p>
<p>That makes for 29 unique committers. These stats were generated by git-blame.</p>
<p>Just over 200 pull requests</p>
<p>872 commits</p>
<h2>The deal with Heroku</h2>
<p>Crazy! Such good stuff. I’m really excited as to how it’s going, but there were a few rocky points along the way as well. Some people saw me tweeting in frustration with Heroku, so here’s the skinny:</p>
<p>On Sunday, for some reason, our 3 line app.js file, as well as our favicon.ico, started to take 60 seconds to respond. This meant that a dyno was getting tied up, and since Heroku changed the way their mesh works, they kept routing requests to these screwed up dynos. This caused a rash of 500 errors, and I frantically filed a support ticket. This was the first time I’d had a paid app on Heroku, and they’ve always been fantastic. The Hackety Hack site was on the free account, and performed beautifully, with similar traffic.</p>
<p>What it boils down to is this, though: Heroku took about 22 hours to respond to my support ticket, in any way. Not like “we’re looking at it, but it’s Sunday, so it’ll take a while.” Crickets. Meanwhile, the errors kept happening. Eventually, I heard something from them, where they pretty much said ‘I see other errors, try fixing those.’ with no response on the actual root cause. I got an email from a Level2 person who promised to investigate and escalate, but by now, I still haven’t heard.</p>
<p>I also had a smaller issue with MongoHQ. To their credit, I heard back Sunday night, and the issue with them was an annoyance, not a showstopper. I ended up hearing from the CEO, who ended up following up with me multiple times, and in a pretty timely fashion.</p>
<p>So, we moved our hosting away from Heroku and onto Duostack. Now, full disclosure: I’m friends with the guys from DuoStack. That said, they took care of me. I found a small bug in their platform when deploying the app, but everything’s gone really well. Bunches of users have commented on how fast the site is now, and I’ve been really happy with it so far. We’ll see how it goes.</p>
<h2>The Future</h2>
<p>Anyway, here’s what’s next for <a href="http://rstat.us/">rstat.us</a>: The biggest thing is getting to 100% ostatus compliance. We spent lots of time this week getting screwed over by a bug in Google’s hub implementation, but now that it’s all taken care of now. While we work on that, we’re also getting ready to deploy a statusnet compliant API, so people can use statusnet clients on mobile and desktop. This is huge for me, as I’m really missing notifications.</p>
<p>Other than that, just keep using it! Tell your friends! If you code, come join us on the IRC (#rstatus on freenode), and help file bugs and patch them!</p></content:encoded>
</item>
<item>
<title>Announcing rstat.us</title>
<link>https://steveklabnik.com/writing/announcing-rstat-us/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/announcing-rstat-us/</guid>
<pubDate>Wed, 23 Mar 2011 00:00:00 GMT</pubDate>
<content:encoded><p>Whew.</p>
<p>If you’ve been <a href="http://twitter.com/steveklabnik">following me on Twitter</a> at all lately, you’ll know that I’ve been working hard on a new project lately. Tonight, even though it’s late, I’m finally getting it out there. Please welcome <a href="http://rstat.us/">http://rstat.us/</a> to the world! (as soon as the DNS updates. ;) )</p>
<p><a href="http://rstat.us/">rstat.us</a> is a micro-blogging service, just like Twitter. There’s one big difference, though: it is distributed. Oh, and it’s 100% open source. So two things. It’s also integrated with Facebook and Twitter, so you can just log in with either of those accounts, and it’ll syndicate your posts. Well, just to Twitter right now, but we’re getting there.</p>
<p>Here’s the other cool thing: Want to own your data? You can actually <a href="http://github.com/hotsh/rstat.us">get the code</a> and run a copy yourself! But you won’t get left out: you can follow anyone else that uses the <a href="http://ostatus.org/">ostatus</a> protocol, like <a href="http://identi.ca/">http://identi.ca/</a>, and it’ll Just Work. You just have to grab the URL to their feed, put it in, and it’s all good! But you probably shouldn’t do that yet. We’re still working on some features.</p>
<p><a href="http://rstat.us/">rstat.us</a> is still very much alpha quality. 7 or 8 of my best friends and I have been fighting off sleep, food, and tests for the last two weeks to put this together. Please <a href="http://rstat.us/">give it a try</a> and let me know what you think, tell me if you can break it, or anything else that’s particularly interesting. If you’re the coding type, please check out the <a href="http://github.com/hotsh/rstat.us/wiki">wiki</a> for a big old list of stuff we’d like to do.</p>
<p>Oh, and of course, you should <a href="http://rstat.us/users/steveklabnik">follow me on rstat.us</a>.</p></content:encoded>
</item>
<item>
<title>Did you hear? I'm Ruby Mendicant University's first visiting teacher</title>
<link>https://steveklabnik.com/writing/id-you-hear-i-m-ruby-mendicant-university-s-first-visiting-teacher/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/id-you-hear-i-m-ruby-mendicant-university-s-first-visiting-teacher/</guid>
<pubDate>Fri, 18 Mar 2011 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve heard <a href="http://university.rubymendicant.com/">Ruby Mendicant University</a> (also known as RMU, not to confuse you Robert Morris alums!) described as “the best kept secret of the Ruby community.” High praise, indeed. If you haven’t heard about RMU before, check out this presentation by Andrew Cox:</p>
<p>TL;DR: RMU is a free, online ‘unversity’ from Gregory Brown, author of ‘Ruby Best Practices.’ You enter an intermediate Rubyist, and you exit an expert.</p>
<p>Anyway, RMU recently announced that <a href="http://university.rubymendicant.com/changelog/visiting-teacher-2011-t2-steve-klabnik">I’ll be their first visiting teacher</a>. Basically, I’ll be hanging out, giving advice, and just generally being even more accessible to RMU students than I already am. Greg and I have talked a lot about education, Ruby, and a lot of other things… between Hackety’s ‘nothing to intermediate’ and RMU’s ‘intermediate to expert’ approaches to learning, anyone wanting to get up to speed with Ruby should have no shortage of resources at their disposal.</p>
<p>If you’re feeling like you’ve got the hang of Ruby, but want to take it to the next level, <a href="http://university.rubymendicant.com/admissions">applications are open</a>! Hopefully I’ll see you there.</p></content:encoded>
</item>
<item>
<title>I'm quitting Hacker News</title>
<link>https://steveklabnik.com/writing/i-m-quitting-hacker-news/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/i-m-quitting-hacker-news/</guid>
<pubDate>Thu, 03 Mar 2011 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve been thinking about this post for a while.</p>
<p>I freaking love Hacker News. It’s my favorite place on the Internet. It’s incredibly valuable. I have learned so much from my time there. With that said, I feel like it’s taken up too much of my time.</p>
<p>I’m trying to accomplish things. I got accepted to grad school. I’m trying to rebuild Shoes. I’m working on improving Hackety. I’m finding that I don’t have enough hours in the day to accomplish the things I want to accomplish. And arguing for the hundredth time about how artists will survive in a post-IP world, or how to get your first couple of users, or whatever it is that Google or Apple did this week hasn’t gotten boring yet, and that’s a problem.</p>
<p>The only respectable solution is to go cold-turkey for a while.</p>
<p>I’ve been doing good on some of my new years’ resolutions, and bad on others. I haven’t spent a lot of time writing lately, and I have only been doing okay at my writing. Hacker News is cutting into that. I feel like I’m using it in the same way some use TV. Yeah, it’s significantly deeper, but I really should be reading more books and less HN. At least, that’s what’s right for me right now.</p>
<p>I won’t be gone forever. I just need to spend more time elsewhere.</p>
<p>With that said, I’ve been spending a <em>lot</em> of time on Shoes. I’ll just leave this here: <a href="https://github.com/shoes/shoes/tree/feature/gemify">https://github.com/shoes/shoes/tree/feature/gemify</a></p></content:encoded>
</item>
<item>
<title>Living in the cloud</title>
<link>https://steveklabnik.com/writing/living-in-the-cloud/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/living-in-the-cloud/</guid>
<pubDate>Thu, 24 Feb 2011 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve been joking for a while that I’d like to “move to the Internet.” Generally I cite my familiarity and love with online culture as opposed to American culture. I don’t really care what’s going on in “the real world,” as I feel it’s kind of played out. The whole cycle of “fight wars, reduce freedoms, sit in front of your TV” has gotten really boring quickly. I find Internet culture to be more complex, entertaining, and useful. But I’m getting way off topic, here.</p>
<p>As a programmer, one of the key skills I rely on daily is the ability to recognize abstractions. Writing great code requires the ability to see similarities in things, even when they may not be immediately apparent. Lately, I’ve been giving a lot of thought to the idea of “living in the cloud.” Primarily, cloud computing is about providing services on-demand, and in the amount you need. Dynamic allocation. It’s generally great stuff, and even though “the cloud” is a bit buzzwordy, it’s an awesome concept.</p>
<p>So what happens when we apply that to meatspace? What does “cloud living” look like? “Cloud working”?</p>
<p>Living in the cloud would look a lot like the world Tim Ferriss encourages with his whole lifestyle design concept, or at least the lifestyle he’s designed for himself. Move around from place to place, live where you find things interesting, reallocate yourself to a better place when the feeling dictates. The concept of being a “digital nomad” certainly makes me think of “living in the cloud.” Doing this right means giving up most material possessions, as they impede freedom of movement.</p>
<p>Another important aspect of cloud living would be finding a way to make a living while being location independent. There was a great article making the rounds a while back called “<a href="http://romansnitko.posterous.com/jobs-dont-scale">Jobs don’t scale</a>.” It’s only tangentially related, but his point about working for other people certainly relates to freedom of movement. If you have to convince your boss to let you work remotely, it’s much harder than just doing it yourself.</p>
<p>In any case, I’m very attracted to all of this as an idea. I’d love to travel the world, working from a different place every day, seeing new sights and exploring new places. It requires re-thinking a lot of things about what it means to work, how relationships work, friendships. Can you have kids if you keep moving around from place to place? How can you stay connected to friends if you never see them, because you’re halfway across the globe? Can your significant other do the same thing? Can you get month to month leases easily?</p>
<p>I’m still thinking about this as a concept, but I thought I’d share my thoughts so far. I’m not moving around the world yet, but I am trying to increase my own mobility as much as possible. I think I’m finally at a place where I don’t care much for most of my possessions anymore…</p></content:encoded>
</item>
<item>
<title>A few notes on Foucault</title>
<link>https://steveklabnik.com/writing/a-few-notes-on-foucault/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/a-few-notes-on-foucault/</guid>
<pubDate>Sat, 22 Jan 2011 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve almost always been in a habit of reading. When I was younger, I spent a lot of time in the library. I’d just grab an entire series of novels, and read them all, right in a row. Sometimes I’d end up reading non-fiction too. I ended up with a lot of books on software.</p>
<p>Then my local library got Internet access. Slowly, I started to spend less time in the stacks and more in front of a terminal. This wasn’t a bad thing; I still kept reading. What’s more, what was previously a one way street turned into two: I didn’t just read the Internet, I wrote it. I spent hours and hours discussing the things I’d read with others.</p>
<p>In any case, as the years flew by, the things that I’ve been reading have become less and less substantial. Current events are fine and all, and pictures of cute cats are nice, but I feel like the volume of what I’ve been reading has gone up, but the quality has gone down. It happens. I can’t really be snide about not owning a TV while being subscribed to <a href="http://www.reddit.com/r/fffffffuuuuuuuuuuuu">/r/fffffffuuuuuuuuuuuu</a>. Reddit somehow has morphed into a place where you can feel superior, but it’s really just the same exact thing in different clothing. Combine this general unease with my aspirations for grad school in the fall, as well as my renewed interest in political philosophy, and my need to hit the books has become readily apparent.</p>
<p>Luckily, I have an ally in this quest, and Jamie has given me a reading list. Since reading and writing are two sides of a single coin, I’ll be writing about the things that I read here. Like many other things I’ve written about in the past, I’m sure that putting my thoughts down on paper (?) will help to gel my understanding and thoughts. First up: Michel Foucault’s “Discipline and Punish.”</p>
<p>I’ll get right to it: I really enjoyed reading this book. It’s partially because it took a lot of random things that I kind of knew and tied them to some experiences that I’ve had in a meaningful way. It’s also partially because I have a substantial infatuation with the conspiratorial; I re-read “1984” every year, and I often think about its all-knowing, all-seeing telescreen imagery when discussing anything vaguely political. “Discipline and Punish” gave me the same sort of images, but they were fewer, and more firmly rooted in history and reality. The book opens with a section named “Torture,” and the scene of Robert-François Damiens’ punishments for attempted regicide. I’m sure that before the release of “Saw,” these images were even more shocking:</p>
<p>… he was to be ‘taken and conveyed in a cart, wearing nothing but a shirt, holding a torch of burningwax weighing two pounds’; then, ’in the said cart, to the Place de Greve, where, on a scaffold that will be erected there, the flesh will be torn from his breasts, arms, thighs, and calves with red-hot pincers, his right hand, holding the knife with which he committed the said parricide, burnt with sulphur, and, on those places where the flesh will be torn away, poured molten lead, boiling oil, burning resin, wax, and sulphur melted together and then his body drawn and quartered by four horses and his limbs and body consumed by fire, reduced to ashes, and his ashes thrown to the winds.</p>
<p>Let it never be said that the people of 1757 were not thorough. Regardless, by January of 1840, we’d transitioned to a prison system that looks pretty much the same as it does now. The book’s primary theme is discussing how we got from point A to point B, and then examining the ‘why’ to explain how modern society has a new sovereign institution: the ‘carceral system.’</p>
<p>Before we can examine that question, though, we need to ask why we used torture as a form of punishment in the first place. The reasoning is actually straightforward: during the period of monarchy, everything revolves around the monarch. He is sovereign in a more absolute way than we even initially think of; the pop-culture image of a king has more to do with something of a popularity contest or that he’s simply the guy on the top of the pyramid, but the nature of a monarch’s power runs more deeply than that. It was called ‘divine right’ for a reason, the physical body of the sovereign was the representation of God himself, and since the entire world belongs to God, thus it belongs to and is a part of the monarch. It reminds me of the kind of doublethink necessary to grasp the Catholic conception of the Holy Trinity, in this case God the Father, the king his son, and the world rather than a Holy Ghost. All one, yet three at the same time. In any case, if the land itself is the literal body of the king, then any transgression is an act of defiance not only of the rule of the monarch, but is making war upon God himself. And since damage has been done to the body of God, so must an equivalent exchange be made with the body of the aggressor. Torture also has an element of the theatrical to it, and therefore demonstrates to all of those watching that they must also comply with the rule of law or face the consequences.</p>
<p>However, eventually, torture became socially inconvenient. Basically, it was a case of the Streisand Effect: when you place that much attention on someone, you create a forum for sympathizers to create romantic images of their fallen hero. There’s a great historical example of this in the Christian mythos: consider the polarizing effect that Christ’s torture on the cross maintains to this day. History is littered with the songs of fallen heros, and a call to follow in their stead. Eventually, whenever a new convict was to be strung up at a gallows, there’d be a state of disarray. Foucault describes several images of rioters throwing stones and even in some cases killing the executioner.</p>
<p>As a result of this, the nature of punishment slowly changed. Reformists argued that punishment was metered out unevenly, and inconsistently. Thus in the same way that monarchy gave way to democracy, the absolute right of the king to punish became distributed as well. However, centralized and distributed systems are quite different, and require different constructs to operate properly. Therefore, a distributed form of the right to punish would need some mechanism by which to operate. This mechanism is termed “discipline” by Foucault. Discipline creates a certain order all by itself, and he uses a great example of monks and monasteries to illustrate the concept of discipline. Think about all of these things that we consider virtuous:</p>
<ul>
<li>Studying is a disciplined form of reading</li>
<li>Dieting is discipline applied to eating</li>
<li>The image of a soldier is almost entirely one of discipline</li>
<li>Morality is discipline applied to all of life</li>
<li>Exercise is disciplined form of the body</li>
</ul>
<p>But discipline has even greater roots in our society. Think about Taylor’s Scientific Management, for example: it’s a means of imposing discipline on workers to maximize their production. Schooling is a way of giving children a ‘structured environment’ (structure seems to be synonymous with discipline in many cases) to develop in. Churches are places for the soul to become disciplined.</p>
<p>Submitting to discipline has deeper psychological effects as well. It creates the idea of a division: there’s those who follow the rules, and those that disregard them. And since we’ve established that those who follow the rules are virtuous, those who don’t must not be. Since those that don’t follow the rules are doing bad things, they should be subject to punishment, so that they can remain disciplined. And thus the system of rules can be used as the distributed form of this right to punish, replacing the absolute right of the monarch. Submitting to this mentality makes people into ‘docile bodies’ that perfectly fit into this worldview.</p>
<p>As an example of how far this disciplinary approach has gone, Foucault presents the Panopticon, which was a prison shaped like a pie, with a tower in the middle. Prisoners would be able to observe others directly across from themselves, and the guard who may or may not be in the tower would be able to watch all of the prisoners at once. Since you couldn’t tell if the guard was in the tower or not, discipline would become internalized, since you always had to assume that Big Brother is watching you… This way of thinking about society ends up creating an all-encompassing ‘carceral system’ that we now live in.</p>
<p>It’s also important to note that this carceral system is total and absolutely permeating every aspect of society, yet they aren’t presented as such. Foucault specifically mentions that it’s important to consider that the delinquent are still a part of the system, and not outside of it. Yet we’re constantly presented with images that serve to present the idea that there’s a ‘criminal underworld,’ that those who lead a life of crime are part of a shadowy alternate universe. Foucault refers to this idea as ‘enclosure’: “Discipline sometimes requires enclosure, the specification of a place heterogeneous to all others and closed in upon itself. It is the protected space of disciplinary monotony.” The enclosure embodies this separation, since there’s a space both within and outside of the enclosure. A self and an Other.</p>
<p>… so yeah. That’s my summary. I’m still digesting a lot of this stuff, and so I may have more to say about it later.</p></content:encoded>
</item>
<item>
<title>Goals for 2011</title>
<link>https://steveklabnik.com/writing/goals-for-2011/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/goals-for-2011/</guid>
<pubDate>Sun, 02 Jan 2011 00:00:00 GMT</pubDate>
<content:encoded><p>This year I’m happy to report that I have much more upbeat plans than last year’s kind of sad post. With that said, here’s my 2011 goals:</p>
<ul>
<li>I want to work for other people 2 days per week on average for the year.</li>
<li>I want to write something every day</li>
<li>I want to keep building on the initial success that Hackety Hack’s 1.0 launch has brought. I’d like to get a new release out every month or so, with a 2.0 for Christmas 2011.</li>
<li>I want to start exercising again. My current lifestyle is still too sedentary for my liking.</li>
<li>I want to eat better.</li>
<li>I want to start logging my activities every day.</li>
</ul>
<p>Most of these are kinda standard. I used to go to the gym all the time, but I’ve never eaten healthily. Both of these things are going to become more important as I age, however, and so I’m starting to actually give them serious consideration.</p>
<p>Hackety Hack had an impressive 1.0 launch, but I have lots of work to do. Please check out the <a href="http://blog.hackety-hack.com/">Hackety Hack blog</a> tomorrow for a report on that. Basically, I now have a sizable userbase to attend to, and they’ve found a bunch of bugs that I need to iron out before starting to build more stuff.</p>
<p>The logging is an idea that I got from <a href="http://www.sebastianmarshall.com/">Sebastian Marshall</a>. We’ve had some good conversation recently, and we talked briefly about the importance of measuring things. You can only improve things if you know what’s actually wrong… I’m going to be tracking the time that I do various activities, and seeing if I can improve. If I feel this is working, I’ll write about it in a month or so.</p>
<p>I’ve already made some progress on all of these, actually. Obviously, most of them can only truly be realized in a year’s time, but here’s what I’ve got going:</p>
<ul>
<li>I’ve got my logging set up going. I’m trying to figure out how to make it as unobtrusive as possible, but I think I have a good first run at it.</li>
<li>I’ve got a set up for the work situation, and if all goes well, that should be accomplished. I originally thought I’d have to front-load a bit, but it seems like I’ll be doing this most of the year.</li>
<li>Working that little will enable me to really accomplish my Hackety goals.</li>
<li>Now that Maria is graduated, she’s got some time to cook things, and she’s also interested in eating better, and her habits are better than mine already. Hopefully this will lead to success.</li>
<li>Logging should help keep my eyes on the prize for my exercise and writing goals.</li>
</ul>
<p>So, that’s it! Good luck with your goals in the new year!</p></content:encoded>
</item>
<item>
<title>Hackety Hack v1.0</title>
<link>https://steveklabnik.com/writing/hackety-hack-v1-0/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/hackety-hack-v1-0/</guid>
<pubDate>Sat, 25 Dec 2010 00:00:00 GMT</pubDate>
<content:encoded><p>It’s been a long journey. I have a lot to say about this, but it’ll have to wait until I’m not in the middle of nowhere spending some time with Maria’s family. I just wanted to tell you all Merry Christmas, and point you at the <a href="http://blog.hackety-hack.com/post/2457961910/merry-christmas-and-a-1-0-announcement">Hackety Hack v1.0 release announcement</a> and the new <a href="http://hackety-hack.com/">Hackety Hack website</a>.</p>
<p>Merry Christmas!</p></content:encoded>
</item>
<item>
<title>The hardest decision I've ever made</title>
<link>https://steveklabnik.com/writing/the-hardest-decision-i-ve-ever-made/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-hardest-decision-i-ve-ever-made/</guid>
<pubDate>Wed, 17 Nov 2010 00:00:00 GMT</pubDate>
<content:encoded><p>I’ll cut to the chase: I’ve stepped down as CTO of CloudFab. I don’t want to discuss details, but life is too short to not be doing what you want with it. And, after a period of careful thought and consideration, it’s no longer the right thing for me. It was an amicable parting, and my replacements are already in place, and doing cool things. I’m happy with what I’ve accomplished, but it’s time for me to move on.</p>
<p>This raises the question: what am I moving on to? Here’s the answer: Hackety Hack. What I truly care about is teaching, writing, and discussing software. If I could do anything, I’d hack on Hackety, share what I know with others, and keep learning and playing with new things all day long.</p>
<p>However, there’s a certain question of sustainability here: I’m not made of money, and I have lots of bills to pay. However, doing the ‘obvious’ thing would be really, truly wrong: Hackety Hack is not and will never be a startup. It’s a social venture, if anything: it would be absolutely unquestioningly wrong for me to ever ask anyone for a single cent for it. It’d also be counterproductive: how could I share programming with everyone if I excluded those without disposable income?</p>
<p>So to this end, Hackety cannot be all that I do. And while I really want to teach, I also have a lot to learn. Luckily, I have some great allies in this area: I’ve been working alongside two teachers at the University of Pittsburgh over the past month or so: one in the English department, and one in the Computer Science department. In doing so, I’ve come to realize that my place in life is actually in the middle: while I love the theoretical side of CS, I’m actually much more interested in the application. And teachers need to be effective communicators, so investing in writing is absolutely worthwhile for me. At the same time, I’ve really enjoyed every chance I’ve had to write: there’s a reason I keep three blogs now! I’m also incredibly interested in the parallels that exist between computer and human languages. I like the comparison between refactoring and composition. I think the two fields have a lot to learn from one another. It’s something I’d like to explore more.</p>
<p>To do so, I’ll have to finish off my undergrad degree. Luckily, I’m very close, so I’ll be doing that in the spring. Afterward, it’s my intention to go to grad school in the English department, so I’m furiously working on my application, with the help of the aforementioned teacher. At the same time, I’ll possibly help teach or TA classes in the CS department as well. I’m hoping to combine my solid foundations in the hard sciences with the new perspective and improved analysis that the humanities can offer.</p>
<p>I don’t think that academia is where I want to stay forever, but I think it’s a good place for me to grow for now. It’s time for me to invest in myself a little. I think I’ll end up a much better person in the end.</p></content:encoded>
</item>
<item>
<title>Trolling is a art</title>
<link>https://steveklabnik.com/writing/trolling-is-a-art/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/trolling-is-a-art/</guid>
<pubDate>Fri, 24 Sep 2010 00:00:00 GMT</pubDate>
<content:encoded><p>I find it incredibly hard not to be judgmental.</p>
<p>I’m not sure what part of my personality makes this happen, but even when I try to curb tearing down other people, I end up doing it anyway. I’m not sure if it’s just me, but I think part of the problem is that these kinds of things are rewarded.</p>
<p>One of the reasons that I use my real name everywhere is to try to improve the civility of my discourse. When you’re anonymous, it’s really easy to be callous. And when I used a handle, I was even worse. But even now, I can think of three times when I accidentally was mean for basically no reason: That tweet, the Diaspora incident, and the only Hacker News comment I’ve ever deleted.</p>
<p>In all three of these instances, it’s not so much that I was wrong. It’s that I ended up presenting my criticisms poorly. I think I did a decent job with the Diaspora one, but people still took it the wrong way, so I failed to communicate. I’m certainly not infallible. I guarantee you there are n00b errors and security holes in my own codebases. So I really shouldn’t be so quick to judge. That’s all I have to say about that. I’ll just have to do better going forward.</p></content:encoded>
</item>
<item>
<title>80% of success is showing up</title>
<link>https://steveklabnik.com/writing/80-of-success-is-showing-up/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/80-of-success-is-showing-up/</guid>
<pubDate>Tue, 21 Sep 2010 00:00:00 GMT</pubDate>
<content:encoded><p>It’s funny how true this quote is.</p>
<p>There are three people that I really admire: _why, Zed Shaw, and Yehuda Katz. And to simplify my tenses a bit, I’ll speak of _why in the present tense. What do these three guys have in common? They all churn out badass software like nobody’s business. _why started so many projects that it just adds to his mythical qualities; Zed has been cranking out software and blogs left and right, and Yehuda’s “<a href="http://yehudakatz.com/2010/09/14/heres-to-the-next-3-years/">I’m quitting Engine Yard</a>” post basically said “I’m pretty much done rebuilding the server side of software in my own image, now it’s time for me to focus on the client side of the equation.” Not with that much hubris, of course, but it’s pretty much true.</p>
<p>Now, I don’t mean to imply that they’ve ‘just shown up,’ but it seems to me that productivity is directly related to just getting started on things. I’ve had people tell me that they think I’m superhumanly productive, which is funny to me: I’m almost constantly fretting about how much time I waste every day. And while I do have a ton of projects, unlike all those guys, I tend to not fully finish them. It’s something that I’ve been trying to work on lately, because I have so many things that I want to accomplish.</p>
<p>For me, the biggest hold on cranking out even more stuff is just simply starting to work. Often, if i slack off for half an hour, I slack off all evening. Now that there’s a TV with Netflix in my house again, it’s difficult to not watch all of the amazing documentaries that are on there. But if I manage to start working on stuff, I find that I keep doing it. It’s like I get to pick one or the other: projects or slacking. I can’t quite seem to do both. And so I’ve been starting some minor experimentation to see if I can’t work on projects more often than not.</p>
<p>One thing that I’ve sort of half-tried is the <a href="http://en.wikipedia.org/wiki/Pomodoro_Technique">Pomodoro Technique</a>. It’s pretty sweet: basically, you set a timer for 25 minutes, and start working. You can take a break when it goes off. Surely you can wait to refresh Hacker News for 25 minutes, right? That’s something else I’ve done: on Friday, I put Reddit and HN in my /etc/hosts file. They’re totally blocked, for now. After <a href="http://blog.steveklabnik.com/trouble-with-diaspora">last week’s situation</a>, I spent waaaay too much freaking time on HN, even for me. I love it, and I’m learning a ton every day, but it’s becoming an issue for my productivity. So it’s gone for a bit. It’s been rough, but I’ve been getting lots done, and I think when I start reading it again, I’ll be able to keep it at reasonable levels.</p>
<p>Anyway, so that’s what I’m doing to keep productive. Anyone else have any thoughts on this? How do you keep working on side projects? What works for you?</p></content:encoded>
</item>
<item>
<title>Trouble with Diaspora</title>
<link>https://steveklabnik.com/writing/trouble-with-diaspora/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/trouble-with-diaspora/</guid>
<pubDate>Fri, 17 Sep 2010 00:00:00 GMT</pubDate>
<content:encoded><p>So, Wednesday, Diaspora was released.</p>
<p>If you’re not familiar, a few months ago everyone was up in arms about the latest Facebook privacy change. So four college kids started <a href="http://www.kickstarter.com/projects/196017994/diaspora-the-personally-controlled-do-it-all-distr">a Kickstarter project</a> with a dream: let’s make a distributed, private Facebook. They asked for $10,000. They got about $200,000.</p>
<p>They worked on the code all summer, in secret. Wednesday, a ‘developer preview’ came out. They’d promised something by the end of the summer, and the 15th of September is the last day. So <a href="http://github.com/diaspora/diaspora">they put it up on GitHub</a>.</p>
<p>Oh boy.</p>
<p>Basically, the code is really, really bad. I don’t mean to rain on anyone’s parade, but <a href="http://www.theregister.co.uk/2010/09/16/diaspora_pre_alpha_landmines/">there are really, really bad security holes</a>. And they’re there due to things that any professional programmer would never dream of leaving out of their code. I don’t want to disclose too many details, but you can see the code yourself on GitHub.</p>
<p>At first, I found one. So I tried to patch it. And I did, and it got accepted into master. Awesome. But then, the more I read, the more bad things I found. They’re going to need a complete overhaul to fix this. Go over every last piece of code. And don’t even get me started on their encryption code.</p>
<p>But basically, please heed their own warning from <a href="http://www.joindiaspora.com/2010/09/15/developer-release.html">the announcement</a>:</p>
<blockquote>
<p>Feel free to try to get it running on your machines and use it, but we give no guarantees. We know there are security holes and bugs, and your data is not yet fully exportable. If you do find something, be sure to log it in our bugtracker, and we would love screenshots and browser info.</p>
</blockquote>
<p>If you find one of the many, many nodes that people are publicly posting, please don’t use your real passwords, or information.</p>
<p>And if you’re a developer, consider helping out. It’s going to be rough going: the mailing list and issue trackers are full of people that have no idea what they’re doing. Be prepared to wade through tons of crap. But they could really, really use the help. I’ll be submitting another patch or two, but it needs much, much more than I can give.</p>
<p>EDIT: Please check out <a href="http://blog.steveklabnik.com/this-is-why-new-media-hates-old-media">my follow up post</a>, where I talk about being misquoted regarding this post.</p>
<p>EDIT 2: Patrick has a post up describing the exploits, now that there have been some patches applied. If you’re a technically inclined individual, you might be interested. You can find it on <a href="http://www.kalzumeus.com/2010/09/22/security-lessons-learned-from-the-diaspora-launch/">his blog</a>.</p></content:encoded>
</item>
<item>
<title>A word about _why, Whyday, and Hackety Hack</title>
<link>https://steveklabnik.com/writing/a-word-about-why-whyday-and-hackety-hack/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/a-word-about-why-whyday-and-hackety-hack/</guid>
<pubDate>Thu, 19 Aug 2010 00:00:00 GMT</pubDate>
<content:encoded><p>Today is Whyday. A programmer’s holiday. Today is the day that we don’t forget that programming isn’t all Agile and driven by behavior and Serious Business. Inside each cubicle, the hacker spirit is trying to burst free. Today, it will.</p>
<p>Today, I’m also releasing a prerelease version of Hackety Hack 1.0. It’s been a long time coming. The announcement of Whyday got me thinking about the past year of a world without _why, and the ‘your domain is about to expire’ notices got me thinking about how I’ve now been the head of the Hackety Hack project for almost 12 months, too. I don’t like thinking about this, because I almost let _why down.</p>
<p>I never met _why. I can’t really claim to know him, or know what his intentions were, but I think about him a lot. ‘_why’ the persona, really embodies everything that I love about software. The world owes _why a great debt, and I’d like to think that I’m doing what I can to repay him.</p>
<p>At least, I am now. I don’t like to think about roughly 6 or 8 months of the last year of Hackety Hack; I did an exceptionally poor job of being a steward. Being alone, having no help, and having a bit of an identity crisis is an excuse; you can decide if it’s a good or poor one. But things have been looking up, and we’ve got some momentum, I owe a great debt to Fela for helping make this summer happen.</p>
<p>I really do feel that the Little Coder still has a predicament, and I’m almost to the point where they can find some solace. I still feel that Hackety Hack could become an absolutely invauable project, I’ve just got to keep going. I can, I will, and I am. But I wouldn’t be able to if so many people didn’t care, say a nice world, or lend a hand.</p>
<p>So thanks. And as ashbb would say, “Let’s all have fun with Hackety Hack!”</p>
<p>Happy Whyday.</p>
<p>EDIT: The release is now out. See the announcement <a href="http://hackety-hack.com/posts/happy_whyday">here</a>, and get it <a href="http://hackety-hack.com/">here</a>.</p>
<p>Thanks!</p></content:encoded>
</item>
<item>
<title>A blip in time</title>
<link>https://steveklabnik.com/writing/a-blip-in-time/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/a-blip-in-time/</guid>
<pubDate>Sat, 24 Jul 2010 00:00:00 GMT</pubDate>
<content:encoded><p>Sometimes, insight comes from very unlikely places.</p>
<p>I’m keenly interested in the music industry, and how it will evolve, or possibly die. I care deeply about music; it’s had a profound effect on my life over the years. Even though the entire industry has been screaming that they’re dying, I truly believe that music will never die. It’s bigger than money. I’m not exactly sure about the details, even though I have some ideas.</p>
<p>In any case, I was reading about this the other day, and I came across <a href="http://sometimesright.com/2010/05/sir-mick-on-music-profits/">this quote from Mick Jaggar</a>:</p>
<blockquote>
<p>But I have a take on that – people only made money out of records for a very, very small time. When The Rolling Stones started out, we didn’t make any money out of records because record companies wouldn’t pay you! They didn’t pay anyone!Then, there was a small period from 1970 to 1997, where people did get paid, and they got paid very handsomely and everyone made money. But now that period has gone.So if you look at the history of recorded music from 1900 to now, there was a 25 year period where artists did very well, but the rest of the time they didn’t.</p>
</blockquote>
<p>An interesting bit of history, and an interesting perspective from someone who has some skin in the game, to say the least. But it’s the last sentence of that blog post that gives the true insight:</p>
<blockquote>
<p>Don’t stop at 1900, though. If you think of the entire history of the world, the notion that you could make an outsized return on making music is a complete aberration.</p>
</blockquote>
<p>When I first read this sentence, something clicked inside my head, and I had a realization: does this apply to everything?</p>
<p>We always think that we’re moving forward. It’s called ‘progress’ for a reason. But it’s kind of obvious why we’d think this: we’re the ones living it! Of course we’d do know wrong!</p>
<p>But what if we are?</p>
<p>What if we’re not progressing, but regressing? What if the latest and greatest isn’t where things are going… it’s just a small blip in the graph?</p>
<p>Maybe we’re actually going down the wrong path, building towards deflation, and need to turn around and revise our position. Maybe we’ve misstepped. What if we’re doing what we think is right, but it’s actually wrong? What if we’re just on the upswing of some local minima?</p>
<p>I don’t necessarily mean that, for example, the music industry will cause the total collapse of everything. But maybe it was a temporary peak. Things will return back to some even level again, and over the grand sum of history, such a small deviation will be forgotten.</p>
<p>Then I start to apply this line of reasoning to everything. How can we really know if we’re doing better than yesterday?</p></content:encoded>
</item>
<item>
<title>A case study in being excellent: Divvy</title>
<link>https://steveklabnik.com/writing/case-study-in-being-excellent-divvy/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/case-study-in-being-excellent-divvy/</guid>
<pubDate>Wed, 30 Jun 2010 00:00:00 GMT</pubDate>
<content:encoded><p>The images in this post have been lost to time. Oh well. Original post below, images removed.</p>
<hr />
<p>The other day I wrote a post about <a href="http://blog.steveklabnik.com/being-excellent-to-each-other">being excellent to each other</a>. Today, I’ve got a great example of that for you: Divvy.</p>
<p>What would you do if you received an email like this?</p>
<p>Here’s what happened:</p>
<p>Here’s a <a href="http://www.reddit.com/r/programming/comments/ckcbr/mizage_a_small_development_company_that_made/">link to the thread</a>. Check out some of these comments:</p>
<p>… okay, maybe that last one is a bit much. But still, they’ve obviously been repaid many times over for something that <em>didn’t even cost them money</em>. This <a href="http://news.ycombinator.com/item?id=1473770">comment on HN</a> really says it all:</p>
<p>If you’d also like to support a company for being awesome, here’s a link to <a href="http://www.mizage.com/divvy/">Divvy</a>.</p>
<p>EDIT: They’ve posted the sales afterward on Reddit <a href="http://www.reddit.com/r/programming/comments/cmmfg/hey_reddit_a_week_ago_someone_posted_about_divvy/">here</a>.</p></content:encoded>
</item>
<item>
<title>Productivity Waves</title>
<link>https://steveklabnik.com/writing/productivity-waves/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/productivity-waves/</guid>
<pubDate>Tue, 08 Jun 2010 00:00:00 GMT</pubDate>
<content:encoded><p>Right now, I’m having a bit of a ‘darkness before the dawn’ kind of moment. I feel like I might soon start to become productive again.</p>
<p>It’s sort of strange, how these things travel in waves. Just a month or two ago, I was super-ultra-crazy productive. I was working on Hackety, coding Bindlr, writing three days a week on the blog, and more. This doesn’t even count the more-than-a-full-time CloudFab. But lately, all I’ve wanted to do was hang out and play video games.</p>
<p>It seems like this is a cyclic kind of thing. I think 80 hour weeks, even of things that you love, is kind of unsustainable. It was lots of fun, and I was feeling pretty good while I was doing it, but now that I’m off that horse, it’s really hard to get back on.</p>
<p>In any case, here’s the status of some of the stuff I’ve been up to, if you’ve been wondering:</p>
<ul>
<li>Blog: I’ve got a bunch of stuff half-written. I’ll be starting to finish those off and getting them up here. I’ve had lots to say, just no motivation to say it.</li>
<li>Watch.steve: I want a better design, so I’m not doing anything with the site ’till that’s done. I don’t want to mess with Ryan’s look and feel. But I’ve got episodes written up, they just need to be recorded.</li>
<li>Hackety Hack: Fela is making great progress, and I’m going to start reaching out to Mac development communities to see if I can get some help on the issue that’s ruining my ability to fix things.</li>
<li>Bindlr: Up, running, and working. Needs more marketing/outreach.</li>
<li>Twitter: started using it again, you read that post.</li>
<li>Selling my stuff: most of it is in boxes or thrown away. Time to start selling the things that were worth money.</li>
<li>Maria: currently missing her.</li>
<li>Friends: started actually making it back to XOmB meetings on Saturdays, and actually going out with some people I haven’t talked to in a while. I was really really bad at this in some cases.</li>
</ul>
<p>This post is kind of random, and I apologize. I’ve got a good mix of technical things, book reviews, and social commentary waiting in the wings.</p>
<p>I’m excited for what June will bring.</p></content:encoded>
</item>
<item>
<title>A Declaration of the Independence of Cyberspace</title>
<link>https://steveklabnik.com/writing/you-are-terrified-of-your-own-children-since-they-are-natives-in-a-world-where-you-will-always-be-immigrants/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/you-are-terrified-of-your-own-children-since-they-are-natives-in-a-world-where-you-will-always-be-immigrants/</guid>
<pubDate>Sat, 01 May 2010 00:00:00 GMT</pubDate>
<content:encoded><p>I just saw this today. It kind of goes along with my “Move to the Internet” post from a few days ago.</p>
<p>Enjoy.</p>
<p><a href="https://www.eff.org/cyberspace-independence">A declaration of the Independence of Cyberspace</a></p></content:encoded>
</item>
<item>
<title>A break with the past</title>
<link>https://steveklabnik.com/writing/a-break-with-the-past/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/a-break-with-the-past/</guid>
<pubDate>Tue, 27 Apr 2010 00:00:00 GMT</pubDate>
<content:encoded><p><a href="http://www.countdowntooauth.com/">Pretty soon</a>, Twitter is going to turn off Basic Authentication and switch entirely to OAuth. <a href="http://www.scripting.com/stories/2010/04/26/theToxicCoralReef.html">People are upset</a>. It’s natural. If apps aren’t updated, they’ll stop working, entirely. This could be bad.</p>
<p>But in the long run, it won’t be.</p>
<p>In any sort of long-term endeavor, things can go sour, due to accumulating cruft over time. You see this in married couples that argue every day, you see this in old-guard companies that have six layers of management, you see this in software of all kinds. You don’t want to break old stuff, so you support it. Rocking the boat may cause some problems. It could bring up old wounds. You might have to ask someone to improve. Things won’t ever be the same, and you’re gambling with the outcome, hoping that in the end, everything will be better.</p>
<p>Sometimes, this gamble pays off big.</p>
<p>Apple’s managed to do this three times in its history, in a huge way. They’ve reinvented themselves a few times, but there were three huge, backward- compatibility breaking changes that could have killed the company. The first was the transition from the 68k architecture to the PowerPC architecture, the eventual change from PowerPC to x86, and the transition from OS9 to OSX.</p>
<p>Apple managed to make all of these transitions successfully, and it’s been one of the greatest benefits to the company. A huge part of why Windows is so horrible is that it goes above and beyond the call of duty with backwards compatibility; Microsoft never wants to rock the boat. When they try to, they get punished. Look at how poorly the transition to Vista went. So what made it go so right for Apple, and so wrong for Microsoft?</p>
<p>The first thing that Apple did right with all of these transitions was being open about them. People were given ample time to move over to the new platform. I was young when the 68k change happened, so my remembering of that time might be fuzzy, but the OSX announcement certainly was made with ample time. An entire year elapsed between the public beta and the release of 10.1, which was widely considered the first version of OSX that was worth using. I remember buying a copy of 10.0, and Apple gave you an upgrade to 10.1 for $30, similar to the Snow Leopard upgrade. 10.0 was just too buggy to be considered a proper replacement. Anyway, that was a year, and that’s not counting the time between the announcement and the beta release. For the transition to Intel, the initial announcement was made in 2005, and Steve Jobs said the transition would happen over the next two years. That gave everyone plenty of time to get their ducks in a row.</p>
<p>The second thing that Apple did correctly was provide ample tools for dealing with the transition. For the first switch, they provided a 68k emulator, and kept it going all the way up until the Intel transition. This meant that people didn’t have to re-write their apps from scratch, and gave them a big window for re-writing apps. Rosetta fulfills the same role for the PowerPC/Intel change. And during the OS9/OSX changeover, Apple not only let you emulate OS9, but also created the Carbon framework to bridge the gap between System 9 and Cocoa.</p>
<p>Finally, Apple made these changes boldly, and didn’t back down. With any kind of massive change like this, unless someone forces the change through, people will stay with the old system, and the transition fails. You can see this happen with both the Python move to Python 3000, the move from Ruby 1.8 to 1.9, and the move from XP to Vista. There’s a fine line between giving people leeway to make the transition and actually forcing them to do it. Apple did a poor job with this when it came to Adobe, who’s being an incredible laggard with Photoshop. It’s a massive application, sure, but Apple announced Carbon’s end of life ages ago, they really should have gotten their shit together. Generally, Apple’s not afraid to end-of-life products after the transition time is over, though.</p>
<p>In any case, those seem to be the three things that make a transitional period work: transparency, tools, and finality. If Twitter wants to survive the Oauthcalypse (as if they won’t), they need to follow the same path. And they have been. They announced the move to Oauth over a year ago (I think), they’ve had libraries and examples out there for ages, and they’ve picked a date and they’re sticking to it.</p>
<p>I’m glad they’re doing it. OAuth is an important part of Twitter’s future, and it’s good that they’re embracing it fully.</p></content:encoded>
</item>
<item>
<title>Book review: Rework</title>
<link>https://steveklabnik.com/writing/book-review-rework/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/book-review-rework/</guid>
<pubDate>Wed, 10 Mar 2010 00:00:00 GMT</pubDate>
<content:encoded><p>I’ve been waiting for this book for a while. “Rework” is the new book by Jason Fried and David Heinemeier Hansson from 37signals. It hit stores on Tuesday. Here’s a (non-affiliate) link to <a href="http://www.amazon.com/gp/product/0307463745/ref=s9_simh_gw_p14_t1?pf_rd_m=ATVPDKIKX0DER&amp;pf_rd_s=center-2&amp;pf_rd_r=1RRQJN39HSJ4SMVA2EM4&amp;pf_rd_t=101&amp;pf_rd_p=470938631&amp;pf_rd_i=507846">Rework on Amazon</a>.</p>
<p>For those of you who don’t know, I’m down in Houston, Texas. I’m here for a few days before heading over to Austin for SXSW. There’s a lot of really cool start up stuff, as well as 3D people, and a friend of mine. It’s been a good trip so far. But even with waking up at 3am, connecting flights, and a few hours of driving, I still managed to find a spare moment to head over to a Borders and grab a copy of Rework. And even though I’m running all over town, Nick is driving, so I’ve been able to read all of Rework in between lunches and networking events.</p>
<p>Rework is interesting. I described it earlier today <a href="http://twitter.com/steveklabnik/status/10281523422">as a philosophy text</a>, and I feel that description is absolutely apt. It’s 37signals in its purest, most potent form. If you’ve read <a href="http://gettingreal.37signals.com/">Getting Real</a>, this territory will be familiar. In fact, a lot of it is basically the same. I’d be lying if I told you otherwise. The real truth is that it doesn’t matter. People who don’t already know and love 37signals won’t have read Getting Real, and so this stuff will be novel to them. People who do won’t mind re-reading this information again, as they’ve bought into the philosophy. And an update isn’t a bad thing, either. What makes Rework interesting is how it’s different from Getting Real, not what’s the same.</p>
<p>I thought it’d be most interesting to talk about Rework in the context of it’s own philosophy. I think there are three points in particular in the book itself that point out just why this book is so good. The first is an old 37signals standby, the other two are new.</p>
<h3>Build half a product, not a half-assed product</h3>
<p>This got <a href="http://gettingreal.37signals.com/ch05_Half_Not_Half_Assed.php">a section devoted to it in Getting Real</a>. Here’s the core idea:</p>
<blockquote>
<p>Throw in every decent idea that comes along and you’ll just wind up with a half-assed version of your product. What you really want to do is build half a product that kicks ass.</p>
</blockquote>
<p>They mention that Rework used to be twice as big. The next to last draft was 57,000 words, and the final draft was 27,000. This is the biggest difference between the two books. It’s the most pure dosage of Kool-Aid I’ve ever read. Each section feels finely honed. They’re all either one, two, or three pages, and an accompanying picture. This book is about what’s worked for the company so far over its lifetime, and this refinement process is clearly demonstrated here.</p>
<p>It’s always easy to ask for more. I’m really curious about the things that were cut. Were there more sections? Was each section twice as long? A little of both? At the same time, this exactly exemplifies the thinking this section is railing against. If the book was twice as long, would I have learned twice as much? Probably not. YAGNI.</p>
<h3>Decommoditize your product</h3>
<blockquote>
<p>Make you part of your product or service. Inject what’s unique about the way you think into what you sell.</p>
</blockquote>
<p>This is something that these guys do really well. It’s part of having such strong opinions, and sharing them with the world. Everyone knows who 37signals is and what they stand for. If I wrote each chapter of Rework into a blog post, you’d still recognize it as their philosophy. It also comes through in the writing. They mention in the notes that Matthew Linderman helped them pull their distinct styles of writing into a more cohesive whole. He did a good job, and didn’t let the voice get lost in the editing.</p>
<h3>Out-teach your competition</h3>
<blockquote>
<p>Teaching is something individuals and small companies can do that big companies can’t.</p>
</blockquote>
<p>Teaching is a topic I’ve been getting more and more into lately. Hackety Hack is about teaching; when I speak, I’m teaching; this blog is about teaching. Rework is about teaching the lessons 37signals have learned about business to the world. A lot of Signal vs. Noise is about teaching. It’s a great way to get people to recognize you, and a great way to give back. The world can always use more great teachers.</p>
<h2>Wrapping up</h2>
<p>There’s a reason I’m up late, writing this review. I couldn’t put Rework down. I enjoyed revisiting the old topics; the new ones are still tumbling around in my brain. I’m sure this book is going to spawn a bunch of posts on this blog in the future, as I add my own thoughts to the stuff I’ve learned. I’d recommend this book to anyone who’s working in the business world or doing a startup, even if they hate that term. It’ll give you a lot of interesting thoughts to chew on.</p></content:encoded>
</item>
<item>
<title>Create a more compelling experience for your users through game mechanics</title>
<link>https://steveklabnik.com/writing/create-a-more-compelling-experience-for-your-users-through-game-mechanics/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/create-a-more-compelling-experience-for-your-users-through-game-mechanics/</guid>
<pubDate>Mon, 08 Mar 2010 00:00:00 GMT</pubDate>
<content:encoded><p>Ever wonder why some websites are so addictive? Certain sites always keep you going back, time after time after time. Well, I can’t speak for all of them, but there’s a subtle reason that some sites draw your attention on such a repeated basis: They’re actually games.</p>
<h2>Wait, games?</h2>
<p>Try a little thought experiment: If I say, “Yeah, he’s a ______ addict,” what are the first few things that pop into your mind? For me, top two are “heroin” and “World of Warcraft.” I’m not sure what that says about me as a person, but ignore that for now. What makes these two things so addicting? Why are they basically synonymous with the word “addict”? Lots of people smoke pot. Lots of people play Call of Duty. Lots do both, and in copious amounts. So why don’t they get the same label?</p>
<h2>Heroin: it’s a hell of a drug</h2>
<p>Yeah, that reference is to cocaine, another famously addictive substance. Oh well.</p>
<p>Heroin is the poster child for addiction because it’s got a built-in viral loop. That sentence sounds ridiculous, but it’s true. It’s very easy to start out with, as it’s snorted. No scary needles or anything. You get high really quickly, due to its chemical properties combined with the fact that your nose is already close to your brain. It gives a really intense high that is also fairly short. As you do it, you develop both a psychological addiction as well as a tolerance. You simultaneously develop a deep desire for more of the drug as you need a larger quantity of the drug to get the same high. Eventually, it becomes more and more difficult, but you’re so addicted that you get over your fear of needles and start mainlining.</p>
<p>World of Warcraft works the same way. It’s easy to try, as there are mechanisms to invite your friends, and the system requirements are fairly low for a video game. The first few quests are super easy, and so you hit that quick reward. You get addicted to “Ding!” but it takes longer and longer every time you do it. Eventually, you max out on levels and have to start doing other things to get your fix. It may sound funny, but it’s absolutely true. People talk about “relapsing.” They speak of “craving.” That’s why WoW has so many subscribers.</p>
<h2>How to replicate this success</h2>
<p>I can’t guarantee that you’ll be able to make your site as addictive as heroin is, but many sites use the same basic psychology to keep you coming back. Game mechanics are one of the tools they use to develop that psychological addiction. This is something we’ve been seeing more and more of lately, but it isn’t really being talked about explicitly as a major trend. I really think that this stuff is really important and useful.</p>
<p>There are a couple of different mechanisms that web sites can incorporate that fall under the realm of “game mechanics:”</p>
<ul>
<li>Collectibles: <em>Any sort of item you can accumulate. Sometimes comes in “sets,” which are finite lists.</em></li>
<li>Points: A concrete number that lets you compare two people.</li>
<li>Levels: A target number of points, you gain the “level” when you go over that number.</li>
<li>Trophies: A special kind of level that’s unrelated to points. You get it for some other arbitrary reason.</li>
<li>Rankings: A place where you can go to see how many points, levels, and trophies others have</li>
<li>Tournaments: A competition between people.</li>
</ul>
<p>We’ve all heard these terms used in games. But in web sites? Okay, let’s try those things again:</p>
<ul>
<li>Collectibles: Gowalla items. Facebook “Gifts”</li>
<li>Points: Twitter followers. Facebook friends. Number of feedbacks. Reddit Karma.</li>
<li>Levels: eBay “Power Sellers.” Foursquare “Super Users.”</li>
<li>Trophies: Badges, of any kind. “Achievements”</li>
<li>Rankings: FourSquare’s Leaderboard. Klout. Listorious. Hacker News’ top list.</li>
<li>Tournaments: I actually can’t come up with a good example of this. Thoughts?</li>
</ul>
<p>The same feedback loop happens on these websites. You say something interesting on Twitter, you gain another follower or two. You say something else, another follower. You check in, oh look, you’re the mayor! You sell an extra hundred things and get your Power Seller discount.</p>
<p>That’s the hard stuff. It’ll get you hooked, and coming back for more.</p>
<h2>Where’s all of this going?</h2>
<p>This is the current stuff that’s being done with game mechanics. But where could we go, in the future?</p>
<p>A while back, there was a huge debacle over ReadWriteWeb and Facebook connect. To give you the basic idea, <a href="http://www.readwriteweb.com/">ReadWriteWeb</a> is a blog that talks about everything Web2.0. They wrote an article entitled “<a href="http://www.readwriteweb.com/archives/facebook_wants_to_be_your_one_true_login.php">Facebook Wants to be your One True Login</a>.” Read the comments. Notice something funny? Due to some Google magic, if you were to Google “Facebook login” the day that was posted, that article would appear at the top under the “Google News” results. Now, RWW uses Facebook Connect for their commenting system, and a ton of people apparently don’t know how to use the Internet. So when they said, “Hey, I think I’ll go to Facebook today,” they Googled “facebook login,” clicked the news story, and went to RWW. They then ignored that RWW is a blog completely covered in red that looks nothing like Facebook, scrolled until they found the Facebook icon, clicked it, logged in, and then said “wtf, this isn’t my facebook? Why’d they change the interface again???” This happened a week after a middle-sized interface upgrade on Facebook, for extra hilarity.</p>
<p>Now, I won’t comment on those people or that situation directly. But one of my favorite Hacker News posters, <a href="http://news.ycombinator.com/user?id=patio11">patio11</a>, posted <a href="http://news.ycombinator.com/item?id=1119186">a really interesting comment</a> about the situation. I’m linking to the person he’s responding to, for context:</p>
<blockquote>
<p>Pyre: Facebook can’t improve their interface to make users not type “facebook login” into Google as a way of accessing their site.</p>
</blockquote>
<blockquote>
<p>patio11: That is a failure of the imagination. They certainly could – whether it is worth doing or not is another question, but hey, that is what God gave us A/B testing to figure out.</p>
</blockquote>
<blockquote>
<p>“Hey user, it looks like you came to us today from Google searching for [Facebook login]. Did you know that there is a better way? Type facebook.com into [blah blah blah]. Try it now and we’ll give you 5 free credits for [without loss of generality: FarmVille]!”</p>
</blockquote>
<blockquote>
<p>Great job! You should do that every time. If you do that to log into Facebook the next five days you use the service, we’ll award you a Facebook Diploma and give you another 10 free credits for [without loss of generality: FarmVille]!"</p>
</blockquote>
<blockquote>
<p>On the back end, you show the above prompts to N% of your users who you detect coming to the login page from Google search results (this is trivial – check the referer). You then compare any user metric you want for the “Was Shown Facebook Login Course” population and “Complete Facebook Login Course” population with the population at large. Kill the test if it hurts your metrics, deploy it sitewide if it helps them.</p>
</blockquote>
<p>How cool would that be? Now the game mechanics aren’t being used just to increase engagement, but to actually teach people how to use your site or service. It’s classical conditioning; reward people for doing the right thing, and they’ll keep doing the right thing.</p>
<h2>Game mechanics are your MVP</h2>
<p>So how’s this stuff relevant to your startup? Well, I think this idea ties in really well with the concept of a Minimum Viable Product. Here’s the idea: Build your MVP, and then build game mechanics in. Unlock new features based on game mechanics. This gives you a few advantages:</p>
<ol>
<li>Your new users only get the most basic experience, which is still useful. It’s a simplified, streamlined experience.</li>
<li>Users only get the new features added that are relevant to how they use the site itself.</li>
<li>You can “fake it till you make it” by implementing the features that are most useful to your users. Is everyone getting Badge A and not Badge B? Implement Feature A Level 2 first!</li>
</ol>
<p>I think that this makes for a really strong experience, if done right. Foursquare kind of does this already in a crude way with their Super User features. But I think it could be taken to a whole new level.</p>
<p>Think about this: Facebook, where you can only friend people, update your profile, and send messages at first. Soon you unlock the ability to use applications. Then the ability to create pages and groups. The interface slowly unfolds in front of you. What about Reddit, where posting comments is all you can do at first? A hundred upvotes gives you the ability to downvote. Ten comments lets you post stories. (Hacker News sort of does this already, with a minimum karma before downvoting is enabled.)</p>
<p>If you could pull it off, I think it’d make for a really compelling user experience. It does bring one extra design skill that many people may not have, though: balance. Game designers are used to this already, but your potential “Power Users” might not like having to wait to get more advanced features. Then again, this might also solve some issues, like spam. If you had to have 100 positively moderated comments before posting a story on Digg, it’d be much harder to just sign up for spam accounts to submit bogus stories.</p>
<p>This idea can be taken in a lot of different directions. I’m sure I’m only barely scratching the surface with this idea, but I think it’ll go really far. What do you think? Any interesting mechanics I’ve missed? Any really interesting thoughts for how services can incorporate game mechanics? I’ve decided to re-open comments, but if nobody uses them, I’ll just shut them off again. Let me know what you think.</p>
<p>This post has been featured on the Startup Pittsburgh blog, <a href="http://startuppittsburgh.com/2010/04/create-a-more-compelling-experience-for-your-users-through-game-mechanics/">here</a>.</p></content:encoded>
</item>
<item>
<title>Writing a su feature with Authlogic</title>
<link>https://steveklabnik.com/writing/writing-a-su-feature-with-authlogic/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/writing-a-su-feature-with-authlogic/</guid>
<pubDate>Fri, 05 Mar 2010 00:00:00 GMT</pubDate>
<content:encoded><p>Sometimes, when responding to a support request, it’s nice to see what your users see. At the same time, you don’t want to ask your users for their passwords, out of respect for their privacy. So what do you do?</p>
<p>Well, *NIX systems have a program called su. Here’s what man su has to say:</p>
<pre><code>NAME
su - run a shell with substitute user and group IDs
SYNOPSIS
su [OPTION]... [-] [USER [ARG]...]
DESCRIPTION
Change the effective user id and group id to that of USER.
</code></pre>
<p>su can be thought of as “substitute user” or “switch user.” It’s a command system administrators use to assume the identity of one of their users, or a way for someone with the root password on the system to switch to the root account itself. So how can we incorporate this into a web application?</p>
<p>Well, we want to first log ourselves out, and then log in as the user we’re su-ing to. That’s it. The tricky part, however, comes in when we’re logging in: as we said before, we don’t want to ask for their password. Luckily, Authlogic provides a way to create our UserSession object directly from a User object by just passing it to create.</p>
<p>This lets us write a controller method to do this pretty easily:</p>
<pre><code>def su
@user = User.find params[:id]
current_user_session.destroy
UserSession.create!(@user)
flash[:notice] = "You've been su-d to that user."
redirect_to dashboard_path
end
</code></pre>
<p>Add in a route:</p>
<pre><code>map.admin_su "/admin/su/:id", :controller =&gt; "admin", :action =&gt; "su"
</code></pre>
<p>And to a view somewhere in your administrative tools:</p>
<pre><code>&lt;%= link_to "log in as this user", admin_su_path(@user) %&gt;
</code></pre>
<p>And we’re good to go!</p>
<p>One last thing about this, though: You don’t want to let anyone who’s not an administrator do this, for obvious reasons. My administrative controllers always include a block like this:</p>
<pre><code>access_control do
allow :admin
end
</code></pre>
<p>acl9 makes this really easy, but it’s really important.</p>
<p>So there you have it. Easy as pie.</p>
<p>EDIT: This post made the Rails subreddit, and <a href="http://www.reddit.com/r/rails/comments/cb0da/writing_a_su_feature_with_authlogic/c0rf26w">brettbender posted his code</a> to get you back to admin.</p></content:encoded>
</item>
<item>
<title>Reddit drama: an interesting look at community values</title>
<link>https://steveklabnik.com/writing/reddit-drama-an-interesting-look-at-community-values/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/reddit-drama-an-interesting-look-at-community-values/</guid>
<pubDate>Mon, 01 Mar 2010 00:00:00 GMT</pubDate>
<content:encoded><p>I bet social psychologists are having a field day with the Internet. I know nothing about the field, so I don’t know if there are thousands of papers about it already written or not, but there should be. I can’t image that we’ve ever had a period in history when more communities are being formed; they’re also being formed out in the open.</p>
<p>I’ve been involved in many different online communities over the years. I’ve already written before about the seeming decline of Reddit’s content… but this past week, something really interesting has happened. Reddit has had two major events occur that have caused it to be increasingly self-reflexive on the values and norms that the community desires for itself. This is particularly interesting because normally, cultural norms have to be observed, not codified. But here, we’re seeing completely open discussion about “The community says this is not okay.” It’s really interesting stuff.</p>
<h2>Speed Issues</h2>
<p>I won’t comment on the technical details involved, but Reddit has gotten significantly slower over the past few months. This is normal, as the community is growing. But it’s caused quite a bit of a stir lately. Many users are seeing increased loading times, error messages, missing posts, and various other problems.</p>
<p>What’s a user to do in this situation? Post about it. See the <a href="http://www.reddit.com/search?q=fix+search&amp;sort=hot&amp;t=month">posts about the search feature</a>, or posts about the slowdown.</p>
<p>For example, as I look for the links to these posts, I get this:</p>
<blockquote>
<p>Our search machines are under too much load to handle your request right now. :( Sorry for the inconvenience.</p>
</blockquote>
<p>Amusing. But unfortunate. The interesting part, though, is the backlash against the complaints. See this thread, “<a href="http://www.reddit.com/r/reddit.com/comments/b7jnr/reddit_fix_your_fucking_users_they_are_unbearably/">Reddit, fix your fucking users. They are unbearably bitchy.</a>” (The cursing is emulating the posts made complaining about Reddit’s lack of speed.)</p>
<p>There’s a huge discussion about what it means to be part of a free online community. There’s a contingent of people who say that people aren’t entitled to complain, because Reddit is free. Another points out that complaining is better than silently leaving the site, and that while people don’t pay for Reddit accounts, the eyeballs Reddit users provide enable the site to make money.</p>
<p>Some choice comments:</p>
<blockquote>
<p>Hey, my incredibly expensive Reddit subscription fees entitle me to 100% perfect service! LEAVE ME ALONE!</p>
</blockquote>
<blockquote>
<p>Man, I’ve been loving this particular strawman for over a decade now. Thanks for breaking out the classics.</p>
</blockquote>
<blockquote>
<p>The reality is that Reddit is a business. The owners get paid because the users keep coming back to the site. Having users bitch and moan is not a problem for the admins - this isn’t their personal blog, they’re not mopy teenagers who are gonna cut themselves when they see how mean people are being to them on the interwebs.</p>
</blockquote>
<blockquote>
<p>The problem for the admins is when users stop visiting Reddit altogether because of constant technical problems, or fishy administrative decisions about deleting threads, or too many spammers, or etc. They would much rather have us bitch about these things on the front page, create giant threads about them that keep thousands of posters and tens of thousands of readers engaged and on the site, and provide catharsis and a public forum to address the issues.</p>
</blockquote>
<blockquote>
<p>tl;dr: Reddit is a business, they’d rather have us complain than have us leave.</p>
</blockquote>
<blockquote>
<p>Meta-bitching about bitching time</p>
</blockquote>
<blockquote>
<p>The OP posted a well thought out comment simply suggesting that Reddit’s users should be polite and respectful when addressing each other and the people that make using this site possible, and 90% of the responses here just prove his point. It seems like people are more concerned with gaining comment karma by posting bad one-liners than actually participating in intelligent conversation.</p>
</blockquote>
<blockquote>
<p>Is anyone else absolutely done with the reddit user base? I mean… I was told that reddit was this great intellectual news aggregator that through the use of ‘karma’ was able to bad submissions off the front page.</p>
</blockquote>
<blockquote>
<p>Over the past few months I’ve been here all that has been up on the front page has been Glen Beck, Conan O’Brian, Tiger Woods, weekly “Reddit is slow” posts and now this Saydrah BS.</p>
</blockquote>
<blockquote>
<p>I’ll say what I said when Conan O’Brian was on the front page for a week longer then Haiti ever was: I DON’T CARE. The Reddit admins don’t think Saydrah has abused her power as a moderator. At the very least (I personally don’t think she is a spammer) go flame some of the other 1000000000000000 spammers on reddit FFS. This is boring uneducated dribble.</p>
</blockquote>
<blockquote>
<p>KEEP THIS TRASH OFF THE FRONT PAGE.</p>
</blockquote>
<p>And more. Interesting stuff, overall. There’s still quite a few trash comments, though.</p>
<h2>The Saydrah Situation</h2>
<p>The other event involves a user named Saydrah. She’s been accused of promoting links on Reddit for money, and abusing her moderator powers in conjunction with those offenses. This has spawned a huge amount of discussion on if these actions were inappropriate or not. Here’s <a href="http://www.reddit.com/r/reddit.com/comments/b7e25/today_i_learned_that_one_of_reddits_most_active/">the first big thread</a>. Then <a href="http://www.reddit.com/r/IAmA/comments/b7hpb/re_the_alleged_conflict_of_interest_on_reddit/">an AMA with moderators about their opinions</a>. Lastly, <a href="http://www.reddit.com/r/IAmA/comments/b7tew/fine_here_saydrah_ama_it_couldnt_get_much_worse/">Saydrah does an AMA</a> herself and explains her side of the story.</p>
<p>I won’t show you a bunch of comments, only one. And it echoes my opinion on the matter:</p>
<blockquote>
<p>Two things: Frankly, I don’t care if people are paid to submit links…if they’re interesting, I upvote. If not, I ignore them.</p>
</blockquote>
<blockquote>
<p>Second, I have lurked around AR and RA a fair amount, and consistently find myself thinking, “That’s a helpful and well-reasoned response! Oh, it’s Saydrah again!” Whatever else people may say, I feel that you at least do try to contribute positively to this site.</p>
</blockquote>
<blockquote>
<p>…I guess I don’t have a question, so I’ll just leave my un-asked-for opinion here.</p>
</blockquote>
<p>There’s a lot more, on both sides of the conflict. Regardless, people are hashing it out.</p>
<h2>Growth Hurts</h2>
<p>There’s a reason they’re called “growing pains.” Change is always a painful time in any organism’s life, and even though there’s a lot of drama, the current Reddit staff can take pride that they’ve created something that’s so important to people that they feel the need to scream about it for hours. It’s unfortunate that their baby is being embroiled in a flamewar, but these things happen.</p>
<p>We’ll see what the community ends up deciding is acceptable. I’ve managed to not get involved in these particular conflicts, but it sure is interesting to watch!</p>
<p>Edit: Jen added some interesting links on my facebook feed: <a href="http://j.mp/baRqdy">http://j.mp/baRqdy</a></p></content:encoded>
</item>
<item>
<title>The Hackety Manifesto</title>
<link>https://steveklabnik.com/writing/the-hackety-mainfesto/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-hackety-mainfesto/</guid>
<pubDate>Thu, 31 Dec 2009 00:00:00 GMT</pubDate>
<content:encoded><p>For those of you that don’t know, I’ve taken over a project by someone named _why. He dropped off of the face of the Internet back in March, and the community has picked up his works. Because I’ll be writing about the project in the future, and because I feel these words are still relevant, I’m reprinting the two of his essays that led up to the creation of the project, Hackety Hack, here on my blog. Here’s the second one.</p>
<p>Enjoy.</p>
<h1>The Hackety Manifesto</h1>
<p>Nearly four years ago, I wrote an essay called The Little Coder’s Predicament. It’s not too unusual. Lots of others like it have been written. The point is: programming just isn’t available to people like it was with the Commodore 64. I also outlined my requirements for a new cross-platform programming environment for beginners.</p>
<p>The essay was widely linked on Slashdot, Reddit, Lambda, etc. I got lots of mail from people who both agreed and disagreed. Great. Nice. Good people all of them! And yet, nothing has changed. Not really!</p>
<p>I’ve been told of the Xbox development kit and possible programming of Wii channels. The Playstation actually had a language. But none if it has met my criteria for a proper coding platform.</p>
<h2>An Aside: We Care, But Not Enough</h2>
<p>So, why has nothing been done about this? Maybe our interpreters and IDEs (ewww…) are good enough? Lots of people wrote in to say that HTML and JavaScript are the new BASIC. NO!!! You can’t be serious!!! So people have to write two languages now, which are intertwined in an almost inexplicable and unfathomable way? This doesn’t do it.</p>
<p>Hello world should be one line.</p>
<p>In fact, downloading an MP3 should be one line!!</p>
<p>We just don’t care right now, do we? Programmers have a paid gig. So business is happily slurping them up. Look at our books. Look at the programming sites. Programming is tightly coupled to business. Often the first example is an e-commerce site! Our books are like FIFTY DOLLARS!! For crying out loud.</p>
<p>This diatribe isn’t about business being bad. Of course you need to feed your family and drive an Audi.</p>
<p>This diatribe is about adding some balance to the world of programming. Okay, so, let’s take things into our own hands and bring hacking to the young folks.</p>
<h2>The Bylaws of Hackety</h2>
<p>Here are the rules by which Hackety Hack was established:</p>
<ul>
<li>Beginners should be greeted to Hackety Hack by a cartoon character. (For the sake of argument, let’s call this character: Hacky Mouse.)</li>
<li>Also, helpful sentences. Preferably short and with a period.</li>
<li>Hackety Hack is fundamentally a browser and a programming language. For now, Gecko and Ruby.</li>
<li>I’m only using Ruby because I know it. Hopefully, more languages can be added!</li>
<li>Again, this isn’t about Ruby, it’s about simply offering a place for plainspeople to tinker with code.</li>
<li>IDEs are a disaster. Newbs should see only one non-scary window free of tree controls and pinned windows and toolbars.</li>
<li>As such, we want to stay away from project files and makefiles, the trappings of an IDE.</li>
<li>Hackety Hack also adds simple libraries for common things.</li>
<li>Common things are one-liners.</li>
<li>Keep args and options to a minimum.</li>
<li>In Ruby, blocks should be used to open up a method to more advanced possibilities.</li>
<li>Help files are clean, short, simple. Lots of short examples. No frames.</li>
<li>While all bug tickets are helpful and great, I just value tickets from beginners to a greater degree.</li>
<li>Hackety Hack is free and will remain free henceforth.</li>
</ul>
<p>Beyond that… anything which makes life easy and fun for Hackety Hackers is definitely encouraged, even demanded.</p></content:encoded>
</item>
<item>
<title>The Little Coder's Predicament</title>
<link>https://steveklabnik.com/writing/the-little-coders-predicament/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/the-little-coders-predicament/</guid>
<pubDate>Mon, 28 Dec 2009 00:00:00 GMT</pubDate>
<content:encoded><p>For those of you that don’t know, I’ve taken over a project by someone named _why. He dropped off of the face of the Internet back in March, and the community has picked up his works. Because I’ll be writing about the project in the future, and because I feel these words are still relevant, I’m reprinting the two of his essays that led up to the creation of the project, Hackety Hack, here on my blog.</p>
<p>Enjoy.</p>
<h1>The Little Coder’s Predicament</h1>
<p>Okay, then, children of the modern age (where we live in a world so tied together with wires that Pangaea ain’t goin’ nowhere!), you tell me if this is a predicament or not.</p>
<p>In the 1980s, you could look up from your <a href="http://www.lemon64.com/">Commodore 64</a>, hours after purchasing it, with a glossy feeling of empowerment, achieved by the pattern of notes spewing from the speaker grille in an endless loop. You were part of the movement to help machines sing! You were a programmer! The <a href="http://kl.net/atari/">Atari 800</a> people had BASIC. They know what I’m talking about. And the <a href="http://web.archive.org/web/20070612155724/www.guidry.org/ti994a/systempics/4asystem-2.jpg">TI-994A</a> guys don’t need to say a word, because the TI could <a href="http://www.obsoletecomputermuseum.org/ti99/ti99_syn.jpg">say it</a> for them!</p>
<p>The old machines don’t compare to the desktops of today, or to the consoles of today. But, sadly, current versions of Windows have no immediately accessible programming languages. And what’s a kid going to do with Visual Basic? Build a modal dialog? Forget coding for XBox. Requires registration in the XBox Developer Program. Otherwise, you gotta crack the sucker open. GameCube? GameBoy? Playstation 2?</p>
<h2>Coding Just Isn’t Accessible</h2>
<p>Yes, there are burgeoning <a href="http://openxdk.sourceforge.net/">free</a> <a href="http://gbdk.sourceforge.net/">SDKs</a> for many of these platforms. But they are obscure and most children have no means of actually deploying or executing the code on their own hardware! This is obvious to us all and likely doesn’t seem such a big deal. But ask yourself what might have happened had you not had access to a programming language on an Atari 800 or a Commodore. You tell me if this is a predicament.</p>
<p>It turns out, most of the kids in my neighborhood are exposed to coding through the <a href="http://www.ticalc.org/">TI calculator</a>. A handful of <a href="http://www.ticalc.org/programming/columns/">languages</a> are available on the TI and its processor is interesting enough to evoke some curiousity. But this hasn’t spread to its PDA big brothers, where young people could have more exposure to programming. And undoubtedly the utility of a language on the Palm, Pocket PC and others would be useful to many.</p>
<p>So what’s the problem here? We have no shortage of new languages, but they become increasingly distanced from the populace. Are the companies behind these platforms weary of placing the power of a programming language in the hands of users? Is there not a demand any longer? It’s got to be some kind of greed, power, money thing, right?</p>
<p>Perhaps this is just another reason to push Linux and BSD on consumer systems. Still, are scripting languages easily accessible to beginners on those systems? OSX has made several scripting languages available (including Ruby and Python), but most users are unaware of their presence.</p>
<p>I should mention that Windows is equipped with its own scripting host for developing in JScript and VBScript. But the use of the scripting host is (I believe) under-documented and limited for beginners. Try doing something useful in a script without using Server.CreateObject. Let’s not let kids touch the COM objects, please!</p>
<h2>The Christmas List</h2>
<p>I’m thinking a toy language for consoles and desktops alike could be monumental. I’m ot saying it needs to be cross-platform. A language for GameCube that took advantage of platform-specific features could be more appealing to GameCube users than a language that used a reduced featureset, but could execute on a handheld. Really, we live in a world where both choices should be available.</p>
<p>As for essential features:</p>
<h3>1. Transportable code.</h3>
<p>On my TI-994A, I could make a little, animated Optimus Prime from pixels. Insert cassette. Record. Pass around to friends. Receive high fives from friends. Put on wraparound shades. Thank you, TI! Thank you, Optimus Prime!</p>
<p>A little language for the consoles could be wildly popular if combined with the good ature of sharing code. This could be done by trading memory cards, but would be more effective if code could be easily obtained and posted on the Web. Learning would accelerate and collaborative development could take place.</p>
<p>A suitable language should give coders access to I/O devices, to allow experimentation with network devices and the ability to enhance one’s connectivity with others. For the consoles, games could provide hooks for user mods. This has long proven a successful staple of the desktop gaming world.</p>
<h3>2. Simplicity.</h3>
<p>You’ve got to be able to write a single line of code and see a result. We need some instant results to give absolute beginners confidence. Simple methods for sending an e-mail, reading a web page, playing music. Demonstrable in a one- liner.</p>
<p>Admittedly, as our systems have grown complex, it is difficult to balance simplicity and capability. Most users will be unimpressed by code that emits beeps and bloops from a PlayStation 2. If Ruby were available on the PS2, then I would hope that I could hear rich symphonic sounds from a wee bit of code.</p>
<p><code>Orchestra.play( "A:2", "C:4", "E:1", "G:1" )</code></p>
<p>Access to the graphic engine might require more complex code. But simple drawing methods could be provided for beginners. Or images could be stored alongside code and accessed programmatically.</p>
<p><code>ImageLibrary.load( "GolfingOldMan" ).drawAt( 12, 10 )</code></p>
<p>The trick would be to uncover what small applications might entice novices and still provide the ability to write large applications that would drive developers to master the language and not limit their growth.</p>
<h3>3. Sensible environment.</h3>
<p>Considering that many won’t want to purchase a keyboard for their gaming unit, let’s make sure that a reasonable environment is provided for entry of text. Controllers could be worked like the <a href="http://www.handykey.com/">Twiddler</a>. Or code could be transferred via IR, TCP/IP. (Dare I say cassette? :D)</p>
<h3>4. Give it away!</h3>
<p>It used to be that programming was practically an inalienable right for users. Include a language with the system, situated in a friendly spot. Each of the game consoles I’ve mentioned has launchers. (With the exception of Game Boy and its successors.) Provide a development prompt from the launcher. From desktop software, provide shortcuts for both the command prompt and a development prompt.</p>
<p>Remember, we’re looking for a language that requires no system hacks. No obscure links. No warranty violation. We’ve become so used to these techniques that it seems to be an essential part of getting our way.</p>
<p>And in many ways it is essential. Tinkering with hardware is learning. Lobotomizing and renovating is meaningful, magical. On behalf of those who prefer to code, I make these wishes. Not to take away jobs from the Phillips screwdriver.</p>
<h2>The Ultimatum</h2>
<p>My challenge is to Sony, Nintendo, Microsoft, Apple, and to those who manufacture and develop our interactive technology. Let us interact with these machines more deeply. Provide us a channel for having a dialogue with the entertainment boxes we nurture and care for. I swear to you, the relationship between the public and your product will assuredly blossom. That box will become more of a chest for our personal works.</p>
<p>In addition, if your developers start putting out crap, then you have a whole world of people to pick up the slack.</p>
<p>My challenge is for you to bundle a useful programming language with your product. Ruby, Squeak, REBOL, Python. Take your pick. It will be inexpensive to add any of these languages to your systems. And people will seriously pray to you. You know how geeks get when they pledge allegiance to something. But, yes, Ruby is preferable.</p></content:encoded>
</item>
<item>
<title>About Crows</title>
<link>https://steveklabnik.com/writing/about-crows/</link>
<guid isPermaLink="true">https://steveklabnik.com/writing/about-crows/</guid>
<pubDate>Wed, 02 Dec 2009 00:00:00 GMT</pubDate>
<content:encoded><p>A short time ago, the G20 came right here to Pittsburgh. With an event of such worldwide importance happening in our backyards, several people decided to document it via social media. The Mattress Factory did a site called “MyG20” and it did pretty well. Some smaller groups did their own thing. But IndyMedia did one better: <a href="http://indypgh.org/g20/">http://indypgh.org/g20/</a></p>
<p>The <a href="http://thecaucus.blogs.nytimes.com/2009/09/24/getting-out-the-anti-globalization-message/">New York Times</a> called it “One of the best all-purpose sites with updates from the streets.” It got quite a lot of attention and hits from all over. And it was put together by a good friend of mine, <a href="http://www.twitter.com/edfilo">@edfilo</a> I helped a teeny tiny bit, but he (with the help of LibSyn) really did an awesome job.</p>
<p>So why should this matter to you? The G20 is old news by now.</p>
<p>He’s releasing the code for the site, making it available to anyone who wants to cover events. It’s called “Crows,” and it lives at <a href="http://crowsne.st/">http://crowsne.st/</a> . With Crows, you can quickly and easily build a site, pulling in content from Flickr, Twitter, and elsewhere, and posting it all on a geotagged Google Map.</p>
<p>It’s still in its rough infancy, but after the awesome role that it played during the G20, several other organizations had contacted IndyMedia about getting access to the code, and so it’s already gaining traction in the space. It’s undergoing rapid development, with new features being added quite fast. Ed really wants to make this useful for people, and I’m going to be contributing to the project in what little spare time I can as well. If you’re a coder too, you can <a href="http://github.com/edfilo/crows">fork crows on GitHub</a> to contribute back to the project.</p>
<p>Check it out, and you can follow <a href="http://twitter.com/crowsource">@crowsource</a> on Twitter for updates, too.</p></content:encoded>
</item>
</channel>
</rss>
Raw text
<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Steve Klabnik</title><description>Steve Klabniks blog</description><link>https://steveklabnik.com/</link><item><title>Does unsafe undermine Rust's guarantees?</title><link>https://steveklabnik.com/writing/does-unsafe-undermine-rusts-guarantees/</link><guid isPermaLink="true">https://steveklabnik.com/writing/does-unsafe-undermine-rusts-guarantees/</guid><pubDate>Mon, 17 Mar 2025 00:00:00 GMT</pubDate><content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>When people first hear about <code>unsafe</code> in Rust, they often have questions. A very
normal thing to ask is, "wait a minute, doesn't this defeat the purpose?" And while
it's a perfectly reasonable question, the answer is both straightforward and has a
lot of nuance. So let's talk about it.</p>
<p>(The straightforward answer is "no", by the way.)</p>
<h2>Memory safe languages and their implementations</h2>
<p>The first way to think about this sort of thing is to remember that programming
languages and their implementations are two different things. This is sometimes
difficult to remember, since many programming languages only have one
implementation, but this general principle applies even in those cases. That is,
there are the desired semantics of the language you are implementing, and then
the codebase that makes programs in that language do those things.</p>
<p>Let's examine a "real" programming language to talk about this: <a href="https://en.wikipedia.org/wiki/Brainfuck">Brainfuck</a>.
Brainfuck programs are made up of eight different characters, each one performing
one operation. Brainfuck programs can only do those eight things, and nothing
more.</p>
<p>However, this simplicity leads to two interesting observations about the
differences between languages in the abstract and their implementations.</p>
<h3>Unsafety in implementation</h3>
<p>The first is that properties of the language itself are distinct from properties
of an implementation of the language. The semantics of the Brainfuck language
don't provide an opportunity to call some sort of re-usable function
abstraction, for example. Yet, when we write a Brainfuck interpreter or
compiler, we can use a programming language that supports functions. That
doesn't change that Brainfuck programs themselves can't define or call
functions.</p>
<p>And in fact, we often want to implement a language in another language that has
more abilities than the language we're implementing. For example, while it's
possible to implement Brainfuck in Brainfuck, it's much easier to do so in a
language that lets you define and call functions. Another classic example is
interacting with hardware. Brainfuck has the <code>.</code> instruction that allows you to
produce output. But in order to actually produce output to say, a terminal, you
interact with your platform's API to do so. But inside of that API, eventually
you'll hit a layer where there are no abstractions: you need to talk to hardware
directly. Doing so is not statically able to be guaranteed to be memory safe,
because the hardware/software interface often boils down to "put some
information at this arbitrary memory location and the hardware will take it from
there." Our Brainfuck program is operating in an environment just like any
other; at the end of the day it has to coordinate with the underlying operating
system or hardware at some point. But just because our implementation must do
so, doesn't mean that our Brainfuck programs do so directly. The semantics of
programs written in Brainfuck aren't changed by the fact that the implementation
can (and must) do things that are outside of those semantics.</p>
<p>Let's take Ruby as a more realistic example than Brainfuck. Ruby does not let us modify
arbitrary memory addresses from within Ruby itself. This means that pure Ruby
programs should not be able to produce a segmentation fault. But in the real
world, Ruby programs can segfault. This is possible if there's a bug in the Ruby
interpreter's code. Sure, it's our Ruby code that's making the call to a
function that results in a segfault, but the real fault lies with the code
outside of Ruby's purview.</p>
<p>That this can happen doesn't mean that Ruby's guarantees around memory
manipulation is somehow useless, or suffers segmentation faults at the same rate
as programs in a language that allow you to manipulate arbitrary memory as a
matter of course. But it does also mean that we don't need to look at our entire
program to figure out where the problem comes from: it instead comes from our
non-Ruby code. Ruby's guarantees have helped us eliminate a lot of suspects when
figuring out whodunit.</p>
<h3>Extending languages and unsafety</h3>
<p>The second property is that certain implementations may extend the language in
arbitrary ways. For example, I could write a Brainfuck interpreter and say that
I support a ninth instruction, <code>@</code>, that terminates the program when invoked.
This would be a non-portable extension, and Brainfuck programs written in my
variant wouldn't work in other interpreters, but sometimes, this technique is
useful.</p>
<p>Many languages find that these sorts of extensions are useful, and so offer a
feature called a "foreign function interface" that allows for your program to
invoke code in a different language. This provides the ability to do things that
are outside of the domain of the language itself, which can be very useful.
Brainfuck does not offer an FFI, but if it did, you could imagine that <code>@</code> could
be implemented in terms of it, making programs that use <code>@</code> portable again, as
long as the extended functionality was included somehow, often as a library of
some kind.</p>
<p>Just like our implementation has the ability to do things outside of our
languages' semantics, FFI and similar extension mechanisms also give us the
ability to do arbitrary things. I can write an extension for Ruby that writes to
arbitrary memory. And I can cause a segfault. But we're in the same place that
we were with our implementation issues; we know that if we get a segfault, the
blame lies not with our Ruby code, but instead, with either the implementation
or our FFI.</p>
<h3>So how is this memory safe?</h3>
<p>It may seem contradictory that we can call a language "memory safe" if
real-world programs have the ability to cause memory problems. But the thing is,
it isn't <em>really</em> programs in that language that caused the issue: it was FFI,
which is in a different language entirely, or it was implementation issues, and
implementations must do memory unsafe things, thanks to a need to interact with
the OS or hardware. And so the definition of "memory safe language" is commonly
understood to refer to languages and their implementations in the absence of
either implementation bugs or FFI bugs. In practice, these bugs occur at such a
low rate compared to languages that are clearly not memory safe that this
practical definition serves a good purpose, even if you may feel a bit
uncomfortable with these "exceptions."</p>
<p>But there's actually a deeper reason why these exceptions are acceptable, and
that's due to how we understand properties of programs and programming languages
in the first place. That is, this isn't just a practical "well these exceptions
seem fine" sort of thing, they're actually okay on a deeper level.</p>
<h2>Programs, properties, and proofs</h2>
<p>So how do we know anything about how programs and programming languages work at
all?</p>
<blockquote>
<p>Computer science is no more about computers than astronomy is about
telescopes.</p>
<ul>
<li>Edsger Wybe Dijkstra</li>
</ul>
</blockquote>
<p>A fun thing about computer science is that it's closely related to other
disciplines, including math! And so there's a long history of using math
techniques to understand computers and their programs.</p>
<p>One technique of building knowledge is the idea of proofs. If you get a
degree in computer science, you'll engage with proofs quite a lot. I even
took a class on logic in the philosophy department as part of my degree.</p>
<p>I don't intend to give you a full introduction to the idea of proofs in this
blog post, but there's some high-level concepts that are useful to make sure
we're in the same page about before we go forward.</p>
<h3>Aristotelian syllogisms</h3>
<p>Here is a very classic example of a form of reasoning called a "syllogism",
given by Aristotle in 350 BCE:</p>
<ol>
<li>All men are mortal.</li>
<li>Socrates is a man.</li>
<li>Therefore, Socrates is mortal.</li>
</ol>
<p>These first two lines are called "propositions," and the third is a conclusion.
We can base our conclusion on a logical relationship between the information given
to us by the propositions.</p>
<p>But how do we know propositions are true? Are all men mortal? What's important
here is that we, for the purposes of our proof, assume that propositions are
true. And we do this because, on some level, we cannot know everything. And so
to begin, we have to start somewhere, and make some assumptions about what we
know. It's true that later, we may discover a fact that disproves our
proposition, and now our proof no longer works. But that's just the way that
the world works. It doesn't prevent these sorts of proofs from being useful to
help us gain knowledge about the world and how it works, as best as we can tell
at the current time.</p>
<p>So on some level, this is also why Ruby is a memory safe language even though a
C extension can segfault: there's always some kind of things that we have to
assume are true. Memory safe languages are ones where the amount of code we have
to assume is memory safe, rather than is guaranteed to be by the semantics, is
small, and preferably indicated in the code itself. Put another way, the amount
of code we need to trust is memory safe is large in a memory unsafe language,
and small in a memory safe language, rather than zero.</p>
<h3>Logic and programs</h3>
<p>As time went on, so did our understanding of logic. And, of course, we even have
competing logics! And then this gets fun, because terms can mean something
slightly different. For example, in more recent logics, we'd call something like
"All men are mortal" to be an <em>axiom</em>, rather than a proposition. Same idea:
it's something that we accept without proof.</p>
<p>As computers appeared, people sought to apply the rules for mathematical logic
onto them. We even call circuits that carry out classic logical operations
"logic gates." Number theory and logic were foundational to making computers
work. And so, once high level languages appeared on the scene, there was
interest in applying these mathematical tools to understanding programs as well.
This discipline is called "formal verification," and the general idea is to
describe various properties we wish a system to have, and then use formal
methods from mathematics to demonstrate that this is true.</p>
<p>This area of study is very deep, and I don't plan to cover the vast majority of
it here. However, I do want to pursue one particular thread in this area.</p>
<h3>Hoare Logic</h3>
<p><a href="https://dl.acm.org/doi/10.1145/363235.363259">"Hoare Logic"</a> is, well:</p>
<blockquote>
<p>In this paper an attempt is made to explore the logical foundations of
computer programming by use of techniques which were first applied in the
study of geometry and have later been extended to other branches of
mathematics. This involves the elucidation of sets of axioms and rules of
inference which can be used in proofs of the properties of computer programs.
Examples are given of such axioms and rules, and a formal proof of a simple
theorem is displayed. Finally, it is argued that important advantage, both
theoretical and practical, may follow from a pursuance of these topics.</p>
</blockquote>
<p>Incidentally, C. A. R. Hoare, the author of this paper, and Graydon Hoare, the
creator of Rust, are unrelated.</p>
<p>How does Hoare logic work? Yet again, not going to cover all of it, but the
general idea is this: In order to figure out if a program does what it is
supposed to do, we need to be able to reason about the state of the program
after execution happens. And so we need to be able to describe the state before,
and its relationship to the state after. And so you get this notation:</p>
<pre><code>P { Q } R
</code></pre>
<p>P is a precondition, Q is a program, and R is the result of the program
executing with those preconditions.</p>
<p>But most programs have more than one statement. So how could we model this?
Hoare gives us the Rule of Composition:</p>
<p>If <code>P { Q1 } R1</code> and <code>R1 { Q2 } R</code> then <code>P { Q1; Q2 } R</code>.</p>
<p>This allows us to build up a program by proving each statement in turn.</p>
<p>Hoare logic is very neat, and I've only scratched the surface here. People did a
lot of work to extend Hoare logic to include more and more aspects of programs.
But then, something else happened.</p>
<h3>Separation Logic</h3>
<p>In 2002, <a href="https://www.cs.cmu.edu/~jcr/seplogic.pdf">Separation Logic: A Logic for Shared Mutable Data Structures</a>
was published.</p>
<blockquote>
<p>In joint work with Peter O'Hearn and others, based on early ideas of Burstall,
we have developed an extension of Hoare logic that permits reasoning about
low-level imperative programs that use shared mutable data structure.</p>
</blockquote>
<p>Hmm, shared mutable data structures? Where have I heard that before...</p>
<p>Let's see what they have to say:</p>
<blockquote>
<p>The problem faced by these approaches is that the correctness of a program
that mutates data structures usually depends upon complex restrictions on the
sharing in these structures.</p>
</blockquote>
<p>For sure. Well, what are we to do about this?</p>
<blockquote>
<p>The key to avoiding this difficulty is to introduce a novel
logical operation <code>P * Q</code>, called separating conjunction (or
sometimes independent or spatial conjunction), that asserts
that P and Q hold for disjoint portions of the addressable
storage.</p>
</blockquote>
<p>What disjoint portions of addressable storage might we care about?</p>
<blockquote>
<p>Our intent is to capture the low-level character of machine language. One can
think of the store as describing the contents of registers, and the heap as
describing the contents of an addressable memory.</p>
</blockquote>
<p>Pretty useful!</p>
<p>Before we talk a bit about how separation logic works, consider this paragraph
on why it's named as such:</p>
<blockquote>
<p>Since these logics are based on the idea that the structure of an assertion
can describe the separation of storage into disjoint components,we have come
to use the term separation logics, both for the extension of predicate
calculus with the separation operators and the resulting extension of Hoare
logic. A more precise name might be storage separation logics, since it is
becoming apparent that the underlying idea can be generalized to describe the
separation of other kinds of resources.</p>
</blockquote>
<p>The plot thickens.</p>
<p>Anyway, in Separation Logic, we use slightly different notation than Hoare Logic:</p>
<pre><code>{ P } C { Q }
</code></pre>
<p>This says that we start with the precondition <code>P</code>, and if the program <code>C</code>
executes, it will not have undefined behavior, and if it terminates, <code>Q</code> will
hold.</p>
<p>Furthermore, there is the "frame rule", which I am going to butcher the notation
for because I haven't bothered to install something to render math correctly
just for this post:</p>
<p>If <code>{ p } c { q }</code></p>
<p>then <code>{ p * r } c { q * r }</code></p>
<p>where no free variable in <code>r</code> is modified by <code>c</code>.</p>
<p>Why is it interesting to add something to both sides of an equation, in a sense?
Well what this gives us the ability to do is add any predicates about parts of
the program that <code>c</code> doesn't modify or mutate. You might think of <code>&amp;mut T</code> or
even just ownership in general: we can reason about just these individual parts
of a program, separately from the rest of it. In other words, we have some
foundational ideas for ownership and even bits of borrowing, and while this original
paper doesn't involve concurrency, eventually Concurrent Separation Logic would become
a thing as well.</p>
<p>I think the paper explains why the frame rule matters better than I can:</p>
<blockquote>
<p>Every valid specification <code>{p} c {q}</code> is "tight" in the sense that every cell
in its footprint must either be allocated by <code>c</code> or asserted to be active by
<code>p</code>; "locality" is the opposite property that everything asserted to be active
belongs to the footprint. The role of the frame rule is to infer from a local
specification of a command the more global specification appropriate to the
larger footprint of an enclosing command.</p>
</blockquote>
<p>What this gives us is something called "local reasoning," and local reasoning
is <em>awesome</em>. Before I talk about that, I want to leave you with one other
very interesting paragraph:</p>
<blockquote>
<p>Since our logic permits programs to use unrestricted address arithmetic,there
is little hope of constructing any general-purpose garbage collector. On the
other hand, the situation for the older logic, in which addresses are disjoint
from integers, is more hopeful. However, it is clear that this logic permits
one to make assertions, such as “The heap contains two elements” that might be
falsified by the execution of a garbage collector, even though, in any
realistic sense, such an execution is unobservable.</p>
</blockquote>
<p>Anyway. Let's talk about local vs global analysis.</p>
<h2>Global vs local analysis</h2>
<p>Proving things about programs isn't easy. But one thing that can make it even
harder is that, for many properties of many programs you'd want to analyze, you
need to do a <em>global analysis</em>. As an example, let's use Ruby. Ruby is an
incredibly dynamic programming language, which makes it fairly resistant to
static analysis.</p>
<h3>Global analysis</h3>
<p>Here is a Ruby program. Do you know if this program executes successfully?</p>
<pre><code>class Foo
def self.bar
"baz"
end
end
p Foo.bar
</code></pre>
<p>Yeah, it prints <code>baz</code>. But what about this Ruby program?</p>
<pre><code>class Foo
def self.bar
"baz"
end
end
require "foo_ext.rb"
p Foo.bar
</code></pre>
<p>We can't know. <code>foo_ext.rb</code> may contain no relevant code, but it also might
include something like this:</p>
<pre><code>Foo.class_eval { undef :bar }
</code></pre>
<p>In which case, when we try and call <code>Foo.bar</code>, it no longer exists:</p>
<pre><code>&lt;anonymous&gt;': eval:13:in `block in &lt;main&gt;': undefined method `bar' for class `Foo' (NameError)
eval:13:in `class_eval'
eval:13:in `&lt;main&gt;'
-e:in `eval'
</code></pre>
<p>Ouch. So in this case, we're going to need the whole code of our program in
order to be able to figure out what's going on here.</p>
<p>Incidentally, <a href="https://sorbet.org/">Sorbet</a> is a very cool project to add type checking to
Ruby. They do require access to the entire Ruby source code in order to do their
analysis. But they also made some decisions to help make it more tractable; if
you try Sorbet out on the web, the type checker is <em>fast</em>. What happens when you
try the above code with Sorbet?</p>
<pre><code>editor.rb:5: Unsupported method: undef https://srb.help/3008
5 |Foo.class_eval { undef :bar }
^^^^^^^^^^
</code></pre>
<p>This is a very fair tradeoff! It's very common with various forms of analysis to
choose certain restrictions in order to make what they want to do tractable. Ruby
has so much evil in it, just not supporting some of the more obscure things is
completely fair, in my opinion.</p>
<h3>Local analysis</h3>
<p>The opposite of global analysis is, well, local analysis. Let's consider a Rust
equivalent of our Ruby:</p>
<pre><code>struct Foo;
impl Foo {
fn bar() -&gt; String {
String::from("baz")
}
}
fn main() {
dbg!(Foo::bar());
}
</code></pre>
<p>Can we know if this program works? Sure, everything is here. Now, if we tried the
same trick as the Ruby code, we know that it would work, because Rust doesn't have
the ability to remove the definition of <code>bar</code> like Ruby does. So let's try something
else:</p>
<pre><code>struct Foo;
impl Foo {
fn bar() -&gt; String {
// body hidden
}
}
fn main() {
dbg!(Foo::bar());
}
</code></pre>
<p>Can we know if <code>main</code> is properly typed here? We can, even though we know
nothing about the body. That's because <code>dbg!</code> takes any value that implements
the <code>Debug</code> trait, and we know that <code>Foo::bar()</code> returns <code>String</code>, which
implements <code>Debug</code>. Typechecking <code>main</code> is local to main's implementation, we
don't need to look into the bodies of any function it calls to determine if it's
well-typed. We only need to know their signatures. If we didn't require <code>bar</code> to
have a type signature, we'd have to peek into its body to figure out what it
returns, and so on for any function that <code>bar</code> calls in its body. Instead of
this arbitrarily deep process, we can just look at the signature and be done
with it.</p>
<h2>Composability of proofs</h2>
<p>So why is local analysis so helpful? The first reason is either speed or
scalability, depending on how you want to look at it. If you are running global
analysis checks, they will get more expensive the larger your codebase, since
they will require checking the entire thing to work. Whereas local analysis only
requires a local context, it doesn't get any more expensive when you add more
code to your project, only when you change that local context. So when you're
trying to scale checks up to larger projects, local analysis is crucial.</p>
<p>But I also like to personally think about it as a sort of abstraction. That is,
global analysis is a leaky abstraction: changes in one part of the codebase can
cascade into other parts. Remember this line about the frame rule?</p>
<blockquote>
<p>The role of the frame rule is to infer from a local specification of a command
the more global specification appropriate to the larger footprint of an
enclosing command.</p>
</blockquote>
<p>If we have local reasoning, we can be sure that changes locally don't break out
of the boundaries of those changes. As long as the types of our function don't
change, we can mess with the body as much as we want, and we know that the rest
of the analysis of the program is still intact. This is really useful, in the
same way that abstractions are useful when building programs more generally.</p>
<h2>Rustbelt</h2>
<p>Okay, we have taken a real deep dive here. What's this all have to do with
unsafe Rust?</p>
<p>Well, seven years ago, <a href="https://plv.mpi-sws.org/rustbelt/popl18/paper.pdf">RustBelt: Securing the Foundations of the Rust
Programming Language</a> by Ralf Jung, Jaques-Henri Jourdan, Robbert
Krebbers, and Derek Dreyer, was published:</p>
<blockquote>
<p>In this paper, we give the first formal (and machine-checked) safety proof for
a language representing a realistic subset of Rust.</p>
</blockquote>
<p>But it's more exciting than that:</p>
<blockquote>
<p>Our proof is extensible in the sense that, for each new Rust library that uses
unsafe features, we can say what verification condition it must satisfy in
order for it to be deemed a safe extension to the language.</p>
</blockquote>
<p>Very good!</p>
<p>The paper begins by talking about why verifying Rust is such a challenge:</p>
<blockquote>
<p>Consequently, to overcome this restriction, the implementations of Rust's
standard libraries make widespread use of unsafe operations, such as "raw
pointer" manipulations for which aliasing is not tracked. The developers of
these libraries claim that their uses of unsafe code have been properly
"encapsulated", meaning that if programmers make use of the APIs exported by
these libraries but otherwise avoid the use of unsafe operations themselves,
then their programs should never exhibit any unsafe/undefined behaviors. In
effect, these libraries extend the expressive power of Rust's type system by
loosening its ownership discipline on aliased mutable state in a modular,
controlled fashion</p>
</blockquote>
<p>This is basically the question asked at the start of this post: can you honestly
say that a function is safe if it contains <code>unsafe</code> code as part of its
implementation?</p>
<p>They go on to describe the first challenge for the project: choosing the correct
logic to model Rust in (some information cut for clarity):</p>
<blockquote>
<p>Iris is a language-generic framework for higher-order concurrent separation
logic which in the past year has been equipped with tactical support for
conducting machine-checked proofs of programs in Coq. By virtue of being a
separation logic, Iris comes with built-in support for reasoning modularly
about ownership. Moreover, the main selling point of Iris is its support for
deriving custom program logics for different domains using only a small set of
primitive mechanisms. In the case of RustBelt, we used Iris to derive a novel
lifetime logic, whose primary feature is a notion of borrow propositions that
mirrors the “borrowing” mechanism for tracking aliasing in Rust.</p>
</blockquote>
<p>Separation logic!</p>
<p>So how does this work? There are three parts to the proof. I'm going to
summarize them because I'm already quoting extensively, please go read the paper
if you want the exact details, it is well written.</p>
<ol>
<li>Verify that the typing rules are sound when interpreted semantically.</li>
<li>Verify that if a program is well typed semantically, it does not exhibit
unsafe behavior.</li>
<li>For any library that uses unsafe, check that it satisfies the semantics that
its interface requires.</li>
</ol>
<p>If all three of these things are true, then the program is safe to execute. This
is also exciting because, thanks to 3, when someone writes new unsafe code, they
can get RustBelt to let them know which properties they need to satisfy to make
sure that their unsafe code doesn't cause problems.</p>
<h3>Weaknesses</h3>
<p>Of course, there is more to do following up on RustBelt:</p>
<blockquote>
<p>We do not model (1) more relaxed forms of atomic accesses, which Rust uses for
efficiency in libraries like Arc; (2) Rust's trait objects (comparable to
interfaces in Java), which can pose safety issues due to their interactions
with lifetimes; or (3) stack unwinding when a panic occurs, which causes
issues similar to exception safety in C++.</p>
</blockquote>
<p>However, the results were good enough to find some soundness bugs in the
standard library, and if I remember correctly, widen one of the APIs in a sound
manner as well. Another interesting thing about RustBelt is that it was
published before Rust gained Non-lexical lifetimes, however, it modeled them,
since folks knew they were coming down the pipeline.</p>
<h2>The theoretical vs the empirical</h2>
<p>So, we have some amount of actual proof that unsafe code in Rust doesn't
undermine Rust's guarantees: what it does is allow us to extend Rust's
semantics, just like FFI would allow us to extend a program written in
a language that supports it. As long as our unsafe code is semantically
correct, then we're all good.</p>
<p>... but what if it's not semantically correct?</p>
<h3>When things go wrong</h3>
<p>At some point maybe I'll write a post about the specifics here, but this post is
already incredibly long. I'll cut to the chase: you can get undefined behavior,
which means anything can happen. You don't have a real Rust program. It's bad.</p>
<p>But at least it's scoped to some degree. However, many people get this scope a
bit wrong. They'll say something like:</p>
<blockquote>
<p>You only need to verify the unsafe blocks.</p>
</blockquote>
<p>This is true, but also a bit misleading. Before we get into the unsafe stuff, I
want you to consider an example. Is this Rust code okay?</p>
<pre><code>struct Foo {
bar: usize,
}
impl Foo {
fn set_bar(&amp;mut self, bar: usize) {
self.bar = bar;
}
}
</code></pre>
<p>No unsafe shenanigans here. This code is perfectly safe, if a bit useless.</p>
<p>Let's talk about unsafe. The canonical example of unsafe code being affected
outside of unsafe itself is the implementation of <code>Vec&lt;T&gt;</code>. Vecs look something
like this (the real code is different for reasons that don't really matter in
this context):</p>
<pre><code>struct Vec&lt;T&gt; {
ptr: *mut T,
len: usize,
cap: usize,
}
</code></pre>
<p>The pointer is to a bunch of <code>T</code>s in a row, the length is the current number of <code>T</code>s
that are valid, and the capacity is the total number of <code>T</code>s. The length and the
capacity are different so that memory allocation is amortized; the capacity is
always greater than or equal to the length.</p>
<p>That property is very important! If the length is greater than the capacity,
when we try and index into the Vec, we'd be accessing random memory.</p>
<p>So now, this function, which is the same as <code>Foo::set_bar</code>, is no longer okay:</p>
<pre><code>impl&lt;T&gt; Vec&lt;T&gt; {
fn set_len(&amp;mut self, len: usize) {
self.len = len;
}
}
</code></pre>
<p>This is because the unsafe code inside of other methods of <code>Vec&lt;T&gt;</code> need to be
able to rely on the fact that <code>len &lt;= capacity</code>. And so you'll find that
<code>Vec&lt;T&gt;::set_len</code> in Rust is marked as unsafe, even though it doesn't contain
unsafe code.</p>
<p>And this is why the module being the privacy boundary matters: the only way to
set len directly in safe Rust code is code within the same privacy boundary as
the <code>Vec&lt;T&gt;</code> itself. And so, that's the same module, or its children.</p>
<p>This is still ultimately better than any line of code in the whole codebase, but
it's not quite as small as you may think at first.</p>
<h3>How can we check that things haven't gone wrong?</h3>
<p>Okay, so while RustBelt could give you some idea if your code is correct, I doubt
you're about to jump into Coq and write some proofs. What can you do? Well,
Rust provides a tool, <code>miri</code>, that can interpret your Rust code and let you know
if you've violated some unsafe rules. But it's not complete, that is, miri can
tell you if your code is wrong, but it cannot tell you if your code is right. It's
still quite useful.</p>
<p>To get miri, you can install it with Rust nightly:</p>
<pre><code>$ rustup +nightly component add miri
</code></pre>
<p>And then run your test suite under miri with</p>
<pre><code>$ cargo +nightly miri test
</code></pre>
<p>For more details, consult <a href="https://github.com/rust-lang/miri">miri's documentation</a>.</p>
<h3>Do things go wrong?</h3>
<p>Do people write good unsafe code, or bad?</p>
<p>Well, the obvious answer to the above question is "yes." Anyone who says all
unsafe code in Rust is sound is obviously lying. But the absolute isn't what's
interesting here, it's the shade of grey. Is unsafe pervasive? Does Rust have
memory related bugs at the same rate as other MSLs, or closer to memory-unsafe
languages?</p>
<p>It's still sort of early days here, but we do have some preliminary results that
are helpful. In 2022, <a href="https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html">Google reported</a> that:</p>
<blockquote>
<p>To date, there have been zero memory safety vulnerabilities discovered in
Android's Rust code.</p>
</blockquote>
<p>I am very interested to see a more up to date report about how that's changed,
but as they say:</p>
<blockquote>
<p>We don't expect that number to stay zero forever, but given the volume of new
Rust code across two Android releases, and the security-sensitive components
where it's being used, it's a significant result. It demonstrates that Rust is
fulfilling its intended purpose of preventing Android's most common source of
vulnerabilities. Historical vulnerability density is greater than 1/kLOC (1
vulnerability per thousand lines of code) in many of Android's C/C++ components
(e.g. media, Bluetooth, NFC, etc). Based on this historical vulnerability
density, it's likely that using Rust has already prevented hundreds of
vulnerabilities from reaching production.</p>
</blockquote>
<p>We'll see if these results replicate themselves, in the future, and also outside
of Google. But it at least looks like in practice, most unsafe code has not led
to problems. At least so far!</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3lkqjlfxypk2d" /&gt;</p>
</content:encoded></item><item><title>Choosing Languages</title><link>https://steveklabnik.com/writing/choosing-languages/</link><guid isPermaLink="true">https://steveklabnik.com/writing/choosing-languages/</guid><pubDate>Thu, 13 Mar 2025 00:00:00 GMT</pubDate><content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>I tried to write this post yesterday, but I didn't like my approach there. Last
night, I had a conversation with one of my best friends about it, and he
encouraged me to write this, but in a different way. I think this way is better,
so thanks, dude.</p>
<hr />
<p>The other day, Microsoft announced <a href="https://devblogs.microsoft.com/typescript/typescript-native-port/">that they are re-writing the TypeScript
compiler in Go</a>. A lot of people had a lot of feelings about this. I did
too, but in a different way than many other people.</p>
<p>The biggest questions people had were "Why not C#?" and "Why not Rust?". To
be clear, I do think that asking why someone chooses a programming language
can be valuable; as professionals, we need to make these sorts of decisions
on a regular basis, and seeing how others make those decisions can be useful,
to get an idea of how other people think about these things.</p>
<p>But it can also... I dunno. Sometimes, I think people have little imagination
when it comes to choosing a particular technology stack.</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3lk6p4clry22p" /&gt;</p>
<p>It is true that sometimes, there are hard requirements that means that choosing
a particular technology is inappropriate. But I've found that often, these
sorts of requirements are often more <em>contextually</em> requirements than ones that
would be the case in the abstract. For example, <a href="https://en.wikipedia.org/wiki/Lisp_machine">you can write an OS in
Lisp</a> if you want. But in practice, you're probably not working with
this hardware, or willing to port the environment to bare metal.</p>
<p>But it's also the case that you often don't know what others' contextual
situation actually is. I would have never guessed the main reason that Microsoft
chose Go for, which is roughly "we are porting the existing codebase, rather
than re-writing it, and our existing code looks kinda like Go, making it easy to
port." That's a very good reason! I hadn't really thought about the differences
between porting and re-writing in this light before, and it makes perfect sense
to me.</p>
<p>At the same time,</p>
<p>&lt;BlueskyPost uri="at://did:plc:4eukmtg5kmyjmp6qw3xkpite/app.bsky.feed.post/3lk6t4a4xa22w" /&gt;</p>
<p>people are going <em>way</em> overboard here.</p>
<p>&lt;BlueskyPost uri="at://did:plc:x2p6i7jmdhk4xj2ej3rflp7a/app.bsky.feed.post/3lk6was5n5227" /&gt;</p>
<p>There's some sort of balance here to be struck. But at the same time, I hate
"choose the right tool for the job" as a suggestion. Other than art projects,
has anyone ever deliberately chosen the wrong tool? Often, something that
seems like the wrong tool is chosen simply because the contextual requirements
weren't obvious to you, the interlocutor.</p>
<p>I think there's two other reasons why this situation is making me feel
complicated things. The first is RIIR, and the second is a situation from a long
time ago that changed my life.</p>
<p>Many of you have probably heard of the "re-write it in Rust" meme. The idea is
that there's this plague of programmers who are always suggesting that every
single thing needs to be re-written in Rust. While I am sure this happens from
time to time, I have yet to see real evidence that it is actually widespread. I
almost even wrote a blog post about this, trying to actually quantify the effect
here. But on some level, perception is reality, and if people believe it's true,
it's real on some level, even if that's not actually the case. Regardless of the
truth of the number of people who suggest this, on every thread about Rust,
people end up complaining about this effect, and so it has an effect on the
overall discourse regardless. And while I don't believe that this effect is real
overall, it absolutely has happened in this specific situation. A bunch of
people have been very aggressively suggesting that this should have been in
Rust, and not Go, and I get where they're coming from, but also: really? Go is
an absolutely fine choice here. It's fine. Why do you all have to reinforce a
stereotype? It's frustrating.</p>
<p>The second one is something that's been on my mind for over a decade, but I've
only started talking about it in places other than conversations with close
friends. Long-time readers may remember <a href="/writing/node/">that one time where I was an
asshole</a>. I mean, I've been an asshole more than once in my life, but
this one was probably the time that affected me the most. The overall shape of
the situation is this:</p>
<ol>
<li>Someone wrote <code>grep</code>, but in Node.</li>
<li>I made fun of it on the internet, because why would someone write a CLI tool in Node?</li>
<li>The author found out that I did and felt bad about it.</li>
</ol>
<p>Looking back at my apology, I don't think it's necessarily the best one. It was
better than the first draft or two of what I wrote, but I've gotten better at
apologizing since then. Regardless, this incident changed the trajectory of my
life, and if you happen to read this, Heather, I am truly, unequivocally, sorry.</p>
<p>There was a couple of things that happened here. The first is just simply, in
2013, I did not understand that the things I said had meaning. I hate talking
about this because it makes me seem more important than I am, but it's also
important to acknowledge. I saw myself at the time as just Steve, some random
guy. If I say something on the internet, it's like I'm talking to a friend in
real life, my words are just random words and I'm human and whatever. It is what
it is.</p>
<p>But at that time in my life, that wasn't actually the case. I was on the Rails
team, I was speaking at conferences, and people were reading my blog and tweets.
I was an "influencer," for better or worse. But I hadn't really internalized
that change in my life yet. And so I didn't really understand that if I
criticized something, it was something thousands of people would see. To me, I'm
just Steve. This situation happened before we talked about "cancel culture" and
all of that kind of stuff, but when the mob came for me, I realized: they were
right, actually. I was being an asshole, and I should not have been. And I
resolved myself to not being an asshole like that ever again.</p>
<p>And to do that, I had to think long and hard about why I was like that. I love
programming languages! I love people writing programs that they find
interesting! I don't think that it's stupid to write a tool in a language that
I wouldn't expect it to be written in! So why did I feel the reason to make fun
of this in that situation?</p>
<p>The answer? A pretty classic situation: the people I was hanging out with were
affecting me, and in ways that, when I examined it, I didn't like very much.</p>
<p>You see, in the Rails world at the time, there was a huge <a href="https://blog.aurynn.com/2015/12/16-contempt-culture">contempt
culture</a> at the time, especially around JavaScript and Node. And, when
you're around people who talk shit all the time, you find yourself talking shit
too. And once I realized that, well, I wanted to stop it. But there was a big
problem: my entire professional, and most of my social, life was built around
Ruby and Rails. So if I wanted to escape that culture... what should I do?</p>
<p>If you clicked on the link to my apology above, you probably didn't make note of
the date: Jan 23 2013. Something had just happened to me, in December of 2012: I
had decided to check out this little programming language I had heard of called
Rust.</p>
<p>One of the things that struck me about the "Rust community" at the time, which
was like forty people in an IRC room, was how nice they were. When I had trouble
getting Hello World to compile, and I typed <code>/join #rust</code>, I fully expected an
RTFM and flames. But instead, I was welcomed with open arms. People were chill.
And it was really nice.</p>
<p>And so, a month and a day later, I had this realization about the direction my
life was heading, and how I wasn't really happy about it. And I thought about
how much I enjoyed the Rust stuff so far... and combined with a few other
factors, maybe of which I'll write about someday, this is when I decided that I
wanted to make Rust a thing, and dedicated myself to achieving that.</p>
<p>And part of that was being conscious about the culture, and how it developed,
and the ways that that would evolve over time. I didn't want Rust to end up the
way that Rails had ended up. And that meant a few things, but first of all,
truly internalizing that I had to change. And understanding that I had to be
intentional with what I did and said. And what that led to, among other things,
was a "we talk about Rust on its own merits, not by trash talking other languages"
culture. (That's also partially because of Nietzsche, but I'm already pretty far
afield of the topic for this post, maybe someday.)</p>
<p>Anyway, the Eternal September comes for us all. The "Rust Community," if it ever
were a coherent concept, doesn't really make sense any more. And it is by no
means perfect. I have a lot of criticism about how things turned out. But I do
think that it's not a coincidence that "Rust makes you trans," for example, is
a meme. I am deeply proud of what we all built.</p>
<p>So yeah, anyway: people choose programming languages for projects based on a
variety of reasons. And sure, there may be better or worse choices. But you
probably have a different context than someone else, and so when someone
makes a choice you don't agree with, there's no reason to be a jerk about it.</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3lkblqoytls22" /&gt;</p>
</content:encoded></item><item><title>A Happy Day for Rust</title><link>https://steveklabnik.com/writing/a-happy-day-for-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/a-happy-day-for-rust/</guid><pubDate>Wed, 05 Mar 2025 00:00:00 GMT</pubDate><content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>A while back, I wrote <a href="/writing/a-sad-day-for-rust/">A sad day for Rust</a> about the Actix-web unsafe
controversy. The short of it was, Actix-web was using some unsound unsafe code.
When this was discovered, people responded pretty harshly, and the maintainer
quit. Others picked up the work, but it still wasn't great:</p>
<blockquote>
<p>I’m not sure where we go from here, and I’m not sure what we could have done
to prevent this from happening. But I feel like this is a failure, and it’s set
Rust back a bit, and I’m just plain sad.</p>
</blockquote>
<p>While the Rust community surely isn't perfect, today I'd like to share with you
a different story: one where there was some controvery, but it was handled in
the complete opposite way.</p>
<p><a href="https://rustup.rs/">Rustup</a> is the official installer and toolchain manager for Rust. It
has a much smaller team than Rust itself does, and releases on a much slower
schedule.</p>
<p>Back in August of last year, it was decided to <a href="https://github.com/rust-lang/rustup/pull/3985">change some behavior</a>.
The details aren't super important, what matters is that it's a breaking change
to a critical component of the ecosystem. I'm not here to say if this change is
good or bad; they had good motivations, there's also good reasons to keep the
previous behavior, it's one of those classic "you can't make everyone happy"
sort of deals.</p>
<p>The team knew that they'd need to let people know that this had happened:</p>
<blockquote>
<p>Yes, you've understood it correctly, and this breakage is intentional: #3635 (comment)</p>
<p>Before the actual release, we'll definitely clearly communicate this change with
the community, and we'll use the beta phase to collect user feedback.</p>
</blockquote>
<p>And they did <a href="https://internals.rust-lang.org/t/seeking-beta-testers-for-rustup-v1-28-0/22060">make a post on internals</a> about the change, seeking
feedback. This also landed in a This Week in Rust, in my understanding.</p>
<p>The problem is, not everyone reads internals, and not everyone reads This Week
in Rust. And so yesterday, some folks woke up to a broken CI, leading to <a href="https://github.com/rust-lang/rustup/issues/4211">this
bug report</a> among other upset posts elsewhere on the internet.</p>
<p>I have been on the other side of this before. It's frustrating when you try and
communicate something, and people don't hear about it, and then they find out
later and get mad at you about it. I also remember the Actix debacle, and many
other controversies in the broader open source ecosystem.</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3ljiqodr7q22f" /&gt;</p>
<p>However, this time, things went differently. As you can see, the issue is very
respectful, and the comments are also generally respectful. The team responded
<a href="https://github.com/rust-lang/rustup/issues/4211#issuecomment-2695620307">graciously</a>, and decided to put in the work to <a href="https://blog.rust-lang.org/2025/03/04/Rustup-1.28.1.html">release a new
version</a> to restore the previous behavior.</p>
<p>I was kind of worried yesterday that this would end up being another Actix
situation. The Rust community has changed a lot over time, and while it's still
pretty good overall, sometimes things can go poorly, just like with any random
group of thousands of people on the internet. But today, I'm grateful that
feedback was given constructively, it was taken with the spirit that it was
intended, and everyone should end up happy.</p>
<p>If only every controversy went this way, maybe I wouldn't be wondering about
coyotes.</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3ljnctebzec25" /&gt;</p>
</content:encoded></item><item><title>When should I use String vs &str?</title><link>https://steveklabnik.com/writing/when-should-i-use-string-vs-str/</link><guid isPermaLink="true">https://steveklabnik.com/writing/when-should-i-use-string-vs-str/</guid><pubDate>Wed, 16 Oct 2024 00:00:00 GMT</pubDate><content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>Rust has two main string types: <code>String</code> and <code>&amp;str</code>. Sometimes, people argue
that these two types make Rust code difficult to write, because you have to
think about which one you should be using in a given situation. My experience
of writing Rust is that I don't really think about this very much, and this
post is about some rules of thumb that you can use to be like me.</p>
<h2>Level 1: Don't think about it at all</h2>
<p>The very first thing you can do is follow the simplest rule:</p>
<blockquote>
<p>Always use <code>String</code>, never use <code>&amp;str</code>.</p>
</blockquote>
<p>That looks like this:</p>
<pre><code>struct Person {
name: String,
}
fn first_word(words: String) -&gt; String {
words
.split_whitespace()
.next()
.expect("words should not be empty")
.to_string()
}
</code></pre>
<p>This style means you may need to add <code>.to_string()</code> or <code>.clone()</code>
for things to work sometimes:</p>
<pre><code>fn main() {
let sentence = "Hello, world!";
println!("{}", first_word(sentence.to_string()));
let owned = String::from("A string");
// if we don't clone here, we can't use owned the second time
println!("{}", first_word(owned.clone()));
println!("{}", first_word(owned));
}
</code></pre>
<p>But that's okay, the compiler will let you know when you need to:</p>
<pre><code>error[E0382]: use of moved value: `owned`
--&gt; src/main.rs:21:31
|
18 | let owned = String::from("A string");
| ----- move occurs because `owned` has type `String`, which does not implement the `Copy` trait
19 |
20 | println!("{}", first_word(owned));
| ----- value moved here
21 | println!("{}", first_word(owned));
| ^^^^^ value used here after move
|
note: consider changing this parameter type in function `first_word` to borrow instead if owning the value isn't necessary
--&gt; src/main.rs:5:22
|
5 | fn first_word(words: String) -&gt; String {
| ---------- ^^^^^^ this parameter takes ownership of the value
| |
| in this function
help: consider cloning the value if the performance cost is acceptable
|
20 | println!("{}", first_word(owned.clone()));
| ++++++++
</code></pre>
<p>Hey, compiler, that's a great idea. Let's move on to level 2.</p>
<h2>Level 2: prefer <code>&amp;str</code> for function parameters</h2>
<p>A rule that's a little better is this one:</p>
<blockquote>
<p>Always use <code>String</code> in structs, and for functions, use <code>&amp;str</code> for parameters
and <code>String</code> types for return values.</p>
</blockquote>
<p>This is what the compiler error for level 1 was suggesting we do instead of <code>.clone</code>.</p>
<p>That results in code that looks like this:</p>
<pre><code>struct Person {
name: String,
}
fn first_word(words: &amp;str) -&gt; String {
words
.split_whitespace()
.next()
.expect("words should not be empty")
.to_string()
}
fn main() {
let sentence = "Hello, world!";
println!("{}", first_word(sentence));
let owned = String::from("A string");
println!("{}", first_word(&amp;owned));
println!("{}", first_word(&amp;owned));
}
</code></pre>
<p>We're now doing much less copying. We do need to add a <code>&amp;</code> on
to <code>String</code> values that we wish to pass into <code>first_word</code>, but
that's not too bad, the compiler will help us when we forget:</p>
<pre><code>error[E0308]: mismatched types
--&gt; src/main.rs:20:31
|
20 | println!("{}", first_word(owned));
| ---------- ^^^^^ expected `&amp;str`, found `String`
| |
| arguments to this function are incorrect
|
note: function defined here
--&gt; src/main.rs:5:4
|
5 | fn first_word(words: &amp;str) -&gt; String {
| ^^^^^^^^^^ -----------
help: consider borrowing here
|
20 | println!("{}", first_word(&amp;owned));
| +
</code></pre>
<p>Following this rule will get you through 95% of situations successfully. Yes,
that number was found via the very scientific process of "I made it up, but it
feels correct after writing Rust for the last twelve years."</p>
<p>For 4% of that last 5%, we can go to level 3:</p>
<h2>Level 3: return <code>&amp;str</code> sometimes</h2>
<p>Here's a slightly more advanced rule for certain circumstances:</p>
<blockquote>
<p>Always use <code>String</code> in structs, and for functions, use <code>&amp;str</code> for parameters.
If the return type of your function is derived from an argument and isn’t
mutated by the body, return <code>&amp;str</code>. If you run into any trouble here, return
<code>String</code> instead.</p>
</blockquote>
<p>That would look like this:</p>
<pre><code>struct Person {
name: String,
}
// we're returning a substring of words, so &amp;str is appropriate
fn first_word(words: &amp;str) -&gt; &amp;str {
words
.split_whitespace()
.next()
.expect("words should not be empty")
}
fn main() {
let sentence = "Hello, world!";
println!("{}", first_word(sentence));
let owned = String::from("A string");
println!("{}", first_word(&amp;owned));
println!("{}", first_word(&amp;owned));
}
</code></pre>
<p>This lets us remove a copy, we no longer have a <code>.to_string</code> in the body of
<code>first_word</code>.</p>
<p>Sometimes, we can't do that though:</p>
<pre><code>// because we are going to uppercase the first word, our return type can't
// be &amp;str anymore, because we aren't actually returning a substring: we are
// creating our own new string.
fn first_word_uppercase(words: &amp;str) -&gt; String {
words
.split_whitespace()
.next()
.expect("words should not be empty")
.to_uppercase()
}
fn main() {
let sentence = "Hello, world!";
println!("{}", first_word_uppercase(sentence));
let owned = String::from("A string");
println!("{}", first_word_uppercase(&amp;owned));
println!("{}", first_word_uppercase(&amp;owned));
}
</code></pre>
<p>How do you know that this is the case? Well, in this specific case,
<code>to_uppercase</code> already returns a <code>String</code>. So that's a great hint.
If we tried to return a <code>&amp;str</code>, we'd get an error:</p>
<pre><code>// this can't work
fn first_word_uppercase(words: &amp;str) -&gt; &amp;str {
&amp;words
.split_whitespace()
.next()
.expect("words should not be empty")
.to_uppercase()
}
</code></pre>
<p>would give us</p>
<pre><code>error[E0515]: cannot return reference to temporary value
--&gt; src/main.rs:7:5
|
7 | &amp;words
| ______^-
| | ______|
8 | || .split_whitespace()
9 | || .next()
10 | || .expect("words should not be empty")
11 | || .to_uppercase()
| || ^
| ||_______________________|
| |_______________________returns a reference to data owned by the current function
| temporary value created here
</code></pre>
<p>And that's really it. Following this rule will get you through virtually every
scenario where you need to wonder about <code>String</code> and <code>&amp;str</code>. With some practice,
you'll internalize these rules, and when you feel comfortable with a level, you
can go up to the next one.</p>
<p>What about that last 1% though? Well, there is a next level...</p>
<h2>Level 4: When to use <code>&amp;str</code> in a struct</h2>
<p>Here's the rule for level 4:</p>
<blockquote>
<p>Should you use a <code>&amp;str</code> in a struct? If you're asking that question, use
<code>String</code>. When you need to use <code>&amp;str</code> in a struct, you'll know.</p>
</blockquote>
<p>Storing references in structs is useful, for sure, and it's good that Rust
supports it. But you're only going to need it in fairly specific scenarios,
and if you feel like you're worring about <code>String</code> vs <code>&amp;str</code>, you're just
not in the position to be worrying about the complexity of storing a <code>&amp;str</code>
in a struct yet.</p>
<p>In fact, some people believe in this rule so strongly that they're working
on a language where storing references in structs isn't even possible, and
it's a language I've found very interesting lately:
<a href="https://www.hylo-lang.org/">Hylo</a>. They go a bit farther than that, even:
in Hylo, you think of everything as being values, rather than having references
at all. They think that you can write meaningful programs with this model.
But this isn't a post about Hylo. I'll write one of those eventually. My point
is just that you can really, truly, get away with not storing <code>&amp;str</code>s in structs
for a <em>lot</em> of useful Rust programs. So it's not really worth spending mental
energy on, until you determine that you must do so. That is the case when
you've profiled your program and determined that copying strings to and from
your struct is a big enough issue to bother with lifetimes.</p>
<p>I hope you've found this helpful, and if you have any other rules of thumb
like this, I'd love to hear about them!</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3l6nkx54sij2w" /&gt;</p>
</content:encoded></item><item><title>Against Names</title><link>https://steveklabnik.com/writing/against-names/</link><guid isPermaLink="true">https://steveklabnik.com/writing/against-names/</guid><pubDate>Mon, 12 Aug 2024 00:00:00 GMT</pubDate><content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>There's an old saying:</p>
<blockquote>
<p>There are only two hard things in Computer Science: cache invalidation and
naming things.</p>
<p>― Phil Karlton</p>
</blockquote>
<p>I also appreciate the joke version that adds "and off by one errors."</p>
<p>Lately, I've been thinking about this saying, combined with another old joke:</p>
<blockquote>
<p>"The patient says, "Doctor, it hurts when I do this." The doctor says, "Then
don't do that!"</p>
<p>― Henny Youngman</p>
</blockquote>
<p>Specifically, if naming things is so hard... why do we insist on doing it all
the time?</p>
<p>Now, I am not actually claiming we should stop giving things names. But I have
had at least two situations recently where I previously found names to be kinda
critical, and then I changed to systems which didn't use names, and I think it
improved the situation.</p>
<p>One of the most famous examples of not giving something a name,
lambdas/closures, took some time to catch on. But many folks already recognize
that naming every single function isn't always neccesary. I wonder if there are
more circumstances where I've been naming things where I didn't actually have
to.</p>
<p>Anyway, here's my two recent examples:</p>
<h2>Nameless Branches</h2>
<p>I haven't written much about it on my blog yet, but I'm fully converted away
from git to <a href="https://github.com/martinvonz/jj">jj</a>. I'll say more about this in the future, but one major
difference between the two is that jj has anonymous branches. If, like me,
you are a huge fan of git, this sounds like a contradiction. After all, the
whole thing about branches are that they're a name for some point in the DAG.
How do you have a nameless name?</p>
<p>Here's some output from <code>jj log</code>:</p>
<p>Here's some sample output from jj log:</p>
<pre><code> $ jj log --limit 5
@ pzoqtwuv [email protected] 2024-03-01 15:06:59.000 -06:00 9353442b
│ added some cool new feature
│ ◉ xrslwzvq [email protected] 2024-02-29 23:06:23.000 -06:00 a70d464c
├─╯ create hello and goodbye functions
│ ◉ yykpmnuq [email protected] 2024-02-29 23:03:22.000 -06:00 210283e8
├─╯ add better documentation
◉ ootnlvpt [email protected] 2024-02-28 23:26:44.000 -06:00 b5db7940
│ only print hello world
◉ nmptruqn [email protected] 2024-02-28 23:09:11.000 -06:00 90a2e97f
│ refactor printing
</code></pre>
<p>Here, we are working on change <code>pzoqtwuv</code>. (<code>@</code> means the working copy.) There
are colors in the real CLI to make the differences more obvious, and to show you
unique prefixes, so for example, you probably only need <code>p</code> or <code>pz</code> instead of
<code>pzoqtwuv</code> to uniquely identify the change. I'll use the full IDs since there's
no syntax highlighting here.</p>
<p>We have two anonymous branches here. They have the change IDs of <code>xrslwzvq</code> and
<code>yykpmnuq</code>. The log output shows the summary line of their messages, so we can
see "create hello and goodbye functions" on one branch, and "add better
documentation" on the other.</p>
<p>You don't need an additional branch name: the change ID is already there. If you
want to add even more better documentation, <code>jj new yykpmnuq</code> (or again, likely
<code>jj new yy</code> in practice) and you're off to the races. (jj new makes a new change
off of the parent you specify.)</p>
<p>(And if you're in a larger repo with more outstanding branches, you can ask <code>jj log</code> to show specific subsets of commits. It has a powerful DSL that lets you do
so. For example, say you only want to see your commits, <code>jj log -r 'mine()'</code> can
do that for you.)</p>
<p>That's all there is to it. We already have the commit messages and IDs, giving
an additional identifier doesn't help that much. In practice, I haven't missed
named branches at all. And in fact, I kind of really appreciate not bothering to
come up with a name, and then eventually remembering to delete that name once
the PR lands, stuff like that. Life is easier.</p>
<h2>Utility CSS</h2>
<p>Another technology I have learned recently is <a href="https://tailwindcss.com/">tailwind</a>. But Tailwind is just
one way of doing a technique that has a few names, I'm going to go with "utility
CSS". The idea is in opposition to "semantic CSS." To crib an example from a
<a href="https://adamwathan.me/css-utility-classes-and-separation-of-concerns/">blog post by the author of Tailwind</a> (which does a better job of thoroughly
explaining why doing utility CSS is a good thing, you should go read it),
semantic CSS is when you do this:</p>
<pre><code>&lt;style&gt;
.greeting {
text-align: center;
}
&lt;/style&gt;
&lt;p class="greeting"&gt;Hello there!&lt;/p&gt;
</code></pre>
<p>Whereas, with Tailwind, you end up instead having something like this:</p>
<pre><code>&lt;p class="text-center"&gt;Hello there!&lt;/p&gt;
</code></pre>
<p>We don't have a new semantic name <code>greeting</code>, but instead describe what
we want to be done to our element via a utility class.</p>
<p>So the thing is, as a previous semantic CSS enjoyer, this feels like using
inline styling. But there's a few significant differences. The first one is,
you're not writing plain CSS, you are re-using building blocks that are defined
for you. The abstraction is in building those utility classes. This means you're
not writing new CSS when you need to add functionality, which to me is a great
sign that the abstraction is working.</p>
<p>It's also that there is some sleight of hand going on here, as we do, on another
level. An objection that gets raised to doing things this way is "what happens
when you need to update a bunch of similar styles?" And the answer for that is
components. That is, it's not so much that utility CSS says that semantic names
are bad, it's that semantic names <em>at the tag level</em> are the wrong level of
abstraction to use names. To sort of mix metaphors, consider the lambda/closure
example. Here's a random function in Rust:</p>
<pre><code>fn calculate(list: &amp;[i32]) -&gt; i32 {
list.into_iter()
.filter(|i| **i % 2 == 0)
.sum()
}
</code></pre>
<p>The <code>**</code> is unfortunate, but this function takes a list of numbers, selects for
the even ones, and then sums them. Here, we have a closure, the argument to
<code>filter</code>, but it's inside a named function, <code>calculate</code>. This is what using
Tailwind feels like to me, we use names for higher level concepts (components),
and then keep things semantically anonymous for some of the tasks inside of them
(markup).</p>
<p>Heck, even the most pro-semantic-styles folks don't advocate that you must give
every single element a class. Everyone recognizes the value of anonymous things
sometimes, it's just a matter of what level of abstraction deserves to get
named.</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3kzkadmleqe2m" /&gt;</p>
</content:encoded></item><item><title>How Does BlueSky Work?</title><link>https://steveklabnik.com/writing/how-does-bluesky-work/</link><guid isPermaLink="true">https://steveklabnik.com/writing/how-does-bluesky-work/</guid><pubDate>Sat, 24 Feb 2024 00:00:00 GMT</pubDate><content:encoded><p>import BlueskyPost from "../../components/BlueskyPost.astro";</p>
<p>One of the reasons I am enthusiastic about BlueSky is because of the way that
it works. So in this post, I am going to lay out some of the design and the
principles behind this design, as I understand them. I am not on the BlueSky
team, so these are my takes only.</p>
<p>Let's begin.</p>
<h2>Why does BlueSky exist?</h2>
<p>Here's what <a href="https://bsky.social">the BlueSky Website</a> says right now:</p>
<blockquote>
<p>Social media is too important to be controlled by a few corporations. We’re
building an open foundation for the social internet so that we can all shape
its future.</p>
</blockquote>
<p>This is the big picture.</p>
<p>Okay so that's a great idea, but like, what does that <em>mean</em>? Currently,
BlueSky is a microblogging application, similar to Twitter and Mastodon. How
does that fit into the big picture? Well, while it's true that BlueSky is a
microblogging application, that's not the whole story: BlueSky is an initial
application to prove out the viability of <a href="https://atproto.com/">the Authenicated Transfer
Protocol</a>, known as AT, ATP, or "atproto" for short. BlueSky is the
"building" and atproto is the "open foundation for the social internet."</p>
<p>An important thing to note: BlueSky is also a company. Some people look at a
company saying "hey we're building something that's too big to be controlled
by companies!" with skepticism. I think that's a healthy starting point, but
the answer for me is atproto.</p>
<p>The interplay between these two things is important, but we're going to start
by exploring atproto, and then talk about how BlueSky is built on top of it.</p>
<h2>Is this a cryptocurrency?</h2>
<p>The first thing we have to get out of the way: If you hear "oh it's a
distributed network called 'something protocol'" you may have a "is this
a cryptocurrency?" alarm bell going off in your head.</p>
<p>Don't worry, it's not a cryptocurrency. It does use some technologies that
originated in the cryptocurrency space, but this isn't a blockchain, or a DAO,
or NFTs, or any of that. Just some cryptography and merkle trees and the like.</p>
<h2>What is the big picture with atproto?</h2>
<p>Here's what <a href="https://atproto.com/guides/overview">the AT Protocol Overview</a> says:</p>
<blockquote>
<p>The Authenticated Transfer Protocol, aka atproto, is a federated protocol for
large-scale distributed social applications.</p>
</blockquote>
<p>Let's break that down:</p>
<blockquote>
<p>a federated protocol</p>
</blockquote>
<p>atproto is federated. This means that the various parts of the system can have
multiple people running them, and that they communicate with each other.</p>
<p>Choosing federation is a big part of how atproto delivers on the "can't be
controlled by one organization" promise. There are other parts too, but this
is an important aspect of solving this.</p>
<blockquote>
<p>for large-scale</p>
</blockquote>
<p>If you want to scale, you have to design with scale in mind. atproto makes
several interesting choices in order to distribute the load of running the
system more onto the actors that can handle load, and less on those that can't.
This way, applications running on top of atproto can scale up to large userbases
without issue.</p>
<p>That's the hope, at least. Earlier this week, BlueSky hit five million users,
and is far more stable than Twitter was in the early days. That's not as big
as many social applications, but it's not nothing either. We'll see how this
works out in practice.</p>
<blockquote>
<p>distributed social applications</p>
</blockquote>
<p>atproto is for connecting to others, so it's focused on social applications.
It also is currently 100% public, there are no private messages or similar. The
reasons for this is that achieving private things in a federated system is
very tricky, and they would rather get it right than ship something with serious
caveats. Best for now to only use this stuff for things you want to be public.</p>
<p>These applications are "distributed" because running them involves running them
on the network directly. There's no "BlueSky server," there's just servers
running atproto distributing messages to each other, both BlueSky messages and
whatever other messages from whatever other applications people create.</p>
<p>So that's the high level, but what does that mean concretely?</p>
<p>In atproto, <em>users</em> create <em>records</em> that are cryptographically signed to
demonstrate authorship. Records have a schema called a <em>Lexicon</em>.</p>
<p>Records are stored in <em>repositories</em>. Repositories run as a <em>service</em>, exposing
HTTP and WebSockets. They then can then talk to each other and federate the
records. These are often called PDSes, for "Personal Data Server." Users
either run their own PDS, or use one that someone else hosts for them.</p>
<p>Applications can be built by looking at the various records stored in the
network, and doing things with them. These services all called <em>App Views</em>,
because they are exposing a particular view of the information stored in the
network. This view is created via the Lexicon system: building an application
means that you define a Lexicon, structuring the data that you want to deal with,
and then look at records that use your lexicon, ignoring the rest.</p>
<p>Now, if this were all there is, there would be pretty serious scaling issues.
For example, if every time I post a new update on BlueSky, if I had to send
my post to every single one of my followers' repositories, that would be
extremely inefficent, and make running a popular repository very expensive to
run. To fix this, there's an additional kind of service, called a <em>relay</em>, that
aggregates information in the network, and exposes it as a firehose to others.
So in practice, App Views don't look at Repositories, but instead, look at
Relays. When I make a post, my respository won't notify my followers'
repositories individually. My repository will notify a Relay, and my followers
will use an App View that filters the ouput of the Relay to show only the posts
of people they're following. This does imply that Relays are often huge and
expensive to run, however you could imagine running a smaller relay that only
propogates posts from a smaller subset of users too. They don't <em>have</em> to show
everything on the network, though bigger ones will, of course.</p>
<p>Here this is in ASCII art:</p>
<pre><code> ┌─────┐ ┌──────────┐
│ PDS ├───────┐ │ App View │
└─────┘ │ └──────────┘
┌▼────────┐ ▲
┌─────┐ │ ├───────┘
│ PDS ├──────► Relay │
└─────┘ │ ├───────┐
└▲────────┘ ▼
┌─────┐ │ ┌──────────┐
│ PDS ├───────┘ │ App View │
└─────┘ └──────────┘
</code></pre>
<p>This is all you really need to know to understand the core of atproto: people
create data, it's shared in the network, and applications can interact with
that data.</p>
<p>However, there are additional service types being introduced, with the
possibility of more in the future. But before we talk about those, we have to
explain some ideological commitments to understand why things are shaped the way
they are.</p>
<h2>What is "speech vs reach"?</h2>
<p>Given that atproto is deliberately created to enable social applications, it
needs to consider not just connecting people, but also disconnecting people.
Moderation is a core component of any social application: "no moderation" is
still a moderation strategy. BlueSky handles these sorts of questions by
acknowledging that different people will have different preferences when it
comes to moderation, and also that moderation at scale is difficult.</p>
<p>As such, the protocol takes a "speech vs reach" approach to moderation. The
stuff we've described so far falls under the "speech" layer. It is purely
concerned with replicating your content across the network, without caring
what the semantic contents of that content is. Moderation tools fall under the
"reach" layer: you take all of that speech, but provide a way to limit the
reach of stuff you don't care to see yourself.</p>
<p>Sometimes, people say that BlueSky is "all about free speech" or "doesn't do
moderation." This is simply inaccurate. Moderation tooling is encoded into the
protocol itself, so that it can work with all content on the network, even
non-BlueSky applications. Moreover, it gives you the ability to choose your own
moderators, so that you aren't beholden to anyone else's choice of moderation or
lack thereof. But I'm getting ahead of myself: let's talk about feed generators
and labelers.</p>
<h2>What are feed generators?</h2>
<p>Most social applications have the concept of a "feed" of content. This is broken
out into its own kind of service in atproto, called a <em>feed generator</em>. A classic
example of a feed is "computer, show me the posts of the people I follow in
reverse chronological order." Lately, algorithmic feeds have become popular with
social networks, to the point of where some non-technical users refer to them
as "algorithms."</p>
<p>Feed generators take the firehose produced by a relay, and then show you a list
of content, filtered and ordered by whatever metric the feed generator desires.
You can then share these feeds with other users.</p>
<p>As a practical example, one of my favorite feeds is the <a href="https://bsky.app/profile/did:plc:vpkhqolt662uhesyj6nxm7ys/feed/infreq">Quiet
Posters</a> feed. This feed shows posts by people who don't post
very often. This makes it so much easier to keep up with people who get drowned
out of my main feed. There are feeds like <a href="https://bsky.app/profile/did:plc:vpkhqolt662uhesyj6nxm7ys/feed/followpics">the 'Gram</a>, which shows
only posts that have pictures attatched. Or <a href="https://bsky.app/profile/did:plc:q6gjnaw2blty4crticxkmujt/feed/bangers">My Bangers</a>, which shows
your most popular posts.</p>
<p>This to me is one of the killer features of BlueSky over other microblogging
tools: total user choice. If I want to make my own algorithm, I can do so.
And I can share them easily with others. If you use BlueSky, you can visit
any of those feeds and follow them too.</p>
<p>Feeds are a recent addition to atproto, and therefore, while they do exist,
they may not be feature complete just yet, and may undergo some change in the
future. We'll see. They're working just fine from my perspective, but I haven't
been following the lower level technical details.</p>
<h2>What are labelers?</h2>
<p>A <em>Labeler</em> is a service that applies <em>labels</em> to content or accounts. As a user,
you can subscribe to a particular labeler, and then have your experience change
based on the labels on posts.</p>
<p>A labeler can do this via whatever method it pleases: automatically by running
some sort of algorithm on posts, manually by having some human give a thumbs
up or thumbs down, whatever method the person running the labeling service
wants.</p>
<p>An example of a labeling service would be a blocklist: a label on the posts
authored by people whose content you don't want to see. Another example is
an NSFW filter, which may run some sort of algorithm over pictures in posts,
and labeling them if they believe they contain NSFW content.</p>
<p>Labeling exists, but I do not believe you can run your own labeler yet. BlueSky
runs their own, but there hasn't been an external release that I am aware of.
But once they do, you can imagine communities running their own services, adding
whatever kind of labels they'd like.</p>
<h2>How does moderation work in atproto?</h2>
<p>Putting this all together, we can see how moderation works: Feeds may choose to
transform the feed based on labels, or App Views may take feeds and apply
transformations based on asking a Labeler about it. These can
be mixed and matched based on preference.</p>
<p>This means you can choose your moderation experience, not just in applications,
but also within it. Want a SFW feed, but allow NSFW content in another? You
can do that. Want to produce a blocklist of people and share it with the
world? You can do that.</p>
<p>Because these moderation tools work at the network level, rather than at the
application level, they actually go <em>further</em> than in other systems. If someone
builds an Instagram clone on atproto, that could also use your blocklist
labeller, since your blocklist labeller works at the protocol level. Block
someone in one place, and they can be blocked on every place, if you so choose.
Maybe you subscribe to different moderation decisions in different applications.
It is 100% up to you.</p>
<p>This model is significantly different from other federated systems, because
you don't really have an "account" on an "instance," like in Mastodon. So a lot
of people ask questions like "what happens when my instance gets defederated"
which don't exactly make sense as stated. You can achieve the same goal, by
blocking a set of users based on some criteria, maybe you dislike a certain
PDS and want to ignore posts that come from a certain one, but that is <em>your</em>
choice and yours alone, it is not dictated by some "server owner" that your
account resides on.</p>
<p>So if you don't have a home server, how does identity work?</p>
<h2>How does identity and account portability work?</h2>
<p>There are a LOT of details to how identity works, so I'm going to focus on the
parts that I find important. I am also going to focus on the part that is
controversial, because that is important to talk about.</p>
<p>At its core, users have an identity number, called a "Decentralized Identifier,"
or <em><a href="https://www.w3.org/TR/did-core/">DID</a></em>. My DID looks like this: <code>did:plc:3danwc67lo7obz2fmdg6jxcr</code>.
Feel free to follow me! Lol, of course that's not the interface that you'll see
most of the time. Identity also involves a <em>handle</em>, which is a domain name.
My handle is <code>steveklabnik.com</code>, unsurprisingly. You'll see my posts on BlueSky
as coming from <code>@steveklabnik.com</code>. This system also works well for people who
don't own a domain; if you sign up for BlueSky, it'll give you the ability to
choose a name, and then your handle is <code>@username.bsky.social</code>. I started off
making posts as <code>@steveklabnik.bsky.social</code>, and then moved to
<code>@steveklabnik.com</code>. But because the DID is stable, there was no disruption to
my followers. They just saw the handle update in the UI.</p>
<p>You can use a domain as your handle by getting the DID your PDS generated for
you, and then adding a <code>TXT</code> record in the DNS you use for that domain. If
you're not the kind of person who uses or even knows what DNS is, I envy you,
but you can also use BlueSky's partnership with NameCheap to register a domain
and configure it to use as a handle without any technical knowledge necessary.
You can then log into applications with your domain as the handle, and
everything works nicely.</p>
<p>This is also how BlueSky delivers true "account portability," partially because,
well, there isn't really a concept of an account. The person who uses a given
DID uses cryptography to sign the content they create, and then that content
is replicated across the network. "Your account" can't really be terminated,
because that would mean someone forcibly stopping you from using keys that they
don't even have access to. If your PDS goes down, and you want to migrate to
a new one, there's a way to backfill the contents of the PDS from the network
itself, and inform the network that your PDS has moved. It is real, meaningful
account portability, and that is radically different from any similar service
running today.[^1]</p>
<p>But.</p>
<p>The devil is in the details, and I think this is one of the more meaningful
criticisms of BlueSky and atproto.</p>
<p>You see, there are different "methods" of creating a DID. BlueSky supports
two methods: <code>did:web</code>, which is based on domain names. There are some drawbacks
with this method that I don't personally fully understand well enough to describe,
I'm sure I'll write something in-depth about DIDs in the future.</p>
<p>So because of that weakness, BlueSky has implemented their own DID method,
called <code>did:plc</code>. The <code>plc</code> stands for "placeholder," because even though
they plan on supporting it indefinitely, it too has its weaknesses. And that
weakness is that it involves asking a service that BlueSky runs in order to
resolve the proper information. For example, <a href="https://plc.directory/did:plc:3danwc67lo7obz2fmdg6jxcr">here is my lookup</a>.
This means that BlueSky can ban you in a more serious way than is otherwise
possible thanks to the network design, which some people take to be a very
serious issue.</p>
<p>So, is the flaw fatal? I don't think so. The first reason is, if you really don't
want to engage with it, you can use <code>did:web</code>. Yes that isn't great for other
reasons; that's why <code>did:plc</code> was created. But you do get around this issue.</p>
<p>Another is that the BlueSky team has demonstrated, in my personal opinion,
enough understanding and uncomfortableness with being in control here, and it's
designed in such a way that if other, better systems develop, you can move
to them. They've also indicated that moving governance of <code>did:plc</code> to some sort
of consensus model in the future is possible. There are options. Also, others
could run a <code>did:plc</code> service and use that instead if they prefer, too.</p>
<p>I personally see this as an example of pragmatically shipping something, others
see it as a nefarious plot. You'll have to decide for yourself.</p>
<h2>How is BlueSky built on top of atproto?</h2>
<p>So, now that we understand atproto, we can understand BlueSky. BlueSky is
an application built on top of the atproto network. They run an App View, and
<a href="https://bsky.app/">a web application</a> that uses that App View to work. They also run a PDS
for users that sign up through the web app, as well as a relay that those PDSes
communicate with.</p>
<p>They publish two Lexicons, one as <code>com.atproto.*</code> and one as <code>app.bsky.*</code>. The
former are low level operations that any application on the network will need,
and the ones specific to BlueSky are in the latter.</p>
<p>But one nice thing about BlueSky in particular is that they've taken the product
goals that nobody should know any of this nerd shit to be able to use BlueSky.
The lack of instances means there's no "I need to pick an instance to create an
account" flow, and the portability means that if my host goes down, I can move,
and my followers are none the wiser.</p>
<h2>How will others build applications on top of atproto?</h2>
<p>You can create an atproto app by creating a Lexicon. You'll then want to run
an App View that does things with data on the network involving your lexicon,
and your application will want to give people the ability to write data to their
PDS using your lexicon.</p>
<p>I myself am considering doing so. We'll see.</p>
<h2>Concluding thoughts</h2>
<p>So yeah, on the technical side of things, that's an overview of how atproto and
BlueSky work. I think this design is very clever. Furthermore, I think the
separation of concerns between atproto and BlueSky are very meaningful, as having
a "killer app" for the network gives a reason to use it. It also is a form of
dogfooding, making sure that atproto is good enough to be able to build real
applications on.</p>
<p>I'm sure I'll have more to say about all of this in the future.</p>
<hr />
<p>Here's my post about this post on BlueSky:</p>
<p>&lt;BlueskyPost uri="at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3km6y2x6cxk2f" /&gt;</p>
<p>[^1]:
A commentor points out https://book.peergos.org/, which I had not heard of,
but apparently was known to the creators of BlueSky before they made it. Neat.</p>
</content:encoded></item><item><title>Using the Oxide Console</title><link>https://steveklabnik.com/writing/using-the-oxide-console/</link><guid isPermaLink="true">https://steveklabnik.com/writing/using-the-oxide-console/</guid><pubDate>Sat, 17 Feb 2024 00:00:00 GMT</pubDate><content:encoded><p>A very, very long time ago, I was introduced to <a href="https://en.wikipedia.org/wiki/Gerrit_(software)">Gerrit</a>. To be honest,
I hated it. However, lately I have become interested in divesting from
<code>git</code> and GitHub, and so have decided to re-visit various "forges" to see what's
out there. The "why" for this will come later, and I'm not leaving them just yet,
just, you know, doing some exploring.</p>
<p>Anyway, in order to play around with Gerrit, I need a server somewhere. This
website runs entirely on Vercel at the moment, so I don't happen to have a
more traditional server lying around to give Gerrit a shot. But as it just so
happens, I work at <a href="https://oxide.computer/">Oxide</a>, and so have access to a rack that I can play
around with. So I thought it would be fun to share what that looks like, given
that people are interested in what we do, but don't actually have access to
one of our machines.</p>
<p>If you'd like to poke at the console yourself, there's a <a href="https://oxide-console-preview.vercel.app/">live demo</a>!
Quoting <a href="https://github.com/oxidecomputer/console?tab=readme-ov-file#live-demo">the README</a>:</p>
<blockquote>
<p>At https://oxide-console-preview.vercel.app the console is deployed as a
static site with a mock API running in a Service Worker. You can create mock
resources and they will persist across client-side navigations, but they exist
only in the browser: nobody else can see them and the mock "DB" is reset on
pageload. Request and response bodies in the mock API match the Oxide API's
OpenAPI spec, but behavior is only mocked in as much detail as is required for
development and testing of the console and is not fully representative of the
real API.</p>
</blockquote>
<p>I happen to think this is extremely cool. If you want to, you could stop reading
this post and go play with that instead. It's also, like the rest of Oxide's software
stack, fully open source!</p>
<p>Before we begin, I would like to say that I am going to be doing this "manually"
via the console and a web browser rather than via the CLI. I am doing this for
two reasons:</p>
<ol>
<li>I think the console is pretty and I would like to show it off. The folks who
work on it do a fantastic job.</li>
<li>I don't have many images on this blog and so want to write a post with some
images and this will look nicer than screenshots of a terminal.</li>
</ol>
<p>To be clear, I imagine a lot of real production usage of the Oxide rack will be
driven by <a href="https://docs.oxide.computer/guides/introduction#_overview">the API</a>, either via our CLI tools or terraform or
whatever. Heck, the console is built on those APIs, so on some level, that's the
only way to do it. My point is, don't think of this as being the only workflow
to interact with Oxide. Maybe if people like this post I'll make some more
covering the CLI driven workflow. Let me know.</p>
<hr />
<p>After you SSO into the console, you're put on the Projects page:</p>
<p>&lt;img src="/img/2024-02-17/projects.png"&gt;</p>
<p>We're in a "silo" named Oxide. A silo is a grouping of users, projects, and
resources that are isolated from one another. But we don't care about that:
we want to make a new project. As you can see, I have censored my co-workers'
projects, as I didn't tell anyone I was making this post, and even though
the description here is hilarious, it wouldn't be right to share without asking,
and I'm already like ten layers deep in this yak shave.</p>
<p>Let's make a new project:</p>
<p>&lt;img src="/img/2024-02-17/new-project.png"&gt;</p>
<p>It's just that easy. After we push the button, we're put on the instances page:</p>
<p>&lt;img src="/img/2024-02-17/instances.png"&gt;</p>
<p>There's a nice little toast in the bottom left that I didn't manage to get a
screenshot of because I was re-sizing my window too. Anyway, now that we have
our project, we can add one or more instances to it. Instances are virtual
machines, and so setting one up looks like you might expect. You can give it
a name and description, but for this screenshot I'm only showing off one part
of this page because this is already a ton of screenshots:</p>
<p>&lt;img src="/img/2024-02-17/create-instance.png"&gt;</p>
<p>We can size instances however we'd like. Even though Gerrit is a Java
application these days, I'm going to keep the instance small, because it's just
for me and performance isn't a huge deal. Yes, I have made massive instances for
fun in the past.</p>
<p>We're using a Debian image that's been provided by this silo, because I am lazy
and that's fine. Is four gigabytes of disk enough? I don't know, let's find out.</p>
<p>A few seconds later, our instance is up and running:</p>
<p>&lt;img src="/img/2024-02-17/instance.png"&gt;</p>
<p>I am obscuring the external IP because, well, no reason to give that out.
Anyway, that's it! We can <code>ssh</code> in:</p>
<p>&lt;img src="/img/2024-02-17/login.png"&gt;</p>
<p>... how, you may ask? Well, I didn't show off the entire "Create instance" page:
I have set up my ssh keys in my account previously, and so by default, because
this image uses <code>cloud-init</code> to configure itself, my <code>ssh</code> keys are already
injected into the image. Furthermore, I left it on the default <a href="https://en.wikipedia.org/wiki/Virtual_private_cloud">VPC</a>, which
is currently configured with a firewall rule that allows for incoming <code>ssh</code>
connections. All of this is of course absolutely customizable, but given what
this rack is being used for, these defaults make sense for us, so I didn't have
to actually do much setup at all.</p>
<p>That being said, I do need to open port 8080 so that I can poke at Gerrit in
a web browser. If I go into the VPC configuration, I can add a new firewall rule:</p>
<p>&lt;img src="/img/2024-02-17/add-firewall-rule.png"&gt;</p>
<p>Further down, I can pick port 8080, and that I want to allow TCP. After setting
up Gerrit, we have success!</p>
<p>&lt;img src="/img/2024-02-17/success.png"&gt;</p>
<hr />
<p>And that's it! It's just that easy. There's of course a ton of stuff I didn't
talk about, but easy things should be easy, and for basic usage, it's just that
easy.</p>
</content:encoded></item><item><title>Memory Safety is a Red Herring</title><link>https://steveklabnik.com/writing/memory-safety-is-a-red-herring/</link><guid isPermaLink="true">https://steveklabnik.com/writing/memory-safety-is-a-red-herring/</guid><pubDate>Thu, 21 Dec 2023 00:00:00 GMT</pubDate><content:encoded><p>TL;DR:</p>
<p>I think that a focus on memory safe languages (MSLs) versus non memory-safe
languages is a bit of a red herring. The actual distinction is slightly bigger
than that: languages which have defined behavior by default, with a superset
where undefined behavior is possible, vs languages which allow for undefined
behavior anywhere in your program. Memory safety is an important aspect of this,
but it is necessary, not sufficient. Rust's marketing has historically focused
on memory safety, and I don't think that was a bad choice, but I do wonder
sometimes. Finally, I wonder about the future of the C++ successor languages in
the face of coming legislation around MSLs for government procurement.</p>
<p>Additionally, I would like to thank ixi for discussing some of this stuff with
me. I do not claim that they agree with me, these opinions are solely my own!
But talking about it helped me form those opinions.</p>
<hr />
<h2>ThreadId(1)</h2>
<p>Today I have been programming in Rust for 11 years. I usually write some sort
of blog post reflecting on the year. This year, I am reflecting on some old
questions:</p>
<ul>
<li>Should Rust have focused so much on memory safety in its advocacy?</li>
<li>Does Rust conflate a more general concept of "safety" with "memory safety?"</li>
<li>What do people even mean by "memory safe languages?"</li>
</ul>
<p>But before we get there, we have to talk about some other things.</p>
<h2>ThreadId(2)</h2>
<p>The other day, someone <a href="https://news.ycombinator.com/item?id=38616605">asked an interesting question on Hacker
News</a>:</p>
<blockquote>
<p>Can someone with a security background enlighten me, on why Python is on the
list of "memory safe" languages? Most of the python code I have worked with is
a thin wrapper on C. Wouldnt that make python vulnerable as well?</p>
</blockquote>
<p>This is a very reasonable question! If you click the link, you'd see my answer:</p>
<blockquote>
<p>You are correct that if you call C from Python, you can run into problems. But
the fault there lies with the C, not the Python. Pure Python itself is memory
safe.</p>
<p>Because memory safety is built on top of memory unsafety, it is generally
understood that when discussing things at this sort of level, that we are
speaking about the Python-only subset. (Same as any other memory safe language.)</p>
</blockquote>
<p>But this has been gnawing at me for a while. Like it's one of those things that
"everybody knows what I mean." But isn't that kind of... bullshit? Like something
is wrong with your definitions. But on the other hand, if we accept FFI as
being not safe, and that since you can FFI from anywhere, that means it's also
unsafe, then every single program is unsafe. Your operating system does not
offer memory-safe APIs. To do anything useful, you must call into the operating
system. This means every program would be infected via this conception of safety,
and therefore, as a definition, it would be useless.</p>
<p>If we instead make an exception for a language's runtime, which is allowed to
make unsafe calls, but users' code is not, that would draw an appropriate
boundary: only write code in the guest language, and you don't have to worry
about safety anymore. And I think that this sort of definition has been the
default for many years. You have "managed" languages which are safe, and you have
"unmanaged" languages which are unsafe. Unmanaged languages are used to implement
managed languages. What's that? A Java runtime written in Java? A Ruby runtime
written in Ruby? Yeah those are weird exceptions. Ignore those. And those
"native extensions" that segfault? Well they aren't written in the host language,
so of course that will happen, and yeah it's kinda annoying when it happens, but
it's fine. Sometimes you have to risk it for performance.</p>
<p>I just don't find this satisfying. It's too vague. You just kinda know it when
you see it. Ignore FFI, and we're good. And sometimes, you just can't have safety.
What if you want to write a runtime? You're gonna need to call into the system
directly, and now we're back to unsafe, so just use an unsafe language. It is
what it is.</p>
<h2>ThreadId(3)</h2>
<p>That comment is on an article titled <a href="https://media.defense.gov/2023/Dec/06/2003352724/-1/-1/0/THE-CASE-FOR-MEMORY-SAFE-ROADMAPS-TLP-CLEAR.PDF">The Case for Memory Safe
Roadmaps: Why Both C-Suite Executives and Technical Experts
Need to Take Memory Safe Coding Seriously
</a>.
This document, published by <a href="https://en.wikipedia.org/wiki/Five_Eyes">the Five Eyes</a>,
suggests... that executives and technical experts need to take memory safety
seriously. Call it boring if you want, but it is what it says on the tin.</p>
<p>More specifically, it says:</p>
<blockquote>
<p>Memory safe programming languages (MSLs) can eliminate memory safety
vulnerabilities.</p>
</blockquote>
<p>As well as an appendix, "Memory Safe Languages," which describes C#, Go, Java,
Python, Rust, and Swift as memory safe languages.</p>
<p>Finally, it says:</p>
<blockquote>
<p>Programming languages such as C and C++ are examples of memory unsafe programming
languages that can lead to memory unsafe code and are still among the most widely used
languages today.</p>
</blockquote>
<p>But it also acknowledges the reality that sometimes you can get around memory
safety:</p>
<blockquote>
<p>MSLs, whether they use a garbage collection model or not, will almost
certainly need to rely on libraries written in languages such as C and C++.
Although there are efforts to re-write widely used libraries in MSLs, no such
effort will be able to re-write them all anytime soon.</p>
<p>For the foreseeable future, most developers will need to work in a hybrid model
of safe and unsafe programming languages. Developers who start writing in an MSL
will need to call C and C++ libraries that are not memory safe. Likewise, there
are going to be situations where a memory unsafe application needs to call into
a memory safe library. When calling a memory unsafe component or application,
the calling application needs to be explicitly aware of—and limit any input
passed to—the defined memory bounds.</p>
<p>The memory safety guarantees offered by MSLs are going to be qualified when data
flows across these boundaries. Other potential challenges include differences
in data marshalling, error handling, concurrency, debugging, and versioning.</p>
</blockquote>
<p>Okay, that's enough quoting from the report. And I swear I'm not about to say
nice things about it because I am cited as #42, though to be honest it's real
fucking weird that searching "klabnik" on a page on defense.gov returns 1/1.
But I think this is an excellent formulation of the critical distinction between
the languages on one side of the line versus the other, though they make the
same conceptual mistake that I believe Rust may have: it's not just about
memory unsafety. "safety" itself is basically the idea that most of the time,
you're in a universe where everything is peachy keen, and occasionally, you'll
need to reach across that line into the scary other place, but you gotta do what
you gotta do. But there is a line there. But that line is more abstract than
just memory safety.</p>
<h2>ThreadId(4)</h2>
<p>Rust often describes itself as "data race free." But what the heck is a data
race anyway? Here's <a href="https://blog.regehr.org/archives/490">John Regehr</a>:</p>
<blockquote>
<p>A data race happens when there are two memory accesses in a program where both:</p>
<ul>
<li>target the same location</li>
<li>are performed concurrently by two threads</li>
<li>are not reads</li>
<li>are not synchronization operations</li>
</ul>
<p>There are other definitions but this one is fine; it’s from Sebastian Burckhardt at Microsoft Research.</p>
</blockquote>
<p>Oh. Well, I guess "Here's Sebastian Burckhardt:" instead.</p>
<p>Point is, data races are undefined behavior in Rust, and they're also undefined
behavior in C++. When looking for a succinct citation for that fact, I came
across <a href="https://www.hboehm.info/c++mm/why_undef.html">this 2006 post by Hans Boehm</a>.
Its first line:</p>
<blockquote>
<p>Our proposed C++ memory model gives completely undefined semantics to programs
with data races.</p>
</blockquote>
<p>I believe this was the model that was accepted, so that's the best citation you're
gonna get from me. Okay, as soon as I read that I said "come on Steve, that's stupid,
make sure it's not just trivial to actually find in the standard" and lo and
behold
<a href="https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/n4849.pdf">n4849</a>
says:</p>
<blockquote>
<p>The execution of a program contains a data race if it contains two potentially
concurrent conflicting actions, at least one of which is not atomic, and
neither happens before the other, except for the special case for
signal handlers described below. Any such data race results in undefined
behavior.</p>
</blockquote>
<p>Okay, the real citation was easier. The article by Bohm is still fascinating.</p>
<p>But data races aren't undefined behavior in every programming language. For
example, let's consider both Java and Go. Now I hate that I even have to
preemptively say this, but I am <em>not</em> saying Java and Go are bad. I think
these are good decisions for both of the two languages. And I think their
decisions here are very informative. Language wars are stupid. Stop it.</p>
<p>Java was the first time I remember hearing, "yeah you can have data races but
like, it's fine" about. I kinda lodged that fact in the back of my brain, and
didn't think about it much. But lately, I was wondering. So I went and read
the Java specification, and more specifically, <a href="https://docs.oracle.com/javase/specs/jls/se21/html/jls-17.html">Chapter
17</a>. It contains
this text:</p>
<blockquote>
<p>The semantics of operations other than inter-thread actions, such as reads of
array lengths (§10.7), executions of checked casts (§5.5, §15.16), and
invocations of virtual methods (§15.12), are not directly affected by data
races.</p>
<p>Therefore, a data race cannot cause incorrect behavior such as returning the
wrong length for an array.</p>
</blockquote>
<p>This is morally equivalent to "not undefined behavior" even though the
specification doesn't define "undefined behavior" in the way that the C, C++,
and Rust specifications do. Notably, the specification also defines "data race"
in a slightly different way:</p>
<blockquote>
<p>When a program contains two conflicting accesses (§17.4.1) that are not
ordered by a happens-before relationship, it is said to contain a data race.</p>
</blockquote>
<p>Not just any ordering or synchronization primitive, a happens-before relation
specifically. Fun! We have three technically-slightly different but basically
morally equivalent ways of defining a data race. Let's do a fourth. Go also
allows data races. Sort of. From <a href="https://go.dev/ref/mem">the Go memory model</a>:</p>
<blockquote>
<p>A data race is defined as a write to a memory location happening concurrently
with another read or write to that same location, unless all the accesses
involved are atomic data accesses as provided by the sync/atomic package.</p>
</blockquote>
<p>Given that Rust provides atomic ordering operations in a submodule of its
standard library, Rust and Go agree here, though Go's more specific about naming
it.</p>
<p>But they also have this to say:</p>
<blockquote>
<p>While programmers should write Go programs without data races, there are
limitations to what a Go implementation can do in response to a data race. An
implementation may always react to a data race by reporting the race and
terminating the program. Otherwise, each read of a single-word-sized or
sub-word-sized memory location must observe a value actually written to that
location (perhaps by a concurrent executing goroutine) and not yet overwritten.</p>
</blockquote>
<p>So it's not right, but it's also not exactly UB. It goes on:</p>
<blockquote>
<p>These implementation constraints make Go more like Java or JavaScript, in that
most races have a limited number of outcomes, and less like C and C++, where the
meaning of any program with a race is entirely undefined, and the compiler may
do anything at all. Go's approach aims to make errant programs more reliable and
easier to debug, while still insisting that races are errors and that tools can
diagnose and report them.</p>
</blockquote>
<p>While a Go program may exhibit what a Rust or C++ program would consider
undefined behavior, and it does also consider it an error, the consequences are
very different. You don't get time travel. You get 998 instead of 1,000.</p>
<p>(After publication, Edoardo Spadolini reached out to me and we had a productive
conversation about the above paragraph: while my overall point here is still
valid, there are slightly more serious consequences to UB in a Go program than
slightly incorrect sums. For example, torn writes to a slice can lead to serious
consequences, and there are other ways to write code that exploits UB to be
actively malicious, such as a function with the signature
<code>func Cast[T any](addr uintptr) *T</code> that works. However, it's also worth mentioning
that Go ships with a race detector, available by simply passing <code>-race</code>. For some
interesting usage of this at scale, check out <a href="https://www.uber.com/blog/dynamic-data-race-detection-in-go-code/">this post by Uber</a>. At the end of the day, this is still
very different than the "your whole program is invalid" style of issues you
can end up with in C or C++, I am slightly underselling the danger above.)</p>
<p>Go also provides <a href="https://go.dev/ref/spec#Package_unsafe">an unsafe package</a>:</p>
<blockquote>
<p>The built-in package unsafe, known to the compiler and accessible through the
import path "unsafe", provides facilities for low-level programming including
operations that violate the type system. A package using unsafe must be vetted
manually for type safety and may not be portable.</p>
</blockquote>
<p>Oh yeah, I forgot, Java also has <code>sun.misc.unsafe</code>, essentially the same thing.
They have actually wanted to remove it for a long time, and some progress has
been made, but it's not quite gone yet. <a href="https://blogs.oracle.com/javamagazine/post/the-unsafe-class-unsafe-at-any-speed">The Unsafe Class: Unsafe at Any
Speed</a>
is a good explanation of the topic.</p>
<h2>ThreadId(5)</h2>
<p>One weird thing about suggesting that the borrow checker is only about memory
safety, and that memory safety means the absence of data races, is that memory
safety is more than just the absence of data races. Consider a problem that
requires no threads to become an issue: iterator invalidation. And different
languages have different answers to this issue:</p>
<ul>
<li>In C++, it is undefined behavior.</li>
<li>In Java, some data structures detect this and throw a <code>ConcurrentModificationException</code>.</li>
<li>In JavaScript, Ruby, and other languages, you often get "weird" behavior:</li>
</ul>
<pre><code>numbers = [1, 2, 3, 4]
numbers.each do |number|
p number
numbers.shift(1)
end
p numbers
</code></pre>
<p>prints</p>
<pre><code>1
3
[3, 4]
</code></pre>
<p>But you don't get full-blown time-travel undefined behavior. Sort of like this
code in Rust:</p>
<pre><code>fn main() {
let i = i32::MAX + 1;
println!("{i}");
}
</code></pre>
<p>Okay in this case because it's all at compile time Rust can detect it:</p>
<pre><code>error: this arithmetic operation will overflow
--&gt; src/main.rs:2:13
|
2 | let i = i32::MAX + 1;
| ^^^^^^^^^^^^ attempt to compute `i32::MAX + 1_i32`, which would overflow
|
= note: `#[deny(arithmetic_overflow)]` on by default
</code></pre>
<p>But in the general case, you won't get a compile error. You may get a panic, you
may get <code>-2147483648</code>. It's weird. But it's not full-blown time-travel undefined
behavior.</p>
<p>Anyway, back to iterator invalidation:</p>
<ul>
<li>In Rust, it is prevented at compile time.</li>
</ul>
<p>This is neat! This is cool! This is good! And it's pretty unique to Rust.</p>
<p>It is nice when our programs do what we expect, instead of what we don't expect.
It is even nicer when, if our programs don't do what we expect, we have some sort
of waypost, some sort of sign where the problem might be. What "nice" means may
be a bit different in different languages, but the boundary is the important
part: we're gonna have to potentially violate the rules somewhere, so at least
give us some help when that inevitably goes wrong. Over here, logic bugs may
happen, some weird behavior may result, but over there? There be dragons.
Nobody can save you out there. Tread carefully.</p>
<h2>ThreadId(6)</h2>
<p>If we think about all of these designs, they all are very similar conceptually:
safe code is the default, but you can call into some sort of unsafe facility.
And everyone is very clear on the relationship between the two: while the unsafe
facility exists to be used, it must uphold the rules that the safe world relies
on. And that means that safety and unsafety have a super/sub-set relationship.
In the core is unsafety. But at some point, we draw a line, and on top of that
line, safety exists.</p>
<p>In Rust, we often talk about how unsafe is great, because it clearly draws a
line around code with which serious bugs may happen. More specifically, while
most people describe it as "where memory safety can originate from" or similar,
it's actually slightly broader than that: it's where undefined behavior can
originate from.</p>
<p><a href="https://doc.rust-lang.org/reference/behavior-considered-undefined.html">The nomicon</a>:</p>
<blockquote>
<p>Rust code is incorrect if it exhibits any of the behaviors in the following
list. This includes code within unsafe blocks and unsafe functions. unsafe
only means that avoiding undefined behavior is on the programmer; it does not
change anything about the fact that Rust programs must never cause undefined
behavior.</p>
<p>It is the programmer's responsibility when writing unsafe code to ensure that
any safe code interacting with the unsafe code cannot trigger these behaviors.
unsafe code that satisfies this property for any safe client is called sound;
if unsafe code can be misused by safe code to exhibit undefined behavior, it
is unsound.</p>
</blockquote>
<p>This is not a new insight, but basically, unsafe is like FFI in a managed
language. It's not something you do often. But it is something you sometimes
have to do. And when you do, you can at least contain the danger: the problem
has to lie somewhere behind that veil. You have clear points to begin your
search into where the issues lie, should the abyss claim another soul.</p>
<p>In some ways I'm also repeating another older slogan Rust had: "memory safety
without garbage collection." But it's more like "no UB and no garbage collection."
Gosh, why didn't we put <em>that</em> on the website? Practically rolls right off the
tongue. What makes Rust appealing, and I think especially to the "non-systems"
crowd, is that it shares a property of many managed languages, that of "no
undefined behavior by default," with a property of many systems languages, that
of "no runtime, as little overhead as possible." And sure, there's an escape
hatch, but it's rarely used, and clearly separates the two worlds. This is a
fundamentally different way of viewing the world than "unsafe by default."</p>
<p>Another way in which there's an interesting split here in language design
is between the scripting languages like Ruby, Python, and JavaScript, and
languages like Rust, Go, and Java, is that the boundary in the former is purely
FFI, while in the latter, there are also unsafe faculties in the host language
itself, as well as with FFI. In all three cases, they're behind either a specific
package or <code>unsafe {}</code>, but they give you some interesting tools that don't
require you to use another language to use. I think this is an under-appreciated
part of the overall design space.</p>
<h2>threads.into_iter().for_each(|t| t.join().unwrap());</h2>
<p>So: did Rust focus too much on memory safety? And what should newer languages
do in this area?</p>
<p>Well, I think that in some sense, the answer is obviously "no, Rust did not
focus too much on memory safety." Rust has been wildly successful, reaching
heights that I only could dream about eleven years ago. Yes, usually I have
tried to be fairly tempered when talking about Rust's successes, and some
folks still try to claim Rust is still super niche. But when I think about the
ways that Rust touched even this blog post, it's staggering: I wrote this post
in VS: Code, which uses Rust, on Windows, which has Rust in the kernel. I
published it to GitHub, which uses Rust for code search. I previewed it in
Chrome, which uses some Rust today and may start having more tomorrow. It's
hosted on Vercel, which uses Rust in projects like turbopack, though maybe not
in the critical path for this post. When you read it, the bytes probably passed
through CloudFlare, which uses a ton of Rust, and maybe you're reading this in
Firefox, which is about 12% Rust. And maybe you're on Asahi Linux, where the
GPU drivers are written in Rust. Rust has Made It, and is here to stay. And
the marketing we used got us here. So in some sense, it would be silly to
declare said messaging a failure.</p>
<p>However, I also wonder what else could have been. Could Rust have grown faster
with a different message? Maybe. That said, what would that message be?
"Undefined behavior is scary?" I'm not sure that's as compelling, even if I think
it's actually a bigger deal. While memory unsafety is only part of undefined
behavior, it is the part that is easy to understand: pointers are the bane of
every C newbie. It has consequences that are easy to understand: security
vulnerabilities are bad. Has Rust <em>really</em> conflated "memory safety" and "safety"
that much? I don't actually think so. People still talk about SQL injection,
about Remote Code Execution, about Cross Site Scripting. People don't <em>really</em>
believe that just because it's written in Rust, bugs are impossible. Sure you
can find a few comments on the internet as such, but the big organizations that
are using Rust for meaningful work do not take this position. Given <a href="https://en.wikipedia.org/wiki/Amdahl%27s_law">Amdahl's
Law</a>, if 70% of security
vulnerabilities are related to memory safety, focusing so strongly on it makes
sense, as fixing that will give you a much larger impact than other things.</p>
<p>So what should new languages do here? Well, pretty much the only language that
is gaining developer mindshare that's new and is memory unsafe by default is
Zig. I think Andrew is very sharp, and a tremendous engineer. I am very
interested to see how things shake out there. But as a language, it does not
appeal to me personally, because I think we've sort of moved past "unsafe by
default." That being said, it's also not that Zig is exactly like other
languages in this space either: if we are willing to go beyond "memory unsafe vs
memory safe at compile time" as a strict binary, and instead look at "memory
unsafe at compile time" itself, we see a pretty big gradient. Zig is doing a
bunch of things to mitigate issues where possible. Maybe "safe enough" truly is
safe enough. Time will tell. But beyond that, you also don't have to go full
Rust in the design space. Rust wants to be a language that you can (and at
Oxide, we do) use at the lowest levels of the system. But also memory safe.
And extremely fast. These constraints mean that Rust takes on a lot of
complexity in the type system, where languages that are willing to relax these
things wouldn't have to. Here are three languages that I think are sort of
"post-Rust" in a sense, that are learning from Rust but putting their own
twist on things:</p>
<ul>
<li><a href="https://www.hylo-lang.org/">Hylo</a> (formerly Val) tries to, in their own words,
"strengthens [value semantics], while de-emphasizing reference semantics".
This lets them avoid a lot of Rust's complexity. Hylo looks more explicitly
to Swift than Rust for positive inspiration in many cases, but I'm including
it here anyway, as I've sort of ended up not talking about Swift in this post.</li>
<li><a href="https://austral-lang.org/">Austral</a> is "if affine types are good for systems
programming, linear types are even better." Austral also believes that parsimony
is a virtue in language design, and that linear types can replace a lot of
different features in other programming languages, while also eschewing other
features that they deem as not carrying their weight.</li>
<li><a href="https://www.modular.com/max/mojo">Mojo</a>, "the usability of Python with the
performance of C." It also has this sort of "lower into a more complex language"
idea; you can Just Write Python for the most part and it works, but if you want
to get fancier, you can. <a href="https://github.com/modularml/mojo/blob/main/proposals/lifetimes-and-provenance.md">Lifetimes are a current
proposal.</a></li>
</ul>
<p>I think there are some other languages that are forming the next big cohort of
programming languages, and they're interesting too, but these three I think have
the most explicit connection to Rust in the design and goals of the language
itself. Shout out to Nu, Inko, Gleam, Roc, Oil Shell, Unison, and others. Lots
of cool stuff going on.</p>
<h2>int atexit(void (*func)(void))</h2>
<p>There is one way in which the question "is the focus on memory unsafety
misguided?" is much more serious than battles in the comment sections of
various fora: the government.</p>
<p>We talked about this earlier, but various government organizations have been
slowly putting out recommendations that organizations should be moving towards
MSLs. And the first reactions to these were always "well they're not laws so
they don't have teeth" as well as "laws are inevitably coming." To be honest,
I have mostly been in the former camp, and viewed the later as fearmongering.
Sure, maybe someday, but like, dipping your toes in does not mean that you're
about to cannonball moments later.</p>
<p>But then I got <a href="https://news.ycombinator.com/item?id=38700267">this interesting reply from David Tolnay on Hacker
News</a>. In it, he refers me to
<a href="https://adspthepodcast.com/about/">ADSP: The Podcast</a>, and more specifically,
<a href="https://adspthepodcast.com/2023/12/15/Episode-160.html">Episode 160: Rust &amp; Safety at Adobe with Sean
Parent</a>. In it, Sean
makes reference to two pieces of legislation, <a href="https://www.congress.gov/bill/118th-congress/house-bill/2670">one in the
US</a>
and <a href="https://digital-strategy.ec.europa.eu/en/library/cyber-resilience-act">one in
the EU</a>.
Now, I am not a politican, and these bills are huge, so I wasn't able to figure
out how these bills do this specifically, but to quote Sean:</p>
<blockquote>
<p>The one in the U.S. that's pending basically says that the Department of
Defense is going to within 270 days of the bill passing (and it's a funding bill
which means it will probably pass late this year - early next year) that the
Department of Defense will establish guidelines around safety and security
including memory safety for software products purchased by Department of
Defense. The E.U. has a similar wording in a bill that's slowly winding its way
through their channels. I don't have insight into when that will pass. The U.S.
one will almost certainly pass here within a month or two.</p>
</blockquote>
<p>This sounds much more realistic, of course. Now, this does not mean that
C++ is outlawed or something silly like that, but it does mean that using Rust
could become a more serious competitive advantage when it comes to selling to
government: if you don't need an exception for your product, that's an advantage
over a product which does.</p>
<p>This all also may go the way of another anecdote Sean talks about in the past:
when a similar bill tried to mandate POSIX compliance. What I'm saying is, go
listen to the podcast. And while this still isn't something as wild as "not
using MSLs is illegal," it is really shocking to me how quickly things seem to
be moving in that vague direction, though of course it will never actually
arrive there. There seems to be at least some sort of thinking that's not
immediately lauged out of the room that memory unsafety could be a consumer
safety issue in a similar sense that other unsafe materials are a consumer
safety issue. Is that right? I'm not sure, but it seems to be where the
conversation is heading.</p>
<p>So where does that leave existing widely used memory-unsafe languages? Well,
the C++ community has... had a wide variety of discussions on this topic. I
should also say I mean "community" at large, and the committee feels like it
has a big divide as well. Now I should re-iterate that while I pay attention to
the C++ standardization process, I am very much an outsider: I have never
attended a meeting, I have no direct involvement in these things, I just read
a lot. And while historically most of the community wrote Rust off as a novelty,
I think that at this point, those with their head in the sand about Rust's usage
are mostly a fringe minority. But the big question that remains is, what to do?</p>
<p>Two approaches have appeared:</p>
<ul>
<li>Improve the safety of C++ in some fashion.</li>
<li>Create a "successor language" that will fix C++'s problems.</li>
</ul>
<p>What is interesting to me about these approaches is that they are both good,
for different reasons. The former is a sort of harm reduction approach: don't
let the perfect be the enemy of the good. <a href="https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p3023r1.html">What can we do today to improve the
lives of a ton of
people?</a>
But if you want to move into the realm of an MSL, you have to break backwards
compatibility. This is tantamount to just doing #2 in the first place, and so
some are cutting straight to that approach. But if we look at it in a slightly
different way, an interesting thing happens:</p>
<ul>
<li>breaking backwards compatibility vs not</li>
<li>memory safety by default vs not</li>
</ul>
<p>This leads to four approaches:</p>
<ul>
<li>We do not break backwards compatibility, and we ignore the memory safety
issue (this is the 'do nothing' option).</li>
<li>We do not break backwards compatibility, but we try to make things as memory
safe as possible given the constraints we have.</li>
<li>We break backwards compatibility, and we try to make things as memory safe as
possible but still remain unsafe.</li>
<li>We break backwards compatibility, and we become memory safe by default.</li>
</ul>
<p>The first option is seemingly untenable, but is also where I see C heading.
There's seemingly not nearly as much desire for movement in the C space as the
C++ space, even though I am following what JeanHeyd Meneide is doing and
appreciate it.</p>
<p>The second option is how I view at least one of the possible C++ successor
languages, <a href="https://www.circle-lang.org/">Circle</a>. Circle is incredibly
interesting and is worth its own post. But in short, it is a C++ compiler that
also implements a lot of extensions, both ones that are proposals in the
committee, but also ones that its author is interested in. He has been doing
lifetime experiments.</p>
<p>The third option is where the most famous successor languages live:
<a href="https://github.com/carbon-language/carbon-lang">Carbon</a> and
<a href="https://github.com/hsutter/cppfront">cpp2</a>. I haven't gotten to spend a ton of
time with either of these yet, but they explicitly aim to do to C++ what C++ did
to C, or what TypeScript did to JavaScript: create a new language for new code,
while allowing you to use the older language for older code. This allows them
to break backwards compatibility more easily: new code can more easily be
written under the new constraints. This gives them a larger degree of freedom
to make changes, which may be necessary to move the needle significantly on
the safety issue. But to my knowledge, they do not attempt to be memory safe
by default. They're conceptually similar to Zig in this way.</p>
<p>But what about a fourth option? What if someone did a TypeScript for C++, but
one that was closer to what Rust does? You might argue that this is basically
just Rust? (Interestingly, it is also sort of what Zig is with C: you can use
the same compiler to compile a mixed codebase, which is close enough in my
opinion.) What makes me worry about option #3 is that it doesn't pass the
seeming test that is coming: memory safe by default. I still think moving
to a #3 style solution is better than staying with #1, but is that enough?
I don't know, time will tell. I wonder to what degree you can get "C++ but memory
safe" without just being Rust. As we talked about earlier, most languages that
are trying to improve on Rust in this space are willing to trade off some core
constraints on Rust's design to get them, but C++ and Rust share the same
constraints. That said, it would take a tremendous amount of hubris to declare
that Rust is the final, ultimate, best language in the "memory safe without GC"
space. Rust has made a lot of mistakes too. Maybe reflecting on those will make
for a good twelfth year post. Someone get started on Rust++, please!</p>
<p>The reason that today is my Rust-a-versary is that I usually play with a new
language over Christmas break, and in 2012, that language was Rust. Maybe this
Christmas, I'll give cpp2 a try.</p>
<p>In the final minutes before publishing this, I also found another <a href="https://www.cisa.gov/sites/default/files/2023-12/CSAC_TAC_Recommendations-Memory-Safety_Final_20231205_508.pdf">interesting
report</a>.
This is a report to the CISA director by the Technical Advisory Council on
how to engage with the memory safety question. Rust has 17 mentions. I haven't
read it yet, but I figured I'd include it here.</p>
</content:encoded></item><item><title>Updating Buck</title><link>https://steveklabnik.com/writing/updating-buck/</link><guid isPermaLink="true">https://steveklabnik.com/writing/updating-buck/</guid><pubDate>Mon, 08 May 2023 00:00:00 GMT</pubDate><content:encoded><p>Hey there! A shorter post today, but I wanted to continue my series on Buck
by going over some things that have changed since this series started.</p>
<h2>A series</h2>
<p>This post is part of a series:</p>
<ul>
<li><a href="using-buck-to-build-rust-projects">Using buck to build Rust projects</a></li>
<li><a href="using-cratesio-with-buck">Using Crates.io with Buck</a></li>
<li><a href="#">Updating Buck</a> (you are here)</li>
</ul>
<h2>Updating Buck</h2>
<p>Lots of great stuff has been happening since the initial release of buck2, and
we'd like to take advantage of that. If we try and update things, though,
we'll get (well, you might not, but I did) an error:</p>
<pre><code>~&gt; cargo +nightly-2023-03-07 install --git https://github.com/facebook/buck2.git buck2
&lt;snipped&gt;
Replacing C:\Users\steve\.cargo\bin\buck2.exe
error: failed to move `C:\Users\steve\.cargo\bin\cargo-installYqDCSV\buck2.exe` to `C:\Users\steve\.cargo\bin\buck2.exe`
Caused by:
Access is denied. (os error 5)
</code></pre>
<p>(Why yes, my console prompt has changed...)</p>
<p>See on Windows, we can't replace a program that's running, while it's running.
On other systems, this may work for you. However, it's probably a good idea to
not take advantage of this particular feature, because of the underlying
cause: buck runs a daemon in the background. This daemon is still going. Trying
to replace it while it's running means you'll still have the old one going
around in the back, and while that's fine (is it? I don't actually know), best
to cleanly shut down first. So do this:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; buck2 killall
Killed buck2.exe (4788). C:\Users\steve\.cargo\bin\buck2.exe --isolation-dir v2 daemon --dont-daemonize
Killed buck2.exe (40200). C:\Users\steve\.cargo\bin\buck2.exe --isolation-dir v2 daemon --dont-daemonize
</code></pre>
<p><code>buck2 killall</code> will kill every instance of the daemon. As you can see, it found
two of mine, and shut them down. As you can see, you want to run this command
from within one of your projects.</p>
<p>And now an upgrade works.</p>
<h2>Update Reindeer</h2>
<p>If you're building Rust code and using Reindeer, like we talked about in the
last post, go ahead and grab that too. Lots of good stuff in there:</p>
<pre><code>~&gt; cargo install --git https://github.com/facebookincubator/reindeer/ reindeer -f
</code></pre>
<p>Now that we're all updated, let's fix up our project to make use of the latest
changes that are relevant.</p>
<h2>Update the prelude</h2>
<p>I no longer need my weird toolchain hacks to get MSVC Rust working. Thank you
dtolnay! 🙏</p>
<p>Let's pull in changes to the prelude:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; cd prelude
~\Documents\GitHub\buck-rust-hello\prelude&gt; git fetch origin
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 17 (delta 12), reused 15 (delta 10), pack-reused 0
Unpacking objects: 100% (17/17), 3.60 KiB | 102.00 KiB/s, done.
From https://github.com/facebook/buck2-prelude
920d3f2..370cd4d main -&gt; origin/main
~\Documents\GitHub\buck-rust-hello\prelude&gt; git reset --hard origin/main
HEAD is now at 370cd4d Http_archive execution platform hack
~\Documents\GitHub\buck-rust-hello\prelude&gt; cd ..
~\Documents\GitHub\buck-rust-hello&gt;
</code></pre>
<p>And remove the hack (if you had to do it too) in <code>toolchains\BUCK</code>:</p>
<pre><code>system_cxx_toolchain(
name = "cxx",
visibility = ["PUBLIC"],
)
</code></pre>
<p>No more <code>link.exe</code> stuff!</p>
<h2>Fix some configuration</h2>
<p>Some good changes have landed in the <code>buck2 init</code> subcommand that we'll want
to add ourselves.</p>
<p>First up, we should <a href="https://github.com/facebook/buck2/commit/54e1eb5e29eca8a021579b859d0a34c73754eb2a">add an empty file named <code>.buckroot</code> to the root of our
repository</a>.
Because buck works with your filesystem hierarchy, it can and will traverse
upwards looking for things at times. Adding this file ensures that if it does
so, it will stop before it starts trying to traverse even higher. There's no
need for anything in the file, as the contents are ignored.</p>
<p>I think this kind of change is an interesting way to look at the usability of
various systems. Adding an empty file here is <em>sort of</em> "more complex" than
say, Cargo. But it's also more explicit. Which means it can be more... I want
to say "legible", if you also read insufferable books like I do sometimes. And
therefore, easier. Anyway, more thoughts on all of this in the future.</p>
<p>Next, we have a change that is demonstrated by this example. Can you guess
what it is?</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; buck2 run //src/bin:hello_world
File changed: root//.git/modules/prelude
Build ID: 1a24b418-acdb-4d23-a5e1-6e9b644c01e6
Jobs completed: 83. Time elapsed: 1.9s. Cache hits: 0%. Commands: 3 (cached: 0, remote: 0, local: 3)
</code></pre>
<p>There's no reason for buck to be watching our <code>.git</code> directory for changes. And
that's why <a href="https://github.com/facebook/buck2/commit/2083261102e867bce2b6385b78b598e758b424a8">it is now put in the default
configuration</a>
when you <code>buck2 init --git</code>. But we've gotta make that change ourselves. Open
up your <code>.buckconfig</code> and add this at the bottom:</p>
<pre><code>[project]
ignore = .git
</code></pre>
<p>We want it to ignore the <code>.git</code> directory. Seems good.</p>
<p>... I lied though, there's one more improvement we want to make: we also don't
want buck to bother listening to the <code>target</code> directory either, as those files
are for Cargo's output. So what we <em>actually</em> want is this:</p>
<pre><code>[project]
ignore = .git, target
</code></pre>
<p>After doing that we'll want to <code>buck2 kill</code> to shut the daemon down, so that
it can pick up our new configuration on the next boot.</p>
<h2>Regenerating reindeer</h2>
<p>Since we've got new bugfixes in Reindeer too, let's regenerate our config
for our dependencies:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; reindeer --third-party-dir third-party buckify
[WARN reindeer::fixups] semver-1.0.17 has a build script, but I don't know what to do with it: Unresolved build script at ..\..\..\..\.cargo\registry\src\github.com-1ecc6299db9ec823\semver-1.0.17\build.rs. Dependencies:
</code></pre>
<p>We still have to deal with the build script! We didn't talk about the
contents of <code>third-party\BUCK</code> last time, and we won't this time either. If you
want to see what's changed, you can take a peek though. One change that we
didn't explicitly talk about before, but you may end up noticing, is that it did
not generate a target to try and build our build script.</p>
<p>Let's try it out now:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; buck2 run //src/bin:hello_world
File changed: root//BUCK
File changed: root//third-party/BUCK
File changed: root//third-party
Build ID: 2eb0c121-d1fb-43d2-b8a4-f923d8dda657
Jobs completed: 20. Time elapsed: 1.1s. Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2)
</code></pre>
<p>Nice. Also note that we aren't seeing any more <code>.git</code> or <code>target</code> changes, not
that we've run anything that would inherently change those files, but go ahead,
invoke Cargo or git, and then build again. You shouldn't see notifications about
those directories anymore.</p>
<h2>Fixing some breakage</h2>
<p>Speaking of invoking Cargo, remember how I said this in the last post?</p>
<blockquote>
<p>We do have two different Cargo.tomls now. That is a bit of a bummer. But at
least it is easy to determine if there’s a problem: dependency failures are
loud, and if you’re building with both in CI, you’ll notice if stuff goes wrong.
There also may be a solution to this I’m just not aware of.</p>
</blockquote>
<p>Well...</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; cargo run
Compiling hello_world v0.1.0 (C:\Users\steve\Documents\GitHub\buck-rust-hello)
error[E0432]: unresolved import `semver`
--&gt; src\bin\main.rs:1:5
|
1 | use semver::{BuildMetadata, Prerelease, Version, VersionReq};
| ^^^^^^ use of undeclared crate or module `semver`
For more information about this error, try `rustc --explain E0432`.
error: could not compile `hello_world` due to previous error
</code></pre>
<p>Going back and re-reading my last post, I did have a <code>cargo add semver</code> in
there, so maybe I just forgot to commit that in my last post. Just in case,
we'll fix that:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; cargo add semver
Updating crates.io index
Adding semver v1.0.17 to dependencies.
Features:
+ std
- serde
</code></pre>
<p>With that, <code>cargo build</code> and <code>cargo run</code> are back in business.</p>
<p>We also have... well "breakage" isn't exactly right, but we have a buck
configuration issue. Let's try to build every target:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; buck2 build ...
Error running analysis for `root//:build (prelude//platforms:default#fb50fd37ce946800)`
Caused by:
0: Error looking up configured node root//:build (prelude//platforms:default#fb50fd37ce946800)
1: `root//src/bin:hello_world` is not visible to `root//:build` (run `buck2 uquery --output-attribute visibility root//src/bin:hello_world` to check the visibility)
Build ID: 51657a07-112c-46b4-a2eb-91d60a4b0aed
Jobs completed: 8. Time elapsed: 0.0s.
BUILD FAILED
</code></pre>
<p>We didn't declare that our binary was visible anywhere, and so when we try and
build it, it isn't happy. We do want this to be public, so change <code>src\bin\BUCK</code>
by adding this <code>visibility</code> line near the end. It should look like this:</p>
<pre><code>rust_binary(
name = "hello_world",
srcs = ["main.rs"],
crate_root = "main.rs",
deps = [
"//third-party:semver",
],
visibility = ["PUBLIC"],
)
</code></pre>
<p>And now that will work:</p>
<pre><code>~\Documents\GitHub\buck-rust-hello&gt; buck2 build ...
File changed: root//src/bin/BUCK
Build ID: 9da23da4-3f99-4e78-8c58-ae5e2f1facaa
Jobs completed: 25. Time elapsed: 0.6s. Cache hits: 0%. Commands: 3 (cached: 0, remote: 0, local: 3)
BUILD SUCCEEDED
</code></pre>
<p>Okay, now that we've fixed both Cargo <em>and</em> buck... let's make sure that isn't
gonna happen again. We aren't testing any of this. So it broke. Just like I said
it was easy to not let break. Sigh.</p>
<p>We're going to use GitHub Actions because this is already on GitHub. I'm sure
you can adapt it to your setup of choice.</p>
<p>Put this in <code>.github\workflows\ci.yml</code>:</p>
<pre><code>name: CI
on:
pull_request:
push:
branches: [main]
jobs:
test:
name: ${{matrix.name || format('Rust {0}', matrix.rust)}}
runs-on: ${{matrix.os || 'ubuntu'}}-latest
strategy:
fail-fast: false
matrix:
include:
- rust: nightly
- rust: beta
- rust: stable
- rust: 1.69.0
- name: Cargo on macOS
rust: nightly
os: macos
- name: Cargo on Windows (msvc)
rust: nightly-x86_64-pc-windows-msvc
os: windows
steps:
- uses: actions/checkout@v3
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{matrix.rust}}
- run: cargo check
- run: cargo run
buck:
name: Buck2 on ${{matrix.os == 'ubuntu' &amp;&amp; 'Linux' || matrix.os == 'macos' &amp;&amp; 'macOS' || matrix.os == 'windows' &amp;&amp; 'Windows' || '???'}}
runs-on: ${{matrix.os}}-latest
if: github.event_name != 'pull_request'
strategy:
matrix:
os: [ubuntu, macos, windows]
steps:
- uses: actions/checkout@v3
with:
submodules: true
- uses: dtolnay/rust-toolchain@stable
with:
components: rust-src
- uses: dtolnay/install-buck2@latest
- run: buck2 build ...
- run: buck2 run //src/bin:hello_world
- uses: actions/cache/restore@v3
id: cache
with:
path: ~/.cargo/bin/reindeer${{matrix.os == 'windows' &amp;&amp; '.exe' || ''}}
key: ${{matrix.os}}-reindeer
- run: cargo install --git https://github.com/facebookincubator/reindeer reindeer
if: steps.cache.outputs.cache-hit != 'true'
- uses: actions/cache/save@v3
if: steps.cache.outputs.cache-hit != 'true'
with:
path: ~/.cargo/bin/reindeer${{matrix.os == 'windows' &amp;&amp; '.exe' || ''}}
key: ${{steps.cache.outputs.cache-primary-key}}
- run: reindeer buckify
working-directory: third-party
if: matrix.os == 'ubuntu'
- name: Check reindeer-generated BUCK file up to date
run: git diff --exit-code
if: matrix.os == 'ubuntu'
</code></pre>
<p>This is... you guessed it, based off of <a href="https://github.com/dtolnay/cxx/blob/master/.github/workflows/ci.yml">dtolnay's CI for
cxx</a>.
What can I say, he writes good code. This version is a bit stripped down, since
this is primarily a project for showing off a build system, rather than a
regular project. This has:</p>
<ul>
<li>Builds with Cargo on Linux against nightly, beta, stable, and 1.69.0 as a sort of MSRV check.</li>
<li>Builds with Cargo on MacOS and Windows against nightly Rust.</li>
<li>Buck on Linux, MacOS and Windows against stable Rust.</li>
<li>On Linux, checks that we have comitted any relevant changes to dependencies.</li>
</ul>
<p>This does not have:</p>
<ul>
<li>A "fail if there are any warnings" build</li>
<li>A clippy build</li>
<li>A full matrix of every os/language/build system combination</li>
</ul>
<p>or other things you may want out of a build. Again, you'll probably want to
customize this heavily, but this is what I'm going to do here.</p>
<h2>Wrapping up</h2>
<p>And with that, we are done! Next post we're going to deal with that build
script. And use some crates that have more intense dependencies, and get them
all working.</p>
<p>As always, you can <a href="https://github.com/steveklabnik/buck-rust-hello/commit/914e3aa7cdc269b2536f8118255eae12bf552f8d">check out the code</a> at this point if you'd like.</p>
</content:encoded></item><item><title>Using Crates.io with Buck</title><link>https://steveklabnik.com/writing/using-cratesio-with-buck/</link><guid isPermaLink="true">https://steveklabnik.com/writing/using-cratesio-with-buck/</guid><pubDate>Thu, 27 Apr 2023 00:00:00 GMT</pubDate><content:encoded><p>In <a href="using-buck-to-build-rust-projects">a previous post</a>, I laid out the basics
of building a Rust project with <a href="https://buck2.build/">buck2</a>. We compared and
contrasted it with Cargo. But what about one of the biggest and best features
that Cargo has to offer, the ability to use other Rust packages from crates.io?
They don't use buck, so how can we integrate them into our build?</p>
<h2>A series</h2>
<p>This post is part of a series:</p>
<ul>
<li><a href="using-buck-to-build-rust-projects">Using buck to build Rust projects</a></li>
<li><a href="#">Using Crates.io with Buck</a> (you are here)</li>
<li><a href="updating-buck">Updating Buck</a></li>
</ul>
<p>This post represents how to do this at the time that this was posted; future
posts may update or change something that happens here. Here's a hopefully
complete but possibly incomplete list of updates and the posts that talk
about it:</p>
<ul>
<li><code>build-script-build</code> target is no longer generated, see "Updating Buck"</li>
</ul>
<h2>Depending on semver</h2>
<p>Let's use the <code>semver</code> package example as our program. I am choosing this for
a few reasons:</p>
<ul>
<li>I used to be involved in maintaining it, and I'm sentimental</li>
<li>It's authored by dtolnay, who authors many great Rust crates.</li>
<li>More specifically than that, his excellent <a href="https://github.com/dtolnay/cxx/">cxx library</a> maintains build rules for Cargo, Buck, and Bazel, and examining how he uses Buck in cxx helped me write both the last post and this one. I wanted to make sure to shout that out.</li>
<li>It's pure Rust (this is easier than something that depends on C, since as I mentioned I'm still having an issue or two with getting my own C toolchain working so far).</li>
<li>It has one optional dependecy on serde, but no others. Again, this is just easier, not a fundamental limitation.</li>
</ul>
<p>Here's the example, from <a href="https://crates.io/crates/semver">the page on crates.io</a>:</p>
<pre><code>use semver::{BuildMetadata, Prerelease, Version, VersionReq};
fn main() {
let req = VersionReq::parse("&gt;=1.2.3, &lt;1.8.0").unwrap();
// Check whether this requirement matches version 1.2.3-alpha.1 (no)
let version = Version {
major: 1,
minor: 2,
patch: 3,
pre: Prerelease::new("alpha.1").unwrap(),
build: BuildMetadata::EMPTY,
};
assert!(!req.matches(&amp;version));
// Check whether it matches 1.3.0 (yes it does)
let version = Version::parse("1.3.0").unwrap();
assert!(req.matches(&amp;version));
}
</code></pre>
<p>Let's change <code>src/bin/main.rs</code> to contain this code. What do we need to do
to add this with Cargo?</p>
<pre><code>buck-rust-hello〉cargo add semver
Adding semver v1.0.17 to dependencies.
Features:
+ std
- serde
buck-rust-hello〉cargo run
Compiling semver v1.0.17
Compiling hello_world v0.1.0 (C:\Users\steve\Documents\GitHub\buck-rust-hello)
Finished dev [unoptimized + debuginfo] target(s) in 0.71s
Running `target\debug\main.exe`
buck-rust-hello〉
</code></pre>
<p>Easy enough! We expect no output, becuase the asserts should pass. (Note that
we didn't ask for the <code>serde</code> feature, so we aren't using it, and therefore
don't depend on anything other than <code>semver</code>.)</p>
<h2>How Cargo handles this</h2>
<p>Before we move on, let's talk for a moment about what Cargo actually does here.</p>
<p>First, <code>cargo add</code> has added this to our <code>Cargo.toml</code>:</p>
<pre><code>[dependencies]
semver = "1.0.17"
</code></pre>
<p>This is the latest release of semver at the time of this writing.</p>
<p>When we <code>cargo build</code>, Cargo will figure out all of the crates we need. It will
then check the cache for the source code for that crate, and if it's not there,
download it. On my system, this lives here:</p>
<pre><code> ~\.cargo\registry\src\github.com-1ecc6299db9ec823\semver-1.0.17\
</code></pre>
<p>This means we have a <em>global</em> cache of source code.</p>
<p>Cargo will then compile <code>semver</code>, as you can see in the output above. It places
that output in the <code>target</code> directory, more specifically</p>
<pre><code>.\target\debug\deps
</code></pre>
<p>This directory will contain <code>libsemver-ded1559592aad8f7.rlib</code>,
<code>libsemver-ded1559592aad8f7.rmeta</code>, and <code>semver-ded1559592aad8f7.d</code>. These have
hashes embedded in them so that if we had multiple versions of <code>semver</code> in our
project, they can be disambiguated. If you're not familiar with <code>rustc</code> output:</p>
<ul>
<li>rlib files are simlar to archive files, like Unix <code>ar</code>. The details are not standard, and may change at any time. Object code lives in here, as well as other things.</li>
<li>rmeta files contain crate metadata. This can be used for <code>cargo check</code>, for example.</li>
<li><code>.d</code> files are a dependency file. This format comes from gcc/make, and is sorta standardized. This is in theory useful for use with other build systems, but we won't be using this today.</li>
</ul>
<p>Cargo will then build our project, passing in <code>libsemver-*.rlib</code> as a
dependency.</p>
<p>If you're curious about the exact commands and flags, <code>cargo build -v</code> will
show that. Make sure to <code>cargo clean</code> first, or else you'll get no output, given
that we've already built this project.</p>
<p>For example, here's the <code>rustc</code> invocation that Cargo makes for building this step:</p>
<pre><code>rustc --crate-name main --edition=2021 src\bin\main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=c398fba113f38e08 --out-dir C:\Users\steve\Documents\GitHub\buck-rust-hello\target\debug\deps -C incremental=C:\Users\steve\Documents\GitHub\buck-rust-hello\target\debug\incremental -L dependency=C:\Users\steve\Documents\GitHub\buck-rust-hello\target\debug\deps --extern hello_world=C:\Users\steve\Documents\GitHub\buck-rust-hello\target\debug\deps\libhello_world-c5b898f1ca338cca.rlib --extern semver=C:\Users\steve\Documents\GitHub\buck-rust-hello\target\debug\deps\libsemver-ded1559592aad8f7.rlib
</code></pre>
<p>Since we are not using Cargo, we need to replace all of that stuff, and get
Buck to generate that <code>rustc</code> line, or at least, some equivalent of it.</p>
<p>Let's talk about the various things we need to do:</p>
<h3>The mismatch</h3>
<p>Let's start with one thing that is basically the same: <code>buck-out</code> and <code>target</code>
are both directories in our project that cache the output of our build. Yeah
the name and details are different, but we're not going to try and somehow
unify these, as they're both considered implementaiton details of the
respective systems, and trying to get them to share is a lot of work for not
much gain.</p>
<p>Buck does not have a central registry of packages that we can download code
from.</p>
<p>Buck is interested in reproducable builds, and therefore, a global cache of
source code doesn't make as much sense. You want the code stored locally, with
your project. The dreaded (or beloved) vendoring.</p>
<p>Buck does not understand the crate index, Cargo configuration for a given
package, and other implementation details. As a more general build system,
those are pretty much out of scope.</p>
<p>Lucikly, other people have done a lot of work here.</p>
<h3>Reindeer</h3>
<p>Enter <a href="https://github.com/facebookincubator/reindeer/">Reindeer</a>. Reindeer is a
project that will help us bridge this gap. Here's how this will work: Reindeer
will create and (mostly) manage a <code>third-party</code> directory for us. It will
generate <code>BUCK</code> files that we can then use to depend on these external crates.
We can even choose to vendor our sources or not.</p>
<p>You can install reindeer through Cargo:</p>
<pre><code>&gt; cargo install --git https://github.com/facebookincubator/reindeer/ reindeer
</code></pre>
<p>Let's set this up. First off, we need to create some files:</p>
<pre><code>&gt; mkdir third-party
&gt; code third-party\Cargo.toml # or vim, or whatever
</code></pre>
<p>In that <code>Cargo.toml</code>, we'll need to put this:</p>
<pre><code>[workspace]
[package]
name = "rust-third-party"
version = "0.0.0"
publish = false
edition = "2021"
# Dummy target to keep Cargo happy
[[bin]]
name = "fake"
path = "/dev/null"
[dependencies]
semver = "1.0.17"
</code></pre>
<p>We're creating a fake package here, so nothing truly matters except the
<code>[dependencies]</code> section. Here we depend on <code>semver</code> as usual.</p>
<p>We're also going to configure Reindeer to not vendor our source code, because
that's how I prefer to do things. Vendoring was the default behavior, with
non-vendoring being added very recently, so if you prefer to vendor, that
workflow works very well</p>
<p>Anyway put this in <code>third-party/reindeer.toml</code>:</p>
<pre><code>vendor = false
</code></pre>
<p>So run this inside of your <code>third-party</code> directory:</p>
<pre><code>〉cd third-party
〉reindeer buckify
[WARN reindeer::fixups] semver-1.0.17 has a build script, but I don't know what to do with it: Unresolved build script at vendor\semver\build.rs. Dependencies:
</code></pre>
<p>Oh no, build scripts! Yeah I said that I picked <code>semver</code> because it should be easy,
but it does have a build script, which is another Cargo-specific feature. Now,
the <code>semver</code> crate's build script is used as a feature to support older versions
of the compiler; all it does is detect old versions and then spit out some
configuration to make sure to not use the newer features of the language. This
is why this is a warning, not an error; in this case, we don't actually need
the build script since we are using a new compiler. So, we are going to skip this
<em>for now</em>, but we'll come back and fix it.</p>
<p>At this point, <code>reindeer</code> has generated a <code>BUCK</code> file for <code>semver</code>. Let's see
what targets we have now:</p>
<pre><code>〉buck2 targets //...
Jobs completed: 4. Time elapsed: 0.0s.
root//:build
root//src/bin:hello_world
root//src/lib:hello_world
root//third-party:semver
root//third-party:semver-1.0.17
root//third-party:semver-1.0.17-build-script-build
</code></pre>
<p>We have a few new ones! One for <code>semver</code>, one for the specific version of
<code>semver</code>, and one for the build script of <code>semver</code>. The general <code>semver</code> is
an alias for <code>semver-1.0.17</code>.</p>
<p>Do you know how to modify our build so that buck builds successfully?</p>
<p>Here's the answer: change <code>src\bin\BUCK</code>:</p>
<pre><code>rust_binary(
name = "hello_world",
srcs = ["main.rs"],
crate_root = "main.rs",
deps = [
"//third-party:semver",
],
)
</code></pre>
<p>And now we can build and run:</p>
<pre><code>〉buck2 run //src/bin:hello_world
File changed: root//src/bin/BUCK
Build ID: b18ba58d-8a77-439a-9d95-6051f3cf21d4
Jobs completed: 26. Time elapsed: 1.9s. Cache hits: 0%. Commands: 3 (cached: 0, remote: 0, local: 3)
</code></pre>
<p>Success! Our program has no output, if the assertions failed we would have
gotten something, but this is expected given the example code.</p>
<p>Now, whenever we need to add or remove a dependency, we can modify
<code>third-party\Cargo.toml</code>, re-run the <code>buckify</code> command, and we're good.</p>
<p>We <em>do</em> have two different <code>Cargo.toml</code>s now. That is a bit of a bummer. But
at least it is easy to determine if there's a problem: dependency failures are
loud, and if you're building with both in CI, you'll notice if stuff goes wrong.
There also may be a solution to this I'm just not aware of.</p>
<p>If you'd like to see the whole thing at this point, <a href="https://github.com/steveklabnik/buck-rust-hello/commit/2abd1ada7dbbc7f89cd8678eace1e07b3df2ae2f">this
commit</a>
should have you covered.</p>
<p>This should get you going with building more advanced projects in Rust using
buck2. In the next post, we'll talk about fixups, which you'll need for
dependencies that are more complex than <code>semver</code>.</p>
<p>By the way, if you're interested in this stuff, I've <a href="https://discord.gg/ZTEmwypZ6K">made a Discord for buck
fans</a>. Come hang out, chat about stuff, ask
questions, it's gonna be a good time.</p>
</content:encoded></item><item><title>Using buck to build Rust projects</title><link>https://steveklabnik.com/writing/using-buck-to-build-rust-projects/</link><guid isPermaLink="true">https://steveklabnik.com/writing/using-buck-to-build-rust-projects/</guid><pubDate>Thu, 13 Apr 2023 00:00:00 GMT</pubDate><content:encoded><p>A few days ago, Facebook/Meta/idk <a href="https://engineering.fb.com/2023/04/06/open-source/buck2-open-source-large-scale-build-system/">announced that buck2 is now open
source</a>.</p>
<blockquote>
<p>Buck2 is an extensible and performant build system written in Rust and
designed to make your build experience faster and more efficient.</p>
</blockquote>
<p>As it just so happens, I have taken an increasing interest in build systems
lately. I have mostly ignored the cool kids' build system things, because
I have never worked at a FAANG, or at a true monorepo shop. I also personally
try and avoid the JVM wherever possible, and the first generation of these tools
were all built on top of it. (Yes, that bias may be outdated, I'm not saying
<em>you</em> should avoid the JVM, just stating my own bias up front.)</p>
<p>So this timing was perfect! Let's explore what using buck looks like.</p>
<blockquote>
<p>A brief aside: listen, I have no love for Facebook. I <a href="im-deleting-my-facebook-tonight">deleted my
account</a> almost a dozen years ago. That
doesn't mean I'm not interested in using good tools they produce. If you
feel differently, fine, but that's not what I want to talk about today, so
I'm not going to.</p>
</blockquote>
<p>Oh, one last bit before I begin: I'm not going to do a lot of motivating on
"why would you want to use Buck?" in this post. There's a few reasons for that,
but for now, if that's what you're looking for, this post isn't it. We're doing
this purely for the novelty of trying out some new tech right now. I will
probably end up with a post giving better motivations at some point in the
future, but I think it makes more sense once you see how it works, rather than
starting there.</p>
<h2>A series</h2>
<p>This post is part of a series:</p>
<ul>
<li><a href="#">Using buck to build Rust projects</a> you are here</li>
<li><a href="using-cratesio-with-buck">Using Crates.io with Buck</a></li>
<li><a href="updating-buck">Updating Buck</a></li>
</ul>
<p>This post represents how to do this at the time that this was posted; future
posts may update or change something that happens here. Here's a hopefully
complete but possibly incomplete list of updates and the posts that talk
about it:</p>
<ul>
<li><code>buck2 init</code> also creates a file named <code>.buckroot</code>, see "Updating Buck"</li>
</ul>
<h2>Getting started with buck2</h2>
<p><a href="https://buck2.build/docs/getting_started/">The Getting Started page</a> will give
you instructions on installing buck. As of this moment, the instructions are:</p>
<pre><code>$ rustup install nightly-2023-01-24
$ cargo +nightly-2023-01-24 install --git https://github.com/facebook/buck2.git buck2
</code></pre>
<p>This is mega-convenient for me as a Rust user, but probably not if you don't
have Rust installed. That said, this is a first release, and so I don't expect
anything fancier. This is what <code>cargo install</code> is good for!</p>
<p>Let's make a new directory, <code>hello</code>:</p>
<pre><code>$ mkdir buck-rust-hello
$ cd buck-rust-hello
</code></pre>
<p>To initialize a project, we use this command:</p>
<pre><code>$ buck2 init --git
</code></pre>
<p>Before we move forward, let's examine what this generated for us.</p>
<h2>Initial project files</h2>
<p>We now have this stuff in our directory:</p>
<pre><code>$ git add .
$ git status
On branch main
No commits yet
Changes to be committed:
(use "git rm --cached &lt;file&gt;..." to unstage)
new file: .buckconfig
new file: .gitignore
new file: .gitmodules
new file: BUCK
new file: prelude
new file: toolchains/BUCK
</code></pre>
<p>Let's talk about each of these in turn.</p>
<h3><code>.buckconfig</code></h3>
<p>The <code>.buckconfig</code> file is... a configuration file for Buck, go figure. It looks
like this:</p>
<pre><code>[repositories]
root = .
prelude = prelude
toolchains = toolchains
none = none
[repository_aliases]
config = prelude
fbcode = none
fbsource = none
buck = none
[parser]
target_platform_detector_spec = target:root//...-&gt;prelude//platforms:default
</code></pre>
<p>That <code>none = none</code> is kind of amusing. Regardless of that, this file is
extremely important: it configures the entire thing. In a sense, it's like
<code>Cargo.toml</code>: just like a package is defined by the existence of a <code>Cargo.toml</code>,
a <code>.buckconfig</code> defines the existence of a... cell. Which defines a package.
We'll get there. Point is, this is the top level configuration. We say that
the repository root is in the current directory, we'd like to use the default
prelude and toolchains.</p>
<p>I uh... I don't know what the <code>none = none</code> is for. It might just be a bug.
I haven't seen it in some of the other config files I've poked at. Let's just
leave that alone for now. I do have a suspicion though... and it involves the
next section.</p>
<p>We also have a table for repository aliases. I couldn't find any documentation
on this, but I would imagine this means we could use the name <code>config</code> instead
of <code>prelude</code> later. Looks like we don't have any way to refer to <code>fbcode</code> and
<code>fbsource</code>, which makes sense, and same with <code>buck</code>.</p>
<p>(I wonder if this is what the <code>none = none</code> is about above, defining a sort of
"none repository" that we can then alias these to.)</p>
<p>Finally, we have a parser table, with one entry, pointing out where a thing
exists. I know this configures Buck's parser, but other than that... I'm sure
I'll figure it out eventually.</p>
<h3><code>.gitmodules</code> &amp; <code>prelude</code></h3>
<p>We have a git submodule, pointing to
<code>https://github.com/facebook/buck2-prelude.git</code>,
that lives at the <code>prelude</code> directory. If you poke around in there, you'll
find a bunch of <code>.bzl</code> files that implement useful features for us to use.
We'll get into those in a moment, but the point is that this is sort of like
a 'standard library' if you will. You could also not use it and define your own.
If you're that kind of person.</p>
<h3><code>.gitignore</code></h3>
<p>A very simple <code>.gitignore</code> will be created, that contains one line: <code>/buck-out</code>.
This is where buck stores artifacts produced by your builds, so we don't want
that checked into version control.</p>
<h3><code>BUCK</code></h3>
<p>Now we get to the good stuff. Here's the generated <code>BUCK</code> file:</p>
<pre><code># A list of available rules and their signatures can be found here: https://buck2.build/docs/generated/starlark/prelude/prelude.bzl
genrule(
name = "hello_world",
out = "out.txt",
cmd = "echo BUILT BY BUCK2&gt; $OUT",
)
</code></pre>
<p><code>genrule</code> is like a function, provided by our prelude. If you're curious, the
implementation is in <code>prelude/genrule.bzl</code>. This command, as you may imagine,
sets up a rule, named <code>hello_world</code>, that produces a file called <code>out.txt</code>.
It does this by running the <code>cmd</code>. Nice and straightforward. We'll give that
a try in a moment, but first, one last file:</p>
<h3><code>toolchains/BUCK</code></h3>
<p>This file describes a toolchain. Here's the contents:</p>
<pre><code>load("@prelude//toolchains:genrule.bzl", "system_genrule_toolchain")
system_genrule_toolchain(
name = "genrule",
visibility = ["PUBLIC"],
)
</code></pre>
<p>This loads a certain rule from the prelude, and then defines this as a public
toolchain. We can define as many toolchains as we want here, for example, if
we wanted to build both Rust and Python, we could define both toolchains here
for later use.</p>
<p>The "genrule" toolchain is used to generate files from a shell command, as we
saw before with our rule that produces <code>out.txt</code>. So, in my understanding, here
we are defining that we wish to actually use that. And then, in the <code>BUCK</code> file,
we're using this toolchain to implement our rule.</p>
<h2>Invoking our first build</h2>
<p>Okay, let's actually give this a shot. To instruct Buck to build something, we
invoke it with the "target pattern" as an argument. Let's ask Buck what targets
it knows how to build. To do this:</p>
<pre><code>C:\Users\steve\Documents\GitHub\buck-rust-hello〉buck2 targets //... 04/10/2023 02:01:50 PM
Build ID: cd778a29-2ba4-484b-8956-dc67f6fc0625
Jobs completed: 4. Time elapsed: 0.0s.
root//:hello_world
</code></pre>
<p>The <code>//...</code> is a "target pattern." The <code>/...</code> means "all build targets in build
files in subdirectories", and <code>/</code> means our root directory, so <code>//...</code> means
"all targets in all build files in all subdirectories." By passing this target
to <code>buck2 targets</code>, we can see every target in the project. This shows our
one target we've defined, <code>root://:hello_world</code>. This name was defined in our
<code>BUCK</code> file above. If you change that to</p>
<pre><code>genrule(
name = "lol",
</code></pre>
<p>then <code>buck2 targets //...</code> would show <code>root://:lol</code>.</p>
<p>Let's actually build our target:</p>
<pre><code>〉buck2 build //:hello_world
File changed: root//BUCK
Build ID: 73f4b797-2238-47bc-8e43-7ffcb2b7d9b7
Jobs completed: 36. Time elapsed: 0.0s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)
BUILD SUCCEEDED
</code></pre>
<p>Okay the build succeeded, but where is our <code>out.txt</code>? We can ask buck!</p>
<pre><code>〉buck2 build //:hello_world --show-output
Build ID: 7ce93845-ab1e-4b0a-9274-51fed9f9e295
Jobs completed: 3. Time elapsed: 0.0s.
BUILD SUCCEEDED
root//:hello_world buck-out\v2\gen\root\fb50fd37ce946800\__hello_world__\out\out.txt
</code></pre>
<p>It lives in a deeply nested subdirectory of <code>buck-out</code>, a new top-level directory
that was created for us. If you remember from before, this directory is ignored
in our <code>.gitignore</code>.</p>
<p>If we look at the file, you can see it contains the text we wanted it to
contain.</p>
<p>Let's build a second time!</p>
<pre><code>〉buck2 build //:hello_world
File changed: root//.git/index.lock
File changed: root//.git
File changed: root//.git/modules/prelude/index.lock
31 additional file change events
Build ID: c00e4bfa-a1f8-40c7-a61c-2a394dca5da5
Jobs completed: 5. Time elapsed: 0.0s.
BUILD SUCCEEDED
</code></pre>
<p>Buck has noticed that we've changed some files, but since our rule doesn't
depend on any of them, we're good to go.</p>
<h2>Building some Rust code</h2>
<p>Okay, <code>echo</code> to a file is fun, but let's actually build some Rust. Create a
file, <code>hello.rs</code>:</p>
<pre><code>fn main() {
println!("Hello, world!");
}
</code></pre>
<p>and then update the <code>BUCK</code> file to this:</p>
<pre><code>rust_binary(
name = "hello_world",
srcs = ["hello.rs"],
crate_root = "hello.rs",
)
</code></pre>
<p>This says "hey, we're building a Rust binary, it has this target name, these
source files, and the crate root lives here." Given we only have one file,
there's some reptition. It happens. Let's build:</p>
<pre><code>〉buck2 build //:hello_world
File changed: root//BUCK
Error running analysis for `root//:hello_world (prelude//platforms:default#fb50fd37ce946800)`
Caused by:
0: Error looking up configured node root//:hello_world (prelude//platforms:default#fb50fd37ce946800)
1: Error looking up configured node toolchains//:cxx (prelude//platforms:default#fb50fd37ce946800) (prelude//platforms:default#fb50fd37ce946800)
2: looking up unconfigured target node `toolchains//:cxx`
3: Unknown target `cxx` from package `toolchains//`.
Did you mean one of the 1 targets in toolchains//:BUCK?
Build ID: f126ce07-efe8-41d3-8aae-8b7d31a4dafc
Jobs completed: 4. Time elapsed: 0.0s.
BUILD FAILED
</code></pre>
<p>Oops! We didn't set up a rust toolchain! Let's do that now. Edit
<code>toolchains/BUCK</code>:</p>
<pre><code>load("@prelude//toolchains:rust.bzl", "system_rust_toolchain")
system_rust_toolchain(
name = "rust",
default_edition = "2021",
visibility = ["PUBLIC"],
)
</code></pre>
<p>And.... when we build again, the same error. Now. I am not 100% sure what's
going on here, but this is what we need to do:</p>
<pre><code>load("@prelude//toolchains:rust.bzl", "system_rust_toolchain")
load("@prelude//toolchains:genrule.bzl", "system_genrule_toolchain")
load("@prelude//toolchains:cxx.bzl", "system_cxx_toolchain")
load("@prelude//toolchains:python.bzl", "system_python_bootstrap_toolchain")
system_genrule_toolchain(
name = "genrule",
visibility = ["PUBLIC"],
)
system_rust_toolchain(
name = "rust",
default_edition = "2021",
visibility = ["PUBLIC"],
)
system_cxx_toolchain(
name = "cxx",
visibility = ["PUBLIC"],
)
system_python_bootstrap_toolchain(
name = "python_bootstrap",
visibility = ["PUBLIC"],
)
</code></pre>
<p>I <em>believe</em> that this is because, to compile the Rust compiler, we need
Python and a C++ compiler. Well, I <em>did</em> believe that, but after digging into
things some more, it's that the Rust toolchain from the prelude depends on
the CXX toolchain in the prelude, because the Rust toolchain invokes the C
compiler to invoke the linker. I'm still not 100% sure why Python needs to be
in there. Anyway.</p>
<p>Now, when I run, I got this:</p>
<pre><code>〉buck2 build //:hello_world -v 3
Action failed: prelude//python_bootstrap/tools:win_python_wrapper (symlinked_dir resources)
Internal error: symlink(original=../../../../../../../../../prelude/python_bootstrap/tools/win_python_wrapper.bat, link=C:\Users\steve\Documents\GitHub\buck-rust-hello\buck-out\v2\gen\prelude\fb50fd37ce946800\python_bootstrap\tools\__win_python_wrapper__\resources\win_python_wrapper.bat): A required privilege is not held by the client. (os error 1314)
Build ID: 57a66885-f7e7-474b-a782-b49fc4425be9
Jobs completed: 14. Time elapsed: 0.0s.
BUILD FAILED
Failed to build 'prelude//python_bootstrap/tools:win_python_wrapper (prelude//platforms:default#fb50fd37ce946800)'
</code></pre>
<p>I got this becuase I'm on Windows, and Windows restricts the ability to create
symlinks by default. Turning on "Developer Mode" (which I'm surprised that I
haven't had to turn on so far yet), I get further:</p>
<pre><code>&lt;whole bunch of output&gt;
= note: 'clang++' is not recognized as an internal or external command,
operable program or batch file.
</code></pre>
<p>Here's that "invokes the compiler to get the linker" thing I was referring to
above.</p>
<p>Now... by default, the Rust support is for the GNU version of the Windows
toolchain. I never use that. Upstream has said that they want everything
to be supported, so that change will come at some point, maybe by the time
you read this! But in the meantime, I could get my (pure Rust) projects
building with two small patches:</p>
<pre><code>diff --git a/prelude/toolchains/cxx.bzl b/prelude/toolchains/cxx.bzl
index c57b7b8..dc14ca7 100644
--- a/prelude/toolchains/cxx.bzl
+++ b/prelude/toolchains/cxx.bzl
@@ -39,7 +39,7 @@ def _system_cxx_toolchain_impl(ctx):
CxxToolchainInfo(
mk_comp_db = ctx.attrs.make_comp_db,
linker_info = LinkerInfo(
- linker = RunInfo(args = ["clang++"]),
+ linker = RunInfo(args = ["link"]),
linker_flags = ["-fuse-ld=lld"] + ctx.attrs.link_flags,
archiver = RunInfo(args = ["ar", "rcs"]),
archiver_type = archiver_type,
diff --git a/prelude/toolchains/rust.bzl b/prelude/toolchains/rust.bzl
index 8172090..4545d55 100644
--- a/prelude/toolchains/rust.bzl
+++ b/prelude/toolchains/rust.bzl
@@ -23,7 +23,7 @@ _DEFAULT_TRIPLE = select({
# default when we're able; but for now buck2 doesn't work with the MSVC
# toolchain yet.
"config//cpu:arm64": "aarch64-pc-windows-gnu",
- "config//cpu:x86_64": "x86_64-pc-windows-gnu",
+ "config//cpu:x86_64": "x86_64-pc-windows-msvc",
}),
})
</code></pre>
<p>Now a build works!</p>
<pre><code>〉buck2 build //:hello_world
File changed: root//BUCK
File changed: root//.git/index.lock
File changed: root//.git
6 additional file change events
Build ID: 65fc80aa-7bfa-433a-bfa7-57919147b550
Jobs completed: 65. Time elapsed: 1.0s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)
BUILD SUCCEEDED
</code></pre>
<p>We can run it to see the output:</p>
<pre><code>〉buck2 run //:hello_world
Build ID: 78b0ca23-2c7c-4c02-a161-bba15e3b38bd
Jobs completed: 3. Time elapsed: 0.0s.
hello world
</code></pre>
<p>Same idea as <code>cargo run</code>.</p>
<p>Speaking of <code>cargo run</code>, what might this look like with Cargo? Well, we can create
a <code>Cargo.toml</code>:</p>
<pre><code>[package]
name = "hello_world"
version = "0.1.0"
edition = "2021"
[[bin]]
name = "hello_world"
path = "hello.rs"
</code></pre>
<p>and try it out. Oh, and you'll probably want to put <code>target</code> into your
<code>.gitignore</code>.</p>
<p>Let's build. The "benchmark" command in nushell is sort of like <code>time</code> on a
UNIX system:</p>
<pre><code>〉benchmark { cargo build }
Compiling hello_world v0.1.0 (C:\Users\steve\Documents\GitHub\buck-rust-hello)
Finished dev [unoptimized + debuginfo] target(s) in 0.34s
416ms 490us 100ns
〉benchmark { cargo build }
Finished dev [unoptimized + debuginfo] target(s) in 0.00s
77ms 317us 200ns
</code></pre>
<p>Not too bad, a bit under half a second for the initial build, and near immediate
on a subsequent build. What about buck?</p>
<pre><code>〉benchmark { buck2 build //:hello_world -v 3 }
Running action: &lt;snip&gt;
Build ID: 47ebd9f1-3394-4f72-a0fb-02c936035d2b
Jobs completed: 58. Time elapsed: 0.8s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)
BUILD SUCCEEDED
945ms 557us
〉benchmark { buck2 build //:hello_world -v 3 }
Build ID: 5eed24e8-a540-454f-861a-855464aba3c9
Jobs completed: 3. Time elapsed: 0.0s.
BUILD SUCCEEDED
78ms 274us 100ns
</code></pre>
<p>Not too shabby; Buck is a <em>teeny</em> bit slower on the initial build, but when stuff
is cached, both are the same speed. All of this is fast enough to qualify as
"basically instant."</p>
<h2>Adding a library</h2>
<p>Let's up the complexity a bit, by adding a library that we want to depend on.
Here's a <code>lib.rs</code>:</p>
<pre><code>pub fn print_hello() {
println!("Hello, world!");
}
</code></pre>
<p>We want to change our top-level <code>BUCK</code> to add this:</p>
<pre><code>+rust_library(
+ name = "print_hello",
+ srcs = ["lib.rs"],
+ edition = "2021",
+ visibility = ["PUBLIC"],
+)
+
rust_binary(
name = "hello_world",
srcs = ["hello.rs"],
crate_root = "hello.rs",
+ deps = [
+ ":print_hello",
+ ],
)
</code></pre>
<p>Here, we make a new library, <code>print_hello</code>, and then make our binary depend
on it.</p>
<p>Let's change the code in <code>main.rs</code> to use the library:</p>
<pre><code>fn main() {
println!("hello world");
print_hello::print_hello();
}
</code></pre>
<p>And that's it! Let's examine our targets:</p>
<pre><code>〉buck2 targets //...
Build ID: 4646f2e7-0ea3-4d59-8590-3da0708ce96e
Jobs completed: 4. Time elapsed: 0.0s.
root//:hello_world
root//:print_hello
</code></pre>
<p>They're both there! We can now build one, the other, or everything:</p>
<pre><code># build everything
〉buck2 build //...
# build just the library
〉buck2 build //:print_hello
# build 'just' the binary, this will of course end up building the library too
〉buck2 build //:print_hello
</code></pre>
<p>Let's make sure it still prints our output:</p>
<pre><code>〉buck2 run //:hello_world
Build ID: d76c80fb-dd77-463a-86a1-b6443cea10f6
Jobs completed: 3. Time elapsed: 0.0s.
Hello, world!
</code></pre>
<p>Nice.</p>
<p>Let's compare that to Cargo. Modify <code>Cargo.toml</code>:</p>
<pre><code>[lib]
name = "print_hello"
path = "lib.rs"
</code></pre>
<p>and build:</p>
<pre><code>〉cargo run
Compiling hello_world v0.1.0 (C:\Users\steve\Documents\GitHub\buck-rust-hello)
Finished dev [unoptimized + debuginfo] target(s) in 0.42s
Running `target\debug\hello_world.exe`
Hello, world!
</code></pre>
<p>Nice.</p>
<h3>more cargo-ish</h3>
<p>Before we move forward, this isn't <em>exactly</em> an apples to apples comparison; we've been doing a
lot of configuration for Cargo that we normally wouldn't have to do, and also,
what if you already have a Cargo project, but you want to try out Buck with it?</p>
<p>Do this:</p>
<pre><code>&gt; mkdir src
&gt; mv hello.rs src/main.rs
&gt; mv lib.rs src/main.rs
</code></pre>
<p>And delete the configuration from <code>Cargo.toml</code>, leaving just the <code>package</code> table.
Finally, we need to change <code>src/main.rs</code>, given that we're using the default
crate name for the library crate, which is <code>hello_world</code> and not <code>print_hello</code>:</p>
<pre><code>fn main() {
hello_world::print_hello();
}
</code></pre>
<p>After this, <code>cargo build</code> works just fine. But what about Buck?</p>
<p>So. We have a bit of weirdness here, and I'm not sure if it's actually
work-around-able in Buck or not, since I'm still learning this myself. But
if we do the basic translation, we'll get an error. Let's try it. This is
how you modify the <code>BUCK</code> file:</p>
<pre><code>rust_library(
name = "hello_world",
srcs = glob(["src/**/*.rs"]),
edition = "2021",
visibility = ["PUBLIC"],
)
rust_binary(
name = "hello_world",
srcs = ["src/main.rs"],
crate_root = "src/main.rs",
deps = [
":print_hello",
],
)
</code></pre>
<p>Cargo produces a binary and a library, both called <code>hello_world</code>, but buck
doesn't like that:</p>
<pre><code>〉buck2 run //:hello_world
Error evaluating build file: `root//:BUCK`
Caused by:
Traceback (most recent call last):
* BUCK:8, in &lt;module&gt;
rust_binary(
error: Attempted to register target root//:hello_world twice, re-run the command with `--target-call-stacks` to obtain a call stack of the first registration
--&gt; BUCK:8:1
|
8 | / rust_binary(
9 | | name = "hello_world",
10 | | srcs = ["src/main.rs"],
11 | | crate_root = "src/main.rs",
12 | | deps = [
13 | | ":print_hello",
14 | | ],
15 | | )
| |_^
|
Build ID: d6a8925d-0180-4308-bcb9-fbc888e7eca1
Jobs completed: 4. Time elapsed: 0.0s.
BUILD FAILED
</code></pre>
<p>It's like hey! You have two targets named <code>hello_world</code>! That's confusing!
It also reveals a difference between Buck and Cargo. With Cargo, if you remember
our configuration, we had to point it to the crate root. From there, Cargo just
leans on <code>rustc</code> to load up all of the other files that may be required if you
have a bunch of modules. But with Buck, we need to tell it up front which files we use.
So as you can see above:</p>
<pre><code> srcs = glob(["src/**/*.rs"]),
</code></pre>
<p>We can use the <code>glob</code> command to glob up all of our files, which is nice, but
it's... it's actually wrong. We want to glob everything <em>except</em> <code>main.rs</code>.
If <code>main.rs</code> were to change, this would try and re-build both the binary and
the library, in my understanding. So that's annoying.</p>
<p>It's not just annoying for Buck, though. Having both a <code>src/main.rs</code> and a
<code>src/lib.rs</code> has led to so much confusion from beginners over the years. At some
point, someone puts <code>mod lib;</code> into <code>src/main.rs</code> and everything goes to hell.
The original intention of this layout, to make simple things simple, is a good
idea, but I think that sadly, we missed the mark here. Luckily, Cargo also
supports a <code>bin</code> directory:</p>
<pre><code>&gt; mkdir src/bin
&gt; mv src/main.rs src/bin
</code></pre>
<p>We can still <code>cargo run</code> and <code>cargo build</code> and this all works just fine.
This doesn't fix our glob issue, though, because <code>src/bin</code> is still inside
of <code>src</code>.</p>
<p>I <em>think</em>, in the abstract, I'd prefer a layout like <code>src/{lib,bin}</code>. You want
things to not really be nested. So let's do that. Both Cargo and Buck can
handle it! It's just not as nice as being purely default in Cargo, since that
convention is so strong.</p>
<pre><code># if you didn't do this above
&gt; mkdir src/bin
&gt; mv src/main.rs src/bin
&gt; mkdir src/lib
&gt; mv src/lib.rs src/lib/print_hello.rs
</code></pre>
<p>We have to change <code>src/bin/main.rs</code> to use <code>hello_world</code> again</p>
<pre><code>fn main() {
hello_world::print_hello();
}
</code></pre>
<p>And we have to re-add some configuration into <code>Cargo.toml</code>:</p>
<pre><code>[lib]
path = "src/lib/print_hello.rs"
</code></pre>
<p>Everything should build just fine. But what about Buck?</p>
<p>So, once you start getting into subdirectories, you can also start using
multiple <code>BUCK</code> files. So we can empty out our root <code>BUCK</code> file (I'm leaving it
existing but empty, if you want to delete it you can but you'll recreate it in
the next part anyway), and create two new ones. Here's <code>src/lib/BUCK</code>:</p>
<pre><code>rust_library(
name = "hello_world",
srcs = glob(["**/*.rs"]),
crate_root = "print_hello.rs",
edition = "2021",
visibility = ["PUBLIC"],
)
</code></pre>
<p>and <code>src/bin/BUCK</code>:</p>
<pre><code>rust_binary(
name = "hello_world",
srcs = ["main.rs"],
crate_root = "main.rs",
deps = [
":hello_world",
],
)
</code></pre>
<p>We added in a <code>crate_root</code> to the library as well. Okay, let's try this:</p>
<pre><code>〉buck2 run //:hello_world
File changed: root//src/lib/BUCK
Unknown target `hello_world` from package `root//`.
Did you mean one of the 0 targets in root//:BUCK?
Build ID: d5059fc9-8001-47c4-ba5a-6ba605a4182c
Jobs completed: 2. Time elapsed: 0.0s.
BUILD FAILED
</code></pre>
<p>Oops! Since we moved files around, the names of our targets have changed. Let's
examine them:</p>
<pre><code>〉buck2 targets //...
Build ID: c4165964-cb87-49b4-8afe-4a3fc2c526bc
Jobs completed: 4. Time elapsed: 0.0s.
root//src/bin:hello_world
root//src/lib:hello_world
</code></pre>
<p>We had only seen very basic target patterns, but this is enough to show off:</p>
<p><code>root/src/bin:hello_world</code></p>
<p>can be read as</p>
<blockquote>
<p>The "hello_world" target defined in <code>/src/bin/BUCK</code>.</p>
</blockquote>
<p>Our target names changing also means we made one mistake in our new <code>BUCK</code> files.
Let's give it a try:</p>
<pre><code>〉buck2 run //src/bin:hello_world
Error running analysis for `root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)`
Caused by:
0: Error looking up configured node root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)
1: Cyclic computation detected when computing key `root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)`, which forms a cycle in computation chain: `root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800),root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)`
Build ID: 930ab541-c2dd-44f5-aef1-f6658a2b7c53
Jobs completed: 2. Time elapsed: 0.0s.
BUILD FAILED
</code></pre>
<p>Right. Our binary depends on <code>:hello_world</code>, which it is itself named <code>hello_world</code>,
so there's a problem. But that's just it, we don't want to depend on any old
<code>hello_world</code>, we want to depend on our libary. Can you write out the target
pattern that should go in <code>src/bin/BUCK</code>?</p>
<p>It looks like this:</p>
<pre><code> deps = [
"//src/lib:hello_world",
],
</code></pre>
<p>"The <code>hello_world</code> target in <code>/src/lib/BUCK</code>. Perfect. And now it works!</p>
<pre><code>〉buck2 run //src/bin:hello_world
File changed: root//src/bin/BUCK
Build ID: c6d2fdaa-298a-425a-9091-d3f6b38c4336
Jobs completed: 12. Time elapsed: 0.5s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)
Hello, world!
</code></pre>
<p>It kinda stinks to have to type all of that out. Luckily, Buck supports aliases
for target patterns. Take our top-level <code>BUCK</code> file, and add this:</p>
<pre><code>alias(
name = "build",
actual = "//src/bin:hello_world",
visibility = ["PUBLIC"],
)
</code></pre>
<p>And now we can use it:</p>
<pre><code>〉buck2 build
Build ID: a87ed1e2-cfab-47b0-830e-407217997fd7
Jobs completed: 2. Time elapsed: 0.0s.
BUILD SUCCEEDED
</code></pre>
<p>Fun.</p>
<p>Okay! This is getting a bit long, so let's end there. We have more to learn
before buck can actually replace Cargo in our normal development workflow, but
I hope this helped you see how you could get started with Buck if you wanted to.</p>
<p>If you want to check out this on your own, I've published this on GitHub:
<a href="https://github.com/steveklabnik/buck-rust-hello/tree/024ef54ba45627e87a65aaf2f69c6661198c336c">https://github.com/steveklabnik/buck-rust-hello/tree/024ef54ba45627e87a65aaf2f69c6661198c336c</a></p>
<p>Next up, we'll be tackling other features, like "using crates from crates.io."
No promises on when that'll get published though!</p>
</content:encoded></item><item><title>Rust's Golden Rule</title><link>https://steveklabnik.com/writing/rusts-golden-rule/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rusts-golden-rule/</guid><pubDate>Mon, 27 Mar 2023 00:00:00 GMT</pubDate><content:encoded><p>I find myself thinking about a particular design principle of Rust today. I'm
not sure I've ever seen it named specifically before, but it gets referred to
from time to time, and I think it's an under-rated but very important aspect
of why Rust works so well. I was going to refer to it as "the signature is the
contract" today, but then I ended up changing it. Regardless of that, if someone
else has already written this out somewhere, and used a different name, please
let me know!</p>
<p>Magic: the Gathering is a really interesting project. I say "project" rather
than "card game" because while it is a card game, it also pioneered a whole
bunch of incedental other things that had big effects on related hobbies.</p>
<p>I learned MtG in the late 90s. The rules were a bit different then, but many
of them are the same. The very first rule I was taught was sort of the "Magic
Golden Rule," though in today's Comprehensive Rulebook, there are four of them.
This one is still the first, though:</p>
<blockquote>
<ol>
<li>The Magic Golden Rules</li>
</ol>
<p>101.1. Whenever a card’s text directly contradicts these rules, the card takes
precedence. The card overrides only the rule that applies to that specific
situation. The only exception is that a player can concede the game at any
time (see rule 104.3a).</p>
</blockquote>
<p>This rule is the most important rule because it kind of creates the spaces of
possibilities for cards: many cards exist to tweak, modify, or break some sort
of fundamental rule.</p>
<p>That being said, all these years later, this idea is so pervasive in games like
this that it's barely even considered an actual rule. It's just part of the
physics of the genre, it's how these games work. Yet it's critical to the
entire enterprise.</p>
<p>Rust also has a rule. It's kinda funny, because in some senses, this rule
is almost the opposite of Magic's, if you can even stretch the comparison this
far. Here it is:</p>
<blockquote>
<p>Whenever the body of a function contradicts the function's signature, the
signature takes precedence; the signature is right and the body is wrong.</p>
</blockquote>
<p>This rule is also so pervasive in Rust that we take it for granted, but it
is really, truly important. I think it is also important for Rust users to
internalize the implications of this rule, so that they know why certain things
work the way that they do.</p>
<p>Here is the most famous implication of this rule: Rust does not infer function
signatures. If it did, changing the body of the function would change its
signature. While this is convenient in the small, it has massive ramifications.</p>
<p>Consider this example program:</p>
<pre><code>fn foo(x: i32) -&gt; i32 {
dbg!(x);
x
}
</code></pre>
<p>This function prints out <code>x</code>, and then returns it. Nothing fancy going on here,
this is just random stuff to make an example. This compiles just fine. But let's
imagine that we have a version of Rust that infers our signatures. So we could
type this instead:</p>
<pre><code>fn foo(x) {
</code></pre>
<p>This is what a Ruby-ish Rust might look like; we declare the name of our
argument but not its type, and we don't declare a return type. Now, let's
do a small refactoring, we're gonna comment out the final value there:</p>
<pre><code>fn foo(x) {
dbg!(x);
//x
}
</code></pre>
<p>the final expression has changed; it's no longer <code>x</code>, but instead is <code>()</code>,
which is what <code>dbg!(x);</code> evaluates to. Because of type inference, the inferrred
type of <code>foo</code> is now <code>fn(i32) -&gt; ()</code>. Our function typechecks! It's all good,
right?</p>
<p>Well, no:</p>
<pre><code>error[E0369]: cannot add `{integer}` to `()`
--&gt; src/lib.rs:5:11
|
5 | y + 1
| - ^ - {integer}
| |
| ()
</code></pre>
<p>Wait, what? We don't have a <code>y + 1</code> anywhere in our code?! Where's that
error coming from... <code>src/lib.rs:5</code>. What's at the top of <code>lib.rs</code>?</p>
<pre><code>mod bar {
fn baz() -&gt; i32 {
let y = crate::foo(5);
y + 1
}
}
</code></pre>
<p>Oh. Some other code was using <code>foo</code>. When its signature changed, we broke
this invocation. It's nice that alt-Rust caught this for us, but this error is
(would be, anyway!) really bad: no longer is it telling us that our code
is wrong, but instead points to some other code somewhere else we weren't even
messing with. Sure, we can go "hey why is <code>y</code> <code>()</code>?" and then figure it out,
but compare that to today's error:</p>
<pre><code>error[E0308]: mismatched types
--&gt; src/lib.rs:9:19
|
9 | fn foo(x: i32) -&gt; i32 {
| --- ^^^ expected `i32`, found `()`
| |
| implicitly returns `()` as its body has no tail or `return` expression
10 | dbg!(x);
| - help: remove this semicolon to return this value
</code></pre>
<p>This error is far better: it points out that the body of our function
contradicts the signature. It points out what in our body is generating the
value that contradicts the signature. And it doesn't complain about callers.</p>
<p>So sure, this gets us nicer error messages, but is that really a big deal?
I think it is, but it's not the only implication here. I'd like to talk about
two more: one that's clearly an advantage, and one that has led to some pain
that people would like to resolve. Balance :)</p>
<p>First, the one that's an advantage. The advantage is modularity. That makes
some forms of analysis much more reasonable, or sometimes even possible, as
opposed to super difficult. Because everything you need for memory safety is
described in the signature of the function, Rust doesn't need to examine your
entire program to determine if there's some shenanigans going on elsewhere.
This is far, far less work than just checking the signatures of the functions
you call. Each function can be checked in isolation, and then assembled
together. This is a very nice property.</p>
<p>Second, where this leads to some pain. Users have nicknamed this one "borrow
splitting," or "partial borrowing." It looks like this:</p>
<pre><code>struct Point {
x: i32,
y: i32,
}
impl Point {
pub fn x_mut(&amp;mut self) -&gt; &amp;mut i32 {
&amp;mut self.x
}
pub fn y_mut(&amp;mut self) -&gt; &amp;mut i32 {
&amp;mut self.y
}
}
</code></pre>
<p>I find accessors, and especially mutators, to be where this sort of thing pops
up most often. This is the classic example. The above code is fine, but if we
try to do this:</p>
<pre><code>// this doesn't work
impl Point {
pub fn calculate(&amp;mut self) -&gt; i32 {
let x = self.x_mut();
let y = self.y_mut();
// yes I picked multiplication because this looks kinda funny
*x * *y
}
}
// we would call it like this:
let answer = p.calculate();
</code></pre>
<p>We get this:</p>
<pre><code>error[E0499]: cannot borrow `*self` as mutable more than once at a time
--&gt; src/lib.rs:19:17
|
18 | let x = self.x_mut();
| ------------ first mutable borrow occurs here
19 | let y = self.y_mut();
| ^^^^^^^^^^^^ second mutable borrow occurs here
20 |
21 | *x * *y
| -- first borrow later used here
</code></pre>
<p>However, if we didn't have these accessors, <code>x</code> and <code>y</code> were instead just
public, this very similar free function:</p>
<pre><code>fn calculate(x: &amp;mut i32, y: &amp;mut i32) -&gt; i32 {
*x * *y
}
// called like this:
let answer = calculate(&amp;mut point.x, &amp;mut point.y);
</code></pre>
<p>works just fine. Why? Because of the signatures. This signature:</p>
<pre><code>pub fn calculate(&amp;mut self) -&gt; i32 {
</code></pre>
<p>and these signatures:</p>
<pre><code>pub fn x_mut(&amp;mut self) -&gt; &amp;mut i32 {
pub fn y_mut(&amp;mut self) -&gt; &amp;mut i32 {
</code></pre>
<p>says "hey, I am going to borrow all of <code>self</code> mutably," which implies an
exclusive reference to <code>self</code>. That the body of <code>calculate</code> borrows two
different parts of <code>self</code> independently is 100% irrelevant, that's what the
signature says! And so rustc looks at this and says "hey wait a minute,
calling <code>x_mut</code> borrows <code>self</code> mutably, and calling <code>y_mut</code> borrows <code>self</code>
mutably. That's aliasing! Bad programmer!</p>
<p>Whereas in the second example, this signature:</p>
<pre><code>fn calculate(x: &amp;mut i32, y: &amp;mut i32) -&gt; i32 {
</code></pre>
<p>says "hey, I have two mutable references, to two different integers." And
at the call site, Rust sees that we're creating two different borrows to two
different integers, even though they're both part of our <code>point</code>, and so it
okays things.</p>
<p>This is kind of a pain in the butt! But what it saves us from is that scary
action at a distance in our typecheck example. Imagine that Rust somehow
inferred that the first version was okay, due to the body only being disjoint.
What happens in the future, when we refactor our function, and the borrows need
to change? That would have the same problem as before; we'd get weird errors
elsewhere in our code. There have been some proposals over the years to possibly
fix this pain point, but it's a tough one. Putting "here's the fields I'm
borrowing in the body" into the signature, in order to conform with the Golden
Rule, looks very odd. Maybe something will happen here, maybe not. I have
complicated feelings.</p>
<p>So beyond error messages and making accessors/mutators awkward, how does this
affect you day to day? Well, one way is that I find I end up paying more
attention to signatures and less attention to bodies, when I'm trying to sort
something out. In many cases, what you're doing in the body is irrelevant, I
care only about what your signature affords me. This isn't <em>quite</em> as true
as in some pure functional languages, but it has that vague feel to it. Another
way, and a slightly bigger one, has to do with the relationships between types
and TDD, but I think I'm going to save that for its own post.</p>
</content:encoded></item><item><title>You're probably learning a technology in its seventh season, not its pilot</title><link>https://steveklabnik.com/writing/you-re-probably-learning-a-technology-in-its-seventh-season-not-its-pilot/</link><guid isPermaLink="true">https://steveklabnik.com/writing/you-re-probably-learning-a-technology-in-its-seventh-season-not-its-pilot/</guid><pubDate>Fri, 27 Jan 2023 00:00:00 GMT</pubDate><content:encoded><p>I first heard this adage in the Rails space. I don't know who to attribute this
to, but I also never wrote it down. So, time to write it down, and if you know
of the origin of this aphorism, let me know and I'd love to put it here!</p>
<p>I <em>think</em> that the first formulation of this sentiment goes something like this:</p>
<blockquote>
<p>Rails is like a soap opera. You're getting started with it in a later season.
The cast of characters has already been established, there's a lot of history.
You kinda just gotta dive in and go with the flow and catch up on older stuff
at your leasure.</p>
</blockquote>
<p>That's obviously a bit long, so I think I'll be using something like this
in the future:</p>
<blockquote>
<p>You're probably learning a technology in its seventh season, not its pilot.</p>
</blockquote>
<p>I was thinking about this today because I happened to come across <a href="https://www.reddit.com/r/rust/comments/xj2a23/comment/ipd6tcv/">this reddit post from a few months ago</a>.</p>
<blockquote>
<blockquote>
<p>In fact, there is hardly any analysis of the downsides of Rust’s error model in the first place, which is quite disheartening.</p>
</blockquote>
<p>Not really planning to add much to the convo, but back in mid 2019 when I first
started learning and writing Rust, about half the posts I saw here in the
subreddit were related to errors, how to do them right, how to minimize the
boilerplate, and so on and so forth...</p>
<p>Over the last couple years, the convo has died down because we've all more or
less agreed that the major pain points around error handling have been solved
with a simple rule: thiserror for library code, anyhow for application code.</p>
<p>There's obvs still more things that can be done around error handling. try is
one such example, along with standardizing thiserror and anyhow so its part of
the stdlib.</p>
<p>But to say there is no discussion about Rust and it's error handling story is
very dishonest imo as that discussion was raged for years before I saw it and
only recently came to a proper conclusion for the majority of cases, and thus
theres now much less talk about it. I even learned 3 different "defacto standard
error libs" (that are all now defunct and no one cares about) out of a much
longer list of them that came and went before them due to better and better
patterns being being found over the years!</p>
<p>There was plenty of healthy and constructive discussion on this topic for an
insanely long time and just because you didn't see it doesn't mean everyone was
satisfied with the error handling story as it is in std from the 1.0 of Rust.
Rust users actually engage with the system constantly, so ofc they'd discuss the
problems with it and work towards fixing it! Hell, there's still people that
prefer things other than thiserror and anyhow and thats part of why those havent
been standardized yet too, and why there still might seem to be "no progress" on
fixing things.</p>
<p>TL;DR: Rust developers use Rust's error handling literally all the time. Your
assertion that they do not understand its shortcomings and pain points and
refuse to even discuss them is weird given that there's been a multi-years long
battle between all kinds of error handling crates that only recently resulted in
a proper and clear winner.</p>
</blockquote>
<p>Now, I don't want to take a position on the original thread, or what they have
to say, or if they actually <em>are</em> falling afoul of this aphorism. But I think
that this particular text shows the meaning of what I mean by "you're in the
seventh season" beautifully. Sometimes, when we learn a new technology, we go
"wtf, why is this like this?" And sometimes, that outside perspective is good!
But sometimes, it just means that you weren't there for the discussion, you either
don't know or aren't in the same contexts, or a variety of other reasons why
something is the way that it is, even if that way is surprising to you at first.</p>
<p>In that sense, I think this idea is pretty closely related to Chesterton's Fence.
It may even just be the same thing, formulated in a slightly different way.</p>
</content:encoded></item><item><title>Too many words about Rust's function syntax</title><link>https://steveklabnik.com/writing/too-many-words-about-rusts-function-syntax/</link><guid isPermaLink="true">https://steveklabnik.com/writing/too-many-words-about-rusts-function-syntax/</guid><pubDate>Tue, 03 Jan 2023 00:00:00 GMT</pubDate><content:encoded><p>There's a famous quote by Eleanor Roosevelt:</p>
<blockquote>
<p>great minds discuss semantics, average minds discuss syntax and small minds discuss syntax of comments</p>
</blockquote>
<p>(Apologies, Phil Wadler.)</p>
<p>I'd like to preface this post that I do <em>not</em> consider this to be an actual
proposal to change Rust, even a pre-RFC. I don't think that making changes like
this has enough benefits once a language is as established as Rust is. Consider
this post more of some musings about some things that there wasn't really time
to truly consider when Rust was younger, so that maybe someone who's making a
new language can learn from those things early enough in its life to do so.</p>
<p>Also, I am not the first person to think about this, but I have lost the
previous post to time. There also may be a newfangled language that has this or
similar syntax already I'm not aware of. If you know of similar previous
proposals or languages, please let me know so that I can link to them here.</p>
<p>Anyway, consider this post a "Steve decided to write up a thing instead of
just tweeting, it is not particularly well thought out but at least it's down
on paper" kind of thing. A serious proposal here, even if I thought the idea was
good in general, which to repeat again, I do not, would require examining a lot
more corner cases than I do in this post.</p>
<p>With all of that out of the way, let's begin.</p>
<hr />
<h2>The fn syntax</h2>
<p>Rust does a really good job of being orthogonal in many ways. While it isn't a
tiny language, that things fit together nicely is one of the things that can
make it feel a lot smaller, in my opinion. There's one thing, though, that
is really a bit inconsistent today, and that's the syntax and semantics of
functions.</p>
<p>Let me explain: you declare a function item (read: define a function) like this:</p>
<pre><code>fn foo(x: i32) -&gt; i32 {
todo!()
}
</code></pre>
<p>Simple enough. However, there's a lot more going on for more complex
declarations. <a href="https://doc.rust-lang.org/1.66.0/reference/items/functions.html">Here</a> is the beginning of the current grammar
of functions:</p>
<pre><code>Function :
FunctionQualifiers fn IDENTIFIER GenericParams?
( FunctionParameters? )
FunctionReturnType? WhereClause?
( BlockExpression | ; )
</code></pre>
<p>I read this like this: "A function consists of some function qualifiers, fn,
an identifier, and an optional list of generic parameters. We then have some
optional function parameters inside of parenthesis, an optional return type,
an optional where clause, and then a block expression or a semicolon."</p>
<p>This is in sharp contrast to a C-style function definition, which would
look like this:</p>
<pre><code>int foo(int x) {
// todo
}
</code></pre>
<p>In general, most new languages have tended to settle closer to Rust's syntax
here than C's. There are a few reasons, and it's not like Rust invented this
style in the first place. One of the core nice things about Rust style syntax
is that <code>fn IDENTIFIER</code>, that is, if you want to find out where <code>foo()</code> is
defined, grep for <code>fn foo</code> and you can find it pretty quickly. You can't
do this as well with the <code>int</code> syntax. Some code styles have you write</p>
<pre><code>int
foo(int x) {
// todo
}
</code></pre>
<p>Notably, the GNU project uses this style:</p>
<blockquote>
<p>It is also important for function definitions to start the name of the
function in column one. This helps people to search for function definitions,
and may also help certain tools recognize them.</p>
</blockquote>
<p>You instead grep for <code>^foo</code>, and you can get a similar benefit.</p>
<p>So, what's bad about this syntax? Well, there are useful 'modifiers' to
functions, and Rust has added some and may add more in the future. For example,
if we want <code>foo</code> to be callable in a const context, we'd make it a <code>const fn</code>:</p>
<pre><code>const fn foo(x: i32) -&gt; i32 {
todo!()
}
</code></pre>
<p>That's all well and good, we still even retain the "grep for <code>fn</code>" idea here,
but these are <em>also</em> useful on arbitrary blocks of code:</p>
<pre><code>let x = const {
1 + 2
};
</code></pre>
<p>This is of course a trivial example that's not very useful, but it shows off
the syntax here. My point is... this starts to feel a bit... nonuniform. In
the function delcaration, we put const at the front, but when it's any other
block expression, we put it right before the block. Rust's function bodies
are similar to blocks in other ways, such as the whole "evaluates to the final
expression" thing. But they're different when they're being used to describe
the body of a function. That's a bit less than ideal.</p>
<p>We could try the naive thing:</p>
<pre><code>fn foo(x: i32) -&gt; i32 const {
todo!()
}
</code></pre>
<p>This is... weird, but it does sort of unify some syntaxes. We now can turn
any block of code, including a function body, <code>const</code> by adding the <code>const</code>
right before the opening brace. However... I kinda shudder at <code>i32 const</code>.
There's not enough separation between the return type and the modifier to make
quick sense of things, in my opinion. However, there's another small tweak we
could make here. But first, a small divergence into another corner of Rust's
syntax: variables.</p>
<h2>Variables</h2>
<p>Rust also has a different syntax for variable declaration. Like C, it's similar
to its function declaration syntax:</p>
<pre><code>// Rust
let x: i32 = 5;
</code></pre>
<pre><code>// C
int x = 5;
</code></pre>
<p>We have <code>let</code> instead of <code>fn</code>, we have the <code>name: type</code> instead of <code>type name</code>.
We can also declare a variable but not initialize it:</p>
<pre><code>let x: i32;
</code></pre>
<p>Rust will make us initialize it before we use it, but we can declare the
variable on its own, and then give it an initial value later.</p>
<p>But what's this has to do with functions?</p>
<h2>A marginally better syntax</h2>
<p>Functions also happen to have a "declare the signature but not the body" syntax
in Rust too, though it's almost exclusively used in an FFI context. Remember
the very end of our grammar?</p>
<blockquote>
<p>and then a block expression or a semicolon.</p>
</blockquote>
<p>If we have a function <code>foo</code> that we're calling through FFI, we can define it
like this:</p>
<pre><code>#[link(name = "something")]
extern {
fn foo(x: i32) -&gt; i32;
}
</code></pre>
<p>We don't provide a body, but we do provide a semicolon. This is strikingly
similar to the variable syntax. So why not have the regular function definition
also follow the variable syntax?</p>
<pre><code>fn foo(x: i32) -&gt; i32 = {
todo!()
}
</code></pre>
<p>We've now added one little extra bit to the grammar: a <code>=</code> after the return
type, if any. This one little change allows us to unify the rest of the syntax
around blocks more easily:</p>
<pre><code>// full signature
fn foo(x: i32) -&gt; i32 = {
todo!()
};
// empty return type
fn foo(x: i32) = {
todo!()
};
// const
fn foo(x: i32) -&gt; i32 = const {
todo!()
};
</code></pre>
<p>I happen to <em>really</em> like this. It's a pretty small tweak but I think it
cleans stuff up nicely.</p>
<pre><code>let name = value;
fn name() = value;
</code></pre>
<p>Where both could be values or blocks, with <code>let</code> it's most often a value but
with <code>fn</code> it's most often a block. The symmetry pleases me.</p>
<p>(Sharp readers will also notice the introduction of a semicolon. Leaving it
off would be closer to the old syntax, requring it would be similar to the
variable syntax, but then that would diverge from the other item declaration
syntax... I don't feel strongly about it but just in case anyone noticed I
figured I'd mention that as well, yet another rabbit hole to chase down.)</p>
<h2>Going too far</h2>
<p>This of course raises another divergence. But I'm not sure that fixing this one
is worth it; it starts to get <em>too</em> weird, I think. But maybe I'm wrong. This
one is about the types.</p>
<p>One of the things that makes <code>name: type</code> syntax nicer than <code>type name</code> syntax
is that it degrades nicely in the presence of type inference:</p>
<pre><code>// Rust
let x: i32 = 5;
let x = 5;
</code></pre>
<pre><code>// C23 (https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3007.htm)
int x = 5;
auto x = 5;
</code></pre>
<p>Rust's function syntax isn't like this though, partially because there's no
type inference on function signatures, so there's not a lot of pressure for
it to degrade nicely. The choice not to offer inference is the right one for
Rust, but for a future language where that's not the case, we could unify
them with something like this:</p>
<pre><code>// explicit type
fn foo(x): fn(i32) -&gt; i32 = {
todo!()
}
// inferred
fn foo(x) = {
todo!()
}
</code></pre>
<p>This one... this may be too far afield. And, while Rust doesn't have named
parameters, and possibly never will, I have no idea how this style of syntax
would work with that. There's also some repetition of <code>fn</code> here that might be
worth trying to remove, but I'm trying to make the smallest possible deltas
from existing syntax here, and <code>fn(i32) -&gt; i32</code> <em>is</em> the type of that function
in today's syntax.</p>
<p>What I mostly take away from this part of the exercise is that consistency is
a good goal, but it's not the <em>only</em> goal. Even if Rust didn't allow for
inference, making its function declaration syntax be like this may have simply
been a bridge too far for wide adoption. Rust already takes a lot of flak for
being weird, and spending some <a href="the-language-strangeness-budget">Strangeness Budget</a> to unify these
syntaxes probably wouldn't be the right choice. But maybe I'm too conservative!
Given that Rust already takes that flak, maybe a bigger jump would have been
okay here. Who knows.</p>
<p>... actually, we have one way to find out, I guess. This is <em>very</em> similar
to the syntax that Herb Sutter came up with for <a href="https://github.com/hsutter/cppfront">cppfront</a>:</p>
<pre><code>foo: (x: i32) -&gt; i32 = {
// todo
}
</code></pre>
<p>There's no <code>fn</code> preceding. The type syntax works a bit differently. But it's
close. And Herb is targetting the people who I'd be most worried about hating
this syntax: C++ folks. Maybe this idea isn't so farfetched at all. He's
also got an example of how making it any expression after the <code>=</code> can be
nice for one-liner functions. If we applied this to our fake Rust syntax:</p>
<pre><code>fn main() = println!("Hello, world!");
fn hello(name): fn(&amp;str) = println!("Hello, {name}!");
</code></pre>
<p>That's kinda nice too.</p>
</content:encoded></item><item><title>Ten Years of Ru...ewriting my website</title><link>https://steveklabnik.com/writing/ten-years-of-ru---ewriting-my-website/</link><guid isPermaLink="true">https://steveklabnik.com/writing/ten-years-of-ru---ewriting-my-website/</guid><pubDate>Wed, 21 Dec 2022 00:00:00 GMT</pubDate><content:encoded><p>December 21 is the anniversary of when I first heard about Rust way back in
2012. I used to write yearly posts about it; the last time I did was <a href="../six-years-with-rust">in
2018</a>. That makes today ten years. I thought I'd have
something big to say here, but... I just don't.</p>
<p>Part of that is because I haven't written a blog post since late 2020.
Part of <em>that</em> is well, <em>gestures wildly at everything</em>, but another part of it
is that the way I had set up my website was cool, but fragile, and I didn't have
the time or energy to keep up with it.</p>
<p>You see, last time I had basically taken a starter project for a new framework
I had never used before, and just kinda wrote code until it works. That's one
of the joys of having your own website; you can do whatever you want with it,
including things you wouldn't do in a professional context. The downside is that
you can do whatever you want with it, including things you wouldn't do in a
professional context. My RSS feed has been broken for who knows how long, and
I just couldn't muster up the effort to fix it.</p>
<p>So last night, I did something new: I did the tutorial for a new framework I
had never used before, and wrote code until it works.</p>
<p>Yeah yeah that's funny and all, but I actually feel much better about things
this time. We'll see how I feel in six months. The framework this time is
<a href="https://astro.build/">Astro</a>, and I think it's really neat. At some point after
I do some more with it and digest some more of the way that it works, I'll give
a more thorough writeup, but I find the <a href="https://docs.astro.build/en/concepts/why-astro/">Why Astro?</a> page on their
site does a good job of explaining.</p>
<p>Why not a Rust based framework? I want to do more in that space too, but the goal
here is to stretch my legs, and using a language (TypeScript, in this case) that
I don't use often is more interesting than the thing I use daily.</p>
<p>However, one little thing did come up during the development of this project
that I thought was kind of amusing, in a 'compare and contrast' kind of way.</p>
<p>I like for my blog to have entries grouped by year. There's no database in this
project, so I have to sort things myself. Here's the relevant code as it stands
today, with some small omissions for clarity:</p>
<pre><code>import type { MarkdownInstance } from "astro";
const allPosts = await Astro.glob('../pages/writing/*.md');
let allPostsByDate = (() =&gt; {
let posts: { [key: string]: MarkdownInstance&lt;Record&lt;string, any&gt;&gt;[] } = {};
allPosts.map((post) =&gt; {
let year = new Date(post.frontmatter.pubDate).getFullYear();
let array = posts[year.toString()];
// if this is the first post of this year, we have to create the inner array
if (!array || !array.length) {
posts[year.toString()] = [];
}
posts[year.toString()].push(post);
})
return posts;
})();
const years = Object.keys(allPostsByDate).reverse();
---
&lt;BaseLayout&gt;
&lt;p&gt;This is my blog.&lt;/p&gt;
{years.map((year) =&gt; (
&lt;h3&gt;{year}&lt;/h3&gt;
&lt;ul&gt;
{allPostsByDate[year].map((post) =&gt; &lt;BlogPost url={post.url} title={post.frontmatter.title} /&gt;)}
&lt;/ul&gt;
))}
&lt;/BaseLayout&gt;
</code></pre>
<p>I'm sure there's like, ten thousand things wrong with this code; like I said,
TypeScript beginner over here. I figured this stuff out myself, there's
probably a better way to do these things, but this works. (Okay, actually while
writing this I realized I didn't sort <em>within</em> the years, so the real code now
handles that too, but I left it out because it's not super relevant to the issue
at hand here.)</p>
<p>This is a pretty classic "I'm doing something in a scripting language" style
setup: you make some sort of data structure out of maps (or objects in this
case, of course) and arrays, and then spit it out in some form. But TypeScript
prefers types, it's right there in the name! And I had configured TypeScript
to be in its strictest mode, becuase I like types too. Here's the one big type
annotation I had to use:</p>
<pre><code>let posts: { [key: string]: MarkdownInstance&lt;Record&lt;string, any&gt;&gt;[] } = {};
</code></pre>
<p>If I didn't put a type on <code>posts</code>, it complains later in the program:</p>
<pre><code>`Element implicitly has an 'any' type because expression of type 'string' can't be used to index type '{}'.
No index signature with a parameter of type 'string' was found on type '{}'.
</code></pre>
<p>It needs a bit of help to figure out the type here. Figuring out how to say
"hey this is on object with string keys and arrays of <code>MarkdownInstance</code> as
values" took me a second, but it's not too bad. But I did struggle with it a bit,
first to figure out why I was getting the error in the first place, and second
so that I could fix it. I kinda laughed at struggling with types in this dymanic
feeling context.</p>
<p>So anyway, I also had to do a bit of translating markdown files; I had exported
all of my old posts from Notion, but I couldn't just plop them in and expect
them to work; Notion doesn't give you the metadata in a frontmatter block, for
starters. I have something like 200 posts, so doing this by hand is <em>just</em>
annoying enough that I said "I bet I can code this up faster than if I did it
all by hand, even with a vim macro."</p>
<p>But by now, I was in "get stuff done" mode, so I decided to switch back to Rust.
The program I wrote needed to:</p>
<ol>
<li>Open each markdown file in the directory.</li>
<li>Try to parse out the metadata.</li>
<li>use that metadata to write out a new file, with a new filename, and the
metadata in a more friendly format.</li>
<li>Delete the old file.</li>
</ol>
<p>This is a classic scripting task! But at this point, I know how to do it in Rust
easier than in a scripting language. I started like this:</p>
<pre><code>&gt; cargo new migrate
&gt; cd migrate
&gt; cargo add anyhow
&gt; cargo add walkdir
&gt; code .
</code></pre>
<p>And ended up writing this:</p>
<pre><code>use anyhow::Result;
use std::io::Write;
use std::{
fs::{self, File},
path::PathBuf,
};
use walkdir::WalkDir;
fn main() -&gt; Result&lt;()&gt; {
let from = std::env::args().skip(1).next().unwrap();
println!("from: {}", from);
for entry in WalkDir::new(from) {
let entry = entry.unwrap();
println!("Processing: '{}'", entry.path().display());
if entry.file_type().is_dir() {
continue;
}
let contents = fs::read_to_string(entry.path())?;
if &amp;contents[0..3] == "---" {
println!("already migrated, skipping");
continue;
}
let header: Vec&lt;_&gt; = contents.lines().take(7).collect();
if &amp;header[0][0..2] != "# " {
eprintln!("couldn't read title, skipping");
continue;
}
let title = &amp;header[0][2..];
if &amp;header[3][0..6] != "Blog: " {
eprintln!("couldn't read blog, skipping");
continue;
}
let blog = &amp;header[3][6..].to_ascii_lowercase().replace(' ', "-");
if &amp;header[4][0..6] != "Date: " {
eprintln!("couldn't read date, skipping");
continue;
}
let date = &amp;header[4][6..];
if &amp;header[6][0..6] != "Slug: " {
eprintln!("couldn't read slug, skipping");
continue;
}
let slug = &amp;header[6][6..];
dbg!(slug);
let mut slug = PathBuf::from(slug);
slug.set_extension("md");
let output = entry.path().parent().unwrap();
let output_path = output.join(slug);
println!("writing to: {}", output_path.display());
let mut output = File::create(&amp;output_path)?;
write!(
output,
"---
layout: ../../layouts/MarkdownPostLayout.astro
title: \"{}\"
pubDate: {}
blog: {}
---\n\n",
title, date, blog
)?;
for line in contents.lines().skip(7) {
writeln!(output, "{}", line)?;
}
fs::remove_file(entry.path())?;
}
Ok(())
}
</code></pre>
<p>76 lines. Not too bad. This is <em>quick and dirty</em>, it is not perfect. For
example, like ten of my posts had no 'blog' metadata (that tells me which of
my old blogs the post was originally written for), and so it didn't work on
them. I still needed to do some small cleanup, but I wanted to double check
every post anyway, so that was fine.</p>
<p>But there's a few other things that are kind of amusing to me about this code:</p>
<ol>
<li>I struggled with types in the scripty scenario above, but I did not here. I
am more experienced with Rust so that's probably it, but I needed to annotate
one type, just like in the TypeScript.</li>
<li>I didn't deal with lifetimes at all. Owned values, baby! And passing references
to some functions.</li>
<li>I didn't deal with memory at all. No borrow checker complaints, none of that.
It didn't take me any extra time to "think about ownership and borrowing."</li>
<li>It didn't take me that long to write this. You can actually see a vestigial
thing of how it was implemented, can you spot the <code>dbg!</code> statement? I wrote
each part in sequence, printing out the information to manually validate that
it looked correct, and then eventually adding the "write new file" and
"delete old file" stuff when I was sure the process would complete correctly.
Lots of copy/paste there too, as you can see. I don't care. I'm never using
this code again now that the migration is done. That's also why I didn't try
to do some sort of struct and "deserialize" the contents, instead just
parsing it out like I'd never heard of the term "parsing" before. It's fine.</li>
</ol>
<p>So yeah, I had too many type problems in my 'real' JavaScript program, and no
type problems in my script-y Rust. Kinda funny.</p>
<p>I guess I have ended up making this a post about Rust anyway, even though I
didn't think I had much to say when I started. Now that Rust has grown so much,
people who love it are using it for all kinds of things, and sometimes that
seems to piss some people off. "You can't use a systems language for scripting
tasks, nooo!!" they cry. If you had asked me ten years ago if I should write web
backends in Rust or replace shell scripts with Rust, I'd have said the same
thing. But now that I've used Rust enough, a lot of those sort of generic
complaints about Rust don't speak to my actual experience: I had a problem, I
wrote ~70 lines of code in not very much time, problem solved. Seems like Rust
worked just as well as anything else would have. Yes, it's true that maybe if
you've just picked up the language, you'd have written that code much more
slowly than in another language you're more familiar with, or maybe you'd have
hit some sort of error that was confusing and took up a bunch of time, just like
I did with my TypeScript. But just because you may have those problems, doesn't
mean that everyone does. And, it also means that in the future, if you stick with
it, you won't have those problems either! Or at least, it's possible not to. Not
every tool clicks with every person.</p>
<p>Here's to another ten years.</p>
</content:encoded></item><item><title>Are out parameters idiomatic in Rust?</title><link>https://steveklabnik.com/writing/are-out-parameters-idiomatic-in-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/are-out-parameters-idiomatic-in-rust/</guid><pubDate>Fri, 16 Oct 2020 00:00:00 GMT</pubDate><content:encoded><p>There is a pretty straightforward design pattern called "out parameters" that you may have heard of before. In short, the question is, does your function wear pants... err, I mean, does your function modify data like this:</p>
<pre><code>fn foo() -&gt; i32 {
// body elided
}
let x = foo();
</code></pre>
<p>or like this:</p>
<pre><code>fn foo(out: &amp;mut i32) {
// body elided
}
let mut x = 0;
foo(&amp;mut x);
</code></pre>
<p>That is, does your function return a value, or modify a parameter?</p>
<p>There are a number of different reasons you may or may not use one or the other of these techniques in different languages, and they often depend on the semantics of the language itself.</p>
<p>In Rust, you almost always want to return a value from a function, that is, prefer option one over option two. There are a few reasons for this.</p>
<p>The first is that it's generally semantically more accurate. Your function is producing a value of some kind, so it should, well, produce a value. Additionally, with Rust's relatively rich data type support, it's easy to do things like return a tuple if you need to return more than one thing.</p>
<p>The second is that, in some languages and with some compilers, the out parameter is a performance optimization that you as a programmer do by hand. Imagine if I wasn't returning an <code>i32</code>, which is just a few bytes. The most straightforward way of implementing the function in assembly is to copy the return value from the function to its parent's stack. This also may be required by your ABI. Now, I am not an expert here, so I am not 100% sure of the exact current state of things, but at least in theory, optimizations can be done by the compiler. This is a pretty well-known optimization, see <a href="https://en.wikipedia.org/wiki/Copy_elision">Wikipedia for more</a>. But, I think that we do already do this today in some cases, if I compile both of these functions:</p>
<pre><code>fn returns_huge(out: &amp;mut [u64; 1024]) {
*out = [0; 1024];
}
fn returns_huge() -&gt; [u64; 1024] {
[0; 1024]
}
</code></pre>
<p>I get identical assembly:</p>
<pre><code>playground::returns_huge: # @playground::returns_huge
# %bb.0:
movl $8192, %edx # imm = 0x2000
xorl %esi, %esi
jmpq *memset@GOTPCREL(%rip) # TAILCALL
# -- End function
</code></pre>
<p>However, it is possible that this breaks for other examples, and so you may, may, may want to move to an out parameter if you see extraneous copies appear in your performance profiling.</p>
<p>C++ guarantees this optimization in certain circumstances; it would be really nice for Rust to do so as well. My understanding is that's the intention.</p>
<h2>When you should use out parameters</h2>
<p>There is one case where it is idiomatic to use an out parameter in Rust though. That's when you are writing into some kind of buffer, and you want to be able to allow your caller to provide the buffer to you, rather than making a new one. This would allow the caller to re-use the same buffer for multiple operations.</p>
<p>(I say "buffer" here but really, this generalizes to any sort of grow-able container, to be clear. I am using "buffer" in a loose sense here.)</p>
<p>You can see this play out in the Rust standard library. <code>std::io::Read</code> looks like this:</p>
<pre><code>pub trait Read {
fn read(&amp;mut self, buf: &amp;mut [u8]) -&gt; Result&lt;usize&gt;;
}
</code></pre>
<p>This accepts a <code>&amp;mut [u8]</code> rather than returning a new buffer. This allows for significant savings by re-using a buffer. For example, in <a href="https://doc.rust-lang.org/stable/book/ch02-00-guessing-game-tutorial.html">Chapter 2 of the book</a>, we have this example:</p>
<pre><code>use rand::Rng;
use std::cmp::Ordering;
use std::io;
fn main() {
println!("Guess the number!");
let secret_number = rand::thread_rng().gen_range(1, 101);
loop {
println!("Please input your guess.");
let mut guess = String::new();
io::stdin()
.read_line(&amp;mut guess)
.expect("Failed to read line");
let guess: u32 = match guess.trim().parse() {
Ok(num) =&gt; num,
Err(_) =&gt; continue,
};
println!("You guessed: {}", guess);
match guess.cmp(&amp;secret_number) {
Ordering::Less =&gt; println!("Too small!"),
Ordering::Greater =&gt; println!("Too big!"),
Ordering::Equal =&gt; {
println!("You win!");
break;
}
}
}
}
</code></pre>
<p>We aren't <em>quite</em> properly re-using the buffer here, but we could modify this to look like this:</p>
<pre><code>use rand::Rng;
use std::cmp::Ordering;
use std::io;
fn main() {
println!("Guess the number!");
let secret_number = rand::thread_rng().gen_range(1, 101);
let mut guess = String::new();
loop {
println!("Please input your guess.");
guess.clear();
io::stdin()
.read_line(&amp;mut guess)
.expect("Failed to read line");
let guess: u32 = match guess.trim().parse() {
Ok(num) =&gt; num,
Err(_) =&gt; continue,
};
println!("You guessed: {}", guess);
match guess.cmp(&amp;secret_number) {
Ordering::Less =&gt; println!("Too small!"),
Ordering::Greater =&gt; println!("Too big!"),
Ordering::Equal =&gt; {
println!("You win!");
break;
}
}
}
}
</code></pre>
<p>Here, we create our string outside of the loop, and then on each iteration of the loop, clear it. Clearing it does not deallocate the backing memory; it only changes the length to zero. When the next line is read in, if it's smaller than the existing size of the string, it won't allocate any new memory. This would not be possible if <code>stdin()</code> returned a new <code>String</code> every time, so even though the out parameter is a bit less nice to use, it can be much faster.</p>
<p>It took 26 releases of Rust for <a href="https://doc.rust-lang.org/stable/std/fs/fn.read.html"><code>std::fs::read</code></a> and <a href="https://doc.rust-lang.org/stable/std/fs/fn.read_to_string.html"><code>std::fs::read_to_string</code></a> to be added for this reason; we eventually did because in many cases, you don't care about re-using the buffer, and these functions are nicer to use, but they weren't included originally because they're pretty simple, and are built on top of the re-usable solutions:</p>
<pre><code>pub fn read_to_string&lt;P: AsRef&lt;Path&gt;&gt;(path: P) -&gt; io::Result&lt;String&gt; {
fn inner(path: &amp;Path) -&gt; io::Result&lt;String&gt; {
let mut file = File::open(path)?;
let mut string = String::with_capacity(initial_buffer_size(&amp;file));
file.read_to_string(&amp;mut string)?;
Ok(string)
}
inner(path.as_ref())
}
</code></pre>
<p>That said, I am exceedingly glad we did add them. It is a much nicer user experience.</p>
<p>(You may be wondering "what's up with that inner function there?"... that maybe deserves another post! It's not neccesary, strictly speaking, but is another case of "the compiler does not do an optimization yet so write some code in a strange way by hand.")</p>
</content:encoded></item><item><title>The CXX Debate</title><link>https://steveklabnik.com/writing/the-cxx-debate/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-cxx-debate/</guid><pubDate>Sat, 22 Aug 2020 00:00:00 GMT</pubDate><content:encoded><p>Let's talk about safe and unsafe in Rust.</p>
<p>To review, Rust has a concept of "unsafe code." Unsafe Rust is a superset of safe Rust, letting you do some extra things that are dangerous. It's your job as a programmer to verify that your code is playing by the rules, because the compiler cannot check them for you.</p>
<p>Imagine we have a C program, <code>add.c</code> that looks like this:</p>
<pre><code>#include &lt;stdint.h&gt;
uint32_t add_one(uint32_t x) {
return x + 1;
}
</code></pre>
<p>We can call this function from Rust like this:</p>
<pre><code>extern {
fn add_one(x: u32) -&gt; u32;
}
fn main() {
let x = 5;
let y = unsafe { add_one(x) };
println!("{}", y);
}
</code></pre>
<p>We need to invoke <code>unsafe</code> here because any <code>extern</code> function doesn't come from Rust; it comes from another language, and it's possible that that function does something that would cause undefined behavior. We look at <code>add_one</code>, see that it's not violating any rules, and we're good.</p>
<p>However, we know that we can pass any <code>u32</code> to this function, and it's safe. We don't need to example the value of the argument every single time, they're all fine. So we do this:</p>
<pre><code>mod internal {
extern {
pub fn add_one(x: u32) -&gt; u32;
}
}
fn add_one(x: u32) -&gt; u32 {
unsafe {
internal::add_one(x)
}
}
fn main() {
let x = 5;
let y = add_one(x);
println!("{}", y);
}
</code></pre>
<p>This is a classic way that unsafe code gets used in Rust; we have our binding, <code>internal::add_one</code>, and our wrapper, <code>add_one</code>. The wrapper is safe, but uses unsafe internally. This is totally fine! The idea is that this gives us a sort of "seam" in our code. Imagine that <code>add_one</code> would do bad things if we passed it a number less than five. We could use our wrapper to check this:</p>
<pre><code>// we would use a real error type in real code but we're using () here because we don't
// actually care about the error handling here.
fn add_one(x: u32) -&gt; Result&lt;u32, ()&gt; {
if x &lt; 5 {
return Err(());
}
unsafe {
Ok(internal::add_one(x))
}
}
</code></pre>
<p>But, <code>add_one</code> doesn't need to do this. Every <code>u32</code> we pass is totally fine.</p>
<p>Nothing so far is controversial here, this is just background and setup. The purpose of these examples is for you to read and ask yourself, "Is this okay? How do I feel about this?"</p>
<p>Somewhere after this point, opinions start to diverge.</p>
<hr />
<p>Our C library is growing new functionality! Here it is:</p>
<pre><code>#include &lt;stdint.h&gt;
uint32_t add_one(uint32_t x) {
return x + 1;
}
uint32_t add_two(uint32_t x) {
return x + 2;
}
</code></pre>
<p>We can now add two, not only one. Amazing, I know. Let's write another wrapper:</p>
<pre><code>mod internal {
extern {
pub fn add_one(x: u32) -&gt; u32;
pub fn add_two(x: u32) -&gt; u32;
}
}
fn add_one(x: u32) -&gt; u32 {
unsafe {
internal::add_one(x)
}
}
fn add_two(x: u32) -&gt; u32 {
unsafe {
internal::add_two(x)
}
}
fn main() {
let x = 5;
let y = add_one(x);
let z = add_two(x);
println!("{}, {}", y, z);
}
</code></pre>
<p>Two functions, both have an unsafe binding, and a safe wrapper.</p>
<p>Is this okay? How do you feel about this?</p>
<p>Now, imagine we had 100 of these functions. Writing this all out ends up being a massive amount of error-prone boilerplate. Heck, I almost forgot to switch to <code>internal::add_two</code> after I copy/pasted the new function there. Rust has a tool to abstract over code like this: macros! So let's refactor our code with a macro:</p>
<pre><code>macro_rules! call_c {
// implementation elided
}
call_c! {
fn add_one(x: u32) -&gt; u32;
fn add_two(x: u32) -&gt; u32;
}
fn main() {
let x = 5;
let y = add_one(x);
let z = add_two(x);
println!("{}, {}", y, z);
}
</code></pre>
<p>I am not showing you the macro because the exact contents are gnarly and immaterial to the point. I also think that maybe this needs to be a procedural macro not a <code>macro_rules</code> macro but again, those details don't really matter. The point is, we have a macro, it generates the exact same structure that we had before.</p>
<p>Is this okay? How do you feel about this?</p>
<hr />
<p>This wrapper is also known as <a href="https://github.com/dtolnay/cxx">CXX</a>. The syntax is slightly different, but conceptually, this is what it is doing. The reason it is controversial is that some people do not in fact feel that the above macro is okay. The reason that they feel it is not okay is that you do not write <code>unsafe</code> by hand, yet, if you're not careful, things can go wrong. I think this is a totally understandable and reasonable reaction, but I also don't think that it makes a ton of sense once you actually dig into the details.</p>
<p>We actually don't need C to think about this problem, we can do this with unsafe Rust purely. And I think that using unsafe Rust makes it a bit more clear, so let's talk about this in some more depth, but purely with Rust.</p>
<p>The <code>unsafe</code> keyword has two, complimentary uses. It can be used to require your caller to check invariants, and it can be used to tell the compiler "I have checked the invariants." In other words:</p>
<pre><code>// I am leaving off parameters and return types because it doesn't matter, but real code
// would of course have them.
// When calling foo, please check some invariants.
unsafe fn foo() {
// ...
}
// I have checked the invaraints, and this is okay to do.
unsafe {
foo();
}
</code></pre>
<p>The compiler forces these combinations of things; using a feature that generates a requirement to check something will require you to also declare that you've checked it. These requirements bubble up, in other words, we can also pass that obligation to our callers:</p>
<pre><code>unsafe fn foo() {
// ...
}
fn bar1() {
// I have checked the invariants, and this is okay to do
unsafe {
foo();
}
}
// I have not checked the invariants, so since I'm calling foo, you must check
// the invariants
unsafe fn bar2() {
foo();
}
</code></pre>
<p>Rust's built-in FFI works like <code>bar2</code>; if the contents of <code>foo</code> are written in another language, it has to be unsafe. This means that, even if it is perfectly safe to call <code>foo</code> with any possible arguments, or in any circumstances, we still need to write a wrapper.</p>
<p>CXX's take on FFI is that it works like <code>bar1</code>; by using the library we are saying "hey, I know that <code>foo</code> is safe in all circumstances, so please generate the wrapper for me." In other words, in some sense, it pushes the unsafe block across the other side of the boundary, because that's where it actually belongs in this case.</p>
<hr />
<p>This has generated a lot of feelings on both sides. The people who don't like it are upset that the letters <code>['u', 'n', 's', 'a', 'f', 'e']</code> are not present, and feel that this makes it quite easy to make a mistake and accidentally use the macro on some code that is not actually safe to call in all circumstances. The people who do like it argue that when you have a large number of calls that are actually safe, using a macro to abstract them away is fine, like you'd abstract away any other boilerplate. Sometimes, repeating the same thing over and over again makes mistakes more likely, not less.</p>
<p>In the meantime, <a href="https://www.reddit.com/r/rust/comments/ielvxu/the_cxx_debate/g2i1r7a/?context=3">dtolnay has decided</a> to change the macro and re-add the <code>unsafe</code> token, which helps assuage the fears of the people who are upset. But it also centralizes these sorts of uses of <code>unsafe</code>, which helps achieve the goals of the library in the first place. I am continually impressed by dtolnay's ability to design libraries, and hope he keeps writing Rust code forever.</p>
<p>Anyway, if you were wondering what this all was about, that's my take on it. I personally feel that CXX is fine the way that it is, but I'm also happy with this compromise, and hope that others are too.</p>
</content:encoded></item><item><title>Today is my first day at Oxide Computer Company</title><link>https://steveklabnik.com/writing/today-is-my-first-day-at-oxide-computer-company/</link><guid isPermaLink="true">https://steveklabnik.com/writing/today-is-my-first-day-at-oxide-computer-company/</guid><pubDate>Mon, 15 Jun 2020 00:00:00 GMT</pubDate><content:encoded><p><img src="/img/2020-06-15/oxide.png" alt="Oxide Logo" /></p>
<p>Today is my first day at <a href="https://oxide.computer/">Oxide Computer Company</a>, where I will be doing engineering work.</p>
<hr />
<p>When Oxide introduced themselves to the world, I saw a lot of folks who were pretty confused. At its core, what Oxide is doing is very straightforward: you give Oxide money, and Oxide gives you a computer. This company sells goods, for money. That raises the issue of what kind of computer, and why would you want one?</p>
<p>In the late 90s and early 2000s, if you were running a web service of some kind, you'd have a server sitting in a rack in a data center somewhere. Got a lot of traffic? You'd buy a bigger, beefier server. This was the era of "vertical scaling." Servers were big, expensive, and hard to manage, so you didn't buy more servers, you bought bigger servers.</p>
<p>That started changing. In 2006, you saw a bunch of articles like <a href="https://www.networkworld.com/article/2304459/google-builds-own-servers-for-efficiency.html">this one</a>:</p>
<blockquote>
<p>Google, typically tight-lipped about the technology behind its data centers, builds its own servers to save costs and because standard products don't exactly meet its needs, Google's senior vice president of operations said on Thursday.</p>
</blockquote>
<p>This era saw companies like Google start to reject the notion of "Server grade" hardware, and rely purely on cheaper, consumer-grade hardware. As the quote said, this required building a bunch of software that would make the cluster of servers more reliable.</p>
<p>But eventually, there were downsides to this approach, as well. As these companies grew, so did a new generation of companies building "the cloud" as we know it today. And once you start managing that many computers, possibilities open up to gain certain efficiencies and cut costs. Years later, we see stuff <a href="https://www.geekwire.com/2017/amazon-web-services-secret-weapon-custom-made-hardware-network/">like this</a>:</p>
<blockquote>
<p>It’s not unusual for internet and software giants to design and even make their own hardware, to increase efficiency and build a competitive advantage. Google custom-designs its own servers, filling them with millions of chips from Intel, and announced in May that it has designed its own application-specific integrated circuit (ASIC) for use on neural networks. Facebook uses its own switches in its data centers. But market-leading public-cloud company Amazon Web Services may have gone farthest down this path — designing not only its own routers, chips, storage servers and compute servers but also its own high-speed network.</p>
</blockquote>
<p>The cloud was printing money, and folks were running into scaling issues. The solution was pretty straightforward: hire some people to build amazing infrastructure. Why not just buy it? Well, it simply didn't exist for purchase in the first place. The current crop of computer companies really grew up in the vertical scaling era. But beyond that, you also have to remember that this stuff was pretty <em>new</em> at that time. The Cloud was still cutting-edge, and so the requirements weren't really known. There was a lot of R&amp;D work to figure out what things would actually work in these environments. It turns out that a lot of it was not just hardware, but the software too. There is a ton of junk and waste in off-the-shelf server systems today, in all parts of the system.</p>
<p>Because these developments were proprietary, it also meant that if you weren't willing to build your own hardware, the only way you could get access to this new tech was by renting it. The public cloud's usage-based billing model is pretty standard, and pretty lucrative. Custom hardware helped dig an even deeper moat around cloud providers. If you want to actually own computers, your options are to use previous-generation tech, or hire a team and try to catch up to the big players. So the world kind of split in two: the hyperscalers and everyone else.</p>
<p>There is a <strong>ton</strong> more to say here, but this is the short of it: what Oxide is doing is building computers that are suitable for hyperscalers, but selling them, rather than building another public cloud. These computers will have hardware and software designed together to create excellent systems. Or I should say "will be selling them," it is still early days.</p>
<hr />
<p>I do not come from a hardware background. But there's a lot more to a system than hardware. And as you might guess from the name of the company, Oxide will be building a lot of software in Rust. I also have known two of the three founders of the company for a while, and they are folks I respect and whose work I admire.</p>
<p>I am also looking forward to doing engineering work again. I still love doing product work, but I think I prefer it in a small company setting. In a new company, engineers can still have a lot of ownership over product, so I won't be giving that up completely. But I'm excited to finally have a job where I get to write Rust code as a direct responsibility. It's not like I've not written any Rust over the past few years, but I'm looking forward to writing more of it, regardless.</p>
<p>Finally, I'm excited to continue to help others as well; Oxide cannot only hire folks who are already Rust experts. There just aren't enough of them yet, and there's folks who do fantastic work who don't yet write Rust. I'll be helping to bridge that gap.</p>
<p>There is so much work to do. It's almost 9am. Time to get started.</p>
</content:encoded></item><item><title>The next iteration of my blogs</title><link>https://steveklabnik.com/writing/the-next-iteration-of-my-blogs/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-next-iteration-of-my-blogs/</guid><pubDate>Mon, 08 Jun 2020 00:00:00 GMT</pubDate><content:encoded><p>I've had a few different blogs over the years. Every so often, I change things. One of the nice things about having your own site is that you can do whatever you want! So, I'm folding my old blogs' content into my main website. I'm sick of adding new subdomains, and it all feels better in one place these days. The old URLs will redirect to posts on steveklabnik.com, but I wanted to record what they looked like at the time.</p>
<hr />
<p>My oldest blog that's survived is <a href="https://web.archive.org/web/20200519230707/http://blog.steveklabnik.com/">Literate Programming</a>. It looked like this:</p>
<p><img src="/img/2020-06-08/Untitled.png" alt="untitled" />
<img src="/img/2020-06-08/Untitled%201.png" alt="untitled 01" />
<img src="/img/2020-06-08/Untitled%202.png" alt="untitled 02" /></p>
<p>This was my own static site generator! I loved making that table of contents stuff over on the left.</p>
<hr />
<p>Then, there's <a href="https://web.archive.org/web/20200604152340/https://words.steveklabnik.com/">words</a>. This used <a href="https://svbtle.com/">Svbtle</a>, which is a great platform. I liked the Meeple, and the red, and the Kudos button that made random internet people <em>so mad</em> for some reason. Anyway, it looked like this:</p>
<p><img src="/img/2020-06-08/Untitled%203.png" alt="untitled 03" />
<img src="/img/2020-06-08/Untitled%204.png" alt="untitled 04" />
<img src="/img/2020-06-08/Untitled%205.png" alt="untitled 05" /></p>
<p>The kudos count does nothing, other than record that you did it. The style was really nice, it was easy to write Markdown. Then Medium got popular. Oh well.</p>
<hr />
<p>Now, I'm writing this in <a href="https://www.notion.so/">Notion</a>. I really, really like writing in Notion. Someone put together <a href="https://github.com/ijjk/notion-blog">some code</a> to use Notion like a headless CMS. Exactly what I wanted. I've modified it a bunch, I'm using I'm using <a href="https://concrete.style/">concrete.css</a> as the base of my look, for example. They're using <a href="https://nextjs.org/">Next.js</a> and <a href="https://vercel.com/">Vercel</a>. I've been a fan of those folks (and pretty much everyone in the JAMStack space, to be honest) for a while, so I'm giving that a go.</p>
<p>I think it's going pretty good so far.</p>
</content:encoded></item><item><title>How often does Rust change?</title><link>https://steveklabnik.com/writing/how-often-does-rust-change/</link><guid isPermaLink="true">https://steveklabnik.com/writing/how-often-does-rust-change/</guid><pubDate>Sat, 11 Apr 2020 00:00:00 GMT</pubDate><content:encoded><p>I’ve been thinking a lot lately about how often Rust changes. There are some people that assert that Rust stays fairly static these days, and there are some people who say Rust is still changing far too much. In this blog post, I want to make a <em>data driven</em> analysis of this question. First, I will present my hypothesis. Next, my methodology. Then, we’ll talk about the results. Afterward, we’ll discuss possible methodological issues, and possible directions for further research.</p>
<p>As <a href="https://twitter.com/adam_chal/status/1248705340163620866">said on twitter</a>:</p>
<p><img src="https://svbtleusercontent.com/66pLpg8EDEH5dvt5BGQ7LH0xspap_small.jpg" alt="https://svbtleusercontent.com/66pLpg8EDEH5dvt5BGQ7LH0xspap_small.jpg" /></p>
<h2>The hypothesis</h2>
<p>My personal take on Rust changing: to me, it feels like we had more change in the past than we have today. The changes are also getting more minor. This will be my hypothesis.</p>
<h2>The methodology</h2>
<p>There are tons of different ways that we could measure the question “how often does Rust change?” So before I started collecting data, I had to decide some details.</p>
<p>I realized that my premise here is about how people <em>feel</em> about Rust. So, how do they tend to understand how Rust changes? Well, the primary way that we communicate change to the Rust community is through release blog posts. So I decided to start there.</p>
<p>What I did was, I opened up every release blog post for Rust, from 1.0.0 to 1.42.0, in a new tab. For fun, I also opened the release notes for 1.43, which will come out soon.</p>
<p>I then looked at every single change announced, and categorized them as follows:</p>
<ul>
<li>adds syntax?</li>
<li>language</li>
<li>standard library</li>
<li>toolchain</li>
<li>major</li>
<li>medium</li>
<li>minor</li>
</ul>
<p>I initially included “deprecation” and “soundness” as categories too, but I ended up not including them. More on that later.</p>
<p>From here, I had to decide what these categories <em>meant</em>. What standard would I be using here? Here’s the easier ones:</p>
<p><em>language</em> changes mean some sort of modification to the language definition. These are additive changes, thanks to our stability policy, but they’re new additions.</p>
<p><em>standard library</em> means a change to the standard library, primarily, new functions, new types, new methods, stuff like that. This one is pretty simple as a criteria but there’s some small interesting methodological issues we’ll get into later.</p>
<p><em>toolchain</em> changes related to <code>cargo</code>, <code>rustup</code>, new compiler target support, stuff like that. Not part of the language proper, but part of the Rust distribution and things Rust programmers would use.</p>
<p>Now, for the harder ones:</p>
<p><em>major</em>/<em>medium</em>/<em>minor</em> changes are how big of a change I think that the change is. There’s a few interesting parts to this. First of all, this is, of course, very subjective. I also decided to try and evaluate these as of today, that is, there are some changes we maybe thought were major that aren’t used all that often in practice today, so I categorized those as <em>medium</em> rather than <em>major</em>. This felt more consistent to me than trying to remember how I felt at the time.</p>
<p><em>adds syntax</em> changes <em>sound</em> easy, but are actually <em>really</em> tricky. For example, consider a feature like “<a href="https://github.com/rust-lang/rfcs/blob/master/text/1492-dotdot-in-patterns.md">dotdot in patterns</a>”. This does change the grammar, for example, so you could argue that it adds syntax. But as a programmer, I don’t <em>really</em> care about the grammar. The summary of this feature is “Permit the .. pattern fragment in more contexts.” It goes on to say:</p>
<blockquote>
<p>This RFC is intended to “complete” the feature and make it work in all possible list contexts, making the language a bit more convenient and consistent.</p>
</blockquote>
<p>I think these kinds of changes are actually very core to this idea, and so I decided to categorize them according to my opinion: this is <em>not</em> a new syntax change. You already had to know about <code>..</code>.</p>
<p>I believe this may be the most controversial part of my analysis. More on this later, of course.</p>
<p>Okay, so that’s the stuff I covered. But there’s also stuff I didn’t cover. I was doing enough work and had to draw the line somewhere. I left this stuff out:</p>
<ul>
<li>compiler speedups. This is interesting with numbers, but that means actually compiling stuff, and I don’t have time for that. This is a whole study on its own.</li>
<li>Documentation work. This tends to be not tracked as new features, but sometimes it appears in release notes, for bigger things. Better to just leave it out. I also don’t think that it affects my hypothesis at all.</li>
<li>Library types implementing new traits. These can be hard to number, especially with generics involved. I decided that it would be better to leave them off.</li>
<li>compiler internal news. We sometimes talk about things like “MIR exists now!” or “we ported the build system from make to cargo!” but similar to documentation, we talk about it very little. It’s also not a change that affects end users, or rather, it affects them more directly by stuff that is counted. MIR enabled NLL, and NLL was tracked as a language feature.</li>
</ul>
<h2>The results &amp; analysis</h2>
<p>I am not as good with Google Sheets as I thought, so I asked for help. Big thanks to <a href="https://twitter.com/ManishEarth">Manish</a> for helping me get this together. I should note that he set me up, but then I tweaked a ton of stuff, so any mistakes are mine, not his. For example, I got a little lazy and didn’t realize that the colors aren’t the same across the various charts. That’s my bad.</p>
<p>Here’s some quick totals of how many changes and of each kind I found, with some sparklines. You can click on any of the images in this post to enlarge them:</p>
<p><img src="https://svbtleusercontent.com/i84Jyi8Ku2vJ2VRGfYmyBV0xspap_small.png" alt="https://svbtleusercontent.com/i84Jyi8Ku2vJ2VRGfYmyBV0xspap_small.png" /></p>
<p>And here’s some charts, per release:</p>
<p><img src="https://svbtleusercontent.com/oZTrBYjy26f5AKURN7z3db0xspap_small.png" alt="https://svbtleusercontent.com/oZTrBYjy26f5AKURN7z3db0xspap_small.png" /></p>
<p>First of all, it is fairly clear that, at least numerically speaking, the standard library is the part of Rust that changes most often. It’s a clear outlier by volume. I find this result kind of hilarious, because Rust is well-known for having a small standard library. We’ll talk more about why I think this is in the “Issues and further research” section below.</p>
<p>Let’s look at the chart without the standard library:</p>
<p><img src="https://svbtleusercontent.com/qXu9v6Li5qzg8Ag6HV1j8p0xspap_small.png" alt="https://svbtleusercontent.com/qXu9v6Li5qzg8Ag6HV1j8p0xspap_small.png" /></p>
<p>You can see there’s a <em>ton</em> of toolchain changes early on. We had a lot of work to do early on, and so made a ton of changes! It’s calmed down in recent times, but toolchain changes happen almost twice as often as language changes. I should also note that this reduction in toolchain changes may be due to methodology; this is talked about later in the post.</p>
<p>So, the core question here: it looks like the language is changing <em>more</em>, recently, rather than less. I think it’s pretty clear that this graph is <em>not</em> exactly what I would imagine, that is, I expected a nice gentle curve down, but that’s not what has happened. I want to dig into this a bit more, but first, let’s look at some other graphs:</p>
<p><img src="https://svbtleusercontent.com/3Xa9rr1SM3atm69gAohMkM0xspap_small.png" alt="https://svbtleusercontent.com/3Xa9rr1SM3atm69gAohMkM0xspap_small.png" /></p>
<p>Changes that add syntax are a subset of language changes. There hasn’t been a whole ton of this, overall. Rust 2018 was the large bump. Half of our releases do not add syntax, though 10 out of 43 did introduce language changes. Our first 29 releases, skipping 1.26, had around one or two changes on average, but every since, it’s been between three and four. I believe this has to do with my methodology, but at the same time, this pretty resoundingly refutes my hypothesis. Very interesting!</p>
<p>Here’s major/medium/minor:</p>
<p><img src="https://svbtleusercontent.com/cm4iWVjB9XZw4qp3c5b94o0xspap_small.png" alt="https://svbtleusercontent.com/cm4iWVjB9XZw4qp3c5b94o0xspap_small.png" /></p>
<p>There’s a peak around Rust 2018, and one from 1.12 to 1.19, but otherwise, it’s been pretty steady lately, in terms of overall changes. If we look at just major changes:</p>
<p><img src="https://svbtleusercontent.com/N3Dud8Du3fUEC8LL8a9Fn0xspap_small.png" alt="https://svbtleusercontent.com/N3Dud8Du3fUEC8LL8a9Fn0xspap_small.png" /></p>
<p>Rust 2018 had a huge bump. After 1.0, and after Rust 2018, it calmed. I think this chart is <em>very</em> interesting, and demonstrates the 3 year edition cycle well. We ship an edition, stuff calms down, things slowly build back up, we ship an edition, they calm down again.</p>
<p>So, why was my hypothesis wrong? I think this conclusion is <em>fascinating</em>. I think part of it has to do with my methodology, and that maybe, that difference explains some of the ways people feel about Rust. You see, even if we release extremely often, and a lot of releases don’t have a <em>ton</em> of changes in them (as you can see, it’s vaguely around 8 or 10, when you exclude the standard library, on average), the very fact that we do release often and have a decent, but fairly small amount of change means that things get into the release notes that may not if we had the exact same amount of change, but did yearly releases.</p>
<p>So for example, in 2019, we released Rust 1.32 to 1.40. That involved 35 language changes. Would I have included, say, “you can now use <code>#[repr(align(N))]</code> on enums” if I had been writing a post about all of these changes for the whole year? Probably not. But because there were only eight total changes in that release, with half of those being language changes, it made sense to include in the 1.37 release post.</p>
<p>Does this mean that folks who read the release posts and think Rust changes a lot are wrong? No, it does not. For those of us in the thick of it, a lot of the smaller changes kind of blur into the background. I actually initially typed “<code>#[repr(N)]</code> on enums” when writing the above sentence, because this change isn’t relevant to me or my work, and is pretty small, and makes things more orthogonal, and so it is just kind of vague background noise to me. But to someone who isn’t as in-tune with the language, it’s harder to know what’s big and what’s small. The endless release posts with tons of stuff in them makes it <em>feel</em> like a lot is happening. But the same amount of stuff may be happening in other languages too, you just don’t see them in release posts, because they only put out one once a year. You both see them less often, as well as see less in them.</p>
<p>I don’t think this means that the Rust project should change its release schedule, and I’m not entirely sure if this means we should change how we write release posts. However, maybe there’s a better way to surface “these are big changes” vs “this is a small thing,” or something like that. I’m not sure. But I do think this helps me understand the disconnect a lot better.</p>
<h2>Issues and further research</h2>
<p>There are some big issues with this methodology. No analysis is perfect, but I want to be fair, so I’m calling out all of the stuff I can think of, at least.</p>
<p>The first of which is, this is inherently subjective. There’s different degrees of subjectivity; for example, “total changes in the notes” is fairly objective. But not completely; folks have to decide what goes into the release notes themselves. So there’s already a layer of filtering going on, before I even started looking at the data. Doing this analysis took me two days of real time, but only a few hours of actual work. Something that analyses the git repo in an automated fashion may see other results. I argue that this level of subjectivity is okay, because we’re also testing something subjective. And I think I designed my methodology in a way that captures this appropriately. On some level, it doesn’t matter if the “real” answer is that Rust rarely changes: folks still get this info by reading the posts, and so I don’t think saying that “oh, yeah you feel that way, but this graph says your feelings are wrong” is really going to help this debate.</p>
<p>The second problem expands on this. We sort of had three eras of release note writing: the first few posts were written by Aaron and Niko. As of Rust 1.4, I took over, and wrote <em>almost</em> every post until 1.33. I then stepped back for a while, though I did help write the 1.42 post. The release team wrote 1.34 onward, and it was more of a collaborative effort than previous posts. This means that the person doing the filter from release notes -&gt; blog post changed over time. This can mess with the numbers. For example, there are some blog posts where I would have maybe omitted a small family of features, but the folks writing that post included them all, individually. This may explain the uptick in language changes lately, and why many of them were not rated as “major.” Additionally, the people who did the filter from “list of PRs” to “release notes” <em>also</em> changed over time, so there’s some degree of filtering change there, as well.</p>
<p>To complicate things further, starting with Rust 1.34, Cargo lost its section in the release blog posts. Big changes in Cargo still made it into the text, but the previous posts had more robust Cargo sections, and so I feel like post-1.34, Cargo was under-counted a bit. Cargo changes are toolchain changes, and so the recent calm-ness of the toolchain numbers may be due to this. This still is okay, because again, this analysis is about folks reading the release notes, but it’s worth calling out. Likewise, I think the standards for writing the library stabilizations changed a bit over time too; it’s gone from more to less comprehensive and back again a few times.</p>
<p>This was mentioned above a bit, but for completeness, the structure of Rust releases means there’s a variable criteria for what makes it into a release post. If it’s a smaller release, smaller changes may get featured, whereas if those same feature were in a bigger release, they may not have been. I think this aspect is a significant factor in the rejection of my hypothesis, as I mentioned above.</p>
<p>I think there’s interesting future research to be done here. I initially tried to track deprecations and soundness fixes, but soundness fixes happened so little it wasn’t worth talking about. There were 7 in those 42 releases. This is another weakness / area for future research, because I did not include point releases, only 1.x.0 releases. This would have added a few more soundness fixes, but it also would have added a lot of random noise where not much happened in the graphs, so that’s why I left those out. They also don’t happen on the same regular schedule, so time would get a bit funny… anyway. Deprecations was also just hard to track because there weren’t a lot of them, and sometimes it was hard to say if something was deprecated because of a soundness issue vs other reasons.</p>
<p>Is my critera for adding syntax correct? I think a reasonable person could say “no.” If you have something that’s special cased, and you’re making it more general, there’s a good argument that this is <em>removing</em> a restriction, rather than adding a feature. But reasonable folks could also say you’ve added something. I don’t think that this part is core to my analysis, and so I think it’s fine that I’ve made this decision, but you may not care for this section of results if you disagree.</p>
<p>This analysis doesn’t get into something that I think is a huge deal: the ecosystem vs the language itself. I only track the Rust distribution itself here, but most Rust programmers use many, many ecosystem libraries. When people talk about churn in Rust, do they really mean the ecosystem, not the language? In some sense, doing so is correct: Rust having a small standard library means that you have to use external packages a lot. If those churn a lot, is it any better than churn in the language itself?</p>
<p><em>Does</em> Rust have a small standard library? There were 962 changes in 42 releases, that’s almost 23 changes per release. A lot of them are small, but still. Maybe Rust has a small, but deep standard library? Due to coherence, I would maybe think this is true. Just how big are standard libraries anyway? I’ve only seen discussion of this concept in terms of design philosophy. I’ve never seen a numerical analysis of such. Is it out there? Do you know of one? I’d love to hear about it!</p>
<p>I think there’s a number of reasons that the standard library changes dominate the total change rate. Primarily, this section of the blog post <em>tends</em> to be the most complete, that is, out of the total amount of changes, more standard library changes make it into blog posts than other kinds of changes. Why? Well, it’s really easy to measure: count up stuff stabilized and put it in a big list in the post. It’s also about how these changes are measured; when a method is added to every number type, that’s <code>u8</code> + <code>u16</code> + <code>u32</code> + <code>u64</code> + <code>u128</code> = <code>5</code> changes, even if conceptually it’s one change, in a different sense.</p>
<p>This leads to another question about those standard library changes: what is up with that weird peak around Rust 1.33? Well, we added the <code>const fn</code> feature in Rust 1.31. After that landed, we could <code>const</code>-ify a number of functions in the standard library. The initial huge bulk of changes here landed in 1.33. It had 138 changes, but only 13 were not “this function is now <code>const</code>.” And the feature continues to drive change outside of 1.33; the initial landing of <code>const fn</code> in 1.31 was very limited, and as it expands, more functions that already existed can be made <code>const</code>.</p>
<p>In conclusion, even though I still believe that Rust has slowed down its rate of change a lot, I think that it makes total sense that not everyone agrees with me. By some metrics, I am just flat-out wrong.</p>
</content:encoded></item><item><title>"C is how the computer works" is a dangerous mindset for C programmers</title><link>https://steveklabnik.com/writing/c-is-how-the-computer-works-is-a-dangerous-mindset-for-c-programmers/</link><guid isPermaLink="true">https://steveklabnik.com/writing/c-is-how-the-computer-works-is-a-dangerous-mindset-for-c-programmers/</guid><pubDate>Tue, 31 Mar 2020 00:00:00 GMT</pubDate><content:encoded><p>A long time ago, I wrote <a href="https://words.steveklabnik.com/should-you-learn-c-to-learn-how-the-computer-works">“Should you learn C to “learn how the computer works”?"</a>. This was part 1 in a three part series. A year later, I got around to part 2, <a href="https://words.steveklabnik.com/c-is-not-how-the-computer-works-can-lead-to-inefficient-code">““C is not how the computer works” can lead to inefficient code"</a>.</p>
<p>It’s been five more months. I don’t think I really want to write out the third part; I expected this to be a quick series of blog posts, not something that was on my plate for a year and a half. Here’s the thesis of part 3:</p>
<blockquote>
<p>Part three is going to show what happens if you make a mistake with the ideas from part two. If you incorrectly assume that C’s abstract model maps directly to hardware, you can make mistakes. This is where UB gets dangerous. While you can take advantages of some properties of the machine you’re relying on, you have to know which ones fit within C’s model and which ones don’t.</p>
</blockquote>
<p>I honestly don’t feel like writing more than that, so I’m just gonna leave it there. Basically, the overall thrust of this series has been this: C is not the hardware, it’s an abstract machine. But that machine runs on real hardware, and abstractions are leaky. If you go too far into “purely only the abstract machine,” you may not be able to accomplish your tasks. If you go too far into “C runs directly on the hardware,” you may be surprised when a compiler does something that you didn’t expect.</p>
</content:encoded></item><item><title>A sad day for Rust</title><link>https://steveklabnik.com/writing/a-sad-day-for-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/a-sad-day-for-rust/</guid><pubDate>Fri, 17 Jan 2020 00:00:00 GMT</pubDate><content:encoded><p><code>actix-web</code> is dead.</p>
<p>This situation is bad, from all sides. When Rust was a tiny, tiny community, I thought to myself, “wow, I wonder how long this will last? Probably 1.0 will kill it.” Sort of playing off of <a href="https://en.wikipedia.org/wiki/Eternal_September">Eternal September</a>, I assumed that over time, the community would grow, and we’d encounter problems.</p>
<p>Today is the first day where I say to myself, okay, has that happened?</p>
<p>This story is not super clear-cut. I’m not going to link to a dozen citations, or try to prove that I’m some sort of neutral party here. I’m going to give you account of this story as I remember it and as I felt it. Because this isn’t really about playing judge. This is about thinking about the future.</p>
<p>It’s been very clear from the beginning that the Rust project saw Rust as more than just the language. The community and the people mattered. From the earliest days, leadership explicitly took the position that it wasn’t just the code, but the people around the project were important. Of course, people are also people, and so this wasn’t perfect; we’ve made several fairly large mis-steps here over the years. But Rust has been an experiment in community building as much as an experiment in language building. Can we reject the idea of a BDFL? Can we include as many people as possible? Can we be welcoming to folks who historically have not had great representation in open source? Can we reject <a href="https://blog.aurynn.com/2015/12/16-contempt-culture">contempt culture</a>? Can we be inclusive of beginners?</p>
<p>Rust has a necessary feature, <code>unsafe</code>, that allows you to escape Rust’s guarantees. This is a really important part of Rust, but also a very dangerous one, hence the name. But one of the things that’s important about it is that, since <code>unsafe</code> means “I have checked this code, and it upholds all of the guarantees the compiler cannot check,” you can build a safe interface with unsafe guts.</p>
<p>Interacting with anything outside of the language, for example, using the operating system to print to the screen, or write a file, has to be <code>unsafe</code>, and so without this property, all programs would be <code>unsafe</code>. This premise is treated skeptically by many, but has generally been demonstrated to have worked out. Several folks are in the process of providing tooling to prove that your <code>unsafe</code> code is correct, and provided proofs for a bunch of the standard library’s <code>unsafe</code> code. It not only found a few places where there were bugs, but also a few places where the restrictions were too tight!</p>
<p>But “generally” is doing a lot of work in that previous sentence. This setup creates a gnawing fear in the back of people’s minds: what if some foundational package uses <code>unsafe</code>, but uses it incorrectly? What happens when this causes problems for every package that uses that package? This relationship between safe and unsafe is also a bit hard to understand, and so, when bugs are found around unsafe, people <em>outside</em> of Rust often use it to suggest that all of Rust is a house of cards. While Rust has “made it” in many senses, it has not quite yet in many others, and so I think this produces a fundamental anxiety in the community.</p>
<p>Speaking of making it, actix-web is a good web framework. It came onto the scene, and people generally liked it. It also performed extremely well. Its score on <a href="https://www.techempower.com/benchmarks/">Techempower</a> in particular was impressive, and got more impressive over time. Its author also works at Microsoft, and suggested that Microsoft is using actix-web in production. This was really exciting to people. This was the second project using Rust at Microsoft, and so this seemed really fantastic.</p>
<p>But then, the anxiety.</p>
<p>Before we get into that, though, there’s also the matter of Reddit. The team has basically rejected Reddit for said social reasons. We provide alternate forums for folks, but Reddit is a huge place. The Rust subreddit has ~87,000 subscribers, partially thanks to this. And, for a while, the Rust reddit was a decent place. I still think it’s better than most reddits, but it’s degraded much more quickly than the community at large. “Why Reddit is like this” is a whole other essay; I think it’s built into reddit’s structure itself. But the point is, while Reddit is not official, and so not linked to by any official resources, it’s still a very large group of people, and so to suggest it’s “not the Rust community” in some way is both true and very not true. For the purposes of this story, I think it has to be included in the broader community, and I think that this situation has brought new questions about the relationship of the project and Reddit, though this is already far too long.</p>
<p>So, someone takes a peek under the covers, and it turns out actix-web is using a <em>lot</em> of unsafe code. That’s not inherently a problem. The problem is, a lot of that unsafe code is not actually needed. This is where things start to go wrong.</p>
<p>The author of actix-web basically says “nah it’s fine.” This is basically the perfect storm of unsafe anxiety: a big, important, visible library, a bunch of <code>unsafe</code>, and an unresponsive author.</p>
<p>Now, it’s important to mention that people were not just complaining: they were writing code. It is easy to frame this as a case of open source entitlement, and it still very may well be, but this isn’t just empty complaints. Patches were included, they just weren’t welcome. And that’s also not inherently wrong either; a project maintainer shouldn’t be required to accept a patch just because it exists…</p>
<p>Reddit whips itself into a frenzy. Lots of ugly things are said. More issues are opened, and closed, and locked. Eventually, the author accepts some code that drops the total unsafe count significantly and has more sound usage of what’s left.</p>
<p>People were not happy about how all this played out, for a variety of reasons and in a variety of ways. I haven’t spoken to actix-web’s author, but I can’t imagine that he was very happy about it either.</p>
<p>And then it happens all over again. More unsafe, more patches rejected, more Reddit, more bad blood. I thought that at this point, the author said he was quitting, but that may be faulty memory, or maybe he changed his mind. Regardless, how responsive would you be, if you were the maintainer, after the shit-show of last time?</p>
<p>People were not happy about how all this played out, for a variety of reasons and in a variety of ways.</p>
<p>All was quiet for a while, and actix-web kept climbing the Techempower benchmarks, reaching the top of most of them by a <em>lot</em>. It is wicked fast.</p>
<p>And then, yesterday, a post titled <a href="https://medium.com/@shnatsel/smoke-testing-rust-http-clients-b8f2ee5db4e6">Smoke-testing Rust HTTP clients</a> gets published. It focuses on a, well, smoke test of Rust HTTP clients, but it also comments a lot on the complexity of each project, and the amount of <code>unsafe</code>. The author found and filed a lot of bugs.</p>
<p>But:</p>
<blockquote>
<p>A quick glance at the dependencies reveals that it relies on actix-service, which underpins all of Actix and has a bespoke and unsound Cell implementation. For example, this method violates memory safety by handing out multiple mutable references to the same data, which can lead to e.g. a use-after-free vulnerability. I have reported the issue to the maintainers, but they have refused to investigate it.</p>
</blockquote>
<p>This causes the now-usual Reddit uproar. It’s extra nasty this time. Some people go far, far, far over the line.</p>
<p>And now the maintainer has quit.</p>
<p>This means, on some level, this situation is over: there will not be a fourth huge actix-web drama. But it’s really left a bad taste in my mouth. It’s unfortunate that a well-known project had soundness issues. It’s unfortunate that the maintainer wasn’t receptive to fixing them. At the same time, I’ve been there: being a maintainer is tough. It’s also unfortunate that there’s this <em>style</em> of response, and kind, and volume. It <em>is</em> true that the situation was dangerous. But did it have to be handled this way? You have to understand a lot of nuance here to even know the main points of the story. One version of this story that will certainly be told is “The Rust community says they’re nice but they will harass you if you use unsafe wrong.” Is that what we want? I, for one, do not. If you’re a part of Rust, you gotta ask yourself: are you happy with this outcome? How did your actions (or lack thereof) contribute to it? Could you have done something better?</p>
<p>I’m not sure where we go from here, and I’m not sure what we could have done to prevent this from happening. But I feel like this is a failure, and it’s set Rust back a bit, and I’m just plain sad.</p>
<p>EPILOGUE</p>
<p>actix-web has new maintainers, and will now live on: https://github.com/actix/actix-web/issues/1289</p>
</content:encoded></item><item><title>"C is not how the computer works" can lead to inefficient code</title><link>https://steveklabnik.com/writing/c-is-not-how-the-computer-works-can-lead-to-inefficient-code/</link><guid isPermaLink="true">https://steveklabnik.com/writing/c-is-not-how-the-computer-works-can-lead-to-inefficient-code/</guid><pubDate>Fri, 25 Oct 2019 00:00:00 GMT</pubDate><content:encoded><p>A little over a year ago, I wrote <a href="https://words.steveklabnik.com/should-you-learn-c-to-learn-how-the-computer-works">“Should you learn C to ‘learn how the computer works’”</a>. It was a bit controversial. I had promised two follow-up posts. It’s taken me a year, but here’s the first one.</p>
<p>In that post, I argued that C doesn’t “work like the computer”, but rather, by a concept called the “C abstract machine.” It happens to be close to how computers operate in some sense, but that when you write C, you’re writing for the abstract machine, not the computer itself. Here’s the trick, though: the C abstract machine doesn’t define every single aspect of how computation happens. And so sometimes, you can write code in two different ways, that the C abstract machine says are equivalent, yet are <em>not</em> equivalent in performance. This means that, even if you buy into my original point, you still need to know the boundaries of this definition, and can exploit those boundaries in important ways.</p>
<p>Let’s start with a made-up example, and then we’ll show how this happens in real code. Imagine that we have a very simple programming language, built on an abstract machine. It defines only the semantics for the multiplication of two 32-bit integers. It also ignores things like overflow. You could write a program like “3 * 4” and this machine would say “hey the result is 12.” Let’s also imagine we have a very simple computer: it has a single instruction, “add two numbers together.” This takes one clock cycle to execute. If we wrote a compiler for this machine, and didn’t apply any optimizations, it would execute “4 + 4”, and then take that result and add four to it. We have two add operations, and so our program takes two clock cycles.</p>
<p>Let’s imagine a new version of our computer comes out. It adds a new instruction: multiplication. This also takes one cycle, but can multiply two numbers directly. Our compiler adds a new back-end, and when compiling our “3 * 4” program for this fancy new machine, it emits a single multiplication instruction. Our program now takes one clock cycle.</p>
<p>Now imagine a new version of our compiler. It is able to do “constant folding”, an optimization. Our program does no computation at all, only produces a “12” directly. This now means our program takes zero cycles on both bits of hardware.</p>
<p>Is all of this okay?</p>
<p>Yes! The machine only defines how the <code>*</code> operator works on numbers in a “what number comes out the other side” sense. It doesn’t specify how to actually accomplish this task. Compilers for our little language are free to implement the details in any way they choose, as long as they respect the rules of the abstract machine. Other details are fair game. As long as the program “3 * 4” produces “12”, the rest of the computational process doesn’t actually matter.</p>
<p>This is how you can write “portable” code that is portable in one sense, but not at all in another sense. As long as you respect the rules of the abstract machine, you’ll be guaranteed (we live in a world with no compiler bugs, of course) to get the correct result. If it takes too long, or takes no time at all, that’s not the language’s concern, strictly speaking.</p>
<p>If this idea seems silly to you, well, I can assure you it’s real. For example, let’s consider the actual C language, and some actual hardware, specifically, x86.</p>
<p>Imagine we have an array of arrays, four by four:</p>
<pre><code>0 0 0 0
0 0 0 0
0 0 0 0
0 0 0 0
</code></pre>
<p>There are two naive ways to process it: row by row:</p>
<pre><code>1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
</code></pre>
<p>or column by column:</p>
<pre><code>1 5 9 13
2 6 10 14
3 7 11 15
4 8 12 16
</code></pre>
<p>Here’s a fun question: is one of these faster than the other?</p>
<p>At this small size, probably not. But there’s a way to do this that’s faster than either of these ways, and that’s called “blocking” or “tiling”. It means that you process a sub-set, or “block”, of the array each time. Something like this:</p>
<pre><code>1 2 5 6
3 4 7 8
9 10 13 14
11 12 15 16
</code></pre>
<p>On x86, this is <em>meaningfully</em> faster in many circumstances. This has to do with the way that the CPU caches information. For more, see <a href="https://software.intel.com/en-us/articles/how-to-use-loop-blocking-to-optimize-memory-use-on-32-bit-intel-architecture">this page from Intel</a>, including a visualization that’s probably better than mine.</p>
<p>This raises an interesting point: is this code portable? Imagine a machine that didn’t have the cache behavior of x86 chips here. This code would run on them, but it may be 14 times slower. Strictly speaking, this program is “portable” in the sense that you’ll get the right computation in the end, but maybe this speed is part of the user requirements for this software to be successful, and so being super slow means that it “doesn’t work” according to those requirements, even though from a programming language point of view, it’s all good.</p>
<p>Because C’s abstract machine is so thin, these kinds of details can really, really matter, as we’ve seen. And this is where the good part of the “C teaches you how the computer works” meme comes in. Because the machine is thinner, you can learn more about the details, and as long as they’re not details that the C abstract machine cares about, exploit them. And there’s a good reason I mentioned an optimizing compiler above; it’s also the job of compiler authors to realize the difference between a given computer and the C abstract machine, and then exploit the difference to make your code go as fast as possible. But this really blurs the line between abstract machine and physical machine. And that’s why we all argue online about this all the time.</p>
<hr />
<p>I have a third post to wrap up this series. I hope it won’t take me another year, but no guarantees…</p>
</content:encoded></item><item><title>Porting steveklabnik.com to Workers Sites and Zola</title><link>https://steveklabnik.com/writing/porting-steveklabnik-com-to-workers-sites-and-zola/</link><guid isPermaLink="true">https://steveklabnik.com/writing/porting-steveklabnik-com-to-workers-sites-and-zola/</guid><pubDate>Thu, 26 Sep 2019 00:00:00 GMT</pubDate><content:encoded><p>One fun thing about having a personal domain is that you can keep the contents the same forever, but take a few days every so often to port it to some new stack you want to try out.</p>
<p>A while back, I had my site on <a href="https://pages.github.com/">GitHub Pages</a> using <a href="https://jekyllrb.com/">Jekyll</a>. This has long been the sort of default, go-to stack for me. At some point, I decided to simplify even further, and wrote a small script to take the few Markdown files I have and just make HTML out of them. I then used <a href="https://www.netlify.com/">Netlify</a> to do the hosting. I wanted to try out their legendary ease of use, and it was indeed super simple.</p>
<p>Earlier this year, I <a href="https://words.steveklabnik.com/i-m-joining-cloudflare">took a job at Cloudflare</a> to work on part of <a href="https://workers.cloudflare.com/">Cloudflare Workers</a>. To get more familiar with the project I’d be working on, <a href="https://www.cloudflare.com/products/workers-kv/">Workers KV</a>, I came up with a plan: I’d put the Markdown files into KV, and then write a worker in Rust and WebAssembly that would generate the HTML on the fly, and run that in a Worker. This worked really well, even though it was <em>incredibly</em> overkill for a simple static site.</p>
<p>Today we’re launching a new product, <a href="https://blog.cloudflare.com/workers-sites/">Workers Sites</a>. You see, I wasn’t the only one who was interested in using Workers for static site hosting; I’d see tweets every so often where others were doing the same. But doing so was a completely manual process, you had to write the code on your own, you had to do all the setup yourself, and there was no assistance in doing so. Workers Sites changes this, and makes it easy to host your own site, if you’d like.</p>
<p>The rest of this post is going to walk you through my porting process for <a href="https://www.steveklabnik.com/">steveklabnik.com</a>.</p>
<hr />
<p>I decided to go with <a href="https://www.getzola.org/">Zola</a> for my static site generator, as it’s written in Rust. I’d been wanting to give Zola a try for a while, and this was a good excuse to do so.</p>
<p>I needed these pre-requisites:</p>
<ul>
<li>Zola, instructions <a href="https://www.getzola.org/documentation/getting-started/installation/">here</a>.</li>
<li>Wrangler, instructions <a href="https://github.com/cloudflare/wrangler">here</a></li>
<li>A Cloudflare Workers account, <a href="https://workers.cloudflare.com/">here</a> if you don’t have a Cloudflare account already, and if you do, you can click the Workers tab in your dashboard to set it up.</li>
</ul>
<p>Additionally, you need to have a $5/month subscription to Workers, to get access to Workers KV. I want us to offer a free tier of KV in the future, but we’re not there yet. That $5/month gets you a lot more than one site, to be clear, but this isn’t a completely free solution today.</p>
<p>From there, it was time to make a new site with Zola:</p>
<pre><code>&gt; zola init steveklabnik.com
&gt; cd steveklabnik.com
</code></pre>
<p>Zola places the contents of your site in the <code>contents</code> directory. I only have one level for my site, so I dumped my two pages in there, as well as re-named <code>index.md</code> to <code>_index.md</code> to follow Zola convention.</p>
<p>I had a few YAML headers on pages, I had to convert those to Zola’s TOML format:</p>
<pre><code>+++
title = "Deleuzional"
+++
I really like the writing of [Gilles Deleuze]. I need
</code></pre>
<p>The <code>+++</code>s are like <code>---</code> in Jekyll, and the metadata is in TOML instead of YAML. I only needed <code>title</code> because I am keeping things very simple.</p>
<p>I then needed to set up templates. Like most static site generators, the template forms the outer contents of the HTML on a page, and renders the Markdown into the middle somewhere. I have a slightly different template for my root than the other two pages, and Zola handles this pretty well.</p>
<pre><code>&gt; ls .\templates\
Directory: C:\Users\Steve Klabnik\src\steveklabnik.com\templates
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 9/25/2019 5:28 PM 1288 index.html
-a---- 9/25/2019 5:28 PM 1234 page.html
</code></pre>
<p><code>index.html</code> is for the root page, and <code>page.html</code> is for the other pages. This is a Zola convention.</p>
<p>Zola templates are written in <a href="https://tera.netlify.com/docs/templates/">Tera</a>, which is sorta like <a href="https://shopify.github.io/liquid/">Liquid</a> or <a href="https://jinja.palletsprojects.com/en/2.10.x/">Jinja2</a>. Here’s a snippit of my template:</p>
<pre><code> &lt;body&gt;
&lt;h2&gt;{{ page.title }}&lt;/h2&gt;
{{page.content | safe}}
&lt;/body&gt;
</code></pre>
<p>The <code>| safe</code> bit says that it shouldn’t escape the HTML produced by <code>page.content</code>, which is the rendered contents of the Markdown pages.</p>
<p>Finally, I have some static assets for the site: a CSS file, some JS, and the favicon, etc. These get dumped in the <code>static</code> directory, and Zola will copy them over when I build the site.</p>
<p>Previewing during this process is very useful:</p>
<pre><code>&gt; zola serve
</code></pre>
<p>It took me a bit to figure out the conventions and get everything in place, and being able to change the contents and watch my browser refresh was invaluable.</p>
<p>Once I had my site ported to Zola, it’s time to deploy it to Workers Sites!</p>
<p>Wrangler is the command-line tool for Cloudflare Workers. If you haven’t used it before, you’ll have to log in:</p>
<pre><code>&gt; wrangler config
</code></pre>
<p>To check that your auth is set up correctly, you should run this command:</p>
<pre><code>&gt; wrangler whoami
You are logged in with the email '[email protected]'.
</code></pre>
<p>Of course, this should show your own email, not mine.</p>
<p>Let’s add Workers Sites to our project:</p>
<pre><code>&gt; wrangler init --site website
</code></pre>
<p>This creates two main things: <code>wrangler.toml</code>, used to configure Wrangler, and <code>workers-sites</code>, a directory with your worker in it. I used the extra <code>website</code> parameter to wrangler becuase it defaults to the name of the subdirectory, and the period in <code>steveklabnik.com</code> is invalid in a worker name, so I chose <code>website</code>. You can put whatever you’d like.</p>
<p>We need to set up <code>wrangler.toml</code>:</p>
<pre><code>&gt; code wrangler.toml
</code></pre>
<p>Mine looks like this:</p>
<pre><code>account_id = "f5518bfbf88c230551a64797c221c7da"
name = "website"
type = "webpack"
route = "www.steveklabnik.com/*"
zone_id = "9359b254c46ff9f5c546203686794862"
workers_dev = false
[site]
bucket = "./public"
entry-point = "workers-site"
</code></pre>
<p>All of these IDs are fine to be public; <code>wrangler</code> stores your sensitive stuff in <code>~/.wrangler</code>.</p>
<p>Important bits:</p>
<p>The <code>route</code> needs to be set up properly; this one means that the worker will serve everything on <code>www.steveklabnik.com</code>. My account and zone IDs are needed; you can find these on the Cloudflare dashboard on the bottom right of the main page for it. <code>workers_dev</code> is false; I’m deploying to my site, not to a subdomain of <code>workers.dev</code>. If we did, we wouldn’t need the <code>zone_id</code>, or <code>route</code>. Finally, the <code>bucket</code> setting needs to be set to whatever your site generator’s output directory is, which for Zola is <code>public</code>.</p>
<p>And that’s it! You can test out what this looks like:</p>
<pre><code>&gt; wrangler preview
</code></pre>
<p>If that all looks good, we can deploy:</p>
<pre><code>&gt; zola build
&gt; wrangler publish
</code></pre>
<p>And that’s it! And wow is it fast:</p>
<p><img src="https://svbtleusercontent.com/hS1xpZgYuQ9sRhUxP1iqq20xspap_small.png" alt="https://svbtleusercontent.com/hS1xpZgYuQ9sRhUxP1iqq20xspap_small.png" /></p>
<hr />
<p>… well, we’re <em>almost</em> done. Right now, this means I can only publish from my laptop. I’d prefer to have GitHub deploy this. We don’t have that as a built-in feature today, so I’d have to write an Action to do so. I haven’t yet, so I can’t blog about that. Maybe next time.</p>
<p>Additionally, there’s one more twist. Zola generates different URLs than my previous ones. I had previously had <code>.html</code> on the ends of my pages, but Zola does not do this.</p>
<ul>
<li>old: https://www.steveklabnik.com/deleuzional.html</li>
<li>new: https://www.steveklabnik.com/deleuzional/</li>
</ul>
<p>I think I’d prefer the new style anyway, so rather than figure out how to get Zola to do this, let’s set up some redirects. This is one cool thing about using Workers for this kind of task; while it’s slightly more manual, we can also make customizations very easily. The contents of the Worker that serves up the site lives in <code>workers-sites/index.js</code>. It’s very short.</p>
<pre><code>import { getAssetFromKV } from "@cloudflare/kv-asset-handler";
addEventListener("fetch", event =&gt; {
event.respondWith(handleEvent(event));
});
async function handleEvent(event) {
try {
return await getAssetFromKV(event);
} catch (e) {
let pathname = new URL(event.request.url).pathname;
return new Response(`"${pathname}" not found`, {
status: 404,
statusText: "not found"
});
}
}
</code></pre>
<p>Given that I only have two URLs, rather than doing something big and fancy, I did something dumb and straightforward:</p>
<pre><code>import { getAssetFromKV } from "@cloudflare/kv-asset-handler";
addEventListener("fetch", event =&gt; {
event.respondWith(handleEvent(event));
});
async function handleEvent(event) {
// redirect some old URLs
let pathname = new URL(event.request.url).pathname;
if (pathname === "/security.html") {
return Response.redirect("https://www.steveklabnik.com/security/", 301);
} else if (pathname === "/deleuzional.html") {
return Response.redirect("https://www.steveklabnik.com/deleuzional/", 301);
}
try {
return await getAssetFromKV(event);
} catch (e) {
let pathname = new URL(event.request.url).pathname;
return new Response(`"${pathname}" not found`, {
status: 404,
statusText: "not found"
});
}
}
</code></pre>
<p>I’m constructing <code>pathname</code> twice here, but I just don’t care enough to bother to refactor this out. Additionally, it’s nice to purely add, rather than customize what was generated too much.</p>
<p>But with that, and another <code>zola build &amp;&amp; wrangler publish</code>, I have the redirects set up for my site.</p>
</content:encoded></item><item><title>The culture war at the heart of open source</title><link>https://steveklabnik.com/writing/the-culture-war-at-the-heart-of-open-source/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-culture-war-at-the-heart-of-open-source/</guid><pubDate>Sun, 26 May 2019 00:00:00 GMT</pubDate><content:encoded><p>There’s a war going on. When isn’t there a war going on? But I’m not talking about a physical war here: I’m talking about a war over meaning. This particular war is a fight over what “open source” means.</p>
<p>Let’s take a few steps back.</p>
<h2>The Free Software Foundation</h2>
<p>People organize into groups for many reasons. This story starts with the story of an organization called the “GNU Project.” It was started in 1983, and <a href="https://groups.google.com/forum/#!msg/net.unix-wizards/8twfRPM79u0/1xlglzrWrU0J">here’s the initial announcement on USENET</a>. I’ve pulled out four important paragraphs:</p>
<blockquote>
<p>Starting this Thanksgiving I am going to write a complete Unix-compatible software system called GNU (for Gnu’s Not Unix), and give it away free to everyone who can use it. Contributions of time, money, programs and equipment are greatly needed.I consider that the golden rule requires that if I like a program I must share it with other people who like it. I cannot in good conscience sign a nondisclosure agreement or a software license agreement.So that I can continue to use computers without violating my principles, I have decided to put together a sufficient body of free software so that I will be able to get along without any software that is not free.If I get donations of money, I may be able to hire a few people full or part time. The salary won’t be high, but I’m looking for people for whom knowing they are helping humanity is as important as money. I view this as a way of enabling dedicated people to devote their full energies to working on GNU by sparing them the need to make a living in another way.</p>
</blockquote>
<p>There’s a lot of different ways to talk about this post: it contains all sorts of interesting things. But for right now, I want to talk about the motivations here. There were two main ones: first, to produce a software artifact. The second one is the big one: the artifact needs to be produced because existing ones were not compatible with a particular set of values.</p>
<p>The word “ideology” is a tricky one, but let’s go with this definition, which is <a href="https://en.wikipedia.org/wiki/Ideology">from Wikipedia</a>:</p>
<blockquote>
<p>An ideology is a collection of normative beliefs and values that an individual or group holds for other than purely epistemic reasons.</p>
</blockquote>
<p>The GNU project was formed to produce software according to a particular ideology: one of sharing software. I’m not here to argue if the project has accomplished this goal, or if this goal is good or not. My point is that the origins of the GNU project were specifically motivated by a collection of normative beliefs and values.</p>
<p>Two years later, the Free Software Foundation would be formed to support the GNU project, and promote the concept of Free Software. Free Software was software that aligns with the ideology of the GNU project, and a definition of Free Software was created and published in Feburary of 1986. Here is that definition:</p>
<blockquote>
<p>The word “free” in our name does not refer to price; it refers to freedom. First, the freedom to copy a program and redistribute it to your neighbors, so that they can use it as well as you. Second, the freedom to change a program, so that you can control it instead of it controlling you; for this, the source code must be made available to you.</p>
</blockquote>
<p>Since then, the Free Software Definition has expanded to four points. You can read the current definition <a href="https://www.gnu.org/philosophy/free-sw.en.html">here</a>.</p>
<h2>Open Source appears</h2>
<p>A decade passes, and trouble is brewing. I’ll <a href="https://en.wikipedia.org/wiki/Open_source#Origins">quote wikipedia again</a>:</p>
<blockquote>
<p>[T]he modern meaning of the term “open source” was first proposed by a group of people in the free software movement who were critical of the political agenda and moral philosophy implied in the term “free software” and sought to reframe the discourse to reflect a more commercially minded position. In addition, the ambiguity of the term “free software” was seen as discouraging business adoption. The group included Christine Peterson, Todd Anderson, Larry Augustin, Jon Hall, Sam Ockman, Michael Tiemann and Eric S. Raymond. Peterson suggested “open source” at a meeting held at Palo Alto, California, in reaction to Netscape’s announcement in January 1998 of a source code release for Navigator. Linus Torvalds gave his support the following day, and Phil Hughes backed the term in Linux Journal.</p>
</blockquote>
<p>Here we see the creation of the open source movement. If you’d prefer a primary source, here’s <a href="http://www.catb.org/~esr/open-source.html">Eric S. Raymond</a>:</p>
<blockquote>
<p>Specifically, we have a problem with the term “free software”, itself, not the concept. I’ve become convinced that the term has to go.The problem with it is twofold. First, it’s confusing; the term “free” is very ambiguous (something the Free Software Foundation’s propaganda has to wrestle with constantly). Does “free” mean “no money charged?” or does it mean “free to be modified by anyone”, or something else?Second, the term makes a lot of corporate types nervous. While this does not intrinsically bother me in the least, we now have a pragmatic interest in converting these people rather than thumbing our noses at them. There’s now a chance we can make serious gains in the mainstream business world without compromising our ideals and commitment to technical excellence – so it’s time to reposition. We need a new and better label.</p>
</blockquote>
<p>Shortly after, the <a href="http://opensource.org/">Open Source Initiative</a> was founded. A sort of mirror of the FSF, the OSI would support the promotion of the term “open source,” and the ideology behind it. Like the Free Software Definition, the <a href="https://opensource.org/osd">Open Source Definition</a> was created, derived from Debian’s guidelines for producing Free Software.</p>
<p>Again, we have an organization that’s created along ideological lines. But in this case, slightly different ones. <a href="https://web.archive.org/web/20090413035630/https://opensource.org/history">An older version of the OSI website says</a>:</p>
<blockquote>
<p>The conferees decided it was time to dump the moralizing and confrontational attitude that had been associated with “free software” in the past and sell the idea strictly on the same pragmatic, business-case grounds that had motivated Netscape.</p>
</blockquote>
<p><a href="https://opensource.org/history">Today’s version says</a>:</p>
<blockquote>
<p>The conferees believed the pragmatic, business-case grounds that had motivated Netscape to release their code illustrated a valuable way to engage with potential software users and developers, and convince them to create and improve source code by participating in an engaged community. The conferees also believed that it would be useful to have a single label that identified this approach and distinguished it from the philosophically- and politically-focused label “free software.”</p>
</blockquote>
<p>The idea here is plain: Free Software, but for business.</p>
<p>For more on this topic, I recommend <a href="https://thebaffler.com/salvos/the-meme-hustler">The Meme Hustler</a>.</p>
<h2>Moving forward</h2>
<p>Twenty years have passed, and a lot has changed.</p>
<p>For ideological movements to persist, they need to re-produce their values in new membership. And for a while, the FSF and OSI did a good job of that. These two movements, Free Software and Open Source, produced a lot of software, and gained a lot of new converts. But then… something happened.</p>
<p>I’m not sure exactly how it happened. I think the lazy answer is “GitHub!!!!”. I do think GitHub played a role, but I think the answer is more complex than that. I personally think that gender plays a huge role. But that’s a different essay. Regardless of <em>why</em> it happened, something did happen.</p>
<p>Somewhere along the way, Open Source ran into a problem that many movements face: the members of the movement no longer understood the ideology that created the movement in the first place.</p>
<p>If you ask a random developer what “open source” means to them, you won’t often hear “software that follows the Open Source Definition.” If you ask them “what’s the difference between free software and open source software,” you’ll often hear “aren’t those the same thing?” or “you can charge money for open source software, it’s not always free.” You may even hear “it’s on GitHub.”</p>
<p>In talking to developers about open source, you’ll also hear something else. Something that’s been itching in the back of my brain for a while, and the thing that led me to write this. You’ll often hear developers talk about how the relationship between businesses and open source developers is messy. Complicated. Businesses don’t “give back enough,” won’t pay people to work on open source. That it’s all take and no give. I’ve said a lot of that stuff myself.</p>
<p>But here’s the thing: that’s why the concept of open source was created in the first place. It’s there, plain as day, if you ask the folks who actually created it. Open source was a way to make free software better for businesses. This doesn’t mean that it’s beyond critique, of course. The relationship can improve. I don’t think these developers are stupid, or hypocrites.</p>
<p>This is where we’re at today. The Free Software movement was created, and then Open Source was created as a reaction to that. Today’s developers have never learned about this history, or don’t care about it, or actively think it’s irrelevant. And so we have a war. A war for the meaning of open source. A war fought with tweets, and blog posts, and discussions. A war between the old guard, those who created the idea originally, and the new generation, the developers who have taken the base idea and run with it.</p>
<p>I think history will repeat itself. In the same way that the open source movement said “we’re like free software, but with these changes,” I think we’ll end up with a new movement. For the same reasons that “open source” came up with a new name, I think the movement that will arise from today’s developers will also need a new name. I’m not sure what that movement will look like, and I’ll explore why in another post. To give you a teaser: the problem is in the way that both Free Software and Open Source are formulated. The rot is in the roots, and I’m not yet sure what will replace it.</p>
</content:encoded></item><item><title>I'm joining Cloudflare</title><link>https://steveklabnik.com/writing/i-m-joining-cloudflare/</link><guid isPermaLink="true">https://steveklabnik.com/writing/i-m-joining-cloudflare/</guid><pubDate>Fri, 05 Apr 2019 00:00:00 GMT</pubDate><content:encoded><p><img src="https://svbtleusercontent.com/jpH1brqXUvSAz1tNQ8uVKb0xspap_small.png" alt="https://svbtleusercontent.com/jpH1brqXUvSAz1tNQ8uVKb0xspap_small.png" /></p>
<p>Yesterday, I signed an offer letter from Cloudflare. I’ll be the product manager of Storage, which is the part of the company that’s working on data storage products for the <a href="https://www.cloudflare.com/products/cloudflare-workers/">Cloudflare Workers</a> platform. I’ll be starting at the end of the month.</p>
<p>I think a lot of you will say “that makes perfect sense,” but many more will say “wait, what?”. I hope this post will make things a bit more clear.</p>
<p>Before we start, I also want to say that I’ll still be continuing my work on Rust, though obviously in a more limited capacity. I’ll still be working on revisions to <a href="https://doc.rust-lang.org/stable/book/">The Rust Programming Language</a>, and doing core team work, for example.</p>
<hr />
<p>Before I worked on Rust, I worked on and with Ruby on Rails. When I got out of the web application game, the hot topic was this new “platform as a service” stuff. I’ve been watching all of the latest changes from afar, and they’ve been quite interesting. I’ve always loved the web, and the people building it. In the last 18 months or so, I’ve also been extremely interested in WebAssembly. Working at Cloudflare is the intersection of four things for me:</p>
<ul>
<li>Edge compute</li>
<li>WebAssembly</li>
<li>Rust</li>
<li>Personal and career growth</li>
</ul>
<h2>Edge Compute</h2>
<p>There’s sort of been four eras of “how do I web server”:</p>
<ul>
<li>Physical servers (“on premise”)</li>
<li>Infrastructure as a service</li>
<li>Platform as a service</li>
<li>Function as a service</li>
</ul>
<p>A lot of people see the relationship between these things as “what you control” vs “what does the provider control”, but I don’t think that’s quite right. But I’m getting ahead of myself.</p>
<p>Back in the day, if you wanted a web site, you bought a physical server, and you put it in a data center somewhere. I wasn’t around for those days. The point is, you managed everything: the hardware, the software. All of it.</p>
<p>In <a href="https://www.shapeways.com/blog/archives/305-CloudFab-Distributed-Fabrication-3D-printing,-Nick-Pinkston-interview.html">my days as a startup CTO</a>, my services were deployed to a Virtual Private Server. Instead of owning servers ourselves, we rented a virtual machine, running on someone else’s servers. This is “infrastructure as a service”; I didn’t have to worry about machines, I could just ask for one, and it’d be given to me. I still had to manage the operating system and everything above it.</p>
<blockquote>
<p>Fun side note: CloudFab was a Rails application with a big C extension to do data processing that was too much for Ruby. This later informed a lot of my Rust work.</p>
</blockquote>
<p>Next up came “platform as a service.” To me, Heroku was the platonic ideal of this era. Now, you didn’t even need to manage servers. You got to worry about your application, and that’s it. Maybe you’d need to set a slider to scale during times of demand.</p>
<p>I’ve been reading a lot about “functions as a service” over the past few years. Instead of deploying an entire application, each endpoint becomes an application, basically. This is where I think the previous thinking about the relationships between these categories of service is a little wrong. The actual underlying trend here is something else: what affordances do I have to scale?</p>
<p>In the on-premise days, if I wanted to add capacity, I’d need to buy new servers, install them, set everything up. There’s a <em>lot</em> of latency there. And if I need to remove capacity, well, I have to sell those servers? The move to infrastructure as a service was significant, because it more clearly separated concerns. Each company didn’t need a data center; a data center provides homogeneous compute. As a user, if I need to scale, I can spin up some more virtual servers much more quickly than I can physical ones. Managing that required a lot of tooling, however.</p>
<p>This was the insight that led to platform as a service: by managing this tooling, you could make it even easier for users to scale. There’s a reason why the slider became the ubiquitous synonym with PaaS. There’s nothing that you could do with PaaS that was impossible with IaaS, but it was much, much easier.</p>
<p>This brings us to Functions as a Service, also known as “serverless.” The reason that this architecture matters is one very similar to previous insights. By breaking your application into multiple functions, they can scale independently. This is one of the dreams of microservice architecture, but there’s a bit of a difference. Microservice architecture focuses on how you build your system, and how you split up components internally. FaaS focuses on how people interact with your system, and splits up the components according to that. This framing is a bit more significant to your ability to scale, because it means that during times of high load, you can scale only the parts under that load. In the same way that IaaS said “let’s not force people to buy servers to scale” and PaaS said “let’s not force people to build out infrastructure to scale”, FaaS says “let’s not force people to scale their help page to scale their shopping cart.”</p>
<p>This brings us to edge compute. Edge compute is a particular kind of FaaS, but an interesting take on it. Here’s the concept: your users are all over the world. Why do their packets need to go to <code>us-east-1</code> and back? To which you might say, “sure, that’s why we’ve developed our application to work in multiple regions.” That’s the same relationship between IaaS and PaaS. That is, why should you have to worry about making your application be available across multiple regions? You may not be worrying about individual physical servers anymore, but you’re still worrying about data centers. And some FaaS products, like AWS Lambda, can already run easily across multiple regions, and with some setup, go between them. However, Amazon has about 20 total regions.</p>
<p>So how is edge compute different? Well, at some point, various CDN companies realized “wait a minute. We have severs around the world. Why can’t we run compute on them as well?”</p>
<blockquote>
<p>Side note: Cloudflare is not really a CDN company, though that’s how I always thought of them. They do a lot more stuff than CDNs.</p>
</blockquote>
<p>This is interesting because, well, <a href="https://aws.amazon.com/about-aws/global-infrastructure/">here’s those 20 Amazon regions</a>:</p>
<p><img src="https://svbtleusercontent.com/qxGa9h6TshtRrXZmh6tsRU0xspap_small.png" alt="https://svbtleusercontent.com/qxGa9h6TshtRrXZmh6tsRU0xspap_small.png" /></p>
<p>Compare this with <a href="https://www.fastly.com/network-map">Fastly’s network</a>:</p>
<p><img src="https://svbtleusercontent.com/ePzMAv7Q4NVLARUKambWH80xspap_small.jpg" alt="https://svbtleusercontent.com/ePzMAv7Q4NVLARUKambWH80xspap_small.jpg" /></p>
<p>And finally, <a href="https://www.cloudflare.com/network/">Cloudflare’s network</a>:</p>
<p><img src="https://svbtleusercontent.com/bQzn9XqBY98mw269Q1tsdH0xspap_small.png" alt="https://svbtleusercontent.com/bQzn9XqBY98mw269Q1tsdH0xspap_small.png" /></p>
<p>(the page is animated, so those highlighted ones don’t mean anything)</p>
<p>You’re much likely to be physically closer to a Fastly or Cloudflare server than you are an AWS one. And that’s what we mean by “edge compute”, that is, you’re running your code on the edge, rather than in the data center.</p>
<p>Now, I should also mention that there is a form of Lambda called Lambda@Edge that runs off of CloudFront locations, rather than the above map. So that looks like this:</p>
<p><img src="https://svbtleusercontent.com/vwJvtVvGK7DpGZYEPcrjzW0xspap_small.png" alt="https://svbtleusercontent.com/vwJvtVvGK7DpGZYEPcrjzW0xspap_small.png" /></p>
<p>Much better! My point about regions is more applicable to “edge compute” vs more general compute, than saying something about Lambda as a particular product.</p>
<p>CloudFlare Workers don’t require you to think about where your service is deployed: when you upload it, it’s automatically running on all 175 locations, within minutes.</p>
<p>I think this space is interesting and significant. Cloudflare has <a href="https://www.cloudflare.com/products/cloudflare-workers/">Workers</a>, and Fastly has <a href="https://wasm.fastlylabs.com/">Terrarium</a>. Amazon has <a href="https://docs.aws.amazon.com/lambda/latest/dg/lambda-edge.html">Lambda@Edge</a>. These are all very interesting platforms for building high performance applications.</p>
<p>My role as part of Storage will be to consider “what does data access and storage look like in this world?” If your code is moved to the edge, but your data is still in a central server, you don’t gain the full benefit of having the code close to the client. There’s a lot of interesting stuff in this space!</p>
<h2>WebAssembly</h2>
<p>Both ClouldFlare Workers and Fastly’s Terrarium use <a href="https://webassembly.org/">WebAssembly</a> as a core part of the platform. And that’s significant:</p>
<blockquote>
<p>If WASM+WASI existed in 2008, we wouldn't have needed to created Docker. That's how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let's hope WASI is up to the task! https://t.co/wnXQg4kwa4
— Solomon Hykes (@solomonstre) March 27, 2019</p>
</blockquote>
<p>In order for this stuff to work, you need a runtime for WebAssembly. One interesting thing we’re seeing in this space is sort of two parallel paths emerge: do you want to support JavaScript, or not? Recently, Fastly open sourced <a href="https://www.fastly.com/blog/announcing-lucet-fastly-native-webassembly-compiler-runtime">Lucet</a>, the WebAssembly runtime that powers Terrarium. It’s an impressive bit of tech. There are others too, like <a href="https://wasmer.io/">Wasmer</a> and <a href="https://github.com/CraneStation/wasmtime">Wasmtime</a>. By focusing purely on WebAssembly, you can build specialized tech, and it can be really, really fast.</p>
<p>However, you can’t compile JavaScript to WebAssembly. You can sort of do it with AssemblyScript, a TypeScript subset the compiles to wasm. But JavaScript is arguably the most popular programming language in the world. I personally believe in JavaScript, even with the rise of WebAssembly. And so Cloudflare is taking a different approach than those projects. Instead of building a wasm-only runtime, they’re building on top of Google’s V8. This means that they can support both WebAssembly and JavaScript in workers. Additionally, by leaning on V8, you can take advantage of all of the excellent engineering resources that Google pours into it, similar to how the Rust compiler gets to take advantage of people working on improving LLVM.</p>
<p>Anyway, the point is, WebAssembly is core to this new edge compute world. And so I’m excited to be in the space. I know people that work at Fastly, wasmer, and Mozilla, and they’re all doing great work. I think there’s space for both approaches, but I’m mostly excited to be there and see how it all turns out.</p>
<p>Oh, one other thing I wanted to say: right now, everything around Workers is closed source. Kudos to Fastly for open-sourcing Lucet. I asked about this in my interview, and Cloudflare is very interested in doing more open source work, and so I’m hoping it doesn’t take us long to catch up in this regard. We’ll see, of course, I haven’t actually started yet. But I think that this stuff needs to be open source, personally.</p>
<h2>Rust</h2>
<p>You may remember a little story called <a href="https://en.wikipedia.org/wiki/Cloudbleed">CloudBleed</a>. In short, Cloudflare had a pretty bad security bug in 2017. This bug happened because of a parser, written in Ragel. Here, <a href="https://blog.cloudflare.com/incident-report-on-memory-leak-caused-by-cloudflare-parser-bug/">I’ll let Cloudflare explain it</a>:</p>
<blockquote>
<p>The Ragel code is converted into generated C code which is then compiled. The C code uses, in the classic C manner, pointers to the HTML document being parsed, and Ragel itself gives the user a lot of control of the movement of those pointers. The underlying bug occurs because of a pointer error.</p>
</blockquote>
<p>CloudBleed happened due to memory unsafety. When talking to Cloudflare today, it’s pretty clear that this bug was taken extremely seriously, to the point where it created a culture change inside the company itself. These days, it seems to me (I haven’t started yet, mind you) that you have to use a memory safe language by default, and only use a memory unsafe one if you have an extremely good justification. But Cloudflare needs to be able to have a lot of performance, and control latency, in much of their stack.</p>
<p>Enter Rust. Cloudflare started exploring Rust after CloudBleed, and now uses a significant amount of Rust in production. They host the local Rust meetup here in Austin, and sometimes the one in San Francisco. I’m excited to help Cloudflare be successful with Rust, and to use that experience to help improve Rust too.</p>
<p>My understanding is that I’ll be dealing with a bunch of TypeScript and possibly Go as well, which is also pretty cool.</p>
<h2>Career growth</h2>
<p>Finally, the move to product management. I really enjoy product work, and have had several jobs that were sorta-kinda product jobs. I don’t think that management jobs are inherently better than engineer jobs, of course, but it’s a way for me to grow. My role at Balanced was sorta-kinda-PM-y, and I loved that job.</p>
<p>I don’t have too much more to say about this, just that it’s a bit different, but something I’m really excited to do.</p>
<h2>I’m excited for the future</h2>
<p>So yeah, that’s my extremely long-winded explanation of what’s going on. I’m not starting until the end of the month, so I’ve got a few weeks to purely enjoy. I’ll still be playing a lot of <a href="http://www.celestegame.com/">Celeste</a> and Fortnite, writing blog posts, and working on some Rust open source. But then I get to get back to work. I’m pumped about all of this! New role, building some new tech with tech I already love.</p>
</content:encoded></item><item><title>What comes after open source</title><link>https://steveklabnik.com/writing/what-comes-after-open-source/</link><guid isPermaLink="true">https://steveklabnik.com/writing/what-comes-after-open-source/</guid><pubDate>Tue, 02 Apr 2019 00:00:00 GMT</pubDate><content:encoded><p>In <a href="https://words.steveklabnik.com/the-culture-war-at-the-heart-of-open-source">a previous post</a>, I discussed the history of open source, and ended with this claim:</p>
<blockquote>
<p>Today’s developers have never learned about this history, or don’t care about it, or actively think it’s irrelevant. … For the same reasons that “open source” came up with a new name, I think the movement that will arise from today’s developers will also need a new name.</p>
</blockquote>
<p>We talked about the ideological history of open source, but that’s not what developers object to, really. I don’t think developers are moving back towards a world of making source code private. Instead, it’s something related to a very old discussion in free software. To quote <a href="https://www.gnu.org/philosophy/free-sw.en.html">the FSF</a>:</p>
<blockquote>
<p>“Free software” means software that respects users’ freedom and community. Roughly, it means that the users have the freedom to run, copy, distribute, study, change and improve the software. Thus, “free software” is a matter of liberty, not price. To understand the concept, you should think of “free” as in “free speech,” not as in “free beer”. We sometimes call it “libre software,” borrowing the French or Spanish word for “free” as in freedom, to show we do not mean the software is gratis.</p>
</blockquote>
<p>In a similar fashion, I don’t think that developers are turning against the concept of “free as in free speech”. I think that they don’t believe that the current definitions of free software and open source actually produce software that is “free as in speech.”</p>
<h2>What does “freedom” mean anyway?</h2>
<p>What “free as in speech” means is itself, yet another schism between different camps in the Free Software/Open Source camp. At the root of this schism is a difference of <em>strategy</em>. The specific tactic is straightforward: there are two kinds of licenses, and which do you choose? The two kinds are:</p>
<ul>
<li><em>permissive</em> licenses</li>
<li><em>viral</em> licenses if you don’t like them, <em>copyleft</em> licenses if you do</li>
</ul>
<p>Open Source advocates prefer permissive licenses, and Free Software advocates prefer viral/copyleft licenses. What’s the difference? Well, that’s the topic for a different essay. Because here’s the thing: the jig is up. We’ve already hit the root problem. And you probably didn’t even notice. It took me a long, long time to notice.</p>
<p>Before I expose the trick, a personal interlude.</p>
<h2>Losing my religion</h2>
<p>I used to be a card-carrying, passionate advocate of Free Software. I used to see Stallman lecture. I only installed Free Software on my personal computer to the fullest extent possible. I was considering buying a coreboot-compatible laptop to get rid of the rest.</p>
<p>But then, slowly, I stopped caring. It felt really similar to when I decided to leave Catholicism. I started noticing that these beliefs weren’t really actually helpful, and were frankly, mostly harmful. Now, I’m not saying Free Software is harmful. What I am saying is that refusing to use a wifi connection on your laptop because there aren’t free drivers helps basically nobody, and only harms yourself. Even if you are the kind of person who thinks boycotts work, there just aren’t enough free software diehards to form a large enough boycott. And even then, you’d want to do the boycott <em>together</em>, <em>simultaneously</em>, to send the message.</p>
<p>I realized that a lot of software I used was under a permissive license, and even though I’d prefer if it was GPL’d, the only way to actually realize the dream of improving the software thanks to the source I’d been given was to contribute back, under those terms. So I started to.</p>
<p>And at that point, the permissive licenses were becoming more and more popular. So I found myself in a strange position: even though I was a passionate Free Software advocate, I found myself near-exclusively building Open Source software. All around me, I saw the assault on copyleft, from all sides. And copyleft was losing.</p>
<p>At some point, I just stopped caring.</p>
<h2>Free Software and Open Source are about licenses</h2>
<p>With that out of the way, here’s the problem: note that I seamlessly switched above from talking about what Free Software and Open Source <em>are</em>, to immediately talking about <em>licenses.</em> This is because these two things are effectively synonymous. Quoting <a href="https://www.gnu.org/licenses/licenses.html">the FSF again</a>:</p>
<blockquote>
<p>Published software should be free software. To make it free software, you need to release it under a free software license.</p>
</blockquote>
<p>The <a href="https://opensource.org/osd">OSI</a>:</p>
<blockquote>
<p>Open source doesn’t just mean access to the source code. The distribution terms of open-source software must comply with the following criteria:</p>
</blockquote>
<p>Both Free Software and Open Source are, almost by definition, specific design constraints on software licenses. While this started with the FSF, the OSI inherited it.</p>
<p>I think a quote from <a href="https://www.gnu.org/licenses/licenses.html#WhatIsCopyleft">further down the FSF’s page</a> really drives this home:</p>
<blockquote>
<p>To copyleft a program, we first state that it is copyrighted; then we add distribution terms, which are a legal instrument that gives everyone the rights to use, modify, and redistribute the program’s code or any program derived from it but only if the distribution terms are unchanged. Thus, the code and the freedoms become legally inseparable.</p>
</blockquote>
<p>The FSF sees copyleft, and therefore copyright, as a legal tool to enforce software freedom. This sometimes puts Free Software advocates in a strange position. For example, they will <a href="https://www.fsf.org/campaigns/drm.html">campaign passionately against Digital Rights Management</a>, a software tool to protect copyrighted works, yet also <a href="https://fsfe.org/news/2019/news-20190326-01.en.html">support expanding copyright in some cases</a> when it can further the aims of the GPL. (Due to controversy, they’ve stepped back a bit since, as you can see.)</p>
<h2>Licenses are not sufficient</h2>
<p>So why is it a problem that the concepts of free software and open source are intrinsically tied to licenses? It’s that the aims and goals of both of these movements are about <em>distribution</em> and therefore <em>consumption</em>, but what people care about most today is about the <em>production</em> of software. Software licences regulate <em>distribution</em>, but cannot regulate <em>production</em>. (technically they can, but practically, they can’t. I get into this below.) This is also the main challenge of whatever comes after open source; they cannot rely on the legal tactics of the last generation. I don’t have solutions here.</p>
<p>Let’s talk about what developers want first, and then we’ll get into why licenses can’t accomplish this.</p>
<h2>Developers care about production</h2>
<p>In <a href="https://words.steveklabnik.com/the-culture-war-at-the-heart-of-open-source">my previous post</a>, I claimed that developers are mostly confused about open source. Here’s two concrete examples of what I mean.</p>
<ul>
<li>Imagine a company wants to take some open source software using a permissive license and use it in their product. In the course of using it in their product, they improve it. Are they required to give anything back to the parent project, at all?</li>
<li>Imagine a company wants to take some free software using a copyleft license and use it in their product. In the course of using it in their product, they improve it. In order to comply with the license, they include a line in their 250 page “software license agreement” that says “Certain components of the software, and third party open source programs included with the software, have been or may be made available by $COMPANY on its Open Source web site (http://www.opensource.mycompany.com/”. That web site contains zip files with the contents of the (heavily modified) source. Are they required to do anything more than that?</li>
</ul>
<p>When presenting these questions to most developers today, I suspect you’d get these answers to these two questions, in order:</p>
<ul>
<li>Yes, and that’s why open source is such a mess; companies take and don’t give back.</li>
<li>… that sounds extremely shady.</li>
</ul>
<p>Those of you who have been around the block a few times may recognize situation number two: it’s what Apple did with WebKit. There was a project called KHTML, which was licensed under the GPL. Apple forked it, and in order to comply with the GPL, did exactly the above. They were completely within their rights to do so. Yet, many recognized, even at this time, that this “wasn’t in the spirit of open source.” These tactics are sometimes called “source dumps” or “code bombs.”</p>
<p>But that attitude, that this may be following the letter, <em>but not the spirit</em>, is the crux of it here. Most developers don’t understand open source to be a particular license that certain software artifacts are in compliance with, but an attitude, an ideology. And that ideology isn’t just about the consumption of the software, but also its production. An open source project should have a public bug tracker. There should be a mailing list, for discussion. You should be able to observe, and ideally participate in, the development of the software. Focusing on the code being open is putting the cart before the horse. In fact, this is one of the reasons why there’s such derision for “source available” licenses; it’s not about the source being open. It’s that the source being open is a necessary, but not sufficient, component of being open source. Now, this is normally framed as a problem of distribution, but I think that many also understand it as a problem of production.</p>
<p>I believe that this focus on process is why the GNU project has fallen out of favor as well. The tools that the GNU project uses to develop its Free Software are arcane, ugly, and unique to them. It’s the same with many of the old-school Open Source projects as well. If I never have to look at a Bugzilla instance again, I will die happy. This is why GitHub took off; it provided a significantly nicer developer experience for building software. You may not personally agree, but the numbers speak for themselves. The FSF didn’t move to GitHub because GitHub is proprietary, and they see that as inconsistent with their values. Most developers see that you can use it for no money, and that the software produced with it is open source. They see this as consistent with their values.</p>
<p>When developers talk about problems they see in open source, it’s often that there are production problems. Companies don’t “give back” money or developer hours. Programmers today don’t seem to be upset that, if they’ve developed any proprietary extensions to their open source software, that those extensions are not shared back with the community. They care that the production process is impeded by additional pressure, without providing resources. If a company were to add a proprietary feature to an open source project, yet pays five employees to develop the open source part further, the FSF sees this as a tragedy. The commons has not been enriched. The new generation of open source developers sees this as a responsible company that thankfully is contributing to the development of something they use and care about.</p>
<p>Software licenses can only restrict what people can do when they distribute the source code, and that’s it. It cannot force someone to have a bug tracker, or a code of conduct, or accept your patch. Copyleft can force an absolute minimal “contribution” back to your project, but it can’t force a good-faith one. This makes it an inadequate tool towards building something with the kinds of values that many developers care about.</p>
<h2>The challenges</h2>
<p>How would one go about building a software movement around the open <em>production</em> of software? There are a number of challenges.</p>
<p>First of all, do you even want the legal system to enforce such a thing? There are pros and cons. Without legal teeth, companies are unlikely to comply. That’s part of why we’re in this mess to begin with!</p>
<p>Okay, so you want the legal system to somehow enforce this kind of control over the production of software. But how? If we look closer at the strategy used by Free Software and Open Source, they use licenses, which are a form of intellectual property law, which is modeled after property law. Earlier, I said that you can’t use licenses to regulate production, and that’s <em>technically</em> not true. For example, say that I own a brand, like McDonalds. I own the intellectual property surrounding that brand. I can licence that intellectual property to others, contingent on them producing hamburgers (and whatever else) in a certain way, according to my specification.</p>
<p>This doesn’t really work with the way that open source is set up. The entities are in reverse here; it’s the software developers that want to be able to dictate things, but it’s the project that sets the license terms.</p>
<p>That got me thinking about a different pattern for doing this kind of thing. Have you ever seen one of these?</p>
<p><img src="https://svbtleusercontent.com/wWhQCoC3mGGKwQ1nS6CC9H0xspap_small.gif" alt="https://svbtleusercontent.com/wWhQCoC3mGGKwQ1nS6CC9H0xspap_small.gif" /></p>
<p>This image on a product is part of a process called “certification.” The image itself is referred to as a “certification mark.” In order to use this image on your product, you apply to a “certification body”, in this case, the <a href="https://www.usda.gov/">USDA</a>. This body has set up some kind of tests, and if your product passes them, you gain the ability to say that you’ve passed the certification. I chose organic food on purpose here; most aspects of this certification are about the process by which the food is produced.</p>
<p>Technology is no stranger to these kinds of processes:</p>
<p><img src="https://svbtleusercontent.com/fMstP8GhZwpMhHxNdZ3aTn0xspap_small.png" alt="https://svbtleusercontent.com/fMstP8GhZwpMhHxNdZ3aTn0xspap_small.png" /></p>
<p>So in theory, one could imagine an organization that produces a different kind of document. Instead of a license for the source code, they would provide a way to say uh, let’s go with “Open Development Certified.” Projects could then submit for certification, they’d get accepted or rejected.</p>
<p>I’m not confident this solution would work, however.</p>
<p>For one, even though some parts of our industry have accepted certifications, I feel like software developers have a pretty big bias against them. Beyond that, there’s two other major questions: who would do this certification, and how would they determine the criteria? It’s not clear who has the moral authority to suggest that they are the arbiter of what is correct here, and that a majority of people would agree. And then they would have a big job of actually determining what those sets of rules would be. It does have a nice property of a built-in business model; you can charge for application for the certification. But that also has a host of issues. And even if you sort all of that out, it runs afoul of the same “boycott” problems that I talked about above with Free Software. This certification only makes sense if people demand that the software they use is Open Development Certified. I’m not sure that this would be the case.</p>
<p>Another option is some sort of “Developer Union,” which would put pressure on the companies that those developers work at to contribute back to open source projects. Many developers seem <em>rabidly</em> anti-union, and tech companies are as well. I’m not sure this is a viable path, today.</p>
<h2>So where do we go from here?</h2>
<p>I’m still, ultimately, left with more questions than answers. But I do think I’ve properly identified the problem: many developers conceive of software freedom as something larger than purely a license that kinds in on redistribution. This is the new frontier for those who are thinking about furthering the goals of the free software and open source movements. Our old tools are inadequate, and I’m not sure that the needed replacements work, or even exist.</p>
<p>Something to think about, though.</p>
</content:encoded></item><item><title>Rust has finally outgrown me</title><link>https://steveklabnik.com/writing/rust-has-finally-outgrown-me/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rust-has-finally-outgrown-me/</guid><pubDate>Fri, 29 Mar 2019 00:00:00 GMT</pubDate><content:encoded><p>Today is a very interesting day. It’s giving me a lot of feels. Today is <a href="https://rustlatam.org/">Rust Latam</a>, a Rust conference in Montevideo.</p>
<p>And I’m not there.</p>
<p>This is significant because, as far as we know, I’m the only person who has been to every Rust conference in existence so far. I went to RustCamp, all three RustConfs, all five RustFests so far, all three Rust Belt Rusts. One RustRush. Am I forgetting any? Thirteen Rust conferences in the past four years.</p>
<p>I really wanted to attend Rust Latam. The main organizer, Santiago, has been a friend for a very long time. It’s amazing that we’re getting a new conference in this part of the world. There’s been a lot of energy with Rust there for quite a while. It would have been nice to meet all those people whose tweets I’ve seen.</p>
<p>But in the end, I’m still between jobs (news on that soon, hopefully!), and spending that money would be irresponsible. I’m just getting settled in Austin, and it’s nice to stay in one place for a little while. I didn’t only attend Rust conferences last year, I attended many more. I missed Thanksgiving with my friends, because I was at a conference. I made five trips in October last year; Eastern Europe, then San Francisco, then Michigan, then New York (for a conf, not to stay at home), then Berlin. I did three more in November. I did about twenty conferences, and a few personal trips, last year. I am very tired.</p>
<p>Don’t get me wrong, I love travel, and plan to do more of it in the future. But I’ve been taking a bit of a breather; I’ve only done four trips so far this year, twice to Europe, twice in the US. But it’s been nice to take a break. For now.</p>
<p>More importantly, it’s time to let go of the pressure I felt to keep that streak up. Some of those trips really took a toll on me, mentally and physically. Nobody should ever fly from New York to Belarus for 30 hours, then immediately fly to San Francisco for three days, then back to New York. I’m not old, but I’m not as young as I once was, either.</p>
<p>It’s also time to acknowledge that I cannot do it all on my own. This conference streak is one thing, but Rust has outgrown me in other ways. I used to be on <code>#rust</code> every day, answering almost every question. Then it was both <code>#rust-beginners</code> and <code>#rust</code>. Not to mention all the other channels. I’m not even on IRC anymore. I was on the core team when the core team approved every RFC; now my team very rarely has RFCs. I used to read every single GitHub email on <code>rust-lang/rust</code>, and then <code>rust-lang/cargo</code>, and then all their other libraries. Now I skim <code>rust-lang/rust</code>, and have unsubscribed from a lot of the other repos. I used to know every single production user of Rust. Those times are long gone; I’m starting to get used to hearing “oh yeah our team has been using Rust in production for a while now, it’s great.” The first time that happened, it felt <em>very</em> strange. Exciting, but strange.</p>
<p>Back when I first decided I wanted to devote my time to Rust, I used to submit every single blog post about Rust to Hacker News and Reddit, and then answer any question anyone had about things. I diligently did this for <em>years</em>. I’m slowly letting go of this one, too. We’ll see.</p>
<p>All of these things are excellent. This is the future I wanted. This is why I spent so much time over the years doing all of this stuff. I did it, mission accomplished. It feels good. But it also feels a little weird. That Rust is bigger than <em>me</em> is an amazing, wonderful thing.</p>
<p>I wonder what the next stage of Rust’s growth will feel like.</p>
</content:encoded></item><item><title>Learning Ada</title><link>https://steveklabnik.com/writing/learning-ada/</link><guid isPermaLink="true">https://steveklabnik.com/writing/learning-ada/</guid><pubDate>Tue, 26 Mar 2019 00:00:00 GMT</pubDate><content:encoded><p>I decided to try and learn Ada. I love programming languages. This post documents my attempt, and explains what I learned. This is pretty stream of consciousness and basically is a bunch of random notes, so if you hate that, better close the tab. I typed my thoughts up pretty raw, so I may be incorrect in places too. Let me know if you know the answer to something I don’t!</p>
<hr />
<p>The first step, of course, is to google:</p>
<p><img src="https://svbtleusercontent.com/72NhELV1orufu8jasf5iiT0xspap_small.png" alt="https://svbtleusercontent.com/72NhELV1orufu8jasf5iiT0xspap_small.png" /></p>
<p>Great! I know that Adacore are sort of the stewards of Ada? Vaguely, that might be wrong. Let’s look at all three of those.</p>
<blockquote>
<p>Learn.adacore.com is an interactive learning platform designed to teach the Ada and SPARK programming languages.</p>
</blockquote>
<p>Ah, that’s awesome! No installation needed.</p>
<p>The second one looks <strong>extremely</strong> like spam. It does have a page of learning materials that look legit enough. I wonder if this is a low-effort attempt to make money through ads. Or maybe it’s actually good and real and feels outdated, it does have a high google ranking. Upon doing some more research, it seems like the latter. Oops…</p>
<p>The third one is a parked domain, it apparently expired. Oops.</p>
<p>Anyway, I’m going to pursue learn.adacore.com</p>
<h2>learn.adacore.com</h2>
<p>Here’s the initial code presented:</p>
<pre><code>with Ada.Text_IO; use Ada.Text_IO;
procedure Learn is
subtype Alphabet is Character range 'A' .. 'Z';
begin
Put_Line ("Learning Ada from " &amp; Alphabet'First &amp; " to " &amp; Alphabet'Last);
end Learn;
</code></pre>
<p>Okay! This clearly has some style to it. Seems like <code>Pascal_Snake_Case</code>. I’m not super hyped about this, but it’s fine. Syntax discussions are like the tabloids of programming language discussion. They’re fun, but they don’t really have any educational comment. One thing I will point out: are apostrophes used for method calls? Otherwise, this seems to be fairly Pascal like, which I think makes sense given what I know about Ada.</p>
<p>I can click run and see it execute right there! Awesome.</p>
<p>There seems to be three sections here: About, Courses, and Books. About is a single page of information that reinforces what I already know:</p>
<blockquote>
<p>The Ada programming language was designed from its inception to be used in applications where safety and security are of the utmost importance.The SPARK programming language is a formally verifiable subset of the Ada language which allows developers to mathematically prove program correctness through static means.Founded in 1994, AdaCore is the leading provider of commercial and open-source software solutions for Ada</p>
</blockquote>
<p>Cool.</p>
<p>There are two courses, introduction to Ada, and introduction to SPARK. I think SPARK is cool but I’ll learn that after I learn Ada itself. I also do like books, but let’s do that first.</p>
<p>I’m not going to write everything about the book as I read it, but I will say that I like the basic structure here: first some history, then imperative programming stuff, then talking about types. After that it gets into packages, more type stuff, privacy, generics, exceptions, tasks, interfacing with C, the standard library, and some appendices. This feels pretty logical. Time to do some reading, I’ll jump back here at random points with notes.</p>
<blockquote>
<p>The first Ada standard was issued in 1983; it was subsequently revised and enhanced in 1995, 2005 and 2012, with each revision bringing useful new features.</p>
</blockquote>
<p>I heard that Ada 2020 is a thing; seems like they’re on a roughly 7-8ish year timeframe generally. Cool.</p>
<blockquote>
<p>In terms of modern languages, the closest in terms of targets and level of abstraction are probably C++ and Rust.</p>
</blockquote>
<p>We got a shout-out! That’s nice.</p>
<blockquote>
<p>Readability is more important than conciseness. Syntactically this shows through the fact that keywords are preferred to symbols, that no keyword is an abbreviation, etc.</p>
</blockquote>
<p>This is interesting; it seems to suggest that readability and verbosity are synonymous. I often find concise things easier to read, personally. This is, of course, subjective, and I wonder how I’ll feel after I learn more of the language.</p>
<blockquote>
<p>Ada is a multi-paradigm language with support for object orientation and some elements of functional programming, but its core is a simple, coherent procedural/imperative language akin to C or Pascal.</p>
</blockquote>
<p>This makes a lot of sense, especially given the time it was created.</p>
<p>I’m finding it a <em>little</em> hard to understand what <code>main</code> is. The docs say</p>
<blockquote>
<p>Greet is a procedure, and the main entry point for our first program. Unlike in C or C++, it can be named anything you prefer. The builder will determine the entry point. In our simple example, gprbuild, GNAT’s builder, will use the file you passed as parameter.</p>
</blockquote>
<p>It will use it as … parameter? Like, is the fact that we called the file <code>greet.adb</code> the reason it calls <code>Greet</code>? It’s not clear to me at the moment.</p>
<blockquote>
<p>Integer’Image is a function that takes an Integer and converts it to a String. It is an example of a language construct known as an attribute, indicated by the “’” syntax, which will be covered in more detail later.</p>
</blockquote>
<p>Ah, it’s not a method call, it’s an attribute.</p>
<blockquote>
<p>In Ada, an integer type is not specified in terms of its machine representation, but rather by its range. The compiler will then choose the most appropriate representation.</p>
</blockquote>
<p>Very interesting. I wonder how well-defined these mappings are, for example, if you need to call a C function, you’re gonna need to give it the correct sized type… I’m sure I’ll get to that.</p>
<blockquote>
<p>Mainly for efficiency reasons, while machine level overflow always results in an exception, type level overflows will only be checked at specific boundaries, like assignment:</p>
</blockquote>
<p>Okay, <em>this</em> is the good stuff. Ranged integers are something people always say Ada is better at than Rust; I wondered if this was a static or dynamic check. It <em>is</em> a dynamic check. Cool. I’ve always wanted to know!</p>
<blockquote>
<p>As mentioned previously, every “built-in” type in Ada is defined with facilities generally available to the user.</p>
</blockquote>
<p>Pretty cool!</p>
<blockquote>
<p>While unconstrained arrays in Ada might seem similar to variable length arrays in C, they are in reality much more powerful, because they’re truly first-class values in the language. You can pass them as parameters to subprograms or return them from functions, and they implicitly contain their bounds as part of their value. This means that it is useless to pass the bounds or length of an array explictly along with the array, because they are accessible via the ’First, ’Last, ’Range and ’Length attributes explained earlier.</p>
</blockquote>
<p>Awesome. This is the way to do it, for sure.</p>
<blockquote>
<p>Here is how the string type is defined in Ada:type String is array (Positive range &lt;&gt;) of Character;</p>
</blockquote>
<p>Hmm, what’s a <code>Character</code> though? Let’s try to mess with the example:</p>
<pre><code>with Ada.Text_IO; use Ada.Text_IO;
procedure Greet is
Message : String (1 .. 2) := "😊";
-- ^ Pre-defined array type.
-- Component type is Character
begin
for I in reverse Message'Range loop
-- ^ Iterate in reverse order
Put (Message (I));
end loop;
New_Line;
end Greet;
</code></pre>
<p>Here’s the output:</p>
<pre><code>Run...
greet.adb:4:04: warning: "Message" is not modified, could be declared constant
greet.adb:4:33: warning: wrong length for array of subtype of "Standard.String" defined at line 4
greet.adb:4:33: warning: "Constraint_Error" will be raised at run time
raised CONSTRAINT_ERROR : greet.adb:4 length check failed
</code></pre>
<p>Hm, wrong length. What is the right length? This particular emoji is four bytes, so let’s try to set it to 5, since we start at 1. Nope that doesn’t work either. What about 4?</p>
<pre><code>Run...
the machine running the examples is not responding, please try again later
</code></pre>
<p>Very suspicious. Anyway, I’m guessing it’s ASCII, maybe it’ll tell me later.</p>
<blockquote>
<p>One last feature of Ada arrays that we’re going to cover is array slices. It is possible to take and use a slice of an array (a contiguous sequence of elements) as a name or a value.</p>
</blockquote>
<p>Hell yeah, I love slices.</p>
<blockquote>
<p>Ada has multidimensional arrays, which are not covered in this course. Slices will only work on one dimensional arrays.</p>
</blockquote>
<p>Ah, bummer. I guess I’ll have to read up on those separately.</p>
<blockquote>
<p>Returning variable size objects in languages lacking a garbage collector is a bit complicated implementation-wise, which is why C and C++ don’t allow it, prefering to depend on explicit dynamic allocation / free from the user.The problem is that explicit storage management is unsafe as soon as you want to collect unused memory. Ada’s ability to return variable size objects will remove one use case for dynamic allocation, and hence, remove one potential source of bugs from your programs.Rust follows the C/C++ model, but with safe pointer semantics. However, dynamic allocation is still used. Ada can benefit from an eventual performance edge because it can use any model.</p>
</blockquote>
<p>Hmmm, I wonder <em>how</em> it is doing this. Doesn’t that still dynamically allocate in Ada?</p>
<blockquote>
<p>Ada doesn’t have a tuple construct and does not allow returning multiple values from a subprogram (except by declaring a full-fledged record type). Hence, a way to return multiple values from a subprogram is to use out parameters.</p>
</blockquote>
<p>I love tuples, oh well.</p>
<blockquote>
<p>While reading an out variable before writing to it should, ideally, trigger an error, imposing that as a rule would cause either inefficient run-time checks or complex compile-time rules. So from the user’s perspective an out parameter acts like an uninitialized variable when the subprogram is invoked.</p>
</blockquote>
<p>Hmmmmmmmmmmmmmmmmmm. This seems <em>extremely</em> against Ada’s safety focus.</p>
<pre><code>with Ada.Text_IO; use Ada.Text_IO;
procedure Outp is
procedure Foo (A : out Integer) is
begin
Put_Line (Integer'Image (A));
end Foo;
A : Integer := 12;
begin
Foo(A);
end Outp;
</code></pre>
<p>gives</p>
<pre><code>Run...
outp.adb:4:19: warning: formal parameter "A" is read but never assigned
outp.adb:10:08: warning: "A" modified by call, but value might not be referenced
32514
2 errors.
</code></pre>
<p>Each time I run it, I get a different number. Seems bad?</p>
<blockquote>
<p>There are two ways in which Ada helps shield programmers from the dangers of pointers:One approach, which we have already seen, is to provide alternative features so that the programmer does not need to use pointers. Parameter modes, arrays, and varying size types are all constructs that can replace typical pointer usages in C.Second, Ada has made pointers as safe and restricted as possible, but allows “escape hatches” when the programmer explicitly requests them and presumably will be exercising such features with appropriate care.</p>
</blockquote>
<p>Seems great! I’m very here for this.</p>
<p>… but they have <code>null</code>. Of course they do, given the timeframe Ada was developed. Oh well.</p>
<p>Okay, so <code>new</code> is a keyword, and is in the language. But freeing is an unsafe standard library API? I guess this makes sense, similarly to how leaking memory isn’t unsafe in Rust. Feels weird, though.</p>
<p>I guess, reading more, the idea is that you should basically never need to do this yourself. I think this is where I was getting caught up earlier with the “return an unknown type” bit too. It <em>is</em> allocating, but since you’re not typing <code>new</code>, you’re not responsible, and so that’s safer. That makes sense. I hope my understanding is correct here, but I’m not totally sure.</p>
<p>This does mean that it feels like allocations exist, but are hidden from you, the user. That’s okay, but it’s very different than Rust.</p>
<blockquote>
<p>Ada’s variant records are very similar to Sum types in functional languages such as OCaml or Haskell. A major difference is that the discriminant is a separate field in Ada, whereas the ‘tag’ of a Sum type is kind of built in, and only accessible with pattern matching.There are other differences (you can have several discriminants in a variant record in Ada). Nevertheless, they allow the same kind of type modeling as sum types in functional languages.</p>
</blockquote>
<p>This is pretty cool.</p>
<p>The “Character Types” section still does not explain to me what valid characters are. Oh well. Googling it gives me <a href="https://www.adaic.org/resources/add_content/standards/05rm/html/RM-3-5-2.html">this</a> which says</p>
<blockquote>
<p>The predefined type Character is a character type whose values correspond to the 256 code positions of Row 00 (also known as Latin-1) of the ISO/IEC 10646:2003 Basic Multilingual Plane (BMP). Each of the graphic characters of Row 00 of the BMP has a corresponding character_literal in Character. Each of the nongraphic positions of Row 00 (0000-001F and 007F-009F) has a corresponding language-defined name, which is not usable as an enumeration literal, but which is usable with the attributes Image, Wide_Image, Wide_Wide_Image, Value, Wide_Value, and Wide_Wide_Value; these names are given in the definition of type Character in A.1, “The Package Standard”, but are set in italics.</p>
</blockquote>
<p>Ah. Yeah so that’s why the emoji didn’t work. Did that crash the compiler earlier, is that what was happening?</p>
<blockquote>
<p>Generic subprograms or packages can’t be used directly. Instead, they need to be instantiated, which we do using the new keyword, as shown in the following example:</p>
</blockquote>
<p>Wait, is this the same <code>new</code> keyword as before? Do generics require heap allocation? That is a reasonable design, I’m just a bit surprised.</p>
<blockquote>
<p>Ada exceptions are not types, but instead objects, which may be peculiar to you if you’re used to the way Java or Python support exceptions.</p>
</blockquote>
<p>I found this sentence confusing; aren’t the objects in Java or Python as well? When I was reading this sentence, halfway through, I thought to myself “oh, like the stuff I know, cool” and then the second half was “this is unlike the stuff you know”. Hrm.</p>
<blockquote>
<p>A task can be thought as an application that runs concurrently with the main application. In other programming languages, a task can be called a thread, and tasking can be called multithreading.</p>
</blockquote>
<p>Cool, this is a big topic I’ve been wondering about.</p>
<blockquote>
<p>As we’ve just seen, as soon as the main task starts, its subtasks also start automatically. The main task continues its processing until it has nothing more to do. At that point, however, it will not terminate. Instead, the task waits until its subtasks have finished before it allows itself to terminate.</p>
</blockquote>
<p>Ah interesting, this is different than Rust. Well, Rust’s standard library, anyway.</p>
<p>I wish it explained if tasks <em>are</em> system threads, or if they’re green threads.</p>
<p>I did some googling, and apparently this is implementation-defined. GNAT uses system threads for some of the same reason that Rust decided to expose system threads. Seems good.</p>
<blockquote>
<p>The pragma Assertion_Policy statement is used to force the compiler to generate code to check the precondition.</p>
</blockquote>
<p>Design by contract works at runtime. This is something I always thought was true, but was never actually sure of. Cool.</p>
<blockquote>
<p>To interface with C’s built-in types, we use the Interfaces.C package, which contains most of the type definitions we need. For example:</p>
</blockquote>
<p>Ah ha, this answers my question from earlier. Cool.</p>
<blockquote>
<p>Object-oriented programming (OOP) is a large and ill-defined concept in programming languages and one that tends to encompass many different meanings because different languages often implement their own vision of it, with similarities and differences from the implementations in other languages.However, one model mostly “won” the battle of what object-oriented means, if only by sheer popularity. It’s the model used in the Java programming language, which is very similar to the one used by C++. Here are some defining characteristics:</p>
</blockquote>
<p>Spicy! But not incorrect.</p>
<p>Dot notation exists! But only if you’re actually using objects, er, “tagged types”.</p>
</content:encoded></item><item><title>What's next for SemVer</title><link>https://steveklabnik.com/writing/what-s-next-for-semver/</link><guid isPermaLink="true">https://steveklabnik.com/writing/what-s-next-for-semver/</guid><pubDate>Mon, 11 Feb 2019 00:00:00 GMT</pubDate><content:encoded><p>On December 18, 2009<a>1</a>, the Semantic Versioning specification was announced. More commonly known as SemVer, it provided guidance for developers regarding software versions:</p>
<blockquote>
<p>I propose a simple set of rules and requirements that dictate how version numbers are assigned and incremented. For this system to work, you first need to declare a public API. This may consist of documentation or be enforced by the code itself. Regardless, it is important that this API be clear and precise. Once you identify your public API, you communicate changes to it with specific increments to your version number. Consider a version format of X.Y.Z (Major.Minor.Patch). Bug fixes not affecting the API increment the patch version, backwards compatible API additions/changes increment the minor version, and backwards incompatible API changes increment the major version.I call this system “Semantic Versioning.” Under this scheme, version numbers and the way they change convey meaning about the underlying code and what has been modified from one version to the next.</p>
</blockquote>
<p>The author came from the Ruby world, and around the same time, some big things were brewing. <a href="https://bundler.io/">Bundler</a> was being brought into the world. Bundler brought a few new things, but the most important features involved specifying dependencies for your projects. Normally, you’d specify which version of a gem your project needed. Instead, Bundler allowed you to specify a range of versions you’d accept, and it would help figure out which version you should use, according to the SemVer rules.</p>
<p>In January of 2010, npm, a similar program for the shiny new Node.js, also shipped, and also used SemVer.</p>
<p>Thanks to these tools, Ruby and Node enjoyed (and still do, to this day) a vibrant open source ecosystem. SemVer is the underlying concept that made this possible.</p>
<p>However, SemVer is not perfect. In some ways, I think of SemVer as the <code>gofmt</code> of versioning:</p>
<blockquote>
<p>Gofmt’s style is no one’s favorite, yet gofmt is everyone’s favorite.Go Proverbs</p>
</blockquote>
<p>SemVer has its flaws, but it empirically works better than previous attempts at such a thing. That such a thing exists is far more important than how perfect it is. Fundamentally, version numbers are a communication tool, and SemVer provides a framework for said communication.</p>
<p>That said, SemVer <em>does</em> have flaws. Bundler, <code>npm</code>, Rust’s Cargo, NuGet, and a variety of tools have implemented the specification. But there are edge cases, as well as gaping holes, in the spec, and so in some ways, these tools are incompatible.</p>
<p>Over the last few months, several of us<a>2</a> have been talking, and today, we have an announcement to make. We’ve formed a semver team, and we intend to start work on producing a new version of the spec.</p>
<p>Who is “we”? The team roster will be <a href="https://github.com/orgs/semver/teams/maintainers/members">in a GitHub team</a><a>3</a>, but for posterity’s sake, here’s the initial list of members, in the same order GitHub lists them:</p>
<ul>
<li>Anand Gaurav, NuGet</li>
<li>Dave Herman, Notion</li>
<li>André Arko, Bundler</li>
<li>isaacs, npm</li>
<li>Samuel Giddins, CocoaPods</li>
<li>Steve Klabnik, Cargo</li>
</ul>
<p>(I’m putting myself as “Cargo,” because Cargo is the main client of my SemVer implementation. In most of these projects, the person who maintains the SemVer implementation is the same as the project, but in Rust, they’re separate. Cargo is the higher-order bit here, though.)</p>
<p>We have not yet started the work on the next iteration of the specification, but we have agreed on a governance model for SemVer. Described in the new <code>[CONTRIBUTING.md](https://github.com/semver/semver/blob/master/CONTRIBUTING.md)</code>, we’re going with an RFC process, similar to the Rust programming language and many other projects. In short, “substantial” changes to the SemVer spec need to go through a process by which a written description of the changes are proposed, and the team reaches consensus on acceptance.</p>
<p>It also contains some principles we have for the future of SemVer. I’ve copied the important ones for the specification itself here:</p>
<ul>
<li>No RFC will be approved if it is deemed to cause significant breakage to any of the SemVer-using communities represented by the SemVer Maintainers group.</li>
<li>RFCs will be considered formally adopted only when they are approved by the SemVer Maintainers group, and implemented in a simple majority of represented implementations.</li>
<li>Implementations may add functionality in advance of an approved RFC but all such functionality must be flagged as “experimental”, so that users understand it may change in the future.</li>
</ul>
<p>In essence, we don’t intend to re-write the spec, but instead, fill in holes in the spec, find out where our implementations differ, and unify all of our implementations as much as is feasible. We believe that this will not only help all of us, but also help new tools as well. Being compatible with each other is an important part of the value of these tools.</p>
<p>I look forward to letting you know when our work is done! If you’d like to get involved, please watch that governance repository, and submit RFCs! While we’re all excited about the future of SemVer, it’s nobody’s job to do this work. As such, we’ll be moving fairly slowly. Please be patient. Thanks :)</p>
<hr />
<p>You may also want to read <a href="https://haacked.com/archive/2019/02/11/semver-collective/">Phil’s post</a>, reflecting on this transition from the other side. Thanks for being so much of a good steward of SemVer so far, Phil!</p>
<hr />
<ol>
<li>At least <a href="https://web.archive.org/web/20091218170740/http://semver.org/">according to the WayBack Machine</a><a>↩</a></li>
<li><a href="https://crates.io/crates/semver">I maintain Cargo’s <code>semver</code> implementation</a>.<a>↩</a></li>
<li>Ugh, I’m told apparently this can’t be made public? Even better that I put the list here, then.<a>↩</a></li>
</ol>
</content:encoded></item><item><title>thank u, next</title><link>https://steveklabnik.com/writing/thank-u-next/</link><guid isPermaLink="true">https://steveklabnik.com/writing/thank-u-next/</guid><pubDate>Sun, 06 Jan 2019 00:00:00 GMT</pubDate><content:encoded><p>I started working at 15, when I took a job as a pizza cook. Over the next seven years, I moved up the ranks, to a driver, shift manager, and then as part of the “new store opening team.” The franchise was growing, and we needed to help new franchisees open their new stores. I’d travel to where the new store was a week before they would open, help train the new staff, and then work their opening weekend. It was really fulfilling work; if pizza paid as well as tech, I’d seriously consider doing it forever.</p>
<p>One time, somewhere in Maryland, I got a bit ill. It wasn’t a good idea for me to work the rest of the week, but the company’s top brass would usually visit to cut the ribbon on the store. It was about a four hour drive back to home, and our COO graciously agreed to let me come back to the ’burgh with him, so I could recover at home. On the drive, I asked him what he did before working here, and the answer really surprised me. “Oh, I was the CEO of American Eagle. I grew them from 20 locations to 200, and then decided to move on.” To a kid from the suburbs, being in charge of one of the most popular clothing brands among my peers seemed like a great gig. Why’d he quit? The answer is something that’s really stuck with me; this happened about fifteen years ago. What he told me was this: at each stage of a company’s growth, they have different needs. Those needs generally require different skills. What he enjoyed, and what he had the skills to do, was to take a tiny company and make it medium sized. Once a company was at that stage of growth, he was less interested and less good at taking them from there. And that was what he was trying to do for our pizza chain.</p>
<p>I’ve mostly worked at small companies. I did a startup, then consulted, then worked for a series of companies with five to 15 people. I quit one of those jobs when I found Rust. What I saw in Rust was something that the world really <em>needed</em>. And I wanted to help it get there. Beyond that, the only real way to get a job working on Rust was to work at Mozilla. And independently of Rust, that was something I’m really excited about.</p>
<p>Here’s a poster on my wall:</p>
<p><img src="https://svbtleusercontent.com/x35Th9VqQFuuZ4SFBEsZkU0xspap_small.jpeg" alt="https://svbtleusercontent.com/x35Th9VqQFuuZ4SFBEsZkU0xspap_small.jpeg" /></p>
<p>I’ve been using Firefox before it was called Firefox. This poster is of a two-page ad that Mozilla took out in the New York Times to announce Firefox 1.0. I’ve long believed in Mozilla’s mission. This was an exciting opportunity!</p>
<p>Fast forward a few years. Rust has been growing, we’re doing great things. I love my team. But every time that this means interacting with Mozilla more broadly, I end up being frustrated. I could say a lot of things here, but I won’t get into the details. What’s really important is this: I’m not proud to be a Mozillian anymore. A variety of incidents contributed to this, but at the end of the day, it’s what’s true. Furthermore, I don’t have any personal opportunity at Mozilla; I recently discovered I’m the lowest-paid person on my team, and Mozilla doesn’t pay particularly well in the first place. In order to have any kind of career growth, I would have to not do the work that aligns with my skills, and what I’m doing now is really how I can have the most impact on Rust moving forward.</p>
<p>I don’t believe that Mozilla’s direction has any chance of changing; when I’ve tried to express my frustrations, I’ve only gotten disciplined. Mozilla is not interested in hearing what I have to say. And that’s fine, but when I take a step back and think about things, that means it’s time to go, for both my sake and Mozilla’s. So I’ve just put in my two weeks’ notice.</p>
<p>So what’s next? Well, the first thing is that I don’t plan to stop working on Rust. How much I’ll be able to depends on what’s next, but that’s the great part about open source; I can continue to help the thing I love, even if it might not be full-time anymore.</p>
<p>I’ve also been enamored with another technology recently: WebAssembly. 2019 is going to be a <em>huge</em> year for WebAssembly, even if many people don’t know it yet, and may not see the effects until 2020. Once again, I find myself seeing something that the world needs, and I think I have the skills to help make that happen. Unlike with Rust, I don’t think that WebAssembly needs my help to actually work on it itself; I think it’s not yet found its killer application. I’d like to help find out what that is, and help both wasm and that application grow.</p>
<p>There’s something similar in Google’s Fuchsia. I’ve worked with the web for a long time, but doing Rust for the last few years, I’ve also really enjoyed systems-level work. Fuchsia is very interesting and a bit of an enigma, but I can’t wait to see how it turns out. They use a lot of Rust, and plan to use more. I’m not really a kernel hacker, and I’m not sure that Fuchsia is far enough along to really need my skills, but it also really piques my interest.</p>
<p>Additionally, I’m always open to other things. Before Rust, I worked at Balanced Payments, which was one of my favorite jobs I’ve ever had. I would have never thought I’d want to work at a payments company, but by being open to it, I found an awesome role, with fantastic co-workers. Maybe what I do next will be something completely different!</p>
<p>In terms of the actual <em>work</em> I would like to do, I don’t think a traditional engineering role really suits me. Don’t get me wrong, I love to write some code, but I don’t think that those kinds of roles really play to my unique strengths. What I really love to do is teaching, evangelizing, and growing something. I’ve done it with Rust, and I can do it again. I could see several roles or hybrid ones here, with titles like “developer evangelist,” “product manager”, or possibly even “VP of Growth.” I want something that challenges me, and where I can grow my career.</p>
<p>So if you’re:</p>
<ul>
<li>A company that wants to invest in Rust, or at Rust in your company</li>
<li>A company that thinks they can take webassembly (and their product built on it) to the next level</li>
<li>The Fuchsia team at Google</li>
</ul>
<p>or something completely different that you think I’d be psyched about, please <a href="mailto:[email protected]">email me</a> and let’s see where things go!</p>
</content:encoded></item><item><title>Thoughts on Rust in 2019</title><link>https://steveklabnik.com/writing/thoughts-on-rust-in-2019/</link><guid isPermaLink="true">https://steveklabnik.com/writing/thoughts-on-rust-in-2019/</guid><pubDate>Thu, 27 Dec 2018 00:00:00 GMT</pubDate><content:encoded><p>Each year, <a href="https://blog.rust-lang.org/2018/12/06/call-for-rust-2019-roadmap-blogposts.html">we ask the community to write blog posts about what they want to see on Rust’s roadmap for the next year</a>. This is my post for Rust in 2019.</p>
<h2>Rust 2021: Maturity</h2>
<p>This year is also a bit special; in 2018, we introduced “editions” to Rust, on a roughly three-year schedule. So now is not just a good time to think about 2019, but about 2020 and 2021 as well. Rust 2015 was about “stability”. Rust 2018 was about “productivity.” I’d like Rust 2021 to be about “maturity.”</p>
<p>In order to get there, here’s what we need in 2019.</p>
<h2>No <em>new</em> features</h2>
<p>Emphasis on “new” here. What do I mean by this? Well, there are a few features that are in the pipeline that I <em>do</em> think should land:</p>
<ul>
<li><code>async</code>/<code>await</code></li>
<li>GATs</li>
<li><code>const</code> generics</li>
</ul>
<p>And <em>possibly</em></p>
<ul>
<li>Specialization</li>
</ul>
<p>None of these features are new; we’ve got their basic designs in place. These features are also <em>significant</em> and <em>foundational</em>; we need <code>async</code>/<code>await</code> (and to some degree, GATs) for a great networking story, we need <code>const</code> generics for an excellent numerics story.</p>
<p>But after that? I’d like to see a moratorium on major features until 2020, and I’d prefer if we limited 2020 to two or three, if any.</p>
<p>We’ve hit a sweet spot. We’ve always said that Rust 1.0 was about stability, not completeness. I think we’re fast approaching completeness.</p>
<p>That said, I don’t think that the language team should be disbanded or anything; I think their work should transition to working on <em>specifying</em> what we already have, in more detail. I’m not sure if we can have the reference be done in 2019 (more on that later), but I’d like it to be much further along. That can only happen with the help of the language team, and they can only do that work if they have the time.</p>
<h2>Refine the RFC process</h2>
<p>The RFC process needs to be significantly re-vamped. Niko wrote a great post on this <a href="http://smallcultfollowing.com/babysteps/blog/2018/06/20/proposal-for-a-staged-rfc-process/">back in June</a>, and I think it’s really, really, really important. I’d like to work on an RFC to propose this, so if you’re interested, we should talk.</p>
<p>Niko already makes the case and lays out some foundations, so I won’t say more there.</p>
<h2>Pay down organizational debt</h2>
<p>Consider this section an endorsement of <a href="https://boats.gitlab.io/blog/post/rust-2019/">everything boats said</a>. I cannot say it better, so I will just leave it at that.</p>
<h2>Figure out documentation sustainability</h2>
<p>This year was a bad year for the docs team. The book got shipped, and that was great. We had some people pitch in on the reference, and their work was amazing. Some of the other documentation writers moved on to work on <code>rustdoc</code>, which was important.</p>
<p>But a lot of the goals we wanted to do about writing more docs themselves never came to fruition. For example, contributing to major ecosystem crates? Never happened. The cookbook isn’t done. Rust by Example still languishes. The standard library could use a lot of love.</p>
<p>We just don’t have the people to make this happen. We’ve tried, but nothing has worked. It’s possible that this is simply unfixable, after all, most programmers don’t like writing docs. But I also don’t want to give up on it either. I’m not sure what to do here, but I do know that it is a major problem.</p>
<h2>Conclusion</h2>
<p>There’s a lot of work to do, and I’m very excited to do it. I think Rust is in an <em>excellent</em> place, and with some work, we can make Rust even more amazing in a year’s time.</p>
</content:encoded></item><item><title>Six years with Rust</title><link>https://steveklabnik.com/writing/six-years-with-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/six-years-with-rust/</guid><pubDate>Fri, 21 Dec 2018 00:00:00 GMT</pubDate><content:encoded><p>Today is six years since I first heard of Rust. <a href="https://words.steveklabnik.com/five-years-with-rust">I wrote a post last year</a> about it.</p>
<p>This past year was… intense. Rust 1.31 was basically Rust 2.0, at least in the marketing sense. I burned myself out getting the first edition of the book together for Rust 1.0, and I burned myself out getting the edition shipped.</p>
<p>Let’s talk about the bad and the good. Bad first so we end on the high notes.</p>
<h2>The Bad</h2>
<p>This year, the docs team sort of… died. Not entirely, but it’s… different now. We used to have a single docs team that handled most everything docs related. The core of that for the past few years has been me and two others. But early this year, we split the sub-teams into fewer individual teams. The old docs team became the docs team, the <code>rustdoc</code> team, the Rust By Example team, and the Reference team. But we didn’t really attract many more people. We did to some degree! The reference team did a <em>ton</em> of amazing reference work. RBE has still struggled. I stayed on the docs team, and the other two went to the <code>rustdoc</code> team. That’s fine, and they’re doing great work, but it’s a little lonely over here now. I can’t decide how much of this issue is “it’s really hard to get people to volunteer to write docs” and how much of this issue is “I mis-managed the team(s) somehow and so it’s my fault.”</p>
<p>Shipping the edition was a monumental effort that caused a lot of debt. We did a good job at avoiding <em>technical</em> debt, but we have a lot of social and organizational debt. <a href="https://boats.gitlab.io/blog/post/rust-2019/">See here for more</a>. This has to be paid off now…</p>
<p>There’s a few other things, but let’s stop there.</p>
<h2>The Good</h2>
<p>We shipped the edition! In fact, we invented the entire edition system and then shipped our first one. Whew! This is a <em>massive</em>, <em>massive</em> accomplishment. Figuring out how we are going to manage Rust’s evolution decades into the future is, frankly a pretty amazing thing to do in twelve months. Non-lexical lifetimes finally landed. Procedural macros finally landed. Rocket is almost on stable. <code>async</code>/<code>await</code> is almost here, and <code>Futures</code> are finally where they should be after two years of work. We did <em>so much</em> good work this year, technically speaking. I’m very proud.</p>
<p>We shipped the new website! This was met with some controversy, but every single website redesign in the history of redesigns has been met with controversy. Doing so in less than a year is also an incredible accomplishment. It almost always takes way longer. We now have the foundations for Rust web properties going forward. This will also pay off a ton later.</p>
<p>We made wasm a thing! I really really wanted wasm to be a thing. A lot of people put in a ton of work, and Rust’s wasm support is bar-none best-in-class. I think this is still a <em>massive</em> growth area for Rust, and we’re almost a full year ahead of almost everyone else on this.</p>
<p>Speaking of growth, Rust’s commercial uptake skyrocketed this year. I met someone for the first time who told me “I learned Rust because it’s what we use at work.” A ton of companies, big and small, have started deploying Rust in critical places, and report good things. Out of the FAANGs, we’ve got Facebook, Amazon, and Google. Three out of five ain’t bad. This tweet is wild:</p>
<blockquote>
<p>Introducing #HyperledgerSawtooth version 1.1, an enterprise-grade #blockchain platform with support for #WebAssembly smart contracts and great performance with @rustlang. Empower your business. https://t.co/wl9OYVnUsR #HyperledgerForum pic.twitter.com/5vurpbVMP1
— Intel Security (@IntelSecurity) December 12, 2018</p>
</blockquote>
<p>So is this one:</p>
<blockquote>
<p>Discover Rust, a C-like language that supports imperative, functional, and object-oriented programming. https://t.co/fCgY4x6zqc
— IBM Developer (@IBMDeveloper) June 10, 2018</p>
</blockquote>
<p>There’s still lots more to do, but things are headed in the right direction.</p>
<p>Speaking of heading in the right direction, tomorrow will be the compliment to this post; my entry for <code>#rust2019</code>. Catch you then!</p>
</content:encoded></item><item><title>Why is Clojure so stable?</title><link>https://steveklabnik.com/writing/why-is-clojure-so-stable/</link><guid isPermaLink="true">https://steveklabnik.com/writing/why-is-clojure-so-stable/</guid><pubDate>Tue, 18 Dec 2018 00:00:00 GMT</pubDate><content:encoded><p>There’s been some Programming Language Discourse lately, and I have some thoughts. It’s mostly centered around <a href="https://news.ycombinator.com/item?id=18702802">a comment on the orange website</a>:</p>
<blockquote>
<p>I would put it simply: Clojure was designed by a thinker, who creates when he is away from the keyboard, not in front of it. When one releases and breaks the code in his head first, very few breaking changes are left for the public releases.</p>
</blockquote>
<p>I think, regardless of this person being right or wrong, there’s an interesting question here. Let’s look at <a href="https://news.ycombinator.com/item?id=18701834">the comment this is ultimately responding to</a>:</p>
<blockquote>
<p>From the perspective of a (fairly large-scale at this point) app developer: I find it great that Clojure places such emphasis on backwards compatibility. In general, migration to newer Clojure versions is completely painless.The language has been designed by experienced and mature people and doesn’t go through “let’s throw everything out and start again” phases like so many other languages do.</p>
</blockquote>
<p>Clojure has a well-deserved reputation for stability. The argument is not about that, but about <em>why</em> Clojure is so stable.</p>
<h2>Individuals and their greatness</h2>
<p>A lot of people, like the original commentator, believe that Rich, the BDFL of Clojure, is responsible. I think Rich deserves a lot of credit, but at the same time, this explanation is a little bit too close to the “great man theory of history.” The Great Man theory was originally proposed in the mid 1800s by Thomas Carlyle, and goes something like this: "“the history of the world is but the biography of great men.”</p>
<blockquote>
<p>Don’t get me started about Carlyle, who was a pretty terrible guy even in his own day. A lot of the alt-right, “Dark Englightenment”, and Neo-reactionary folks consider him a really important thinker, and that’s all I’ll say about that.</p>
</blockquote>
<p>What this <em>means</em> is that Important Men make big decisions, and that the lives of the rest of us are irrelevant in the scheme of things. He cited people like Shakespeare, Martin Luther, and Napoleon. In Carlyle’s mind, history was changed by these men and them alone. The French Army didn’t win wars, Napoleon won wars.</p>
<p>I tend to subscribe to the opposite, that is, “history from below,” or “a people’s history,” which emphasizes the army over Napoleon, the worker over the CEO. A critic of Carlyle’s, Herbert Spencer, said this:</p>
<blockquote>
<p>You must admit that the genesis of a great man depends on the long series of complex influences which has produced the race in which he appears, and the social state into which that race has slowly grown. … Before he can remake his society, his society must make him.</p>
</blockquote>
<p>That is, we have to look at historical context to see what created these changes, not just a few famous individuals.</p>
<blockquote>
<p>Two sidebars here: first, Spencer is the person who coined the phrase “survival of the fittest.” Second, there is an irony in framing this particular philosophical debate about if individuals or history cause historical change by citing well-known opinions of two individual men. This idea was so popular, and still is, that it’s something many of us latch onto, sometimes subconsciously, even those who don’t believe it!</p>
</blockquote>
<p>So, beyond Rich, who as I said before, does deserve a lot of credit… why might Clojure be so stable? I can think of three reasons. There may be more, but these are what comes to my mind.</p>
<h2>Clojure is a Lisp</h2>
<p>First of all, Clojure is a Lisp. This has a wide array of reasons why stability is a bit easier than in other languages.</p>
<p>The first is Lisps’ lack of syntax. Most languages have updates that introduces new syntax. As this grows larger and larger, the design space gets more and more constrained, making updates harder and harder. Lisps, on the other hand, rarely grow syntactically. Clojure has more syntax than most Lisps, but it still has far less syntax than non-Lisp languages.</p>
<p>Another perspective of this is homoiconicity. This property means that “new syntax” for Lisps can be introduced through macros, and not through the base language. In most other languages, users cannot extend the language to provide new syntax, but in Clojure, they can.</p>
<p>Lesson: the smaller your core language, the more stable you can be.</p>
<h2>Clojure is dynamically typed</h2>
<p>There are many, many, many ways to debate the virtues of static typing against those of dynamic typing. However, one area in which dynamically typed languages clearly win out, in my mind, is that they have a wider set of tools at their disposal to make changes without breaking code. This cuts both ways; they also make it much, much harder to know if you’ve broken code as well. Those are able to be mitigated through a variety of techniques, and we can argue about the tradeoffs here, but I think the possibility of advantage is plainly clear.</p>
<p>Lesson: it can be easier to change a dynamic system than a static one. This is both good and bad, but for now we’re focused on the good.</p>
<h2>Clojure’s development is pretty slow</h2>
<p>Clojure famously optimizes contributions around Rich and his process, at the expense of others. Debating this has been a real sore spot lately, and I don’t mean to pile onto this part of the discussion. I’m not interested in suggesting, in this blog post, if this is good or bad. I’m interested in the effects it has.</p>
<p>If we ask <a href="https://github.com/clojure/clojure/compare/1.0...master">GitHub how many commits were between 1.0 and <code>master</code></a>, the answer is 1,957. If we look at <a href="https://github.com/clojure/clojure/compare/clojure-1.9.0...clojure-1.10.0">commits between 1.9.0 and 1.10.0</a>, we get 138 commits. Clojure 1.9 was released December 8, 2017. Clojure 1.10 was released December 17, 2018. That’s 138 commits for that release, for a year.</p>
<p>In contrast, <a href="https://github.com/rust-lang/rust/compare/1.30.0...1.31.0">between 1.30 and 1.31</a>, six weeks of development, Rust had 1,694 commits. that’s almost as many as Clojure has had in the last nine and a half years.</p>
<p>Lesson: It’s much easier to keep things stable when you don’t change as often.</p>
<hr />
<p>All of these factors tie into each other. Clojure doesn’t need to change as much, since the core language is a Lisp. When it does need to change, it has good tools at its disposal to change. Also, note that Rich was the one who made a lot of these decisions! He absolutely deserves credit for a job well done, even if I think that there are many other factors at play.</p>
<p>Furthermore, ascribing this entirely to Rich makes it harder for other languages to learn good lessons from Clojure’s successes; they can’t just get a Rich. By digging in a bit deeper, we can apply these lessons a bit more broadly. Good language design requires taking a sober look at various tradeoffs, and making the best decision you can make, regardless if you’re a BDFL or part of a language design team.</p>
</content:encoded></item><item><title>Borrow checking, escape analysis, and the generational hypothesis</title><link>https://steveklabnik.com/writing/borrow-checking-escape-analysis-and-the-generational-hypothesis/</link><guid isPermaLink="true">https://steveklabnik.com/writing/borrow-checking-escape-analysis-and-the-generational-hypothesis/</guid><pubDate>Wed, 10 Oct 2018 00:00:00 GMT</pubDate><content:encoded><p>There’s a fairly common argument that pops up on programming forums, and it’s about the nature of what “garbage collection” means. In the research world, <a href="http://www.memorymanagement.org/glossary/g.html#term-garbage-collection">this</a> is what GC means:</p>
<blockquote>
<p>Garbage collection (GC), also known as automatic memory management, is the automatic recycling of dynamically allocated memory(2). Garbage collection is performed by a garbage collector which recycles memory that it can prove will never be used again. Systems and languages which use garbage collection can be described as garbage-collected.</p>
</blockquote>
<p>Historically, there has been two major forms of GC: reference counting, and tracing. The argument happens because, with the rise of tracing garbage collectors in many popular programming languages, for many programmers, “garbage collection” is synonymous with “tracing garbage collection.” For this reason, I’ve been coming around to the term “automatic memory management”, as it doesn’t carry this baggage.</p>
<hr />
<p>I’ve often seen people make statements like this one, from the Rust subreddit this morning:</p>
<blockquote>
<p>Manual memory management requires more work than garbage collected. Its a trade off of course to be more performant or use lower resources. When and where should Rust be used or not used according to you?</p>
</blockquote>
<p>While I don’t <em>completely</em> disagree with this sentiment, it’s also never quite sat right with me. Yes, Rust is a bit harder <em>at the start</em>, but once you get over a hump, I don’t generally find writing Rust to be significantly harder than using a GC’d language. I’ve been trying to figure out why that is.</p>
<blockquote>
<p>The idea that it’s easy for me because I’m an amazing programmer is not an answer I’m willing to accept here, both because I don’t think that’s particularly true, but also because I’ve seen Rust have a pretty wide appeal to a fairly large group of people, and they can’t all be amazing programmers. It’s just math.</p>
</blockquote>
<hr />
<p>My first clue comes from my own programming history. Some of my first programming languages were C, C++, and Java. These languages are <em>statically</em> typed, that is, they require you to declare (or have inferred) the type of variables up front, and check that the type holds. That is, they’re known at compile time. I can remember the first time I came across Perl, and the concept of dynamic typing. Wow, this is <em>so much easier</em>! I could do <em>anything</em>. From then on, I pretty much focused on dynamically typed languages.</p>
<p>While doing my work in Ruby, one thing I worked hard on was testing. Rubyists are big on tests; with a language that’s <em>so</em> dynamic, you really need them to make sure your stuff is working properly.</p>
<p>When I decided to start writing a bunch of Rust, I had to get used to using static types again. But I also wrote less tests. Many of the things that I was testing for were things that were taken care of by the type system. In other words, my tests were sort of like “dynamic type checking.” Instead of being checked by a compiler at compile-time, these behaviors of my system were checked at run time, by my tests. This isn’t a novel insight; many others have stated this relationship between tests and types.</p>
<p>We often acknowledge that writing tests is a <em>skill</em>. When I first started with Ruby, I was bad at testing. I got better with practice. But I don’t think we collectively acknowledge that working with types is <em>also</em> a skill. At first, it’s tough. But as you develop those skills, it becomes easier and easier.</p>
<hr />
<p>Let’s switch gears a little, to garbage collection.</p>
<p>In garbage collection research, there’s a term, the <a href="http://www.memorymanagement.org/glossary/g.html#term-generational-hypothesis">“generational hypothesis”</a>. It states</p>
<blockquote>
<p>In most cases, young objects are much more likely to die than old objects.</p>
</blockquote>
<p>If that succinct description doesn’t make sense, here’s a slightly longer one: there’s a kind of GC called a “generational garbage collector.” There are a few options when implementing one, but I’m going to describe one form in particular. What it does is this: new objects are allocated in a section of memory called the “nursery,” an analogy to where human children go shortly after they’re born. When the GC runs, it looks through the nursery. Any objects that survive are promoted to a new section of memory. Some collectors may have multiple of these “generations.” The generational hypothesis states that most objects die in the nursery. The GC can then take advantage of this fact by scanning the nursery more often than the older generations, which allows you to not scan every object every collection.</p>
<p>Empirically, the generational hypothesis is true. But I haven’t seen much investigation into <em>why</em> it is true. That said, I’m also not a GC expert. If you have references here, please get in touch.</p>
<hr />
<p>There’s another concept related to GCs called “escape analysis.” This is an optimization which several languages do. The idea is fairly straightforward: if something is declared in a function, and never leaves (“escapes”) that function, you don’t need to put the object on the heap, you can stack allocate it instead.</p>
<blockquote>
<p>“It isn't that the generational hypothesis isn't true for Go, it's just that the young objects live and die young on the stack.The result is that generational collection is much less effective than you might find in other managed runtime languages” #golang https://t.co/aXEleY2yQ6
— Dave Cheney (@davecheney) July 18, 2018</p>
</blockquote>
<p>If you squint a little at this tweet, you can almost think of escape analysis as a <em>stronger</em> version of the generational hypothesis. The claim is that escape analysis applies so often to Go code that any object that reaches the heap is effectively in the older generation.</p>
<p>Why is this? I’m not totally sure, but I have a feeling that is has to do with the core concept of functions and a call stack. Additionally, the value/reference type distinction. The more you program with values in functions, rather than references, the less likely objects are to escape past the call stack. in generational hypothesis terms, values allocated lower in the call stack tend to live longer, and ones higher in the call stack tend to live shorter.</p>
<hr />
<p>There’s an interesting area of type research called “gradual typing.” The idea is that typing is not an either/or, static vs dynamic. It’s a spectrum. Some things are statically typed, some things are dynamically typed, and it all works together. The idea is that you can start off writing things that are dynamically typed, and then once you’re more sure of the behaviors you want, you can start to statically type them.</p>
<p>TypeScript is a great example:</p>
<pre><code>// person is dynamically typed
function greeter(person) {
return "Hello, " + person;
}
// person is statically typed
function greeter(person: string) {
return "Hello, " + person;
}
</code></pre>
<p>TypeScript is really, really growing in popularity. I think one of those factors is the gradual typing; you can slowly work on your type-using skills, and you don’t have to use them everywhere.</p>
<hr />
<p>Interestingly enough, many JIT compilers rely on the fact that in practice, programmers tend to pass the same types of things to functions, even if the language is dynamically typed. This is called <a href="https://hacks.mozilla.org/2017/02/a-crash-course-in-just-in-time-jit-compilers/">“type specialization”</a>. This is similar to the generational hypothesis to me, but in a different domain. And as I said about it, empirically, this is true. But I haven’t seen much investigation into <em>why</em> it is true. That said, I’m also not a JIT expert. If you have references here, please get in touch.</p>
<hr />
<p>So what’s all this have to do with Rust?</p>
<p>Similar to the dichotomy between static and dynamic typing, I think that Rust is proving out a new niche in the concept of garbage collection. That is, historically, we think of GC as something <em>dynamic</em>, that is, it involves a run-time component that does stuff at runtime. However, the idea of automatic memory management doesn’t <em>inherently</em> mean that it has to execute at runtime.</p>
<blockquote>
<p>I should mention that C++ paved a lot of the path here too, so maybe the above is worded too strongly. I do think that Rust is significantly different here, but you may not. If you don’t, please replace the above with “Rust is continuing to prove out a niche pioneered by C++,” and that’s fine by me.</p>
</blockquote>
<p>This also opens up two more things about why I don’t feel that “Rust is too hard and so you should use a GC’d language when you can afford it.” First of all, Rust’s strong, static checking of memory provides me a similar peace of mind to a runtime GC. To make an analogy to types and tests, when I program in a language like C, I have to think a lot about the details of memory management, because I don’t really have anything at all checking my work. Its memory management is completely unchecked, similar to (compile time) types in a Ruby program. But with automatic memory management, I don’t have to worry about this, as the memory management system has my back. I won’t get a use after free, I won’t get dangling pointers.</p>
<p>I further think this line of thought is on the right track because lifetimes are generic types, and the ownership system is affine types.</p>
<hr />
<p>This also relates to the “difficulty” of Rust. In the same way that learning to use static types takes time, learning to use static memory management takes time. But once you’ve developed that skill, you can rely on those static checks to free up your brain to instead focus on your program itself, and not the house-keeping tasks that these systems are checking for.</p>
<p>And, just like with static types, different people will make different choices of tradeoff for effort. Some people refuse to use dynamically typed languages, because they’ve invested the time into effectively using static types and prefer to have these things checked. Others don’t want to waste time on learning these kinds of things, and are more focused on shipping things. They have their own set of skills to ensure reliability. I think both approaches make sense depending on what you’re doing.</p>
<hr />
<p>In some ways, you can think of the borrow checker as an even stronger form of escape analysis; that is, instead of saying “oh, this escapes, so allocate it on the heap”, the borrow checker says “oh, this escapes, so error at compile time.” For example, this code:</p>
<pre><code>fn bar() -&gt; i32 {
5
}
fn foo() -&gt; &amp;'static i32 {
&amp;bar()
}
</code></pre>
<p>gives this error:</p>
<pre><code>error[E0716]: temporary value dropped while borrowed
--&gt; src/lib.rs:6:6
|
6 | &amp;bar()
| ^^^^^ creates a temporary which is freed while still in use
7 | }
| - temporary value is freed at the end of this statement
|
= note: borrowed value must be valid for the static lifetime...
</code></pre>
<p><code>bar()</code> produces a value, and so <code>&amp;bar()</code> would produce a reference to a value on <code>foo()</code>’s stack. Returning it would be a dangling pointer. In a system without automatic memory management, this would cause a dangling pointer. In a system with dynamic automatic memory management, this would heap allocate the return value of <code>bar()</code>, so that everything works out. But Rust’s automatic memory management is fully static, and so it has to error in this situation.</p>
<hr />
<p>Similar to gradual typing, this means that you can also have gradual GC. For example, Rust does have <code>Rc&lt;T&gt;</code> and <code>Arc&lt;T&gt;</code>, which are reference-counted types, clearly GC according to the literature. This moves the memory management from static to dynamic.</p>
<blockquote>
<p>I don’t think that the transition here is as seamless as in TypeScript, though that’s a whole other post… additionally, it’s not like gradual typing is perfect.</p>
</blockquote>
<hr />
<p>One last thing before I go: the lifetime elision rules are also an empiric statement about how programmers write code, but I’m not sure what the underlying reasons are. From <a href="https://github.com/rust-lang/rfcs/blob/master/text/0141-lifetime-elision.md">the RFC</a>:</p>
<blockquote>
<p>We found that of the 169 lifetimes that currently require annotation for libstd, 147 would be elidable under the new rules, or 87%.</p>
</blockquote>
<p>If it were 100%, we could never require you to write annotations. But with 87%, we can let you not write annotations, and most code that follows most normal coding practices never needs to write explicit lifetimes.</p>
</content:encoded></item><item><title>Should you learn C to "learn how the computer works"?</title><link>https://steveklabnik.com/writing/should-you-learn-c-to-learn-how-the-computer-works/</link><guid isPermaLink="true">https://steveklabnik.com/writing/should-you-learn-c-to-learn-how-the-computer-works/</guid><pubDate>Tue, 02 Oct 2018 00:00:00 GMT</pubDate><content:encoded><p>I’ve often seen people suggest that you should learn C in order to learn how computers work. Is this a good idea? Is this accurate? I’m going to start with my conclusion right upfront, just to be crystal clear about what I’m saying here:</p>
<ul>
<li>C is not “how the computer works.”</li>
<li>I don’t think most people mean this phrase literally, so that is sort of irrelevant.</li>
<li>Understanding the context means that learning C for this reason may still be a good idea for you, depending on your objectives.</li>
</ul>
<p>I plan on making two follow-up posts as well, exploring more implications of this idea, but this is already quite a lot. I’ll update this post with links to them when I make them.</p>
<hr />
<p>I’ve often seen people suggest this:</p>
<blockquote>
<p>By learning C, you can learn how computers work.</p>
</blockquote>
<p>I think that this idea is not inherently wrong, but does come with some caveats. As long as you keep those caveats in mind, I think this can be a viable strategy for learning new and important things. However, I rarely see people discuss this slogan in detail, so I’m writing this post so that I can provide that context that I believe is sorely needed. If you’re thinking about learning C for this reason, this post is written for you. I hope it’ll assist you as you learn.</p>
<blockquote>
<p>Before we truly begin, I’d like to say one more thing: if you want to learn C, then learn C! Learning things is great. Learning C was quite important to my understanding of computing and my career. Learning C and about its place in programming language history will make you a better programmer. You don’t need a justification. Learn stuff for the sake of learning it. This post is intended to be a guide to your quest for understanding, not a suggestion that you should or should not learn C.</p>
</blockquote>
<p>First of all, let’s talk about who is generally recommended this idea. If you’re trying to “learn how computers work,” then it stands to reason that you don’t currently understand that. Which programmers don’t understand how computers work? I’ve mostly seen this sentiment come from people who primarily program in dynamically typed, “scripting” languages, like Ruby, Python, or JavaScript. They “don’t know how computers work” because these languages operate inside of a virtual machine, and therefore, the semantics of the virtual machine are what actually matters. After all, the whole idea of a virtual machine is to provide portability. The goal is to only require knowledge of what’s portable, rather than rely on the details of the hardware that the VM is running on.</p>
<p>There’s just one problem with this: C <em>also</em> operates inside of a virtual machine.</p>
<h2>The C abstract machine</h2>
<p>From the <a href="http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf">C99 spec</a>, Section 5.1.2.3, “Program Execution”:</p>
<blockquote>
<p>The semantic descriptions in this International Standard describe the behavior of an abstract machine in which issues of optimization are irrelevant.</p>
</blockquote>
<p>In my opinion, this is the most important concept to understand when learning C. C does not “describe how the computer works,” it describes how the “C abstract machine” works. Everything else of importance flows from this concept and its implications.</p>
<blockquote>
<p>A further note: I’ve chosen C99 here, which is not the latest C standard. Why? Well, MSVC has an …interesting amount of C support, and I’m a Windows user these days. Yes, you can run clang and gcc on Windows. There is not a ton of difference between C89, C99, and C11 with regards to what we’re talking about in these posts. At some point you have to pick. The version I’ve linked here includes some amendments to the initial spec.</p>
</blockquote>
<p>You may have heard another slogan when talking about C: “C is portable assembler.” If you think about this slogan for a minute, you’ll also find that if it’s true, C cannot be how the computer works: there are many kinds of different computers, with different architectures. If C is like a version of assembly language that works on multiple computers with multiple architectures, then it cannot function exactly how each of those computers work simultaneously. It <em>must</em> hide details, or else it wouldn’t be portable!</p>
<p>That said, I think this fact is sort of irrelevant, because I don’t think people mean the phrase “C is how the computer works” literally. Before we can talk about that, let’s talk about the C abstract machine, and why many people don’t seem to understand this aspect of the C language.</p>
<h2>Aside: why do people get this wrong?</h2>
<p>I can only relate my own experience, though I think it generalizes to many others.</p>
<p>I learned GW-BASIC, then C, then C++, then Java. I had heard about Java before I started writing it in roughly 1999, four years after it was introduced. Java’s marketing at the time was very anti-C++, and it focused on the JVM as a platform, and how the machine model was what differentiated it from C++, and therefore C. Sun Microsystems no longer exists, but a <a href="https://tech-insider.org/java/research/1996/0123.html">mirror of the press release</a> has this to say:</p>
<blockquote>
<p>Java-based applications are platform-independent; only the Java Virtual Machine needs to be ported to each platform. It acts as an interpreter between an end user’s computer and the Java-based application. An application written in the Java environment can run anywhere, ending the need for porting applications to multiple platforms.</p>
</blockquote>
<p>“Write once, run everywhere” was the big slogan. These two sentences were how I (and many others) came to understand Java, and how it differed from C++. Java has an interpreter, the Java Virtual Machine. C++ does not have a Virtual Machine.</p>
<p>With this huge marketing push, “virtual machine” became synonymous with “large runtime and/or interpreter” in the minds of many. Languages without this feature were too tied to the particular computer, and required porting between platforms, because they are not truly platform independent. Java’s very reason to exist was to change this aspect of writing applications in C++.</p>
<blockquote>
<p>“Runtime”, “virtual machine”, and “abstract machine” are different words for the same fundamental thing. But they’ve since gained different connotations, due to non-essential variance in different implementations of these ideas.</p>
</blockquote>
<p>I personally believe that this marketing, in 1995, is why programmers still misunderstand C’s nature today.</p>
<p>So is this claim false? Why would Sun Microsystems spend millions and millions of dollars promoting a lie? If C is also based on an abstract machine that offers portability between platforms, why did Java need to exist? I think this is key to understanding what people <em>really</em> mean when they say “C is how the computer works.”</p>
<h2>What do people actually mean?</h2>
<p>Even though C operates in the context of a virtual machine, it still is significantly different than something like Java. Sun was not lying. In order to understand why, you have to understand C’s history.</p>
<p>An operating system was written in assembly language for a computer called the “PDP-7” in 1969 at Bell Labs. In 1970, it was christened UNIX. As time went on, Bell Labs bought more and newer computers; including the PDP-11.</p>
<p>When it came time to port Unix to the PDP-11, they considered using a higher-level language, which was a pretty radical idea at the time. Imagine that I said to you “I’m going to write an OS in Java” today, you’d probably laugh, even though <a href="https://en.wikipedia.org/wiki/JavaOS">it is possible</a>. The situation (in my understanding, I wasn’t alive then) was pretty analogous. They considered a language called B, but it didn’t support some of the features that the PDP-11 had, and so they created a successor, naming it “C”, since C was the next letter in the alphabet.</p>
<blockquote>
<p>There was no “A”; B was a successor to BCPL, the “Basic Combined Programming Language.”</p>
</blockquote>
<p>In 1972, on a PDP-11, they wrote the first C compiler, and simultaneously re-wrote UNIX in C. Initially, portability wasn’t the actual goal, but C did resonate with a lot of people, and C compilers were ported to other systems.</p>
<p>In 1978, the first edition of “The C Programming Language,” a book on C, was published. Affectionately known as “K&amp;R,” after its authors, the book was very much not a specification, but provided enough of a description of the language that others attempted to write C compilers. This would later be called “K&amp;R C.”</p>
<p>As UNIX spread, so did C, and both were ported to many computers. In the 70s and 80s, the ports grew and grew. In the same way that C was created because B couldn’t support all of the features of the PDP-11, many compilers had extensions to the language. Since there was only K&amp;R, and not a spec, as long as they were close enough, that was considered acceptable. By 1983, the lack of any sort of standardization was causing issues, and so a group was created at ANSI to produce a spec. C89 was published in 1989, and is sometimes called “ANSI C.”</p>
<p>As such, the C specification was attempting to unify these diverse implementations, on diverse hardware. And so the C abstract machine is sort of the minimal possible specification that would allow the same code to run the same on all platforms. Implementations of C were compiled, not interpreted, and so there was no interpreter, so there was no “VM” in that 1995 sense. However, C programs are written against this abstract, non-existent machine, which is then translated into assembly that is specific to the actual computer that the program is running on. You could not rely on some specific details in order to write portable C code. This makes writing portable C very tricky, as you may have made a platform-specific assumption when writing the initial version of your code.</p>
<p>This is best illustrated with an example. One of the fundamental data types in C is a <code>char</code>, named after “character.” However, the C abstract machine does not define how many bits a <code>char</code> is. Well, it defines it, but not with a number; it defines it as <code>CHAR_BIT</code> big, which is a constant. Section 5.2.4.2.1 of the spec:</p>
<blockquote>
<p>The values given below shall be replaced by constant expressions suitable or use in #if preprocessing directives. … Their implementation-defined values shall be equal or greater in magnitude (absolute value) to those shown, with the same sign.CHAR_BIT: 8</p>
</blockquote>
<p>So in other words, you know that a <code>char</code> must be at least 8 bits in size, but implementations are allowed to be larger. In order to properly code against the “C abstract machine”, you must use <code>CHAR_BIT</code> instead of <code>8</code> as the size when doing <code>char</code>-by-<code>char</code> processing. But this isn’t some sort of interpreter feature, in the way that we think of virtual machines; it’s a property of how the compiler turns your source code into machine code.</p>
<blockquote>
<p>Yes, there are systems where CHAR_BIT isn’t 8.</p>
</blockquote>
<p>So this “abstract machine”, while still technically being the same idea as the Java VM, is more of a compile-time construct to guide compilers in the task of emitting assembly code than some sort of runtime check or property. The equivalent type in Java is <code>byte</code>, which is always 8 bits, and it’s up to the implementation of the JVM to decide how to make that happen on platforms where a byte is larger. (I am not sure if the JVM runs on any of those platforms, but that’s how it would work.) The C abstract machine was created to be a minimal papering over of various hardware, not to be some sort of platform created from whole cloth that’s written in software that your code targets.</p>
<p>So while Sun is technically incorrect, in practice, they mean something different than what they’re literally saying, and what they <em>mean</em> is accurate. It’s the same with “learn C to learn how computers work.”</p>
<h2>Learn C to learn <em>more</em> about how computers work</h2>
<p>What <em>do</em> people actually mean? In the context of “should a Rubyist learn C to learn about how computers work”, this desire to drop “down to the metal,” as it were, is an interest in understanding not only their program, and how it works inside the VM, but to understand how the combo of their program + their VM operates in the context of the machine itself.</p>
<p>Learning C <em>will</em> expose you to more of these kinds of details, because the abstract machine maps so much more closely to hardware, as well as abstractions provided by operating systems. C is very different than these sorts of languages, and so learning C can teach you a lot.</p>
<p>But it’s also important to remember that C is fundamentally an <em>abstraction</em> of hardware, and abstractions are leaky. Don’t conflate what C does or how it operates with the machine itself. If you do, you’re bound to run afoul of these differences, which can cause problems. Most learning resources for C, especially today, as hardware becomes more and more homogeneous, will promote the idea that this is how the computer works. So it can be hard, as a learner, to know what’s going on under the hood, and what’s an abstraction provided by C.</p>
<p>We haven’t even touched on other factors in this discussion, like that due to C’s tremendous popularity, hardware has become more homogeneous because it tends towards being similar to the semantics of C’s abstract machine. If your architecture deviates too much from C’s semantics, C programs may run much more slowly than others, and C programs are often how the speed of hardware is tested. This post is long enough…</p>
<p>For this reason, I think a more accurate version of this statement would be “By learning C, you can learn <em>more</em> about how computers work.” I do think that a rough familiarity with C can be useful to many programmers, and will serve them well even if they don’t write any C themselves. Learning more about C can also give you insight into the history of how our field has developed.</p>
<p>There are other ways to learn this stuff as well; C is not <em>inherently</em> the way to learn these topics, but it is a good option. My next two posts will explore some of the implications of this intersection between portability, the C abstract machine, and how people think about C programs.</p>
<p>There’s so many things to learn in computing. I wish you well on your journey.</p>
</content:encoded></item><item><title>You can't "turn off the borrow checker" in Rust</title><link>https://steveklabnik.com/writing/you-can-t-turn-off-the-borrow-checker-in-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/you-can-t-turn-off-the-borrow-checker-in-rust/</guid><pubDate>Fri, 14 Sep 2018 00:00:00 GMT</pubDate><content:encoded><p>Every once in a while, someone will talk about <code>unsafe</code> in Rust, and how it “turns off the borrow checker.” I think this framing leads to misconceptions about <code>unsafe</code> and how it interacts with safe code.</p>
<p>Here’s some code that causes a borrow checker error:</p>
<pre><code>fn main() {
let mut x = 5;
let y = &amp;x;
let z = &amp;mut x;
println!("{}", y);
}
</code></pre>
<p>And the error:</p>
<pre><code>error[E0502]: cannot borrow `x` as mutable because it is also borrowed as immutable
--&gt; src/main.rs:5:18
|
4 | let y = &amp;x;
| - immutable borrow occurs here
5 | let z = &amp;mut x;
| ^ mutable borrow occurs here
6 | }
| - immutable borrow ends here
</code></pre>
<p>Rust won’t let us have a <code>&amp;T</code> and a <code>&amp;mut T</code> to the same <code>T</code> at the same time.</p>
<p>If <code>unsafe</code> "turned off the borrow checker, we’d expect this code to work:</p>
<pre><code>fn main() {
let mut x = 5;
unsafe {
let y = &amp;x;
let z = &amp;mut x;
println!("{}", y);
}
}
</code></pre>
<p>However, we get the same error, plus a new warning:</p>
<pre><code>error[E0502]: cannot borrow `x` as mutable because it is also borrowed as immutable
--&gt; src/main.rs:6:22
|
5 | let y = &amp;x;
| - immutable borrow occurs here
6 | let z = &amp;mut x;
| ^ mutable borrow occurs here
7 | }
| - immutable borrow ends here
warning: unnecessary `unsafe` block
--&gt; src/main.rs:4:5
|
4 | unsafe {
| ^^^^^^ unnecessary `unsafe` block
|
= note: #[warn(unused_unsafe)] on by default
</code></pre>
<p>So what does <code>unsafe</code> actually do? Unsafe Rust is a <em>superset</em> of Safe Rust. <code>Unsafe</code> never changes the semantics of Rust code. It instead adds new features that you can only use inside an <code>unsafe</code> block.</p>
<p>One of those features is the ability to de-reference a “raw pointer”. This code <em>does</em> compile:</p>
<pre><code>fn main() {
let mut x = 5;
let y = &amp;x as *const i32;
let z = &amp;mut x as *mut i32;
unsafe {
println!("y: {} z: {}", *y, *z);
}
}
</code></pre>
<p><code>y</code> and <code>z</code> here have the types <code>*const i32</code> and <code>*mut i32</code>, both kinds of raw pointers. In order to print <code>y</code> and <code>z</code>, we must de-reference them, which means that we need an unsafe block.</p>
<blockquote>
<p>Incidentally, this code is safe. Raw pointers are allowed to alias. And we have no &amp;T or &amp;mut Ts here. So we’re good.</p>
</blockquote>
<p>Why does this matter? Well, a lot of people think that as soon as you drop into <code>unsafe</code>, you’re missing the point of Rust, and that you lose all of its guarantees. It’s true that you have to do a lot more work when writing unsafe, since you don’t have the compiler helping you in certain situations, but that’s only for the unsafe constructs.</p>
<p>For example, let’s look at Rust’s standard library, and the <code>[LinkedList&lt;T&gt;](https://doc.rust-lang.org/stable/std/collections/linked_list/struct.LinkedList.html)</code> that it contains. It looks like this:</p>
<pre><code>pub struct LinkedList&lt;T&gt; {
head: Option&lt;NonNull&lt;Node&lt;T&gt;&gt;&gt;,
tail: Option&lt;NonNull&lt;Node&lt;T&gt;&gt;&gt;,
len: usize,
marker: PhantomData&lt;Box&lt;Node&lt;T&gt;&gt;&gt;,
}
</code></pre>
<p>I’m not gonna go into <em>too</em> much detail here, but it has a head pointer, a tail pointer, a length. <code>[NonNull&lt;T&gt;](https://doc.rust-lang.org/stable/std/ptr/struct.NonNull.html)</code> is like <code>*mut T</code>, but asserts that it will never be null. This means that we can combine it with <code>Option&lt;T&gt;</code>, and the option will use the <code>null</code> case for <code>None</code>:</p>
<pre><code>fn main() {
println!("{}", std::mem::size_of::&lt;*mut i32&gt;());
println!("{}", std::mem::size_of::&lt;Option&lt;std::ptr::NonNull&lt;i32&gt;&gt;&gt;());
}
</code></pre>
<p>prints this on my system:</p>
<pre><code>8
8
</code></pre>
<p>This optimization is guaranteed due to guarantees on both <code>Option&lt;T&gt;</code> and <code>NonNull&lt;T&gt;</code>.</p>
<p>So now, we have a sort of hybrid construct: <code>Option&lt;T&gt;</code> is safe, and so we can do some operations entirely in safe code. Rust is now forcing us to handle the null checks, even though we have zero runtime overhead in representation.</p>
<p>I think a good example of how this plays out in practice is in the implementation of <code>[append](https://doc.rust-lang.org/stable/std/collections/linked_list/struct.LinkedList.html#method.append)</code>, which takes two <code>LinkedList&lt;T&gt;</code>s and appends the contents of one to the end of the other:</p>
<pre><code> pub fn append(&amp;mut self, other: &amp;mut Self) {
match self.tail {
None =&gt; mem::swap(self, other),
Some(mut tail) =&gt; {
if let Some(mut other_head) = other.head.take() {
unsafe {
tail.as_mut().next = Some(other_head);
other_head.as_mut().prev = Some(tail);
}
self.tail = other.tail.take();
self.len += mem::replace(&amp;mut other.len, 0);
}
}
}
}
</code></pre>
<p>Here’s some comments to help understand what’s going on:</p>
<pre><code> pub fn append(&amp;mut self, other: &amp;mut Self) {
// do we have a tail?
match self.tail {
// If we don't, then we have no elements, and so appending a
// list to this one means that that list is now this list.
None =&gt; mem::swap(self, other),
// If we do have a tail...
Some(mut tail) =&gt; {
// the first thing we do is, check if the other list has
// a head. if it does...
if let Some(mut other_head) = other.head.take() {
// we set our tail to point at their head, and we
// set their head to point at our tail.
unsafe {
tail.as_mut().next = Some(other_head);
other_head.as_mut().prev = Some(tail);
}
// We set our tail to their tail
self.tail = other.tail.take();
// finally, we add their length to our length, and
// replace theirs with zero, because we took all
// their nodes!
self.len += mem::replace(&amp;mut other.len, 0);
} // there's no else case for that if let; if they don't
// have a head, that list is empty and so there's
// nothing to do.
}
}
}
</code></pre>
<p>We only need <code>unsafe</code> when mutating the <code>next</code> of the <code>tail</code> of the first list, and the <code>prev</code> of the <code>head</code> of the second list. And that’s even only in the case where the <code>tail</code> exists in the first place. We can do things like “set our tail to their tail” in safe code, because we’re never de-referencing pointers. Safe Rust is helping us manage many, but not all, of our guarantees. We <em>could</em> move the <code>unsafe</code> block to encompass the entire body of this function, and Rust would still make us check those <code>Option&lt;T&gt;</code>s. It would still check that <code>self</code> and <code>other</code> are not aliased by <code>&amp;T</code> or <code>&amp;mut T</code>s in the body of the function.</p>
<p><code>unsafe</code> is a powerful tool, and should be used with care. But that doesn’t mean that you throw out many of the things that make Rust what it is.</p>
</content:encoded></item><item><title>Is WebAssembly the return of Java Applets & Flash?</title><link>https://steveklabnik.com/writing/is-webassembly-the-return-of-java-applets-flash/</link><guid isPermaLink="true">https://steveklabnik.com/writing/is-webassembly-the-return-of-java-applets-flash/</guid><pubDate>Wed, 25 Jul 2018 00:00:00 GMT</pubDate><content:encoded><p>In my <a href="http://words.steveklabnik.com/webassembly-is-more-than-just-the-web">last post on WebAssembly</a>, I made the following claim:</p>
<blockquote>
<p>Some have compared WebAssembly to Java applets; in some ways, they’re very right, but in some ways, they’re very wrong. Eventually I’ll write a post about the wrong, but for now, the right: in some sense, WebAssembly is a different way of accomplishing what the JVM set out to do: it’s a common virtual machine that can be used to build very cross-platform software.</p>
</blockquote>
<p>A lot of people expressed interest in me elaborating, so let’s get to it! For this post, I’m going to make three comparisons: to Flash, to Java Applets, and <em>occasionally</em> to PNaCL. Secondly, this post is going to focus on the <em>web</em> use-case for WebAssembly, even though the previous post was about non-web uses. We’ll make <em>that</em> comparison in the next post. Finally, this post is kind of like eating tapas, there’s a bunch of little sections. I feel like it’s a bit short, but at the same time, I’m trying to get back into the habit of blogging, and if I try to keep expanding it, it’ll take forever, so it is what it is.</p>
<hr />
<p>For what it’s worth, I think this comparison is very <em>natural</em>; that is, when you hear this pitch for WebAssembly:</p>
<blockquote>
<p>WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable target for compilation of high-level languages like C/C++/Rust, enabling deployment on the web for client and server applications.</p>
</blockquote>
<p>It sounds quite a bit like previous technologies. Asking how they relate, or assuming that they’re a slightly different take on the problem makes a lot of sense. But WebAssembly is significantly different, for a few reasons.</p>
<h2>Wasm won</h2>
<p>The first reason that WebAssembly is different is that it’s succeeded, and those technologies did not. When I say this, I mean it in a very specific way; if you look at the number of Flash applications out there, and compare it to the number of WebAssembly apps out there, Wasm loses, for sure. Those of us that are excited about WebAssembly still have a lot of work to do before it’s widely used.</p>
<p>When I say “WebAssembly won”, I mean that it’s become a part of the Web Platform. Flash, Java Applets, and PNaCL used the browser’s plugin functionality to work. They were, in a sense, <em>outside</em> of the web ecosystem. They were never standardized, and that matters a lot.</p>
<p>In some sense, this is the final point, really. But just saying that it’s true doesn’t explain <em>why</em>. The rest of this post digs into <em>why</em> this occurred, and those points sort of sum up to this one: other technologies didn’t make it, but WebAssembly did. There’s a <em>lot</em> of overlap here, but I’m trying to reasonably separate it out into individual points.</p>
<h2>Other technologies didn’t fit nicely into the platform</h2>
<p>Remember this?</p>
<p><img src="https://svbtleusercontent.com/gY9tj6D62jnC9Fc6RncFhD0xspap_small.gif" alt="https://svbtleusercontent.com/gY9tj6D62jnC9Fc6RncFhD0xspap_small.gif" /></p>
<p>Or this?</p>
<p><img src="https://svbtleusercontent.com/qaipxL2e5a4gLH5cAhRHDn0xspap_small.jpg" alt="https://svbtleusercontent.com/qaipxL2e5a4gLH5cAhRHDn0xspap_small.jpg" /></p>
<p>What about this?</p>
<p><img src="https://svbtleusercontent.com/9mGgBDzvwYYLGHFg4Pafzt0xspap_small.png" alt="https://svbtleusercontent.com/9mGgBDzvwYYLGHFg4Pafzt0xspap_small.png" /></p>
<p>If you built an applet in one of these technologies, you didn’t really build a <em>web application</em>. You had a web page with a chunk cut out of it, and your applet worked within that frame. You lost all of the benefits of other web technologies; you lost HTML, you lost CSS, you lost the accessibility built into the web. These platforms never gained the ability to interact with the rest of the platform provided by your browser. Well, <a href="https://docs.oracle.com/javase/tutorial/deployment/applet/manipulatingDOMFromApplet.html">technically they might have</a>, but that’s not how these technologies were used in practice.</p>
<p>How could the web sustain an ecosystem that doesn’t integrate with it? It was never going to happen. And users ended up solidly rejecting it. Outside of games, users <em>hated</em> Flash. Java Applets were heavy and slow. Enshrining these technologies into the web platform wasn’t going to work.</p>
<p>WebAssembly, on the other hand, is much closer to JavaScript. It doesn’t inherently require taking over a part of your screen. It doesn’t expect to be its own little closed off world. Via JavaScript today, and <a href="https://github.com/WebAssembly/host-bindings/blob/master/proposals/host-bindings/Overview.md">on its own in the future</a>, it’s able to interact with the surrounding environment. It just… fits.</p>
<h2>Other technologies were owned by companies</h2>
<p>Java was owned by Sun Microsystems, Flash was owned by Adobe. But why does this matter?</p>
<p>Corporate influence on the web is a complicated topic. But <em>in general</em>, the web runs on a psuedo-consensus model. Java and Flash, on the other hand, were controlled by their respective companies. These companies have significant motive to make profit, not to make the web better. And that’s partially what led to the above situation; these companies didn’t care that they didn’t integrate properly with the rest of the platform. Why would they? It’d be much better for business if you got locked into their platform, and abandoned the rest of the web entirely. The incentives are fundamentally misaligned.</p>
<p>WebAssembly is a joint venture between Mozilla, Google, Apple, Microsoft, and others. It’s not trying to push anyone’s particular platform, but instead represents the shared interests of a wide number of stakeholders, both corporate and individual.</p>
<h2>WebAssembly followed the web process</h2>
<p>This <em>also</em> meant that these technologies never really followed the process that we use to standardize the web. That being said, the web standards process is healthier than its ever been, and at the time these technologies were big, things worked a bit differently. Regardless, WebAssembly followed the way that we do things on the web.</p>
<p>First, <code>asm.js</code> was created, proving that we could do more impressive things with the web. Its performance was good enough to demonstrate what could be done, though it wasn’t fantastic. The secret sauce of <code>asm.js</code> was that it was just JavaScript; it was fully backwards compatible with the existing ecosystem. “Don’t break the web” is a very, very important rule for browser vendors.</p>
<p>WebAssembly was sort of <code>asm.js</code> done better. After building some degree of consensus around <code>asm.js</code>, a variety of people came together to make WebAssembly a reality. Given that it wasn’t just JavaScript, it had to go through the usual process for browsers to implement it, and it did so. Now a W3C spec, it went <em>with</em> the grain of web standards, not against them.</p>
<h3>Corollary: other technologies were too big and unstable</h3>
<p>There’s another reason Wasm succeeded: it’s tiny. Wasm takes the approach of other web technologies: start small, and build on those foundations. Stay backwards compatible. These technologies also didn’t follow that particular social convention. For Flash and Java Applets, you’d be downloading the particular runtime you needed, and so there was no need for compatibility. PNaCl built on top of LLVM IR, which is completely unstable. They were going to make it a stable subset, but then when LLVM introduces changes, it would have ended up diverging, which isn’t a great situation to be in.</p>
<p>These technologies were also far too large to ever make it through the stabilization process. Can you imagine the big four browser vendors specifying the JVM in full and then agreeing to those semantics for all time? For all of ActionScript, itself a similar-to-but-not-quite version of ECMAScript? For all of LLVM-IR?</p>
<p>Large technologies introduce an unacceptable amount of risk in this situation. It’s an all-or-nothing kind of deal, and the safe bet is to go with “nothing.” WebAssembly, on the other hand, does almost nothing. It’s pretty much math and calling into JavaScript. This means it’s <em>much</em> easier to develop consensus around.</p>
<h3>Collary: other technologies required an entire separate virtual machine</h3>
<p>I haven’t mentioned Dart in this post, but it sort of fits here too. One big issue with “let’s just put the JVM in every browser” is that browsers <em>already</em> contain a language runtime: JavaScript’s. It’s hard enough to maintain one runtime, let alone two. And then how do you integrate them?</p>
<p>WebAssembly, on the other hand, was designed to be a small extension to existing JavaScript virtual machines. While you <em>can</em> implement your own WebAssembly VM, and non-web use cases often do, for browsers, the maintenance cost is much, much lower.</p>
<h2>Other technologies were too specific</h2>
<p>WebAssembly is fundamentally language-agnostic. Flash and Java Applets were built first and foremost to run ActionScript and Java. They’re deeply tied to their relative semantics. Even PNaCl suffers from this to some degree; LLVM is really designed for C-like languages, though not quite the same amount.</p>
<p>Do we really want to christen one language as the next language of the web? We already have JavaScript. Are we someday going to introduce a third language? A fourth? An agnostic approach is significantly better for longevity’s sake.</p>
<h2>Wasm has a strong approach for safety</h2>
<p>Java Applets and Flash were security <em>nightmares</em>. Even if there were attempts to secure them, in reality, they constantly had problems.</p>
<p>WebAssembly, on the other hand, piggybacks on that JavaScript VM again; all of the effort going into creating its sandbox also applies to Wasm. Wasm is missing some features of regular assembly language that can cause security vulnerabilities, like stack smashing. Wasm is memory-safe, which is huge!</p>
<p>Additionally, WebAssembly was designed for <em>validation</em> in mind; its fully typed, and can be checked without running any of its code. It spec includes instructions on how to do validation. This stuff is quite useful!</p>
<h2>Conclusion</h2>
<p>One thing I’ll say about this post is that it’s quite <em>implementer</em> focused. But implementers are important; they control the web. Providing a solution that works with their goals is just as important as something that works for users. In a future post, I’d like to try to go over user concerns some as well. I have lots of posts to write!</p>
<p>At the end of the day:</p>
<ul>
<li>Other technologies were not integrated into the platform, and commercial interests didn’t point towards that happening</li>
<li>Other technologies required too much; too much to agree on, too much to implement</li>
<li>Wasm has a really strong sandboxing and verification story that others simply did not.</li>
</ul>
</content:encoded></item><item><title>WebAssembly is more than just the web</title><link>https://steveklabnik.com/writing/webassembly-is-more-than-just-the-web/</link><guid isPermaLink="true">https://steveklabnik.com/writing/webassembly-is-more-than-just-the-web/</guid><pubDate>Fri, 13 Jul 2018 00:00:00 GMT</pubDate><content:encoded><p>While WebAssembly was given its name for good reason, I also think it’s quite the misnomer. When you hear about WebAssembly, don’t just think of the web: think bigger.</p>
<p>Many programmers know that WebAssembly is a technology that lets you run non-JavaScript code in the browser. That is true, but there’s a lot more to it. You see, the spec is written in a very interesting way: there’s the core, what WebAssembly actually is. Here’s <a href="https://webassembly.github.io/spec/">the opening section</a>:</p>
<blockquote>
<p>To support the embedding of WebAssembly into different environments, its specification is split into layers that are specified in separate documents.</p>
</blockquote>
<blockquote>
<p>Core specification - Defines the semantics of WebAssembly modules and its instruction set, independent from a concrete embedding.API specifications - Define application programming interfaces enabling the use of WebAssembly modules in concrete embedding environments.</p>
</blockquote>
<p>It goes on to say:</p>
<blockquote>
<p>Currently, two APIs are specified: JavaScript API … Web API.</p>
</blockquote>
<p>So, WebAssembly itself, the core, is completely independent of the web. The web is a particular environment that Wasm is embedded <em>in</em>, but not actually required to use wasm. Other embedding APIs can be created, with their own interfaces, to do various things.</p>
<p>Let’s make this concrete: on the web, we can write a program that draws graphics using WebGL. But we could also write a desktop program that embeds a WebAssembly interpreter, and provides an OpenGL (or whatever else) API to render to the screen. And now our WebAssembly program is running on the desktop.</p>
<p>Why would you want to do this? Well:</p>
<blockquote>
<p>I wonder if wasm, in being sufficiently minimal, well defined, practical and widely supported, will wind up as an archival format of choice for emulation-based software preservation. https://t.co/33OUOrqiWg
— Graydon Hoare (@graydon_pub) July 11, 2018</p>
</blockquote>
<p>These emulators, with a teeny bit of work, could be made to run on the desktop as well. “Cross-platform” doesn’t just mean “Windows, Linux, and MacOS”, but can also mean running to the web as well.</p>
<p>There’s so many more things that can be done, too. All it requires is a bit of re-framing of WebAssembly in your mind:</p>
<p>WebAssembly is <em>not</em> a way to run code other than JavaScript on the web; WebAssembly is a tiny, embeddable language that you can use almost anywhere.</p>
<p>Some have compared WebAssembly to Java applets; in some ways, they’re very right, but in some ways, they’re very wrong. Eventually I’ll write a post about the wrong, but for now, the right: in some sense, WebAssembly is a different way of accomplishing what the JVM set out to do: it’s a common virtual machine that can be used to build very cross-platform software.</p>
<p>There’s a couple of other interesting applications of WebAssembly outside of the browser; one of my personal favorites is <a href="https://github.com/nebulet/nebulet">Nebulet</a>:</p>
<blockquote>
<p>A microkernel that implements a WebAssembly “usermode” that runs in Ring 0.</p>
</blockquote>
<p>I’d describe it like this:</p>
<blockquote>
<p>Nebulet is an operating system that runs WebAssembly programs.</p>
</blockquote>
<p>None of this has anything to do with a web browser; just like ELF is the standard binary format on many Linuxes, WebAssembly is the standard binary format on Nebulet. It’s a new implementation of a quite old idea, but I’m really excited to see where it goes.</p>
<p>A second interesting example is Parity. I’m a cryptocurrency skeptic, but from a tech perspective, this is quite cool. Parity is a company who is building a VM for the Ethereum cryptocurrency. Etherium’s VM has a programming language, Solidity, that lets you script their blockchain. Writing a programming language is a tough task on its own, and Solidity has had some issues, leading to pretty serious bugs. And those bugs are affecting people’s money.</p>
<p>So, rather than keep their custom language, Parity is experimenting with <a href="https://wiki.parity.io/WebAssembly-Design">embedding WebAssembly into the Ethereum VM instead</a>. This gets them a well-tested language that has pretty decent toolchain support, and lets them use a variety of other languages to do said scripting. This is very cool!</p>
<p>These are a few of the possibilities for WebAssembly, and I’m really excited about where it could go in the future.</p>
</content:encoded></item><item><title>"The Expressive C++17 Coding Challenge (in Rust)" revisited</title><link>https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust-revisited/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust-revisited/</guid><pubDate>Wed, 14 Feb 2018 00:00:00 GMT</pubDate><content:encoded><p>In October of last year, I wrote a post, <a href="http://words.steveklabnik.com/the-expressive-c-17-coding-challenge-in-rust">“The Expressive C++17 Coding Challenge (in Rust)”</a>. For various reasons, it got brought up again in the D world, and <a href="https://seb.wilzba.ch/b/2018/02/the-expressive-c17-coding-challenge-in-d/">seb has written a new post</a>. It’s good, you should check it out!</p>
<p>However, it links to my gist, not my blog post. As I said back then:</p>
<blockquote>
<p>I held myself to the same constraints as the original contest; no external packages is a bit painful in Rust, but it’s not too bad. Mostly it would let me eliminate boilerplate while also improving correctness, and making the code a bit shorter.</p>
</blockquote>
<p>So, that got me thinking: What <em>would</em> this look like if I could use external packages? I took about an hour, and knocked it out. I have two versions to show you today, one where I pay no attention to allocations, and one where it’s zero-allocation.</p>
<p>First, the “whatever just allocate” version:</p>
<pre><code>extern crate csv;
#[macro_use]
extern crate serde_derive;
extern crate serde;
#[macro_use]
extern crate structopt;
use std::error::Error;
use std::fs::File;
use structopt::StructOpt;
#[derive(Debug,Serialize,Deserialize)]
struct Record {
name: String,
surname: String,
city: String,
country: String,
}
#[derive(StructOpt, Debug)]
#[structopt(name = "basic")]
struct Opt {
filename: String,
column_name: String,
replacement: String,
output_filename: String,
}
fn run(args: &amp;Opt) -&gt; Result&lt;(), Box&lt;Error&gt;&gt; {
let input = File::open(&amp;args.filename)?;
let output = File::create(&amp;args.output_filename)?;
let mut rdr = csv::Reader::from_reader(input);
let mut wtr = csv::Writer::from_writer(output);
for result in rdr.deserialize() {
let mut record: Record = result?;
let replacement = args.replacement.clone();
match &amp;*args.column_name {
"name" =&gt; record.name = replacement,
"surname" =&gt; record.surname = replacement,
"city" =&gt; record.city = replacement,
"country" =&gt; record.country = replacement,
_ =&gt; panic!("incorrect column name"),
}
wtr.serialize(record)?;
}
wtr.flush()?;
Ok(())
}
fn main() {
let opt = Opt::from_args();
if let Err(err) = run(&amp;opt) {
eprintln!("error: {}", err);
std::process::exit(1);
}
}
</code></pre>
<p>This uses the <code>[csv</code> crate](https://crates.io/crates/csv/1.0.0-beta.5) to do the heavy lifting with regards to the CSV stuff, and the <code>[strucopt</code> crate](https://crates.io/crates/structopt) for argument parsing. Not only is this code much shorter than my “no crates” version, but it’s also got more features!</p>
<p>There’s a few things that aren’t the best here: first, the <code>match</code>. Rust doesn’t have reflection, exactly, so saying “hey I have this string and I want to index it in this struct” is not as nice. I’m just panic-ing here, but it’d be nicer to validate it up front, and possibly even do some custom derive shenanigans so that I don’t have to write it myself, but I wanted to present a fairly <em>realistic</em> version of what I’d write today. This is good enough.</p>
<p>Second, there’s that one <code>clone</code>. We need it because we’ve said that our <code>Record</code> type holds <code>String</code>s, and so has ownership over the string. We end up copying <code>London</code> each iteration of the loop. Is copying a city name a problem? Not <em>really</em>, in the context of this example. But what if it <em>was</em>? Fundamentally, there’s no reason to do any allocation here: We’re reading from a file, and we’re writing the same string over and over again.</p>
<p>Enter version 2:</p>
<pre><code>extern crate csv;
#[macro_use]
extern crate serde_derive;
extern crate serde;
#[macro_use]
extern crate structopt;
use std::error::Error;
use std::fs::File;
use structopt::StructOpt;
#[derive(Debug,Serialize,Deserialize)]
struct Record&lt;'a&gt; {
name: &amp;'a str,
surname: &amp;'a str,
city: &amp;'a str,
country: &amp;'a str,
}
#[derive(StructOpt, Debug)]
#[structopt(name = "basic")]
struct Opt {
filename: String,
column_name: String,
replacement: String,
output_filename: String,
}
fn run(args: &amp;Opt) -&gt; Result&lt;(), Box&lt;Error&gt;&gt; {
let input = File::open(&amp;args.filename)?;
let output = File::create(&amp;args.output_filename)?;
let mut rdr = csv::Reader::from_reader(input);
let mut wtr = csv::Writer::from_writer(output);
let mut raw_record = csv::StringRecord::new();
let headers = rdr.headers()?.clone();
while rdr.read_record(&amp;mut raw_record)? {
let mut record: Record = raw_record.deserialize(Some(&amp;headers))?;
match &amp;*args.column_name {
"name" =&gt; record.name = &amp;args.replacement,
"surname" =&gt; record.surname = &amp;args.replacement,
"city" =&gt; record.city = &amp;args.replacement,
"country" =&gt; record.country = &amp;args.replacement,
_ =&gt; panic!("incorrect column name"),
}
wtr.serialize(record)?;
}
wtr.flush()?;
Ok(())
}
fn main() {
let opt = Opt::from_args();
if let Err(err) = run(&amp;opt) {
eprintln!("error: {}", err);
std::process::exit(1);
}
}
</code></pre>
<p>This is <em>slightly</em> more involved, as you no longer get access to the nice iterator, and have to do things a little more manually. But it took me about five minutes to port this bit over, so it’s not <em>too</em> onerous.</p>
<p>So, as you can see, once you let people use external packages, the Rust gets <em>significantly</em> nicer. The code pretty much mirrors the task: open the two files, iterate over them, replace the column, write it out. Nice.</p>
<p>I’m not going to do any performance comparisons here because that’s not really what the contest was about. I am mildly curious… if I really wanted to optimize performance, I’d also skip the string stuff and go for bytes directly. This isn’t a super tough transformation, but moves things to be slightly lower level.</p>
<p>If you manipulate CSVs often, you should check out <code>[xsv](https://crates.io/crates/xsv)</code>, a command-line tool in Rust that lets you slice and dice these files with speed and ease.</p>
<hr />
<p>I should also mention that several people have been working on <code>quicli</code>, a package to make making CLIs super easy. One of its authors <a href="https://gist.github.com/killercup/049d759118e3b5029737eb77e157ea42">posted a gist of what this looks like with quicli</a>, you should check that out too!</p>
</content:encoded></item><item><title>0b100000</title><link>https://steveklabnik.com/writing/2-sup-5/</link><guid isPermaLink="true">https://steveklabnik.com/writing/2-sup-5/</guid><pubDate>Thu, 25 Jan 2018 00:00:00 GMT</pubDate><content:encoded><p>As someone who has <a href="https://www.inc.com/jeff-bercovici/peter-thiel-young-blood.html">significantly different beliefs than Peter Thiel</a>, today is likely my second-to-last power of two years alive.</p>
<p>Two years ago, <a href="http://words.steveklabnik.com/a-eulogy-for-my-20s">I wrote a post about turning 30</a>, and I pretty much feel the same way. Never enough time, tick tock, tick tock. I have today off (thanks Mozilla!), but am still considering working anyway, because I really care about what I’m working on, and am excited about it. We’ll see.</p>
<p>I feel like I have more to say here, but also don’t. So maybe let’s leave it at that.</p>
</content:encoded></item><item><title>Looking back at Rust in 2018</title><link>https://steveklabnik.com/writing/looking-back-at-rust-in-2018/</link><guid isPermaLink="true">https://steveklabnik.com/writing/looking-back-at-rust-in-2018/</guid><pubDate>Tue, 09 Jan 2018 00:00:00 GMT</pubDate><content:encoded><blockquote>
<p>A careful reader will note the date of this post; it’s fundamentally in response to this call for blog posts, with a rhetorical device of looking backwards as if my personal plans for Rust came true. If you’re reading this in 2019 or beyond, this may be a work of fiction.</p>
</blockquote>
<p>It’s been an exciting year for Rust. Well, it depends on what you mean by “exciting”, exactly. In a word, this year has been about <em>polish</em>. I find this kind of thing exciting! It’s a very different kind of exciting than the “new and shiny” exciting. It’s “take what we have and make it shiny.”</p>
<p>Let’s recap the major achievements of the year.</p>
<h2>Rust 2018</h2>
<p>First of all, we shipped our first epoch! If you type <code>cargo new</code> today, you’ll see a new shiny entry in your <code>Cargo.toml</code>:</p>
<pre><code>[package]
name = "new-program"
version = "0.1.0"
authors = ["steveklabnik &lt;[email protected]&gt;"]
epoch = "2018"
[dependencies]
</code></pre>
<p>What does this mean in practical terms? Well, a few things. First, let’s get the big one out of the way: Try to compile this code with no epoch, or with <code>epoch = "2015"</code>:</p>
<pre><code>fn main() { let catch = 5;}
</code></pre>
<p>You’ll see this:</p>
<pre><code>&gt; cargo build
Compiling new-program v0.1.0 (file:///C:/Users/steve/tmp/new-program)
warning: `catch` will become a keyword in the 2018 epoch
--&gt; src\main.rs:2:9
|
2 | let catch = 5;
| ^^^^^
|
= note: #[warn(new_keyword)] on by default
= note: to avoid this warning, consider using `_catch` instead
</code></pre>
<p>And if you change it to <code>epoch = "2018"</code>, you’ll see this:</p>
<pre><code>&gt; cargo build
Compiling new-program v0.1.0 (file:///C:/Users/steve/tmp/new-program)
error: expected pattern, found keyword `catch`
--&gt; src\main.rs:2:9
|
2 | let catch = 5;
| ^^^^^
error: aborting due to previous error
</code></pre>
<p>Success! However, the specifics here aren’t as interesting as what this change means:</p>
<p>First, <em>this is one of only two changes that relies on the epoch</em>. When we announced our plans for epochs, many people were skeptical that we would do as we said, and only included small, parser-level changes here. But we did, and will continue to do so. <code>catch</code> and <code>dyn Trait</code> are the only two changes here.</p>
<p>Second, <em>we’ve demonstrated that this doesn’t split the ecosystem</em>. All of the crates that had no epoch are treated as 2015, and everything still works. Mixing 2015 and 2018 code works just fine.</p>
<p>Finally, <em>mrustc has not had trouble implementing this split</em>. <code>[mrustc</code> is an alternative implementation of Rust](https://github.com/thepowersgang/mrustc), and some of the worries about epochs would be that they would unnecessarily burden alternative implementations. This fear has not borne fruit yet though.</p>
<h2>We did ship <em>some</em> new stuff</h2>
<p>Even though this year was about polish, that doesn’t mean that we had no new features. The key is, these features were all <em>designed</em> in 2017 or before. That is, features like non-lexical lifetimes, <code>impl Trait</code>, and Macros 2.0 didn’t land in stable until 2018, but they’ve been worked on for years. At the end of 2017, we had preliminary implementations already landed, it was just about polishing them up so we could ship.</p>
<p>Cargo, however, did gain some new abilities. Cargo was sort of neglected in 2017, or at least, wasn’t worked on as much as the rest of the project was. Cargo grew the ability to use alternative registries, and at least one company is offering “crates.io as a service” and “crates.io behind your firewall”. Cargo also became easier to use offline, and the “integrate Cargo into other build systems” work bore some more fruit.</p>
<h2>WebAssembly and Embedded platforms</h2>
<p>At the end of 2017, Rust was poised to take advantage of WebAssembly. And take advantage, we did. WebAssembly has not taken over the world yet, but that was also never its intention. It is a common aspect of the Web Platform these days, and Rust’s support for it is so excellent that it’s the primary language with which you write new code for WebAssembly today. Tons of JavaScript programmers use Rust for their various systems-level tasks on the web, and we’re just starting to see completely new kinds of applications form. Adding Rust to your Node or front-end project is trivial, thanks to excellent integration from JavaScript’s build tools, or as we now call them, “compilers”.</p>
<blockquote>
<p>An aside: in the 2014-2017 era, even though JavaScript was JIT compiled, many programmers used various “build tools” or “transpilers” to do things like minify and bundle their JavaScript together. This was useful for things like “let me use new ES2017 features today but transpile to ES5.” In 2017, lots of JavaScript programmers, through tools like Rollup, started to appreciate that what they’re doing, on a fundamental level, is compiling. Once these tools realized this, they started taking advantage of decades of existing compiler research, and these tools started doing amazing things. Oddly enough, this makes JavaScript feel more like Java: you compile to a thing that ends up being JIT-ed. And that’s not even half of it; stuff like Ember’s GlimmerVM took this approach to new heights.</p>
</blockquote>
<p>Furthermore, it’s considered standard practice to test your crate against the WebAssembly target, just like you would Windows, Linux, and macOS, so the vast majority of the crates.io ecosystem Just Works.</p>
<p>But this work on wasm has a good side effect: wasm is basically a very minimal embedded platform. By doing the legwork to make wasm awesome, we also made embedded platforms of various kinds awesome. There’s also specific work that had to be done, for example, remember <code>xargo</code>? Cargo just does that now. We’ve elevated ARM to a Tier 1 platform these days, and provide easy instructions for getting a new architecture going.</p>
<p>This also implies stabilizing some of the lower-level things that are needed to do OSdev work. You can’t write a “real” OS in stable Rust yet, but you can write toy ones, and real OSes should be usable by next year.</p>
<h2>Stabilization</h2>
<p>One way in which we did polish was to ship a ton of stuff that was previously in flight. I mentioned non-lexical lifetimes earlier, as well as some other features, but there’s also a sort of “long tail” of stabilizations that we addressed.</p>
<p>At the start of the year, the teams went through each of their feature gates, and voted: stabilize, put on hold, or remove. Things put on hold this year will get decided next year, but cannot be put on hold for another year. Of course, there were some small course-corrections over the year, but in general, it let us go “oh hey, slice patterns has been unstable since before Rust 1.0, let’s finally ship it.”</p>
<h2>Bugs &amp; new contributors</h2>
<p>Rust’s bugtracker was in a much better state in 2017 than in 2016, but got even better in 2018. 2017 saw the infrastructure team actually ensure that all bugs get triaged, that is, they all have tags within two days of being opened. In 2018, however, there was a focus to knock out a lot of the long-standing, papercut style bugs. At the start of 2018, there were ~3100 bugs on the tracker, but today, there are only ~1500. That’s four bugs per day! We couldn’t have gotten there without all of the new contributors and team members we’ve added this year; in 2017, we had about 75 people on our various teams, and now we have 100. Your average Rust release in 2017 had about 120 contributors, it’s 180 for 2018.</p>
<h2>Community and Job growth</h2>
<p>It’s not just the teams that have grown, but also the community. For example, in 2016, crates.io had ~7500 crates. In 2017, ~13,000. In 2018, we’re up to 35,000. That’s a lot of packages! One major driver of this growth is jobs; once <strong>REDACTED</strong> announced that they’d been using Rust in a production setting, and that compared to the old system written in <strong>REDACTED</strong>, it saved them millions of dollars in server costs per year, a ton of companies started using Rust. It’s not <em>super</em> easy to get a Rust job, but they’re out there. We’ve also seen our meetups continue to grow, RustConf had two tracks this year, and there’s a new RustFest in South America, with rumors of a conference in China next year.</p>
<h2>IDEs</h2>
<p>While I don’t really use IDEs, we put a lot of work into making them awesome, and 2018 was the first time that people who <em>do</em> use IDEs love using one. Specifically, they love using the new IDE that JetBrains has released for Rust. In 2017, JetBrains was sponsoring the plugin for IntelliJ, but in 2018, they’ve released a whole product. Other IDEs are coming along swimmingly; Visual Studio now has really excellent support, for example.</p>
<h2>Documentation</h2>
<p>I work on docs, so I’m going to give this a section, even though maybe other things are more “exciting”.</p>
<p>2018 saw the release of $REDACTED, a sort of “next-generation <code>rustdoc</code>.” It focused on not just API docs, but all of your docs holistically; you can publish guides as well, structure things how you’d like, etc. In some ways, it’s “<code>rustdoc</code> + <code>mdbook</code>” in a nice integrated package. This has led to even more people writing even better docs, since it’s so nice to use.</p>
<p>There were two other major initiatives this year: focusing on intermediate documentation, and helping major ecosystem crates get excellent docs. In 2017, there was sort of a “what do I do after TRPL” situation, but now we’ve got some great resources to help you grow as a Rust programmer. Additionally, many of the more popular crates have docs that rival the standard library’s, and smaller crates have decent docs.</p>
<p>Oh, and TRPL saw its initial paper release early in the year. Yay!</p>
<h2>Epilogue</h2>
<p>Whew, that’s a lot of stuff! What does 2019 hold for Rust? That post is for another time.</p>
</content:encoded></item><item><title>Five years with Rust</title><link>https://steveklabnik.com/writing/five-years-with-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/five-years-with-rust/</guid><pubDate>Thu, 21 Dec 2017 00:00:00 GMT</pubDate><content:encoded><p>Today is five years since I first heard of Rust. <a href="http://words.steveklabnik.com/four-years-with-rust">I wrote a post last year</a> about it. This year, I have something slightly different in mind.</p>
<p>Five years is a long time in tech. It’s also a long time for me to be focused on one specific thing; other than the pizza job I held in high school and college, I’ve never stayed in a single job for five years. (This is true today as well; it’s not like I got a job with Rust the first day I learned of its existence.) I started with Ruby and Rails in 2008/2009, and my final commits to Rails were in August/September of 2013. So at this point, my career has basically been split in half, rather than dominated by Ruby.</p>
<p>Five years ago tomorrow, I registered <a href="http://www.rustforrubyists.com/">http://www.rustforrubyists.com/</a>.</p>
<p>Yesterday, I had lunch with someone I’m very honored to call a friend. We talked about a lot of things, but also about Rust’s past, present, and future. One thing I’ve always felt a little awkward about is that there was a crucial point back in 2011/2012 where I wasn’t sure what to do with myself. I was very frustrated by the industry, and considered doing other things instead. But, as they and I talked about, sometimes, it’s not about your interests. If what you want to do is change the world, caring a lot isn’t enough; you have to analyze what you can actually <em>do</em>. While I think our industry is pretty broken, I at least have the ability to try and improve it, thanks to my stature. (God, even just typing that makes me feel gross.)</p>
<p>I decided to put my time into Rust because I believed, and still believe, that it’s an important thing. And I saw the work that needed to be done, and saw that it’s the kind of work that I could do. And so, it’s maybe not something bigger, but I resolved myself to doing the work.</p>
<p>Five years in, I think my journey has been successful so far. I’m proud of how far Rust has come. That said, there’s a lot more work to do. I’m still committed as ever to getting that work done, and feel proud to be able to work alongside people as amazing as the Rust community.</p>
<p>Here’s to five more years. ?</p>
</content:encoded></item><item><title>User-Agent: moz://a</title><link>https://steveklabnik.com/writing/user-agent-moz-a/</link><guid isPermaLink="true">https://steveklabnik.com/writing/user-agent-moz-a/</guid><pubDate>Tue, 12 Dec 2017 00:00:00 GMT</pubDate><content:encoded><p>Every six months, Mozilla gets all of its employees from around the world together in one place, for a week. This morning was the plenary session, with Mozilla leadership talking about our successes this year, as well as plans for the future. I noticed a theme that, while not super overt, I find very interesting, and bodes well for Mozilla’s future.</p>
<p>The words “user agent” kept coming up in various aspects of the plenary. The <code>User-Agent</code> header has a… shall we say <a href="https://webaim.org/blog/user-agent-string-history/">storied history</a>. When most people hear the term “user agent”, they mostly think of the “<a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent/Firefox">user agent string</a>”. This is passed through the <code>User-Agent</code> header to identify which browser you’re using.</p>
<p>Well, that’s how most people think of it, but that’s not the whole story. If it is supposed to identify which browser you’re using, why is it not the <code>Browser</code> header? What is this “user agent” term anyway?</p>
<p>The concept is a bit more broad: a “user agent” is a bit of software that acts on behalf of its user. A web browser is a “user agent” for the web. But an email program is also a “user agent” for email, for example. So “user agent” and “web browser” are not synonymous; a web browser is a specific form of user agent.</p>
<p>Words have power, however. Framing matters. The way you describe a problem affects the way you think about the solutions. So the shift in language I picked up on was, Mozilla is not building a web browser, Mozilla is building a user agent. So what’s the difference here?</p>
<p>A web browser is a dumb terminal. A user agent advocates and works for you, to better your experience.</p>
<p>There’s a lot more to say about what all this framing implies: for security, for privacy, for customization, for openness, for advocacy, for social impact. But I have some meetings to go to, so I’ll just leave you with a question:</p>
<p>What if your browser worked <em>for</em> you, rather than <em>against</em> you?</p>
<p>In my mind, this “user agent” framing is a very useful and interesting way to think about Mozilla’s mission, and the goals we pursue in the future.</p>
</content:encoded></item><item><title>"The Expressive C++17 Coding Challenge" in Rust</title><link>https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust/</guid><pubDate>Wed, 25 Oct 2017 00:00:00 GMT</pubDate><content:encoded><p>A few weeks ago, two C++ bloggers initiated a challenge (<a href="https://www.fluentcpp.com/2017/10/23/results-expressive-cpp17-coding-challenge/">here</a> and <a href="http://www.bfilipek.com/2017/09/the-expressive-cpp17-challenge.html">here</a>): use the shiny new features of C++17 to write a small program that manipulates CSV files. The link is to the results; you can see the examples people came up with.</p>
<hr />
<p>I’ve written <a href="http://words.steveklabnik.com/the-expressive-c-17-coding-challenge-in-rust-revisited">a follow-up post</a> too!</p>
<hr />
<p>I decided to implement it in Rust for fun. You should give it a try, in whatever language you use, it’s a nice sample problem. I held myself to the same constraints as the original contest; no external packages is a bit painful in Rust, but it’s not too bad. Mostly it would let me eliminate boilerplate while also improving correctness, and making the code a bit shorter.</p>
<p>Anyway, here it is:</p>
<p>https://gist.github.com/steveklabnik/ad0a33acc82e21ca3f763e4278ad31a5</p>
<p>I implemented <code>From</code> for everything so I could use <code>?</code>, but then <code>main</code> just has a ton of <code>unwrap</code>s. I normally would remove those too, but the challenge does not include good error handling, so I decided to just stick to the text of the challenge.</p>
<p>I’m not going to say that this is the <em>best</em> Rust code in the world, nor compare it directly to the C++ examples. But it was a fun hour or two to knock it out, and figured I’d at least share my results.</p>
</content:encoded></item><item><title>Want to join the Rust docs team?</title><link>https://steveklabnik.com/writing/want-to-join-the-rust-docs-team/</link><guid isPermaLink="true">https://steveklabnik.com/writing/want-to-join-the-rust-docs-team/</guid><pubDate>Fri, 24 Mar 2017 00:00:00 GMT</pubDate><content:encoded><p>One of Rust’s major goals this year is to <a href="https://github.com/rust-lang/rust-roadmap/issues/13">provide mentoring at all levels of the project</a>. This is a goal I’m very on-board with. There’s lots of ways to get involved with Rust, but I’d like to talk about the bit I’m most involved in: documentation.</p>
<p>There’s many ways to contribute to Rust’s docs. Some people contribute occasional PRs, some people end up contributing a lot. No matter what your interest, I’d like to help. But before I get into those details, <em>why</em> should someone get involved with documentation? I have my own reasons, but I also have a pitch for you:</p>
<blockquote>
<p>Working on documentation is a fantastic way to learn something in a deeper way.</p>
</blockquote>
<p>In order to document something, you have to understand it, even if only a little bit! It’s a good excuse to dig into the details. And I’ve found that many people are eager to help explain something to you in return for you writing that down and synthesizing it into documentation.</p>
<p>If that sounds intriguing to you, let’s talk! There’s a number of different ways to do this, and I’m interested generally in helping anyone work on Rust’s documentation, in any form. However, if you don’t have something in mind, I do have a suggestion.</p>
<p>Back in October of 2015, I made a <a href="https://github.com/rust-lang/rust/issues/29329">big GitHub issue</a> to track giving the whole standard library documentation a once-over. I then made sub-issues for each main module. Slowly, myself and others have been knocking them out, but there’s still a number of issues open.</p>
<p>Today, I went through all of the remaining ones, and made fine-grained bullet points of what needs to be done. I’ll admit that some of them may not be as explicit as they could be, but it at least gives a starting point to talk about. Here’s all the sections that are currently open:</p>
<ul>
<li><a href="https://github.com/rust-lang/rust/issues/29349">convert</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29354">ffi</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29381">macros</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29378">thread</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29377">sync</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29375">str</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29371">ptr</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29370">process</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29367">os</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29365">ops</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29363">net</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29357">hash</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/29355">fmt</a></li>
</ul>
<p>Some of the work that needs to be done is very small: add a link here, re-word a sentence there. Other bits are large; “this whole thing needs re-written.”</p>
<p>So here’s my pitch to you: have you ever wanted to learn a bit more about Rust? Do any of those names sound interesting to you? Maybe you’ve wanted to learn a bit more about threads in Rust. Check out that ‘thread’ issue above, poke around, learn some Rust, and then let’s make the docs better! We can work together on the issues, through email, or on IRC. Whatever works. Maybe you’ll find it as addicting as I do. And I find that people who are non-experts write way better docs than experts. So don’t worry about that!</p>
<p>Worst case, you’ll have learned a bit of the standard library a bit better, and that’s always a good thing, too.</p>
</content:encoded></item><item><title>An overview of macros in Rust</title><link>https://steveklabnik.com/writing/an-overview-of-macros-in-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/an-overview-of-macros-in-rust/</guid><pubDate>Tue, 24 Jan 2017 00:00:00 GMT</pubDate><content:encoded><p>Rust has had a lot of stuff going on with “macros”. There are a few, interconnected aspects of these features, many different names for the same thing, and a lot of development work going on. This post will give an overview of the past, present, and future of macros in Rust.</p>
<p>Ultimately, macros allow you to write things like this:</p>
<pre><code>let map = hashmap!{
"a" =&gt; 1,
"b" =&gt; 2,
};
</code></pre>
<p>or</p>
<pre><code>#[derive(Serialize, Deserialize)]
struct S {
#[serde(rename = "a")]
f: i32,
}
</code></pre>
<p>That is, some macros look like functions: <code>foo!</code>, but allow you to write nearly arbitrary syntax inside of the <code>()</code>s. Some let you control the ability to write custom attributes. Both generate some other kind of valid Rust code for the compiler to interpret. In the first case, the <code>hashmap!</code> macro will expand to a call to <code>HashMap::new</code> and a bunch of <code>insert</code> calls, and the second case, the <code>Serialize</code> and <code>Deserialize</code> traits will be automatically implemented for this struct, along with a bit of custom logic where the field <code>f</code> maps to the name <code>a</code> in the data that’s being {de,}serialized.</p>
<p>These features are extremely powerful, but also must be done right: metaprogramming is very useful, but can also make code inscrutable, or give poor error messages.</p>
<p>Oh, and I’d like to give a shout-out to <a href="https://danielkeep.github.io/tlborm/book/index.html">The Little Book of Rust Macros</a>, which is basically <em>the</em> resource for doing this kind of work. If you are interested in writing macros, you should check it out.</p>
<h2>Pre-1.0</h2>
<p>As Rust developed, two major forms of macros evolved. One was called “macros” and the other was called “compiler plugins”. Both of these systems were pretty good, but both of them also had a lot of problems. We’ll get into the details of each of these kinds of macros below, but there’s one thing we should talk about first. On the way to 1.0, it was decided to stabilize one kind of macros, but not the other. From <a href="https://blog.rust-lang.org/2014/10/30/Stability.html">Stability as a Deliverable</a>: (also, this uses some terms that we will cover shortly)</p>
<blockquote>
<p>After extensive discussion, we plan to release globs and macros as stable at 1.0. For globs, we believe we can address problems in a backwards-compatible way. For macros, we will likely provide an alternative way to define macros (with better hygiene) at some later date, and will incrementally improve the “macro rules” feature until then. The 1.0 release will stabilize all current macro support, including import/export.On the other hand, we cannot stabilize syntax extensions, which are plugins with complete access to compiler internals. Stabilizing it would effectively forever freeze the internals of the compiler; we need to design a more deliberate interface between extensions and the compiler. So syntax extensions will remain behind a feature gate for 1.0.Many major uses of syntax extensions could be replaced with traditional code generation, and the Cargo tool will soon be growing specific support for this use case. We plan to work with library authors to help them migrate away from syntax extensions prior to 1.0. Because many syntax extensions don’t fit this model, we also see stabilizing syntax extensions as an immediate priority after the 1.0 release.</p>
</blockquote>
<p>We’ll discuss in each section how this ended up working out, but it’s important to remember that only <em>one</em> form of these macros was available on stable Rust for 1.0, and in fact, as of the time of this writing. That’s going to change soon…</p>
<p>It’s also important to note that we weren’t entirely happy with macros. They were fine, but they have some flaws. We wanted to fix them, but we also didn’t want to ship 1.0 without macros. Language design is hard. I’ll talk about this more in the appropriate section.</p>
<h2>Declarative Macros</h2>
<p>The first form of Macros in Rust, and the one that’s most widely used, are called “declarative macros”, sometimes “macros by example”, sometimes “macro_rules macros”, or sometimes just “macros”. At their core, declarative macros allow you to write something similar to a Rust <code>match</code> statement:</p>
<pre><code>match x {
4 =&gt; println!("four!"),
5 =&gt; println!("five!"),
_ =&gt; println!("something else"),
}
</code></pre>
<p>With <code>match</code>, <code>x</code>’s structure and value will be evaluated, and the right arm will execute based on what matches. So if <code>x</code> is five, the second arm happens. Etc.</p>
<p>These kinds of macros work in the same way: you set up some sort of pattern, and then, if that pattern matches, some code is generated. One important difference here is that in <code>match</code>, <code>x</code> gets evaluated. With macros, <code>x</code> does not get evaluated.</p>
<p>There are two kinds of declarative macros: ones defined with the <code>macro_rules!</code> keyword, and ones defined with the <code>macro</code> keyword.</p>
<h3><code>macro_rules!</code></h3>
<p>These are most commonly known as “macros” in Rust world, and are the only thing that’s stable in Rust at the time of this writing. Here’s a macro you’ll see when using Rust:</p>
<pre><code>let x: Vec&lt;u32&gt; = vec![1, 2, 3];
</code></pre>
<p>This macro creates a new vector, with three elements inside. Here’s what the macro could look like, written with <code>macro_rules!</code>:</p>
<pre><code>macro_rules! vec {
( $( $x:expr ),* ) =&gt; {
{
let mut temp_vec = Vec::new();
$(
temp_vec.push($x);
)*
temp_vec
}
};
}
</code></pre>
<p>Whew! That’s a bunch of stuff. The most important line is here:</p>
<pre><code> ( $( $x:expr ),* ) =&gt; {
</code></pre>
<p>This <code>pattern =&gt; block</code> looks similar to the <code>match</code> statement above. If this pattern matches, then the block of code will be emitted. Given that this is the only pattern in this macro, there’s only one valid way to match; any other will be an error. More complex macros will have more than one rule.</p>
<p>The <code>$x:expr</code> part matches an expression, and gives it the name <code>$x</code>. The <code>$(),*</code> part matches zero or more of these expressions. Then, in the body of the macro, the <code>$()*</code> part is generated for each part that matches, and the <code>$x</code> within is replaced with each expression that was matched.</p>
<p>These macros are fine, but there’s a number of bugs and rough edges. For example, there’s no namespacing: if a macro exists, it’s everywhere. In order to prevent name clashes, this means that you have to explicitly import the macros when using a crate:</p>
<pre><code>#[macro_use]
extern crate serde;
</code></pre>
<p>Otherwise, you couldn’t import two crates that had the same macro name. In practice this conflict doesn’t come up much, but as we grow, it becomes more likely. The hygiene of <code>macro_rules</code> is there, but not perfect. (Only local variables and labels are hygienic…) Et cetera. The plan is that with the new macro systems, you’ll be able to use the regular module system, and use macros just like you use functions today.</p>
<p>Even with these problems, macros were useful enough to include in Rust 1.0. But plans for a better macro system were already being worked on, and so, they were given the name <code>macro_rules!</code> so that we could eventually deprecate all of this. This is one of the few features that we decided to do this with Rust 1.0; a vestigial thing that was always planned for obsolescence. Doing it hurts, but not having any metaprogramming at all would hurt much more.</p>
<h3><code>macro</code></h3>
<p>As such, the <code>macro</code> keyword was reserved for a new macro system. We’re still not 100% sure if we’re calling these “macros by example” (which you’ll sometimes see with a “2.0” at the end) or “declarative macros”. I prefer the latter, because it pairs nicely with “procedural macros.” It seems like most agree with me. We’ll see.</p>
<p>These macros are an incremental improvement over the existing <code>macro_rules!</code> macros: they’re fundamentally the same thing, but with some nicer features.</p>
<p>At the time of this writing, <a href="https://github.com/rust-lang/rfcs/pull/1584">an initial RFC is in FCP</a>, laying out some high-level aspects of this feature. An initial implementation is underway, but it hasn’t landed yet, so you can’t use <code>macro</code> in real code yet.</p>
<h2>Procedural Macros</h2>
<p>In opposition to the pattern-based declarative macros, the second form are called “procedural macros” because they’re functions: they accept some Rust code as an input, and produce some Rust code as an output. I say “code” but I don’t mean that literally; we’ll get into the exact details in each section.</p>
<h3>Syntax Extensions</h3>
<p>Syntax extensions, also sometimes just called “procedural macros” or “compiler plugins”, were not stabilized for Rust 1.0. Here’s (part of) a syntax extension that implements roman numeral literals:</p>
<pre><code>#![feature(plugin_registrar, rustc_private)]
extern crate syntax;
extern crate rustc;
extern crate rustc_plugin;
use syntax::parse::token;
use syntax::tokenstream::TokenTree;
use syntax::ext::base::ExtCtxt;
use rustc_plugin::Registry;
fn expand_rn(cx: &amp;mut ExtCtxt, sp: Span, args: &amp;[TokenTree])
-&gt; Box&lt;MacResult + 'static&gt; {
// implementation goes here
}
#[plugin_registrar]
pub fn plugin_registrar(reg: &amp;mut Registry) {
reg.register_macro("rn", expand_rn);
}
</code></pre>
<p>You’d use it like this:</p>
<pre><code>#![feature(plugin)]
#![plugin(roman_numerals)]
fn main() {
assert_eq!(rn!(MMXV), 2015);
}
</code></pre>
<p>That is… a lot of code. And frankly, probably doesn’t even work on today’s Rust right now. Why not? Well, that’s the big problem with syntax extensions:</p>
<pre><code>extern crate syntax;
extern crate rustc;
extern crate rustc_plugin;
</code></pre>
<p>This literally loads up bits of the Rust compiler and uses it as a library. Then, it uses Rust’s internal AST to do the processing. This is extremely powerful, but also extremely unstable. We don’t and can’t freeze Rust’s internal AST, or any of its other internal APIs, at least not at this time. Not to mention that this means any alternative Rust compiler would need to either use our AST or map ours to theirs, which puts a damper on alternative Rust implementations. This means that syntax extensions haven’t had a good path towards stability. It’s also why they couldn’t be made stable as-of Rust 1.0. But “run arbitrary Rust code to produce more Rust code” is a useful feature, and so, some projects needed them. In addition, while declarative macros can produce function-style <code>foo!</code> calls, they cannot do the custom attribute stuff. And those are very useful. We <em>could</em> make this work, but there hasn’t been a real reason to, nor much demand.</p>
<p>A side note: there’s also the term “compiler plugin.” Syntax extensions are a particular kind of compiler plugin; there are others, like lint passes, LLVM pass plugins, MIR plugins… we’re only talking about syntax extensions in this post, however.</p>
<p>The current plan is to never stabilize syntax extensions, and instead, stabilize “procedural macros.” Again, this isn’t for all compiler plugins, only syntax extensions.</p>
<h3>Procedural Macros</h3>
<p>Accepted in <a href="https://github.com/rust-lang/rfcs/blob/master/text/1566-proc-macros.md">RFC 1566</a>, procedural macros will give you the same power as compiler plugins, without needing to reach into the guts of <code>rustc</code>. How? Well, they look like this:</p>
<pre><code>#[proc_macro]
pub fn foo(TokenStream) -&gt; TokenStream {
// do stuff to implement the foo macro
}
</code></pre>
<p>This creates a <code>foo!</code> procedural macro. You’ll note that it accepts a <code>TokenStream</code> and produces a <code>TokenStream</code>. To quote the RFC:</p>
<blockquote>
<p>The first argument is the tokens between the delimiters in the macro use. For example in foo!(a, b, c), the first argument would be [Ident(a), Comma, Ident(b), Comma, Ident(c)].The value returned replaces the macro use.</p>
</blockquote>
<p>Why tokens? Well, while an AST can change, the actual tokens will not. This approach will allow us the flexibility to change <code>rustc</code>’s internal AST, while giving us the full power of compiler plugins.</p>
<p>So, how do you work with one of these kinds of macros? You can of course work with the token stream directly, but we also expect to see libraries that will provide easier-to-use interfaces on top of this. You could imagine a package that converts a <code>TokenStream</code> to its own AST, and then provides an interface to that AST. This is nicer to use than working with tokens directly. There are already some libraries that do similar kinds of things, like <a href="https://github.com/dtolnay/syn">syn</a>. More on that in a moment.</p>
<h3>Function-like vs attribute like</h3>
<p>The above is for a macro that gives you a <code>foo!</code> macro, that is, one that looks like a function. What about one that gives you an attribute? That’d look like this:</p>
<pre><code>#[proc_macro_attribute]
pub fn foo(Option&lt;TokenStream&gt;, TokenStream) -&gt; TokenStream {
// implementation goes here
}
</code></pre>
<p>To quote the RFC:</p>
<blockquote>
<p>The first argument is a list of the tokens between the delimiters in the macro use. Examples:#[foo] =&gt; None#[foo()] =&gt; Some([])#[foo(a, b, c)] =&gt; Some([Ident(a), Comma, Ident(b), Comma, Ident(c)])The second argument is the tokens for the AST node the attribute is placed on.</p>
</blockquote>
<p>Internally, it’s the same thing: use a <code>TokenStream</code>, produce a <code>TokenStream</code>.</p>
<h3>“Macros 1.1”</h3>
<p>You may have also heard a lot of commotion about “Macros 1.1.” In fact, they will be stable as of the next release of Rust, 1.15. Macros 1.1 are the <a href="https://en.wikipedia.org/wiki/Pareto_principle">Pareto principle</a> applied to procedural macros: as it turns out, the vast majority of people who use procedural macros today use it for “custom derive”, that is, the ability to say <code>#[derive(MyTrait)]</code> and implement a trait automatically. As such, a plan was hatched: we could stabilize <em>just the very very very basics</em> of procedural macros, enough to get the ability to write custom derives, and stabilize that, without needing to implement everything. What does that look like? Here’s one that implements <code>#[derive(Double)]</code></p>
<pre><code>extern crate proc_macro;
use proc_macro::TokenStream;
#[proc_macro_derive(Double)]
pub fn double(input: TokenStream) -&gt; TokenStream {
let source = input.to_string();
// Parse `source` for struct/enum declaration, and then build up some new
// source code representing a number of items in the implementation of
// the `Double` trait for the struct/enum in question.
let source = derive_double(&amp;source);
// Parse this back to a token stream and return it
source.parse().unwrap()
}
</code></pre>
<p>You’ll notice that this still looks like <code>TokenStream</code> -&gt; <code>TokenStream</code>, but there’s something more subtle here: <code>input.to_string()</code>. That is, for Macros 1.1, <code>TokenStream</code> has two methods; all it has is the ability to be converted to a string, with the original Rust code inside of it, and a <code>parse</code> method, which can take a string of Rust code and produce another <code>TokenStream</code>. What you do with that <code>String</code> is your own business.</p>
<p>What most people will do is use <a href="https://github.com/dtolnay/syn">syn</a> to parse the Rust code, and <a href="https://github.com/dtolnay/quote">quote</a> to generate Rust code from that. These libraries will make it feasible to build real-world custom derive code, without us needing to stabilize anything we don’t want to. Of course, once full procedural macros are implemented, it’s not likely that people will want to turn <code>TokenStream</code>s into and parse <code>TokenStream</code>s from strings, but it’s not <em>totally</em> unheard of, and will be the tiniest wart. It’s worth it to move a significant chunk of the ecosystem onto the stable compiler.</p>
<p>For more details, see <a href="https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md">RFC 1681</a>.</p>
<h3>“Macros 1.2” ?</h3>
<p>There’s also been some small talk about possibly doing a “macros 1.2”, that is, another step towards full procedural macros without needing the whole thing. This would be “procedural macros without hygiene”, though it’s still early days for this idea; there isn’t even an RFC yet. We’ll see if this happens. There’s still some controversy around this idea.</p>
</content:encoded></item><item><title>Announcing Rust Contributors</title><link>https://steveklabnik.com/writing/announcing-rust-contributors/</link><guid isPermaLink="true">https://steveklabnik.com/writing/announcing-rust-contributors/</guid><pubDate>Thu, 12 Jan 2017 00:00:00 GMT</pubDate><content:encoded><p>When I was working with Ruby on Rails, someone once sent me this link:</p>
<p>http://contributors.rubyonrails.org/</p>
<p>In some ways, this link changed my life. You see, I respond <em>really</em> strongly to integers going upwards. I am a sucker for points, hits, whatever. That web page, however, puts that integer towards something useful: open source contributions. It’s fairly well known that open source contribution is a long tail, but that’s also <em>perfect</em> for this kind of gamification. One contribution, as of today, will put you in 2461th place. Your second? 1473! That’s almost a thousand places! Just <em>one</em> more contribution puts you at number 1114. Still three hundred spots. Your forth commit will let you get in the top 900, #893. You get the idea. (I ended up at #47.)</p>
<p>Today, I’m working on Rust. And one of the things that I do is write the release announcements, or at least, their first draft. And one of my favorite things to do is the last bit: thanking all of the people who have put in work toward this release. It’s one of my favorite things because I really do appreciate all of the patches, but also because I remember how hooked I was when people gave me a little bit of credit for my contribution. Thanking people is powerful.</p>
<p>So to write those blog posts, I have to run some <code>git</code> commands. And I always forget <em>exactly</em> what I have to write, given our branching strategy. So I wrote a little <code>Rakefile</code> almost a year ago that will generate a new post with the appropriate thank yous and statistics. But I never made it general enough; I have to tweak the numbers every release. It’s annoying. I’ve always wanted to fix it up. There’s one more way in which I don’t like this setup: we only thank people who contribute to <code>rust-lang/rust</code>. Even those with a commit to Cargo don’t make it in there. It’s a historical artifact of the release process, but I don’t like it. We should be thanking those people too.</p>
<p>While all of this has been rolling around in the back of my brain, I’ve been thinking about contributors.rubyonrails.org. I’ve said for a while I wanted it in Rust. So, over the last few days, I hacked one together.</p>
<p>https://rust-contributors.herokuapp.com/</p>
<p>Right now, it only replicates what we say in the blog posts. It doesn’t show commit counts, or order by it, or anything. But the foundation is there. Also, since this is a fun hack, I didn’t take too much care when writing it: there’s a number of low-hanging fruit kinds of things to do. I didn’t use a fancy web framework. I didn’t focus on refactoring relentlessly. I didn’t write any tests. The error handling is basically non-existent. I just knocked it out as fast as I can. But it’s a web app, written in Rust, and that’s fun.</p>
<p>If you’d like to help me hack on this, head over to https://github.com/steveklabnik/contributors and take a look at the issues. I’ve already identified a number of things, and if you do web work but have been meaning to mess with Rust, this might be a good chance to give it a shot.</p>
<p>Thank you to everyone who submits patches to Rust; you’re all great, and I’m lucky to be able to work with you all.</p>
</content:encoded></item><item><title>Fire Mario not fire flowers</title><link>https://steveklabnik.com/writing/fire-mario-not-fire-flowers/</link><guid isPermaLink="true">https://steveklabnik.com/writing/fire-mario-not-fire-flowers/</guid><pubDate>Thu, 29 Dec 2016 00:00:00 GMT</pubDate><content:encoded><p><a href="http://words.steveklabnik.com/rust-is-more-than-safety">My post yesterday</a> got a lot of great responses. Two of them are from people I greatly respect, <a href="https://thefeedbackloop.xyz/safety-is-rusts-fireflower/">Dave Herman</a> and <a href="http://graydon2.dreamwidth.org/247406.html">Graydon Hoare</a>. You should go read those two posts; this one is in response to them. I’m going to respond to both at the same time by mixing them together.</p>
<p>Graydon says:</p>
<blockquote>
<p>Safety in the systems space is Rust’s raison d’être. Especially safe concurrency (or as Aaron put it, fearless concurrency). I do not know how else to put it.</p>
</blockquote>
<p>“Fearless concurrency” is a different way to put it! That is, to use Dave’s metaphor, “safety” is the fire flower. But “fearless concurrency” is Fire Mario. This is where I think Dave misses the mark slightly as well. I agree with his metaphor, but disagree with his conclusion.</p>
<p>I’m not saying that we get rid of fire flowers. I’m saying that we focus on “Rust makes you Fire Mario,” not “Rust is made of fire flowers”, when talking to people about Rust.</p>
<p>Dave also misses my mark slightly, but in a different way:</p>
<blockquote>
<p>I don’t think the Rust community should worry about shying away from talking about safety, but I hope we all remember that it’s in service of something much more exciting: making more badass systems programmers!</p>
</blockquote>
<p>We are so close here that it feels like nitpicking, but I think the difference is important. I think one of Rust’s greatest potentials, like Dave does, is bringing a ton of new people into systems programming. But my point is this: “memory safety without garbage collection” doesn’t speak to that <em>at all</em>. That is, memory safety without garbage collection is the mechanism of how that’s accomplished, but it’s not the accomplishment itself. It’s the Fire Flower, not Fire Mario.</p>
<p>I’d like to make sure we’re talking about being Fire Mario, rather than being Fire Flower enthusiasts.</p>
</content:encoded></item><item><title>Rust is more than safety</title><link>https://steveklabnik.com/writing/rust-is-more-than-safety/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rust-is-more-than-safety/</guid><pubDate>Wed, 28 Dec 2016 00:00:00 GMT</pubDate><content:encoded><p>Rust is most well-known for its features around writing low-level code in a safer way than its predecessors. But it’s also a lot more than that. The Rust community is slowly evolving the way that we talk about Rust to people who haven’t yet given Rust a try, but it can take time. But how did we get here?</p>
<p>In Rust’s initial release announcement, Graydon used a phrase that is amazing:</p>
<blockquote>
<p>[Rust is] Technology from the past come to save the future from itself</p>
</blockquote>
<p>He goes on to elaborate a bit:</p>
<blockquote>
<p>Many older languages [are] better than new ones. We keep forgetting already-learned lessons.</p>
</blockquote>
<p>This is one of the many things that the “Rust” name evokes. Rust was not intended to be a cutting-edge PLT (programming language theory) research language. It was challenging an assumption that I think many of us hold dear: that “progress” means that things always get better. Reality is a bit more complex than that. Yes, we’ve improved on some things, but there are some older things that were actually “better” in some sense, but never quite made it for other reasons. I’ve always phrased this sentiment like this:</p>
<hr />
<p>Let’s look at the initial release dates of some of the top programming languages:</p>
<ul>
<li>C: 1972</li>
<li>C++: 1983</li>
<li>C#: 2000</li>
<li>Java: 1995</li>
<li>JavaScript: 1995</li>
<li>Python: 1991</li>
<li>PHP: 1995</li>
<li>Ruby: 1995</li>
</ul>
<p>The first thing you’ll notice is: what the hell were they putting in the water in 1995?</p>
<p>The second is that, other than C# in 2000, 1995 or earlier is where most of our programming languages come from. Have we really not made any advancements significant enough in the last 20 years to have gotten people’s attention?</p>
<hr />
<p>Now, there are a multitude of newer languages, and people do use them. And these languages have had more releases since then. But none of these languages has fundamentally changed. They’re all pretty much the same, but better. And specifically, you’ll notice that many of these languages are scripting or higher-level languages: if we restrict ourselves to “what most operating systems are programmed in”, we haven’t had a new generation of programming languages in thirty or forty years!</p>
<p>PLT research and industry live in two different worlds. Industry is effectively using ancient PLT tech. There’s lots of <em>good</em> reasons for this too, mind you. But if you’re looking to make a new hit programming language, you don’t actually have to be innovative in the PLT sense. You can take PLT tech from the 2005-2010 era and you’ll still be ten years ahead!</p>
<p>So in some senses, this is what Rust is trying to do: not be a cutting-edge programming language in a PLT sense, but advance the industry by mining “older” research.</p>
<p>However, plans don’t always work out the way you want to. <a href="https://www.youtube.com/watch?v=79PSagCD_AY">As Rust evolved</a>, many of its original bits were thrown out, and replaced with new ones. Rust <em>has</em> advanced a small bit of the universe in PLT with the ownership/borrowing system. Like most research, it’s built on top of older work. But that’s exciting!</p>
<p>Another thing about the borrow checker is that it’s “unique” about Rust as well. So if someone asks you, “hey, what’s cool about Rust?”, it’s extremely tempting to answer with the thing that makes it most clearly different than other programming languages. And for some people, this pitch is compelling! But not for everyone…</p>
<h2>Shifting priorities</h2>
<p>Marketing is hard. But there’s one thing that I’ve learned about it that I think applies to Rust:</p>
<blockquote>
<p>Your marketing should be focused on what problems your product solves for the consumer.</p>
</blockquote>
<p>Rust’s marketing, historically, hasn’t been as good at this as it could have been. That is, we’ve been too focused on the details, and not enough on the big picture. We’ve been worried about what’s interesting in a PLT aspect, and not what problems Rust solves for users. I can think of two big examples where this has happened:</p>
<ul>
<li>Rust is safer than C or C++. (“I never have problems with safety in the C++ code I write” is a common reply.)</li>
<li>Rust is as low-level of a programming language as C! (“I never write C. Rust is not for me.”)</li>
</ul>
<p>Both of these pitches for Rust focus on the <em>how</em>, and not on the problem itself. And that means that sometimes people draw the opposite inference as to what you were intending with the pitch in the first place.</p>
<p>I have some ideas on a better pitch for Rust, and I even had a draft of it here, but I’ve decided to delete it for now. Instead, I’d like to ask all of you who love Rust to give this some thought, and write your own blog posts about what problems Rust solves for you. I’ll edit this post and link to yours if you ask me to. I’ve seen several people do a good job at articulating this kind of thing before. And I’ll work on my own post as well, as a follow-up.</p>
<p>Let’s figure out the best way to share our love of Rust with the rest of the programming world.</p>
<hr />
<p>Here’s some of the responses generated from this:</p>
<p>I’ve written <a href="http://words.steveklabnik.com/fire-mario-not-fire-flowers">Fire Mario, not Fire Flowers</a>, as a response to two responses to this post.</p>
<p>The <a href="https://www.reddit.com/r/rust/comments/5krhr0/rust_is_more_than_safety/">/r/rust</a>, <a href="https://www.reddit.com/r/programming/comments/5krw7r/rust_is_more_than_safety/">proggit</a>, and <a href="https://news.ycombinator.com/item?id=13272474">orange website</a> threads are full of good replies.</p>
<ul>
<li><a href="https://medium.com/@Hisako1337/rust-in-2017-8f2b57a67d9b">Rust in 2017</a></li>
<li><a href="https://insanitybit.github.io/2016/12/28/why-rust-introduction">Why Rust?</a> and <a href="https://insanitybit.github.io/2016/12/28/why-rust-sum-types">Why Rust? Sum Types</a></li>
</ul>
</content:encoded></item><item><title>Four years with Rust</title><link>https://steveklabnik.com/writing/four-years-with-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/four-years-with-rust/</guid><pubDate>Wed, 21 Dec 2016 00:00:00 GMT</pubDate><content:encoded><p>Today is four years since I first learned about the existence of Rust. I know this because Rust 0.5 was the first release I used. Rust has changed a lot in that time. For a brief overview of its history, <a href="https://www.youtube.com/watch?v=79PSagCD_AY">watch this talk of mine</a>. But for today, I thought it would be fun to check out <a href="https://mail.mozilla.org/pipermail/rust-dev/2012-December/002787.html">the release announcement</a> and see what’s the same, and what’s changed.</p>
<h2>rust-dev</h2>
<p>First up, the announcement itself. The <a href="https://mail.mozilla.org/listinfo/rust-dev">rust-dev mailing list</a> used to be your one-stop shop for talking about Rust in a persistent form. But in <a href="https://mail.mozilla.org/pipermail/rust-dev/2015-January/011558.html">January of 2015</a>, we decided to close it down. Why? Let’s see what Brian said at the time:</p>
<blockquote>
<p>You likely have already noticed, but traffic to rust-dev has decreased dramatically in recent months. This was a result of natural changes in project coordination at first, and then an intentional effort to phase out the list.In the beginning of the project there were only two places to talk about rust: #rust and rust-dev, and they served us well for all purposes for several years. As Rust grew though the coordination of the project moved to different venues, conversations were happening in a number of other places, and the purpose of rust-dev became less clear. At the same time there were also a number of heated and essentially unmoderatable discussions that caused many to lose faith in the effectiveness of the mailing list.</p>
</blockquote>
<p>I love mailing lists, but I know I’m fairly unique in that. But they do have one big problem, which is that the tools for moderating them are extremely thin. You can pretty much ban someone, and that’s about it. No removing only certain posts, no cooldown periods, no shadow banning. While Rust has a generally positive reputation for community and moderation, that doesn’t mean things were always perfect. We learned some lessons the hard way.</p>
<p>More than that, mailman represents something of the “old guard” of open source. People see it as a sign of something backwards. Very few people want even more email. The culture is changing. We recognized this, and decided to move to <a href="http://www.discourse.org/">Discourse</a> instead. rust-dev was survived by two children, <a href="http://users.rust-lang.org/">users</a> and <a href="https://internals.rust-lang.org/">internals</a>. This created a clear split between “discussion about Rust” and “discussions about building Rust.” It’s still free/open source software, it has good moderation tools, and you can even have it send you email!</p>
<p>We also don’t make release announcements on these forums; we do it on <a href="https://blog.rust-lang.org/">our blog</a>.</p>
<p>Let’s dig into those notes!</p>
<h2>900 changes, numerous bugfixes</h2>
<p>900 changes is a lot! At the time, Rust was on a roughly six-month release schedule, so that’s about five patches merged per day. Rust 1.14 <a href="https://github.com/rust-lang/blog.rust-lang.org/pull/141">will have</a> 1230 patches over its six-week release cycle: that’s just over 29 per day. That’s a much, much higher velocity. And that only counts the main repository; we’ve since added many more, like <a href="https://github.com/rust-lang/cargo/">Cargo</a> and <a href="https://github.com/rust-lang/crates.io">crates.io</a>. This is actually something I’ve been worried about for a while now; we only credit people for commits to <code>rust-lang/rust</code> in release announcements, basically due to this history, but there’s a lot more work that goes on here these days. I want <a href="http://contributors.rubyonrails.org/">Rails Contributors</a>, but for the Rust project. Anyone want to work on it with me sometime?</p>
<h2>Syntax changes</h2>
<ul>
<li>Removed <code>&lt;-</code> move operator</li>
</ul>
<p>Rust went through several iterations of “when do you move, when do you copy” over its lifetime. Today, that distinction is made between types that implement <code>Copy</code> and ones that don’t. But older Rusts had various schemes. Given that this one was removed in the first version of Rust I used, I don’t <em>exactly</em> remember this, but I think it was</p>
<pre><code>x &lt;- y; // move
x = y; // copy
</code></pre>
<p>Rust also had this at one point in time, I believe:</p>
<pre><code>x = move y; // move
x = y; // copy
</code></pre>
<p>We decided to unify the two syntactically for two reasons: one, the annotations felt like busywork, the compiler would tell you what to put if you got it wrong. Secondly, at least in today’s Rust, move and copy are identical operations, with the exception of what you can do with the older variable after the move. Sharing the syntax reinforces that.</p>
<p>Niko’s blog has a lot of great history in it, and <a href="http://smallcultfollowing.com/babysteps/blog/2012/10/01/moves-based-on-type/">this topic is no exception</a>.</p>
<ul>
<li>Completed transition from <code>#fmt</code> extension syntax to <code>fmt!</code></li>
</ul>
<p>Long ago, Rust had a “no keywords can have more than five letters” rule, and so many things were abbreviated. This particular extension lives on today, but with the longer name <code>[format!](https://doc.rust-lang.org/stable/std/macro.format.html)</code>.</p>
<ul>
<li>Removed old fixed length vector syntax - <code>[T]/N</code></li>
</ul>
<p>Rust’s vector and array types, as we know them today, went through a <em>lot</em> of internal iteration, across many axes. The moral equivalent of <code>[T]/N</code> today is <code>[T; N]</code>, but I’m sure there are/were lots of subtle differences between the two. I can’t quite remember.</p>
<ul>
<li>New token quasi-quoters, <code>quote_tokens!</code>, <code>quote_expr!</code>, etc.</li>
</ul>
<p>These live on in some sense; they’re still not stable, so much so that they’re note even shown in the official docs. <a href="https://manishearth.github.io/rust-internals-docs/syntax/ext/quote/index.html">Manish hosts a copy though</a>. These are tools for “syntax extensions”, Rust’s most powerful form of metaprogramming. A final design for them was <a href="https://github.com/rust-lang/rfcs/pull/1566">just accepted eight days ago</a>, so it’ll be a while still before you see this on stable Rust.</p>
<ul>
<li>Macros may now expand to items and statements</li>
</ul>
<p>I don’t know why macros couldn’t do this before, but they still can today, and it’s very useful.</p>
<ul>
<li><code>a.b()</code> is always parsed as a method call, never as a field</li>
</ul>
<p>This is a fun edge case. Here’s the problem:</p>
<pre><code>struct Env&lt;F: Fn(i32)&gt; { f: F,}let e = Env { f: |i| println!("Hello, {}", i) };e.f(); // what does this do?
</code></pre>
<p>According to this entry, this would be a method call. In today’s Rust, this would be an error, you need to write <code>(e.f)();</code> At least the error tells you exactly what to do!</p>
<pre><code>error: no method named `f` found for type `Env&lt;[closure@&lt;anon&gt;:6:22: 6:50]&gt;` in the current scope
--&gt; &lt;anon&gt;:8:7
|
8 | e.f(); // what does this do?
| ^
|
note: use `(e.f)(...)` if you meant to call the function stored in the `f` field
--&gt; &lt;anon&gt;:8:7
|
8 | e.f(); // what does this do?
| ^
</code></pre>
<ul>
<li><code>Eq</code> and <code>IterBytes</code> implementations automatically generated with <code>#[deriving_eq]</code> and <code>#[deriving_iter_bytes]</code> respectively</li>
</ul>
<p>We now have a more generic way to derive traits: <code>#[derive(Eq)]</code>, for example. <code>Eq</code> is still here today, but <code>IterBytes</code> is not. I can’t recall exactly what it did.</p>
<p>Today, the list of traits you can derive is still only blessed ones by the compiler, and it’s a huge cause of people using nightly today, for better ergonomics with libraries like <a href="http://serde.rs/">Serde</a> and <a href="http://diesel.rs/">Diesel</a>. But as of Rust 1.15, this restriction will be lifted, and one of the largest blockers of people using stable Rust will be eliminated! ???</p>
<ul>
<li>Removed the special crate language for <code>.rc</code> files</li>
</ul>
<p>The moral equivalent of <code>.rc</code> files today are <code>.crate</code> files, which Cargo will upload to crates.io. But unlike in the 0.5 days, you almost never need to worry or think about them, since Cargo makes it Just Work (tm).</p>
<ul>
<li>Function arguments may consist of any irrefutable pattern</li>
</ul>
<p>This is true today, and is a little known fact! It works like this:</p>
<pre><code>struct Point { x: i32, y: i32,}fn takes_point(Point {x, y}: Point) { println!("({}, {})", x, y);}fn main() { let origin = Point { x: 0, y: 0 }; takes_point(origin);}
</code></pre>
<p>That is, arguments to functions are <code>PATTERN: TYPE</code>, not <code>NAME: TYPE</code>. Inside of <code>takes_point</code>, <code>x</code> and <code>y</code> are in scope, rather than a whole point.</p>
<h2>Semantic changes</h2>
<ul>
<li><code>&amp;</code> and <code>~</code> pointers may point to objects</li>
</ul>
<p>Rust did have objects, long ago, but I thought they were removed at this point. This one is unclear to me.</p>
<p>Worth mentioning that <code>~</code> is <code>Box&lt;T&gt;</code> now, though. That’s a long story…</p>
<ul>
<li>Tuple structs - <code>struct Foo(Bar, Baz)</code>. Replace newtype enums.</li>
</ul>
<p>These are <a href="https://doc.rust-lang.org/stable/book/structs.html#tuple-structs">still in Rust today</a>.</p>
<ul>
<li>Enum variants may be structs</li>
</ul>
<p>This is true today:</p>
<pre><code>enum Foo {
Variant { x: i32, y: i32 },
}
</code></pre>
<ul>
<li>Destructors can be added to all nominal types via Drop trait</li>
</ul>
<p>This is <a href="https://doc.rust-lang.org/stable/std/ops/trait.Drop.html">still true</a>!</p>
<p>I think the “nominal” here is alluding to something older in Rust’s type system; we don’t make these kinds of distinctions today, as IIRC, all types are nominal. Not 100% sure.</p>
<ul>
<li>Structs and nullary enum variants may be constants</li>
</ul>
<p>This is true today.</p>
<ul>
<li>Values that cannot be implicitly copied are automatically moved without writing <code>move</code> explicitly</li>
</ul>
<p>I alluded to this above with <code>&lt;-</code>.</p>
<ul>
<li><code>&amp;T</code> may now be coerced to <code>*T</code></li>
</ul>
<p><code>*T</code> is <code>*const T</code> or <code>*mut T</code> today, but this is still true:</p>
<pre><code>let x = &amp;5;
let y: *const i32 = x;
</code></pre>
<ul>
<li>Coercions happen in <code>let</code> statements as well as function calls</li>
</ul>
<p>We’ve gone through many different iterations of “what coerces and what doesn’t” over the years; I actually expected the example with <code>*const i32</code> above to require an <code>as</code>.</p>
<ul>
<li><code>use</code> statements now take crate-relative paths</li>
</ul>
<p>This is true today. I really like it, but a lot of people find it confusing. Basically, “use” always starts from the root of the crate. So</p>
<pre><code>mod bar { struct Foo; mod baz { use bar::Foo; // from the root use super::Foo; // if you want to start from the parent use self::super::Foo; // self makes it be relative. Not even sure if self + super works, but you get the idea. }}
</code></pre>
<ul>
<li>The module and type namespaces have been merged so that static method names can be resolved under the trait in which they are declared</li>
</ul>
<p>Not sure on this one, it’s too in the weeds.</p>
<h2>Improved support for language features</h2>
<ul>
<li>Trait inheritance works in many scenarios</li>
</ul>
<p>This has always been a bit of a misnomer, inheritance means:</p>
<pre><code>trait Foo: Bar {
}
</code></pre>
<p>If you implement <code>Foo</code>, you must also implement <code>Bar</code>. That’s it. This line makes me smile: “it works more often.”</p>
<ul>
<li>Support for explicit self arguments in methods - <code>self</code>, <code>&amp;self</code> <code>@self</code>, and <code>~self</code> all generally work as expected</li>
</ul>
<p>Today’s Rust doesn’t just support this; it’s required! However:</p>
<ul>
<li><code>self</code> is still <code>self</code></li>
<li><code>&amp;self</code> is still <code>&amp;self</code></li>
<li><code>@self</code> is <code>Rc&lt;self&gt;</code>, sorta kinda.</li>
<li><code>~self</code> is <code>Box&lt;self&gt;</code></li>
</ul>
<p><code>@self</code> was supposed to be a garbage-collected type, but was never really more than refcounting.</p>
<p>In addition, the last two don’t actually work like that: they would be <code>self: Rc&lt;Self&gt;</code> and <code>self: Box&lt;self&gt;</code>. The <code>Box</code> version will compile today, but the <code>Rc</code> version does not, because <code>Box</code> is magical. Eventually this will be rectified.</p>
<ul>
<li>Static methods work in more situations</li>
</ul>
<p>Another “yay more features work” line. :)</p>
<ul>
<li>Experimental: Traits may declare default methods for impls to use</li>
</ul>
<p>This works today, and is very useful. Consider <code>[Iterator](https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html)</code>: all of those methods have default implementations, so you only need to define <code>next()</code>, and you get the rest automatically.</p>
<h2>Libraries</h2>
<ul>
<li>New condition handling system in <code>core::condition</code></li>
</ul>
<p>Conditions were a Lisp way of handling errors that Rust supported for a while. They were really cool, but nobody really knew how to effectively use them, so they never got used, so they’re gone now.</p>
<ul>
<li>Timsort added to <code>std::sort</code></li>
</ul>
<p><code>std::sort</code> doesn’t exist anymore, though we do have <a href="https://doc.rust-lang.org/stable/std/primitive.slice.html#method.sort">a sort function on slices</a>. We don’t declare it to have a particular algorithm, though we do say “This sort is stable and O(n log n) worst-case but allocates approximately 2 * n where n is the length of self.”</p>
<p><a href="https://github.com/rust-lang/rust/pull/38192">Recently it was improved a ton</a>, by a first-time contributor, no less! Extremely well-done. In the PR, he explains the algorithm:</p>
<blockquote>
<p>However, if we take the main ideas from TimSort (intelligent merging strategy of sorted runs) and drop galloping, then we’ll have great performance on random inputs and it won’t be bad on partially sorted inputs either.</p>
</blockquote>
<p><code>TimSort</code> is still around, kicking.</p>
<ul>
<li>New priority queue, <code>std::priority_queue</code></li>
</ul>
<p>Today this is <code>[std::collections::binary_heap](https://doc.rust-lang.org/std/collections/binary_heap/)</code>.</p>
<ul>
<li>Pipes for serializable types, `std::flatpipes’</li>
</ul>
<p>No idea where this went.</p>
<ul>
<li>Serialization overhauled to be trait-based</li>
</ul>
<p>This is true today. See the above comments about Serde, though. <code>[rustc-serialize](https://crates.io/crates/rustc-serialize)</code> got to cheat though, and the compiler understands <code>RustcEncodable</code> and <code>RustcDecodable</code>. Can’t wait for Rust 1.15.</p>
<ul>
<li>Expanded <code>getopts</code> definitions</li>
</ul>
<p>We’ve <a href="https://github.com/rust-lang-nursery/getopts">moved <code>getopts</code> out of tree</a>, but it’s still there.</p>
<ul>
<li>Moved futures to <code>std</code></li>
</ul>
<p>Hoo boy! I forgot that we used to have futures in the standard library. Futures are one of the hottest topics in Rust today. <a href="https://github.com/tokio-rs">Tokio</a> is one of the most anticipated releases of all time in Rust world. For a taste, <a href="https://www.youtube.com/watch?v=bcrzfivXpc4">watch this talk</a>. A little birdie told me that Tokio should have a <code>0.1</code> release soon…</p>
<ul>
<li>More functions are pure now</li>
</ul>
<p>Rust no longer has a notion of purity, though <code>[const fn](https://github.com/rust-lang/rfcs/blob/master/text/0911-const-fn.md)</code> (still unstable) sorta feels like it in some senses. For some history on purity in Rust, see <a href="http://smallcultfollowing.com/babysteps/blog/2012/10/12/extending-the-definition-of-purity-in-rust/">Niko’s blog</a>, or <a href="https://mail.mozilla.org/pipermail/rust-dev/2013-April/003926.html">this explanation by Graydon on why it was removed</a>.</p>
<ul>
<li><code>core::comm</code> renamed to <code>oldcomm</code>. Still deprecated</li>
</ul>
<p>Very, very long gone.</p>
<ul>
<li><code>rustdoc</code> and <code>cargo</code> are libraries now</li>
</ul>
<p>Rustdoc still exists, though not <em>really</em> as a library. Cargo however, is not the cargo you’re thinking of. <a href="https://github.com/rust-lang/rust-wiki-backup/blob/73816d6f888c24fb8115d78078a1601805cbecb5/Doc-using-cargo-to-manage-packages.md">It’s a totally different one</a>. Rust went through many iterations of package managers before coming up with today’s Cargo; there was also <code>[rustpkg](https://github.com/rust-lang/rust/blob/3e39e3e80dcf726a96ec0fe778f96e2a9dde620b/doc/guide-rustpkg.md)</code>, for example, which was <em>after</em> the Cargo referenced here, but before the Cargo we’re using today.</p>
<h2>Misc</h2>
<ul>
<li>Added a preliminary REPL, <code>rusti</code></li>
</ul>
<p>We removed the repl from the tree, as it never <em>really</em> worked, and was a maintenance nightmare. <a href="https://github.com/murarth/rusti">This one</a> is the only one I know of today. Some people ask about one, but nobody has done the work to get a good one together yet. It’s tough!</p>
<ul>
<li>License changed from MIT to dual MIT/APL2</li>
</ul>
<p>This is still true today, with no plans on changing.</p>
<h2>Contributors to Rust 0.5:</h2>
<p>We had 41 people contribute to 0.5. 1.14 will have 144 contributors, and that’s around average for a release these days. Out of those 41 people, at a glance, I recognize the names of about 17 individuals, and roughly six of them are still involved with Rust and Servo today. What about the other eleven? Some were interns that got jobs at other places and can’t work on Rust anymore, some are contributors that left for various reasons, and then of course, there’s Graydon.</p>
<p>I personally opened <a href="https://github.com/rust-lang/rust/pull/4305">my first pull request</a> six days after discovering it. You’ll note this PR wasn’t merged due to a procedural issue: I sent it into the wrong branch! Today, GitHub lets you change this, but I had to <a href="https://github.com/rust-lang/rust/pull/4308">open a new PR instead</a>.</p>
<p>As I said in that PR:</p>
<blockquote>
<p>I’ve just started using Rust, and I really like it. One of the things that makes Rust hard to use is its lack of documentation, which is obviously fine, given that Rust isn’t production-ready yet.I’d like to help change that.To get started, I’ve just modified this one little description in the rustdoc. If this works out, and you all like my changes, I’ll start working on more docs like this for the rest of core, and maybe the stdlib after that.</p>
</blockquote>
<p>You might say it did work out. I’m now <a href="https://blog.rust-lang.org/2014/12/12/Core-Team.html">on the core team</a>, I’ve sent in 866 PRs since then, and that first patch landed in <a href="https://mail.mozilla.org/pipermail/rust-dev/2013-April/003427.html">Rust 0.6</a>. Rust 0.10 is the only release since then that I’ve not gotten at least one patch into, and today, I write the release announcements.</p>
<p>I hope you’ve enjoyed this little trip down memory lane! I’m still really loving Rust and its community. Here’s to many more years ?♥</p>
</content:encoded></item><item><title>What's new with "The Rust Programming Language"</title><link>https://steveklabnik.com/writing/whats-new-with-the-rust-programming-language/</link><guid isPermaLink="true">https://steveklabnik.com/writing/whats-new-with-the-rust-programming-language/</guid><pubDate>Mon, 15 Aug 2016 00:00:00 GMT</pubDate><content:encoded><p>It’s been a while since I’ve mentioned my work on “The Rust Programming Language”, and I wanted to give you all an update. Before we get into the news, though, I’d like to take a moment to reflect on where the book has been, and then we’ll talk about it’s future.</p>
<p>I <a href="http://words.steveklabnik.com/rusts-documentation-is-about-to-drastically-improve">previously wrote about my history with Rust and docs</a>. As I said back then:</p>
<blockquote>
<p>But my first area of focus will be on the area of Rust’s documentation that’s the most weak, and simultaneously the most important: the tutorial.</p>
</blockquote>
<p>“The tutorial” eventually turned into “The Rust Programming Language.” But there’s a funny thing about writing a book for a language that isn’t done yet: you don’t really know <em>how</em> to write a good book on something that doesn’t exist, because, well, you haven’t played with it yourself. I could make decent guesses, given that Rust was evolving incrementally, but that’s still very different. I think “technical debt” is a great metaphor here: the evolution of the language meant that I had to keep up with the work of many, many people to try and update the documentation. Furthermore, since nobody had tried to learn Rust 1.0 yet, since it didn’t exist, I couldn’t write a book that would do an amazing job of teaching them, since I hadn’t used it either!</p>
<p>In some ways, this mirrors the compiler itself, which accrued a <em>massive</em> amount of technical debt until 1.0. And so, the story since 1.0 has largely about been paying off this debt; culminating in <a href="https://blog.rust-lang.org/2016/04/19/MIR.html">MIR</a>. By the same token, since 1.0, a lot of people have read the book, and a lot of people have tried to learn Rust. I now know so much more about how people go about doing so, and so am much, much better situated to write something that’ll be great for them.</p>
<p>And so, now that time has passed, re-writing the book is what I’ve been spending a lot of my time on. I’ve been mostly quiet about this work, since I’ve wanted to get a chunk finished before starting to talk about it, and so today is finally that day.</p>
<p>Let’s talk about what’s going on with the book today.</p>
<h2>A new coauthor.</h2>
<p>The first bit of news is something that I’ve been wanting to tell you all for a few weeks now, and is the bit that I’m most excited about: I’m bringing <a href="https://twitter.com/carols10cents">Carol (Nichols || Goulding)</a> on as a coauthor. TRPL is an incredibly personal work for me, one that I’ve poured my life into over the last few years. Carol and I have been friends for a long time, and there’s nobody else I’d rather share an authorship with. Not only is Carol a friend, but she’s been doing Rust longer than most. You might know her from <a href="https://github.com/carols10cents/rustlings">Rustlings</a>, or the <a href="http://rust-belt-rust.com/">Rust Belt Rust conference</a>, or as a cofounder of <a href="http://integer32.com/">Integer32</a>, the first Rust-focused consulting company. Carol knows her stuff, and I’m proud to have her name next time mine on the cover. The book will be far better through her influence.</p>
<h2>A new beginning</h2>
<p>One of Carols’ first tasks was taking a bunch of the stuff I’d already written and checking it out herself, making improvements. She’s taken the first six chapters of the book and whipped them into shape. Some of the content you might recognize from the first edition of TRPL, but much of it is brand-new.</p>
<ul>
<li><a href="http://rust-lang.github.io/book/">Read the first six chapters online</a></li>
<li><a href="https://github.com/rust-lang/book">Source and issue tracker</a></li>
</ul>
<p>As an example of how much better this book is, can, and will be than the first edition, compare the <a href="http://rust-lang.github.io/book/ch04-00-understanding-ownership.html">Ownership chapter</a> to <a href="https://doc.rust-lang.org/stable/book/ownership.html">the current one</a> ( plus <a href="https://doc.rust-lang.org/stable/book/references-and-borrowing.html">borrowing</a> ). Instead of an abstract set of rules, we took one of the biggest things facing a new Rustacean, strings, and used them as a way to make these concepts concrete.</p>
<p>If you have the time, please check it out, and file issues with any suggestions!</p>
<h2>More to come</h2>
<p>If you’d like to see chapters as we work on them, just follow the repo: everything is open source. Initial PRs are already up for the start of the next three chapters, and hopefully we’ll get them merged and move forward soon. I’m really proud of how the book is shaping up, and hope you enjoy it too.</p>
</content:encoded></item><item><title>Structure literals vs. constructors in Rust</title><link>https://steveklabnik.com/writing/structure-literals-vs-constructors-in-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/structure-literals-vs-constructors-in-rust/</guid><pubDate>Tue, 26 Jul 2016 00:00:00 GMT</pubDate><content:encoded><p>Learning the basics of a language and its syntax is easy. Learning how all those bits fit together is a bit harder. There’s a neat intersection between three of Rust’s features that I’ve seen people use, but never seen written down. I was explaining this technique to someone in <code>#rust-beginners</code> the other day, and thought I’d write it down in case it helps you, too.</p>
<p>A small review: If you have a <code>struct</code> in Rust, like this:</p>
<pre><code>struct Point {
x: i32,
y: i32,
}
</code></pre>
<p>You can use ‘<code>struct</code> literal syntax’ to create a new instance of the <code>struct</code>:</p>
<pre><code>let origin = Point { x: 0, y: 0 };
</code></pre>
<p>However, this syntax only works if you have the proper access to the <code>struct</code> and its members via Rust’s privacy rules.</p>
<pre><code>mod foo {
pub struct Point {
x: i32,
y: i32,
}
pub fn foo(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y } // this is fine, as we're in the same module
}
}
fn main() {
let origin = foo::Point { x: 0, y: 0 }; // this is not fine
}
</code></pre>
<p>We can’t use <code>struct</code> literal synatx in <code>main</code> because <code>x</code> and <code>y</code> are also not public. But within the same module, we have access, so it works. So how would we let <code>main</code> instantiate a <code>Point</code> if we can’t use the literal syntax? Well, our <code>foo</code> function does this, so we could expose it. More conventionally, we’d make it an associated function and call it <code>new</code>:</p>
<pre><code>mod foo {
pub struct Point {
x: i32,
y: i32,
}
impl Point {
pub fn new(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y } // this is fine, as we're in the same module
}
}
}
fn main() {
let origin = foo::Point::new(0, 0);
}
</code></pre>
<p>Great. But what if we wanted <code>x</code> and <code>y</code> to be public for some reason, yet we still wanted to force people to use the <code>new</code> function to create <code>Point</code>s? Maybe the initial creation does some sort of side effect that’s important. If we changed our code to this:</p>
<pre><code>mod foo {
pub struct Point {
pub x: i32,
pub y: i32,
}
impl Point {
pub fn new(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y } // this is fine, as we're in the same module
}
}
}
fn main() {
let origin = foo::Point::new(0, 0);
// but so does this:
let origin = foo::Point { x: 0, y: 0 };
}
</code></pre>
<p>By making all of the elements of <code>Point</code> public, we’ve re-enabled the literal syntax, which isn’t what we wanted. So what do we do?</p>
<p>Fixing this requires two insights. The first is “zero-sized types”. In Rust, certain types only have values that don’t require any storage. Take, for example, <code>()</code>, the “unit” type. It only has one possible value, also <code>()</code>. Since it only has one value, there’s no need to actually store anything in memory to represent <code>()</code>; if we have a valid value, we already know what it is. That means that once we compile down to actual assembly, <code>()</code> just goes away entirely. So we can do this:</p>
<pre><code>mod foo {
pub struct Point {
pub x: i32,
pub y: i32,
_secret: (),
}
impl Point {
pub fn new(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y, _secret: () }
}
}
}
fn main() {
let origin = foo::Point::new(0, 0);
}
</code></pre>
<p>Now, we have a new, non-public field, <code>_secret</code>. I’ve given it a name that starts with an underscore because we don’t intend to use it for anything, and so Rust won’t warn us about it. <code>_secret</code> has the type of <code>()</code>, and so it’s entirely a compile-time construct; it doesn’t materially affect <code>Point</code>’s representation. But it being private <em>does</em> affect how we’re allowed to construct our <code>Point</code>s. <code>main</code> can no longer use the <code>struct</code> literal syntax, since not all of the fields are public.</p>
<p>However, remember that privacy is a module-level thing in Rust. Therefore, we can still use the <code>struct</code> literal syntax inside of the <code>foo</code> module:</p>
<pre><code>mod foo {
pub struct Point {
pub x: i32,
pub y: i32,
_secret: (),
}
impl Point {
pub fn new(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y, _secret: () }
}
}
fn foo() -&gt; Point {
Point: { x: 0, y: 0, _secret: () } // this is still allowed!
}
}
fn main() {
let origin = foo::Point::new(0, 0);
}
</code></pre>
<p>To prevent <code>foo</code> from being able to use the literal syntax, we need one more concept: <code>pub use</code>. Check this out:</p>
<pre><code>mod foo {
mod point {
pub struct Point {
pub x: i32,
pub y: i32,
_secret: (),
}
impl Point {
pub fn new(x: i32, y: i32) -&gt; Point {
Point { x: x, y: y, _secret: () }
}
}
}
pub use foo::point::Point;
fn foo() -&gt; Point {
Point::new(0, 0) // must use `new` here, as we're no longer in the same module!
}
}
fn main() {
let origin = foo::Point::new(0, 0);
}
</code></pre>
<p>By giving <code>Point</code> its own module, everything that’s private to it is private. But typing <code>foo::point::Point</code> would be redundant and an ergonomic regression; <code>pub use</code> saves the day! We re-export the <code>Point</code> structure into <code>foo</code>, so we can still use <code>foo::Point</code>, but since one of its members is private, literal syntax isn’t allowed.</p>
<p>To me, understanding things like this is when I really start to feel like I’m getting to know a language: putting three or four disparate concepts together to achieve some goal. It’s when a language stops being a bunch of disjoint parts and starts becoming a cohesive whole.</p>
</content:encoded></item><item><title>A eulogy for my 20s</title><link>https://steveklabnik.com/writing/a-eulogy-for-my-20s/</link><guid isPermaLink="true">https://steveklabnik.com/writing/a-eulogy-for-my-20s/</guid><pubDate>Sun, 24 Jan 2016 00:00:00 GMT</pubDate><content:encoded><p>Today is the last day of my 29th year of existence. I’ve been thinking about it a lot, which is extra strange because I don’t generally care about birthdays. I’m not much of one for holidays in general. So why do I care about this one?</p>
<p>My 20s have been an extreme period of growth. At the start of it, I thought that I was going to live in my little farming town, get married to The Girl, go to mass every Sunday, and pretty much do exactly what everyone else I knew had done. If you’re reading this, well, you know a very different me. Before my 20s, I had only left the country once, a short trip to Toronto. I had barely even left my state. I was smart, but cocky, and incredibly sheltered. I was the very embodiment of a stereotype.</p>
<p>I’ve come a long way.</p>
<p>The root of my anxiety about being 30 comes from something that my brain is constantly whispering into my ear: “You don’t have enough time.” There’s sort of two sides to this; I think: the first is how much I’ve changed in this decade, and the second is a particularly toxic attitude of the industry I’ve immersed myself in.</p>
<p>I feel pretty dumb about this post overall, but whatever. Please just take this as a brain dump from this morning, not some sort of grand, planned essay about how I know all the things or something.</p>
<h2>Identity is a function</h2>
<p>Have you ever played around with <a href="http://www.utdallas.edu/~gupta/courses/apl/lambda.pdf">lambda calculus</a>? I myself only know the basics, I’m not a particularly math-y person. Here’s a short introduction: say we have a function. This function is very simple, it returns its single argument. In other words, the identify function:</p>
<pre><code>fn id(x) {
return x;
}
</code></pre>
<p>(Making this syntax up, though it will look sort of like Rust without types.)</p>
<p>We can write another function, <code>zero'()</code>, like this:</p>
<pre><code>fn zero'(f, g) {
return g();
}
</code></pre>
<p>In other words, <code>zero'()</code> takes two functions, <code>f</code> and <code>g</code>. It then calls <code>g()</code>, ignoring <code>f()</code>. This will make sense in a moment, we just need one more function: <code>succ()</code></p>
<pre><code>fn succ(f, g) {
return f(g());
}
</code></pre>
<p>This function, short for “successor”, applies <code>f()</code> to <code>g()</code>, one time.</p>
<p>With these tools, we can represent all of the natural numbers. This is called <a href="https://en.wikipedia.org/wiki/Church_encoding#Church_numerals">Church numerals</a>, and here’s how it works:</p>
<pre><code>
let zero = zero'(succ, id); // zero any function applied to id zero times.
let one = succ(succ, zero); // one is succ() applied to zero
let two = succ(succ, one); // two is succ() applied to one
let three = succ(succ, two); // three is succ() applied to two
</code></pre>
<p>And so on. Now I have this crippling fear that I’ve messed up the notation, trying to transform it into something that looks more like “normal” programming. I’m sure that someone on the Internet is going to find some kind of flaw and call me an idiot over it.</p>
<p>Ahem.</p>
<p>Anyway, this is the core idea: you can count the number of times you’ve applied a function to identity, and that encodes the number you’re trying to represent. We don’t need state to represent numbers and counting, we can do it all through functions.</p>
<p>The concept of “identity” is a really important one in philosophy. You can take all kinds of approaches here. One classic approach is called “Platonic idealism”, also known as “essentialism.” Essentialism states that you can tell the identity of something by the properties it has. For example, a chair is a chair because it has a platform for you to sit, it has four legs, and a back.</p>
<p>The problem with essentialism is that well, there are things that you or I would reasonably call a chair that has three legs, or five or six. Not all things that have a place to sit can be called a chair, either. In other words, while it sounds reasonable at first, the real world absolutely destroys this kind of categorization, in my opinion.</p>
<p>So what do you do? Another solution to this question is posed by a really broad family called ‘process philosophy.’ The solution to identity there is based around movement, change, becoming. Function application. A “chair” is a particular kind of object that participates in a world like other things that we call “chairs”. In other words, identity is something that comes out of <em>doing</em>, not <em>being</em>.</p>
<p>I am not the person I was ten years ago, quite literally. My body is made up of various kinds of cells, and they die at different rates. At a cellular level, I am quite literally dying all the time, in various microscopic places, and I am also regenerating. New cells are being produced. I am quite literally not my past self at a cellular level. And once I finally die, it’s not like my cells also instantly die either. Some of them will live for a period of time after I’m dead, though not for long. Bits of my body will actually out-live me.</p>
<p>Life is full of growth and change. But at this point, the function has been applied so many times that I barely even recognize my past self. It makes me feel like I’m starting <em>now</em>, so late into the game! I’ve wasted the earlier years of my life on all kinds of dumb and bad things.</p>
<p>I constantly feel that I am running out of time, that I have not done enough, that I will be a failure. That my work will not be meaningful enough to be remembered, or worse, will be remembered as bad. When I express this kind of feeling, people often point this out:</p>
<p><img src="https://svbtleusercontent.com/yteofannex78va_small.png" alt="https://svbtleusercontent.com/yteofannex78va_small.png" /></p>
<p>This doesn’t make me feel better, it makes me feel worse. There are so many things that I want to do that I’m <em>not</em> working on. That hypermedia book still isn’t done. I have been a bad maintainer of Resque and all my other Ruby stuff. I didn’t listen to my friends in Greece, who told me to relax, and then the exact situation they predicted happened. I just hope I’m around long enough to make an impact on people and be a positive change in the world. We’ll see if I make it.</p>
<p>There’s a whole ’nother rant here about Camus and Heidegger and absurd-ism and being-towards-death too. I will save that sophomoric rant for my “I’m turning 40” post.</p>
<h2>Youth Cults</h2>
<p>The second thing that’s making me feel terrible about turning 30 is the goddamn computer industry. It’s extra worse because historically, I have benefited from this particular terrible attitude. And that’s the worship of youth.</p>
<p>I used to teach new programmers, and often give talks to classrooms full of them. They always want to know how I got started, and I always preface it with something like</p>
<blockquote>
<p>My story is one I don’t like to share, because I’m the living embodiment of a stereotype that I think is harmful to new programmers.</p>
</blockquote>
<p>I started with computers when I was seven. That shouldn’t matter. I’ve learned more about software in the last five years than I did from seven till I was 18. But we hold up these young genius whiz-kids (and let’s be serious, it’s not ‘kids’, it’s ‘boys’) as the epitome of what our industry can do. The heroic youth who understands computers better than those silly old people. What do they know, anyway?</p>
<p>Age discrimination is real. It’s stupid, but it’s real. And even when you know that it’s stupid, it can still slip into your brain in a really terrible way. If I’m 30 and this is all I’ve done by now, well, why even bother?</p>
<p>This is dumb. The attitude is dumb. That I disagree with it, yet still feel it at times, is dumb. Ideology is a motherfucker.</p>
<p>So yeah, that’s it. My 20s are almost over. And I wasted some of the last bits of them writing this silly blog post. There’s no stopping it, just watching the clock tick forward, and doing my best as hard as I can until it’s all over. How many of these decades will I even get? My dad certainly thought he had a few more until that night he collapsed and found out that he had stage four, terminal cancer.</p>
<p>Let’s see what I can do in the next ten years.</p>
</content:encoded></item><item><title>Docember</title><link>https://steveklabnik.com/writing/docember/</link><guid isPermaLink="true">https://steveklabnik.com/writing/docember/</guid><pubDate>Tue, 01 Dec 2015 00:00:00 GMT</pubDate><content:encoded><p>There are few things I appreciate more than using software projects that have good documentation. When I’m new to the project, documentation is invaluable for helping me get going. When I’m not new, good documentation helps me get the information I need to :shipit:.</p>
<blockquote>
<p>We say programming is hard, and we hate bad code. So we practice writing code.</p>
</blockquote>
<p>We say writing docs is hard, and hate bad docs…</p>
<p>— Some(@steveklabnik) (@steveklabnik) November 30, 2015</p>
<blockquote></blockquote>
<p>About a month ago, I was tweeting about this topic, and <a href="https://mayeu.me/misc/2015/11/13/docember-one-month-to-improve-your-documentation.html">several of us came up with an idea</a>: similar to how November is <a href="http://nanowrimo.org/">National Novel Writing Month</a>, why not have a similar month for programming?</p>
<p>Here’s the idea: in December, let’s try to work on the documentation for the projects we work on. Make it 50k words like NaNoWriMo, make it <a href="http://24pullrequests.com/">24 pull requests</a>, whatever floats your boat. Your future self will love you for it.</p>
</content:encoded></item><item><title>Rescuing Resque (again)</title><link>https://steveklabnik.com/writing/rescuing-resque-again/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rescuing-resque-again/</guid><pubDate>Wed, 28 Oct 2015 00:00:00 GMT</pubDate><content:encoded><p>A long time ago, there was a big open source project, which was a huge part of the Rails world, but had almost no maintenance. This project was Resque, the go-to way to write background jobs for your Rails application. I attended a talk where the current maintainer was clearly getting burned out, and so, <a href="http://blog.steveklabnik.com/posts/2012-09-22-resque--let-s-do-this">offered to help</a>. The problem is, things have changed.</p>
<p>It wasn’t just me. It was a bunch of people. We all used Resque, and loved it, and wanted to make it better. We made plans for a big 2.0 to pay off a bunch of technical debt, simplify the codebase, and add more features. A ton of people chipped in, I had some awesome remote pairing sessions. Things were good.</p>
<p>But slowly, we all started to drift away. One contributor changed jobs. Another decided to spend more time with family. I got discouraged, and then, there was a strange situation in which I accidentally released an entirely straight-up broken version. So I got scared, released a fix, and basically never touched Resque again.</p>
<p>Resque is used by a <em>lot</em> of people:</p>
<p><img src="https://svbtleusercontent.com/rsoq8peusw08q_small.png" alt="https://svbtleusercontent.com/rsoq8peusw08q_small.png" /></p>
<p>Yet, it hasn’t seen a release in 18 months. The PRs are piling up. 2.0 is basically dead in the water. I’m not much of a fan of 2.0s anymore myself, at least, ones that don’t follow the “stability without stagnation” principles. I don’t use Resque myself anymore, and barely have enough time for my work, let alone my personal projects, let alone a project that I care about but only work on out of the goodness of my heart.</p>
<p>I’m here to tell you that that’s changing a little bit. I’ve been chatting with <a href="https://github.com/hoffmanc/">Chris Hoffman</a>, who’s interested in starting up on work again. I asked him what his intentions were, and what he answered with was exactly what I wanted to hear, as it turns out. So, I’m going to be doing some work again, this time, with the explicit intention of handing off maintenance to Chris and anyone else who wants to chip in. If you use Resque commercially, this is a great time to give some of your people the opportunity to give back. If you’ve never worked on an open source project before, but want to, I’d be more than happy to help mentor you with this. Resque is a big, real project, with a lot of impact. You can do a lot of good. That’s why I started working on it.</p>
<p>Without further ado, here is The Plan, from Chris and I:</p>
<h2>TL;DR</h2>
<ul>
<li>change <code>master</code> from unreleased 2.0 back to <code>1-x-stable</code></li>
<li>clean up tests - make the development experience enjoyable</li>
<li>integrate pending Pull Requests</li>
<li>2.0? Never heard of it. At least, not for a while.</li>
</ul>
<h2>Make <code>1-x-stable</code> master again</h2>
<p><a href="https://github.com/resque/resque/issues/1175">Many</a> <a href="https://github.com/resque/resque/issues/976">have</a> stated that the README on the master branch has led them astray, because the published gem is still at 1.25.2.</p>
<p>We will plan on making this switch in the next couple of weeks. If you feel this is the wrong move, please let us know. But since active development will in fact happen on 1.x, this feels like the correct thing.</p>
<h2>Fix up the test suite</h2>
<p>Tests need to be fast, and clean, so that there is more incentive to contribute.</p>
<p>One possible approach is using one of the available Redis mocking libraries (mock_redis, fakeredis) in non-forking tests, which could speed suite execution up considerably.</p>
<h2>Review Pending <code>1-x-stable</code> PRs</h2>
<p>This is the lion’s share of the work, as there are currently 56 (that’s <em><a href="https://github.com/resque/resque/pulls">fifty six</a></em>) open pull requests…</p>
<p><em>And that’s an awesome problem!</em> Let’s make use of them! These PRs need to be evaluated and closed, so that the project can benefit from the community.</p>
<h2>Decide What to do with 2.0</h2>
<p>The focus of Resque should be on stability, with an eye towards legacy support. 2.0 is off the table, at least in the near term.</p>
<p>If there is a 2.0, it will be a “garbage collection”-style release, where only deprecated functionality is removed, and the upgrade path is clear.</p>
</content:encoded></item><item><title>"The Rust Programming Language" will be published by No Starch Press</title><link>https://steveklabnik.com/writing/the-rust-programming-language-will-be-published-by-no-starch-press/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-rust-programming-language-will-be-published-by-no-starch-press/</guid><pubDate>Tue, 08 Sep 2015 00:00:00 GMT</pubDate><content:encoded><p>I’m happy to bring you some news: <a href="http://doc.rust-lang.org/stable/book/">“The Rust Programming Language”</a> is going to be published, in physical form, by <a href="https://www.nostarch.com/">No Starch Press</a>. I am really excited to work with No Starch on this: they’ve published some of my favorite programming books, like <a href="https://www.nostarch.com/hemingway">“If Hemingway Wrote JavaScript”</a>, <a href="https://www.nostarch.com/rum">“Ruby Under a Microscope”</a>, and <a href="https://www.nostarch.com/rubywizardry">“Ruby Wizardry”</a>.</p>
<p>Another reason I’m keen to work with No Starch on this is that they believe in publishing freely available material: we’re going to take the existing manuscript, whip it into shape, make a nice paper book out of it, and merge the changes back into the official documentation. So the improvements will get back to everyone, even those who can’t afford a physical copy.</p>
<p>The authorship credit is to “The Rust Project Developers,” as other people have contributed significantly: we even have an <a href="https://github.com/rust-lang/rust/pull/28301">active PR to re-write an entire chapter</a>. Rust is a team effort.</p>
<p>Oh, and one last thing: the profit from the book won’t go to me, it will go to <a href="http://openhatch.org/">OpenHatch</a>, “a non-profit dedicated to matching prospective free software contributors with communities, tools, and education,” to use their words about it.</p>
<p>I’ll let you know when we’re closer to actually shipping!</p>
</content:encoded></item><item><title>Community Versions for Rust</title><link>https://steveklabnik.com/writing/community-versions-for-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/community-versions-for-rust/</guid><pubDate>Sun, 23 Aug 2015 00:00:00 GMT</pubDate><content:encoded><p><a href="http://www.rust-lang.org/">Rust</a> has been through a <em>lot</em> of changes in recent years. As such, tutorials get out of date. We’ve occasionally said that including <code>rustc -v</code> in your blog posts would be helpful, but today, I saw this tweet go by:</p>
<blockquote>
<p>Tired of guessing what versions of Ember work with a given blog post? Check out http://t.co/6joJiNdJQ5 #feedbackplz pic.twitter.com/YToBduBVtx
— mixonic (@mixonic) August 23, 2015</p>
</blockquote>
<p>After some quick hacking, I’ve modified it to work with Rust. You can find the project here: <a href="https://github.com/steveklabnik/rust-community-versions">https://github.com/steveklabnik/rust-community-versions</a></p>
<p>TL;DR: Put this into your blog post, substituting the appropriate date and title:</p>
<pre><code>&lt;iframe
width="178" height="24" style="border:0px"
src="http://steveklabnik.github.io/rust-community-versions/2013/10/18/pointers-in-rust-a-guide.html"&gt;
&lt;/iframe&gt;
</code></pre>
<p>Then, make a <a href="https://github.com/steveklabnik/rust-community-versions/pulls">pull request</a> adding a new post with some front-matter:</p>
<pre><code>---
layout: post
url: http://words.steveklabnik.com/pointers-in-rust-a-guide
title: "Pointers in Rust: a Guide"
date: 2013-10-18
start_version: 0.8
end_version: 0.9
---
</code></pre>
<p>(<code>end_version</code> is optional)</p>
<p>And bam! You get a nice looking badge. Check out out on this blog post of mine: http://words.steveklabnik.com/pointers-in-rust-a-guide</p>
<p>What’s nice about this approach is that, if your post ever gets out of date, someone else can modify the YAML in the repo, and your badge will automatically update to include the end date. No remembering to go back and fix it.</p>
<p>Let’s see how this goes!</p>
</content:encoded></item><item><title>Ember Data: ‘DS’ is not defined</title><link>https://steveklabnik.com/writing/ember-data--ds-is-not-defined/</link><guid isPermaLink="true">https://steveklabnik.com/writing/ember-data--ds-is-not-defined/</guid><pubDate>Sun, 05 Jul 2015 00:00:00 GMT</pubDate><content:encoded><p>I’m doing a little Ember app, and if there’s one thing I’ve learned from writing software, it’s to blog about error messages. Two-years-later me has ended up finding my own posts when searching for help!</p>
<p>So today, when getting started with Ember Data 1.13, I was trying to use the new <code>JSONAPIAdapter</code>. I saw this code snippet:</p>
<pre><code>App.ApplicationAdapter = DS.JSONAPIAdapter.extend({
namespace: 'v1',
});
</code></pre>
<p>Using that gave me an error when <code>ember serve</code>-ing, though:</p>
<pre><code>app.js: line 16, col 26, ‘DS’ is not defined.
</code></pre>
<p>Turns out, <code>DS</code> isn’t in scope by default, even though <code>ember-cli</code> installs Ember Data by default.</p>
<p>Fixing this just means importing it at the top of <code>app/app.js</code>:</p>
<pre><code>import DS from 'ember-data';
</code></pre>
<p>Easy enough!</p>
</content:encoded></item><item><title>The language strangeness budget</title><link>https://steveklabnik.com/writing/the-language-strangeness-budget/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-language-strangeness-budget/</guid><pubDate>Fri, 26 Jun 2015 00:00:00 GMT</pubDate><content:encoded><p>I’ve always loved programming languages. I’ve spent plenty of times with many of them, and am fortunate enough that <a href="http://blog.rust-lang.org/2014/12/12/Core-Team.html">language design is now part of my job</a>. In discussions about building Rust, I’ve noticed myself making a particular kind of argument often in design discussions. I like to call it ‘the language strangeness budget’.</p>
<p>When building anything, it’s important to understand why you are building it, and who you are building it for. When it comes to programming languages, <a href="https://github.com/steveklabnik/mojikun">building one is easy</a>, but getting people to use it is much, much harder. If your aim is to build a practical programming language with a large community, you need to be aware of how many new, interesting, exciting things that your language is doing, and carefully consider the number of such features you include.</p>
<p>Learning a language takes time and effort. <a href="http://doc.rust-lang.org/stable/book/">The Rust Programming Language</a>, rendered as a PDF, is about 250 pages at the moment. And at the moment, it really covers the basics, but doesn’t get into many intermediate/advanced topics. A potential user of Rust needs to literally read a book to learn how to use it. As such, it’s important to be considerate of how many things in your language will be strange for your target audience, because if you put too many strange things in, they won’t give it a try.</p>
<p>You can see us avoiding blowing the budget in Rust with many of our syntactic choices. We chose to stick with curly braces, for example, because one of our major target audiences, systems programmers, is currently using a curly brace language. Instead, we spend this strangeness budget on our major, core feature: ownership and borrowing.</p>
<p>On the other hand, if you’re trying to accomplish something different with your language, purposefully blowing this budget can be useful too. I really enjoy Haskell, and for a long time, their slogan has been ‘<a href="http://www.computerworld.com.au/article/261007/a-z_programming_languages_haskell/">avoid success at all costs</a>’:</p>
<blockquote>
<p>As for the second, I don’t know if you know this, but Haskell has a sort of unofficial slogan: avoid success at all costs. I think I mentioned this at a talk I gave about Haskell a few years back and it’s become sort of a little saying. When you become too well known, or too widely used and too successful (and certainly being adopted by Microsoft means such a thing), suddenly you can’t change anything anymore. You get caught and spend ages talking about things that have nothing to do with the research side of things.I’m primarily a programming language researcher, so the fact that Haskell has up to now been used for just university types has been ideal</p>
</blockquote>
<p>This is Simon Peyton-Jones knowing exactly what kind of language he wants to build, and in what way. Haskell’s embrace of strangeness in this sense is what makes Haskell Haskell, and actually furthers its goals.</p>
<p>If you include <em>no</em> new features, then there’s no incentive to use your language. If you include too many, not enough people may be willing to invest the time to give it a try. Language designers should give careful thought to how strange their language is, and choose the right amount to accomplish what they’re trying to accomplish.</p>
</content:encoded></item><item><title>Ouroboros</title><link>https://steveklabnik.com/writing/ouroboros/</link><guid isPermaLink="true">https://steveklabnik.com/writing/ouroboros/</guid><pubDate>Mon, 18 May 2015 00:00:00 GMT</pubDate><content:encoded><p>One of the things that I love about open source is its ability to send improvements across projects. Sometimes, an improvement in one project ends up improving its upstream. This kind of network effect really gets to the core of it for me: steady, constant improvement.</p>
<p>I saw an example of this today that makes me smile. Before <a href="http://blog.rust-lang.org/2015/05/15/Rust-1.0.html">Rust’s 1.0 release</a>, I wrote a <a href="http://doc.rust-lang.org/stable/book/dining-philosophers.html">chapter about the Dining Philosopher’s problem</a>. Incidentally, I’m really happy to use classic CS concepts in the Rust docs, hopefully exposing them to people who didn’t get the opportunity to get a degree in computer science. Anyway, towards that end, I decided to cite <a href="http://www.usingcsp.com/cspbook.pdf">Tony Hoare’s classic CSP paper</a>. Other funny aspect of that: I started my implementation using channels, but ended up using mutexes instead.</p>
<p>Anyway, today, a new contributor <a href="https://github.com/rust-lang/rust/pull/25575">spotted a typo</a>. The PR message made me smile:</p>
<blockquote>
<p>Looks like this was an issue in the source material. I’ve let the editor know and he has told me he is correcting it there as well.</p>
</blockquote>
<p>In my own small way, a thing I did led to someone else improving one of the most classic and well-known CS papers ever written. And it’s just Monday.</p>
</content:encoded></item><item><title>Hope</title><link>https://steveklabnik.com/writing/hope/</link><guid isPermaLink="true">https://steveklabnik.com/writing/hope/</guid><pubDate>Fri, 08 May 2015 00:00:00 GMT</pubDate><content:encoded><p>“The people are unstoppable / Another world is possible”</p>
<p>This has long been my favorite protest chant. For me, it gets to the heart of the matter: another world <em>is</em> possible. If I’m honest with myself, most days, I don’t really believe that the world will get better. We often present history as a kind of positive progression: the world has only been getting better, more tolerant, and less poor as the years go by. I don’t <em>feel</em> that, though. What I see is imperialism, consolidation of power, and rampant racism/sexism/etc. Liberalism presents itself as an end of history, the final stop on the line. That’s unacceptable to me, but I usually feel like the only one.</p>
<p>The start of this post sounds depressing, but I’m only writing it because I actually feel positive, invigorated. I’m writing this post from the Barcelona airport, about to board the plane home. It took me a while to figure this out, but what I realized is this: getting out of the US every once in a while really helps me feel better. This isn’t to say that things are actually <em>better</em> in other places, that’s not my point. My point is that It’s not something like “European-style parlamentary democracy is better than a bicameral legislature” or something like that, just look at the UK election results. It’s that the street signs are different. The money isn’t the same. Familiar foods have changes. I see 漢字 instead of the Latin alphabet. Whatever is unusual about whatever part of the world I am in, it’s the smallest differences that have the largest impact on me and my state of mind.</p>
<p>When I go somewhere else, I can <em>feel</em> the difference. And difference is possibility. And possibility is hope, for a better world, for a better humanity.</p>
</content:encoded></item><item><title>An introduction to economics under capitalism</title><link>https://steveklabnik.com/writing/an-introduction-to-economics-under-capitalism/</link><guid isPermaLink="true">https://steveklabnik.com/writing/an-introduction-to-economics-under-capitalism/</guid><pubDate>Tue, 10 Mar 2015 00:00:00 GMT</pubDate><content:encoded><p>The dismal science a difficult one. Many people have strong opinions about economics, and it determines many of their other political beliefs. As someone who has significantly different thoughts on this topic than many of my peers, I’m writing this post out to give a basic explanation as I see it. That doesn’t mean I’m <em>inventing</em> something here, just explaining the model that I currently prefer. I’ll give sources at the end, if you want to dig deeper. Speaking of ‘deeper,’ this is also an <em>introduction</em>, one could easily write a book or three on this topic. I’ll try to be clear where I’m making an assumption, and why I’m making it.</p>
<p>Finally, this is very much a draft, I may make small, or even maybe major errors. Nobody’s perfect. My spelling is terrible, and there’s a bug where my browser spellcheck isn’t working… anyway.</p>
<hr />
<p>Let’s start from the beginning. And that beginning starts with stuff. People trade things between them. Let’s be a bit more precise: people trade commodities. They trade commodities becuase they want something they do not currently have. In other words, if I’m hungry, and you have some $FOOD, and you’re cold, and see I have a coat, you might trade me your food for my coat and we’d both be better off. This ‘commodity’ is distinct from any useful thing because we trade for it, and because it satisfies some sort of need.</p>
<p>But how do we know how much food we should trade for a coat? What does that even mean? To make this trade equal, we need to settle on a value for food, and a value for coats. That value has two different parts to it. The first: that value represents how much something satisfies our needs. Something that’s really useful will have a lot of value to us. Something that’s not may have less value. We call this component of value ‘use value.’</p>
<p>The second component is more complicated. It’s called ‘exchange value,’ and it’s called that becuase this is the actual value that we trade each other for. These two kinds of value are distinct, but the distinction is important. While I may find a coat midly useful, and therefore, its use value is low to me, that doesn’t mean that if I trade it to someone else, they will agree in how useful it is. We might have to sell this coat for more or less than we expected to. While the usefulness affects the exchange value in a sense, it doesn’t mandate it.</p>
<p>Let’s give a more concrete example of this. If I can trade my single coat for four $UNITs of $FOOD, then the exchange value of one coat is four $FOODs, and the exchange value of a $FOOD is one quarter of a coat.</p>
<p>So if the exchange value of a commodity determines how we trade, and that’s based on a subjective preference for usefulness. So if I told you there was some underlying constant that did determine exchange value, you probably wouldn’t believe me. This is, in fact, one of the root areas in which economists disagree with me. Their first mistake. They claim that this exchange is entirely based on preference, with no underlying reasoning. I (and others) disagree. Consider the equation,</p>
<pre><code>4 * c = 1 * f
</code></pre>
<p>where <code>c</code> is coats and <code>f</code> is food. If we wanted to reduce these equations, we could end up with two variants:</p>
<pre><code>(4 * c) / 1 = f
c = (1 * f) / 4
</code></pre>
<p>Because <code>c</code> and <code>f</code> are defined in terms of each other, it’s hard to get at an underlying property of both things.</p>
<p>Now, remember why we’re trading food for coats: they satisfy some sort of need. Therefore, these equations make sense only in a certian scope, if you will: when we’re considering how to turn some sort of commodity into the satisfaction of our need. This is what produces this relationship between food and coats. But we have a layer of abstraction here: as we said, the exchange value isn’t the actual, underlying use values. Like all abstractions, this one leaks. We’ll get to the leak later.</p>
<p>But in order to turn some food into a satisfied need, you need to do some kind of labor. First of all, this food had to actually be produced by someone. We’re not ordering raw materials, we’re ordering a finished product. Someone, somewhere, had to obtain these raw materials, and then apply themselves. And if you’re buying raw materials, someone had to do the labor to cut down the trees, or mine the ore.</p>
<p>This is an area where I’m going to make a slight hand-wave for now, and argue that this aspect, the labor that’s needed to make and consume a commodity, is the underlying aspect missing from the equation. It might be a little bit of labor, it might be a lot of labor, but somewhere, someone is doing some kind of labor. We can make a better argument here, but for now, I’m just going to leave it as is, because I think it makes intutive sense. Many, many people go to work, and labor to produce things that get exchanged for many other things. The common thread is that we all work. We can dig into a more rigorous argument over this point later.</p>
<p>In other words, we’re trading an abstraction over people’s labor. When I say “I’ll trade you a coat for four food,” I’m making that estimate on how much labor I would need to aquire a coat, and estimating how much to get four food, and deeming them equal. But we’re not trading the hours directly: we’re trading objects. We objectify our labor, and then trade that.</p>
<p>Now, you might have a problem with this. And many economists do. However, their arguments are usually based on a misunderstanding of this point, which is a very important one: If value is based on how many hours of labor someone puts into something, than isn’t a slow worker making more valuable stuff than a fast worker? How does that make any sense? And if I were to do some labor that’s useless, like for example, I dig around in my backyard and make a bunch of pies out of the mud, I did some labor, and therefore, shouldn’t I get paid?</p>
<p>Early economists such as Adam Smith often had this problem with their models. And it’s indeed a problem. But there’s a solution: we’re not talking about just any old labor. We’re only talking about labor that produces something useful. “Useful” in this case is decided by the particular society in which this person is doing this labor. A carriage maker in 1890 does a lot of productive labor, but a carriage maker in 2015 does significantly less.</p>
<p>Our revised statement: the common thread between all of the commodities that get exchanged is that somewhere, sometime, someone had to do some sort of labor that was socially useful to produce that commodity. We exchange commodities so that we can gain the commodities we need to fulfill our desires. We exchange them at a ratio where they’re equal in terms of how much labor went into making them, a fair trade.</p>
<p>This leads to another problem: how do we qualify ‘how much labor’? What is this unit and magnitude? Well, many people have a 40 hour week, so naturally, we gravitate towards ‘hours’ as an answer for this question. Hours are a unit of time, so we can quantify our exchange values as ‘socially neccesary labor time.’ Time, becuse we work in hours. Labor, because that’s what we’re doing. “Socially neccesary” to explain the context of only labor that’s doing something society deems useful.</p>
<p>But back to one of the original problems: If I’m bad at making coats, and I take a hundred hours to make a coat, and you’re good at making coats, so it takes you an hour, is my coat worth one hundred times yours? That doesn’t make any sense. So we’re going to make an assumption. When we use ‘socially neccesary labor time’ in this way, we mean that it’s an average hour, by an average skilled worker, across all of society. Because averages are all you can really reason about here, it’s impossible to compare each individual worker. It’s an approximation. And understanding that it’s just an approximation has led to many successful businesses, as they’re more efficient than the average, and therefore, make more money. For now, we’re going to set that aside, however. We’ll get there.</p>
<p>So! To recap, we have needs and desires, and to fulfill those, we need certain commodities, which we can aquire through exchanging commodities we have with others in our society who have the commodities we want. We make equal trades, in a ratio roughly determined by the average amount of time it would take an average worker in our society to make the thing.</p>
<hr />
<p>Exchange values are a bit unweildy, though. For one thing, there’s a ton of them: they explode in a <code>O(n^2)</code> fashion. If we have coats and food, each commodity has a single exchange value, but if we add just one more item, each commodity can be traded for two others, and therefore has two exchange values, times the three commoddities, meaning our system has six exchange values, five more than before. Adding a fourth means each of our four commodities has three possible trades, meaning twelve exchange values, six more than before. So what’s a society that’s trying to produce use-value to do?</p>
<p>We can add more things to our equation. We have coats and food in our economy, let’s add a third, and call it ‘coins.’ We’ve established that one coat is worth four food, and let’s say that one coin is worth two food. This means that four food is worth two coins is worth one coat:</p>
<pre><code>4 * food = 2 * coins = 1 * coat
</code></pre>
<p>In another sense, one coat is worth two coins is worth four food:</p>
<pre><code>1 * coat = 2 * coins = 4 * food
</code></pre>
<p>We have coins in the middle of both of these equations, which means that we can express the two halves in one system:</p>
<pre><code>1 * coat = 2 * coins
4 * food = 2 * coins
</code></pre>
<p>And when we add another commodity to our equation, we can just say what it’s worth in coins, rather than what it’s worth in both coats and food. This makes it significantly easier to conduct trade, as we only have to know what our commodities are worth in terms of this particular commodity. This commodity plays a special role in our little society: it’s the commodity that we use to determine the rates of all the other commodities. The vocabulary term for this is ‘the money commodity.’ And we call the exchange value of a commodity to the money commodity a ‘price.’</p>
<p>At this point, we’re going to take a small detour. Because our commodity is an abstraction of socially neccsary abstract labor time, I want to talk for a moment about how that abstraction leaks. Becuase objects fulfill our needs, and trading them is such an important part of society, we place a ton of emphasis on this abstraction. In other words, we love to buy things. Why does this happen? What’s the weird obsession, or some might say, fetish, with commodities?</p>
<p>Let’s say you’re a cobbler, and you make shoes. Hopefully, you’re proud of the shoes you make, and so, to some degree, because you’ve put hard labor into those shoes, you see them as a reflection of your own self. And when every commodity is priced equivalant to the money commodity, it’s really easy to see that your shoes trade at $50, but mine may trade at $45. This means you make better shoes, and so you have a higher social standing than I do. Your labor is more socially useful. This sort of collective desire we all have that sets the price of commodities, and can make us quite obsessed. And not just in the classical materialist way, but in a way that shapes our worldview.</p>
<p>You can’t take a microscope to a coat and observe that the coat costs $2. There’s no ‘money atom.’ It’s not some inherent property of coats. It’s something that our society has created as a basic assumption. But it doesn’t have to be that way, we’ve just chosen it.</p>
<hr />
<p>Let’s talk a bit more about exchanging stuff. Let’s use the previous exchange as an example, I have a coat, you have four food. But now, we don’t know each other. What do we do? Well, I can take my coat to a market, and sell it for two money. You can take your food to market, and sell it for two money. I can then buy your food for two money, and you can buy my coat for two money. We’ve exchanged our food and coats succesfully. We both completed a trade in the same way:</p>
<pre><code>commodity -&gt; money -&gt; commodity
</code></pre>
<p>or</p>
<pre><code>C -&gt; M -&gt; C'
</code></pre>
<p>We both had a commodity, we traded it for the money commodity, and then we traded that money commodity for a different commodity. There’s two sides to these trades: a commodity for the money commodity, and then money commodity for the commodity.</p>
<p>It’s important we’re making an assumption here, but it’s one that virtually all economists make: we’re assuming that both of us know the value, and that the intermediary who is holding the money agrees. This simplifying assumption is also a place where many business opportunities are formed, charging a fee to facilitate transactions like this. See the pattern yet?</p>
<p>That’s also something new: there’s an intermediate here, ‘the market.’ We don’t have to know each other, we just have to know the market exists. Software engineers would say that by lowering coupling, we enable scale. When everyone doesn’t need to know each other, we can trade a lot more things. This is another reason why the money commodity is so damn useful.</p>
<p>Anyway, it’s important to recognize that this is four total exchanges, which form two little circuts, which form our overall economy here. It’s a system, like any other. And so we can model it like a system. This is something that we virtually all agree upon. But this assumption is treacherous too: we assume that both ends of the trade always happen. But there are four trades here, and only some of them may occur. This causes disruption in the system, which throws it into crisis. We’ll talk about this case more later.</p>
<p>A moment ago, I mentioned a business model for charging for transactions. This is a very natural reaction to systems like these: by intervening in each trade, by being the mediator, you can argue that you are doing socially neccesary work, and therefore, should be paid in compensation.</p>
<p>Furthermore, since the money commodity mediates all exchange, there’s lots of good reasons to control said mediation. This is why states print currency and then ensure that transactions occur in said currency, as it gives them control over all exchange, and therefore, of what is ‘socially neccesary.’</p>
<p>To summarize up to this point: people have needs. To satisfy those needs, some people do labor to make commodities, which have the ability to satisfy those needs. We call one of these commodities ‘money’, and people trade commodities for it, and it for other commodities. This gets them the things they want. The rate that all of this is exchanged at is based in how much time it takes to make, and how important it is to the society overall.</p>
<p>I have yet to say what any of this has to do with <em>capitalism</em>, however. This is just people trading things. Money, and exchange, have been around longer than capitalism. It’s something a bit different. Before we can get into that, though, we have to define the ‘capital’ in ‘capitalism.’</p>
<hr />
<p>There is a distinction between ‘money’ and ‘capital.’ Capital is a special kind of money. In other words, it holds a special, social distinction about how it’s used. Let’s examine our little trade again:</p>
<pre><code>commodity -&gt; money -&gt; commodity
</code></pre>
<p>Someone has to actually have that money in the middle. To simplify our story, let’s say this shop both buys and sells various commodities. I sell my coat to the shopkeeper, you sell your food to the shopkeeper, then I buy food from the shopkeeper, and you buy a coat from the shopkeeper. From the shopkeeper’s perspective, these trades are backwards:</p>
<pre><code>money -&gt; commodity -&gt; money
</code></pre>
<p>The keeper has four dollars. They give two to me in exchange for the coat, and then they sell the coat for two dollars on the other side. Their circut here is the opposite of yours and mine: they start and end with money, rather than with another kind of commodity.</p>
<p>It’s important to note that this difference is significant. While you are attempting to exchange a commodity for another to gain some kind of use, with the <code>M -&gt; C -&gt; M</code> variant, you’re attempting to gain more exchange value. This underlying need is at odds with yours and mine. No concern is given for use here, just how much something can be exchanged for.</p>
<p>However, as we know, a shopkeeper isn’t exactly happy when they start off with four dollars and end up with four dollars. While this system is balanced, they’re not being compensated for this social function, one of shopkeeper. And in the real world, shopkeers do make money, so something is missing from our model of equal exchanges. But before we get into that, consider the implications of this point: the shopkeeper has no concern for what is useful, just for what can be exchanged. If we optimize for the needs of shopkeepers, we may not be optimizing for the usefullness of society, but instead for the pursuit of exchanging the money commodity.</p>
<p>So, if we can engineer a situation in which we have some money, we exchange it for a commodity, and then we exchange that commodity for money, and we get <em>more</em> money out of it in the end, that extra money is called ‘capital’. In line with our earlier equations:</p>
<pre><code>M -&gt; C -&gt; M'
</code></pre>
<p>The difference between <code>M'</code> and <code>M</code> is how much money we’re making. This is very close to our <code>M -&gt; C -&gt; M</code> circut, and very different than our <code>C -&gt; M -&gt; C</code> circut. Only shopkeepers produce capital, not those who are simply exchanging a commodity in the hopes of attaining another.</p>
<p>But we’ve already established that all trades here are equal. So how can we get <code>M'</code>? Shoudn’t that be impossible?</p>
<p>This is where the mistake we mentioned before rears its ugly head. If all value is subjective, and exchanges are between equals, where can this extra money come from? And if all of these exchanges are equal, then why do both people feel like they’ve won?</p>
<hr />
<p>In order to make this happen, our prospective capitalist shopkeeper needs to find a commodity whose use-value, the act of consuming the commodity, produces value. If they can, they can purchase that commodity, and then use it to get more value than it was exchanged for, due to the extra production. In other words, they don’t need to purchase an object, but something more abstract: a potential source of value.</p>
<p>Previously, we mentioned that the common thread behind exchanges was labor, and so, that labor was the common element of value. Labor, when placed on a market, is called ‘labor power,’ and is a commodity that produces value. We call someone who possesses labor power a worker. They enter into the market, selling their labor power. This is colloquially known as ‘getting a job.’ In order to be willing to sell your labor on the market, though, you need to have some sort of need that is fulfilled. If you don’t need anything, there’s no reason to get a job. Furthermore, you can’t just do work yourself, you need objects to do your work, that will allow you to produce something. We call these objects the ‘means of production.’ A worker also needs enough commodities to keep themselves alive, to work another day. We call this the ‘means of subsistence.’ It’s also important to note that if a worker has other commodities, they could sell those to fulfill their needs, rather than selling their labor power. So someone selling their labor-power in the market needs something, but doesn’t have anything. The only thing they have is the labor-power their body possesses.</p>
<p>It’s important to note that “person with a pile of money” and “person possessing nothing but their ability to work” is not the state of nature. In other words, this stuation arises due to the way we’ve organized society. We like to think that this existance is somehow ‘natural’ or inevitable, but that’s not true. This doesn’t mean older forms of organization are better than what we have now, but the possiblity for change exists. Some people try to paint a picture that those who have the piles of money are virtuous, and those who have only their labor are at fault for their predicament. But the predicament wouldn’t even exist in the first place if not for certian historical developments.</p>
<p>So, if you’re trading labor-power in a market, how do you determine how much to sell it for? Like any other commodity, its value comes down to a socially neccesary labor time. In this case, it’s the amount of money you need for subsistence, the money to pay your rent, to eat food, to stay alive. Having some more is nice, but if you don’t go to work, you can’t pay your bills, and then bad things happen.</p>
<p>One odd thing about labor-power as a commodity is that you don’t just hand it over in the way that you hand over a coat. You put the work in first, and then, eventually, the thing you made gets sold at a totally different time. There’s a gap. Often, you don’t actually get to see the fruits of your labor, as someone else is actually doing the sales. This can lead to some angst, but I just want to make mention of this, we’ll talk more about it later.</p>
<hr />
<p>This commodity, labor-power, allows us to create the trick. Here’s how the story goes:</p>
<p>Let’s change our example slightly: a coat costs two coins, and four spindles of thread costs two coins:</p>
<pre><code>1 * coat = 2 * coin
4 * thread = 2 * coin
</code></pre>
<p>It takes two spindles of thread and an hour to make a coat. By this measure, given that a coat costs two coins, you’d imagine that cost of materials is a coin, so labor must be a coin. Which means an hour of work is worth a coin. if this is true, then the exchange looks like this:</p>
<p>Someone wants a coat. They’re willing to pay two coins. A prospective capitalist sees that the cost of materials is a coin, so they set the price of labor-power to one hour per coin:</p>
<pre><code>1 * coat = 2 * coin
2 * thread = 1 * coin
1 * hour of work = 1 * coin
</code></pre>
<p>I spend my one hour of labor and two threads to make a coat, which costs two coins and is sold for two coins. We’re back at an equilibrium. There’s still no capital being created here.</p>
<p>Now the capitalist is upset. “I provided the job, I provided the thread, yet I made no money. This worker couldn’t have made that coat without me. I deserve to be paid for having the initial capital to set up this business. That’s work!”</p>
<p>But here’s the thing: if I worked for half a coin instead of a full coin for an hour, and I worked eight of those hours a day, I’m still making more than enough money to pay my bills. Because that’s the outcome I’m shooting for, I’m happy to exchange my labor on the market for less than the labor time would be: I’m making a good salary, I can live a good life. So the <em>exchange value</em> of my labor-power is less than the value I create from it. It’s not that I’m being cheated with my wages, it’s just that the exchange value of my labor is worth less than the labor I create. With this set of equations:</p>
<pre><code>1 * coat = 2 * coin
2 * thread = 1 * coin
2 * hour of work = 1 * coin
</code></pre>
<p>the capitalist can extract that difference. He takes his initial four coins, invests two of them in four threads. He then invests a coin in two hours of a worker’s labor. That worker then uses the threads and their labor-power to make two coats. The capitalist then sells the two coats for four dollars. Looking back, he’s seemingly played a trick: he invested three coins, two in threads and one in labor, but gained four coins at the end, for the two coats.</p>
<p>We haven’t broken any rules of exchange here. The capitalist has paid full market price for all commodities, there’s no inherent need to pay less. Each exchange is between understanding, consenting parties. There’s no trickery. Yet an extra coin has moved from one party to another, the worker to the capitalist, just by following the rules.</p>
<p>Once you have a machine that you can put three dollars in and get four dollars out, you can live off of this extra, rather than doing labor yourself. By having the initial capital of three dollars, and by using it in this way, a capitalist can just make money out of thin air. As they say, it takes money to make money. This is why every worker isn’t a capitalist, you need to create this capital somehow, to be able to afford to use the machine. A machine that makes four dollars out of three isn’t much use if you dont have the inital three dollars.</p>
<hr />
<p>In summary: people come together at a market to exchange commodities to fulfill their needs. They exchange these commodities with each other in equal values. One of the commodities which can be exchanged is labor-power, which is a special commodity, because it creates more value. Capitalists purchase this commodity, and then use it to create capital, which they can re-invest to continue to accumulate money. Those who sell their labor-power on the market must because they don’t have the capital to do this themselves.</p>
<p>Now that we’ve established that this system is off-balance, we can figure out how that will eventually become its undoing.</p>
<hr />
<p>Anyway, this is the beginning. There’s obviously much more detail to go into here. If you’d like to read more, this post is basically my own little summary of <a href="https://www.marxists.org/archive/marx/works/1867-c1/">Capital, Volume I</a>, chapters 1-7, by Karl Marx. These basic building blocks of understanding are not without their own problems, but hopefully, I’ve given you a foothold into the topic. I may or may not do this for additional chapters, we’ll see….</p>
</content:encoded></item><item><title>A new introduction to Rust</title><link>https://steveklabnik.com/writing/a-new-introduction-to-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/a-new-introduction-to-rust/</guid><pubDate>Fri, 27 Feb 2015 00:00:00 GMT</pubDate><content:encoded><p>Lately, I’ve been giving a lot of thought to first impressions of Rust. On May 15, <a href="http://blog.rust-lang.org/2015/02/13/Final-1.0-timeline.html">we’re going to have a lot of them</a>. And you only get one chance at a first impression. So I’ve been wondering if our <a href="http://doc.rust-lang.org/intro.html">Intro</a> and <a href="http://doc.rust-lang.org/book/basic.html">Basics</a> are putting our best foot forward. At first I thought yes, but a few days ago, I had an idea, and it’s making me doubt it, maybe. So instead of re-writing all of our introductory material, I’m just going to write the first bit. A spike, if you will. And I’d like to hear what you think about it. This would take the same place as <a href="http://doc.rust-lang.org/book/variable-bindings.html">2.4: Variable bindings</a> in the existing structure: They’ve installed Rust and gotten Hello World working.</p>
<pre><code>$ rustc --version
rustc 1.0.0-dev (dcc6ce2c7 2015-02-22) (built 2015-02-22)
</code></pre>
<h2>Hello, Ownership</h2>
<p>Let’s learn more about Rust’s central concept: ownership. Along the way, we’ll learn more about its syntax, too. Here’s the program we’re going to talk about:</p>
<pre><code>fn main() {
let x = 5;
}
</code></pre>
<p>This small snippit is enough to start with. First up: <code>let</code>. A <code>let</code> statement introduces a <strong>variable binding</strong>. Bindings allow you to associate a name with some sort of value.</p>
<p>Why ‘variable binding’? Rust draws heavily from both systems languages and functional programming languages. The name “variable binding” is a great example of this. Many systems languages let you declare a variable. These variables are called by that name because they can change over time, they’re mutable. Many functional languages let you declare bindings. These bindings are called by that name because they bind a name to a value, and don’t change over time. They’re immutable.</p>
<p>Rust’s variable bindings are immutable by default, but can become mutable, allowing them to be re-bound to something else. In other words,</p>
<pre><code>fn main() {
let x = 5;
x = 6; // error: re-assignment of immutable variable
let mut y = 5;
y = 6; // just fine
}
</code></pre>
<p>You won’t be typing <code>mut</code> that often.</p>
<p>In any case, there’s one way in which <code>let</code> bindings work just like variables in other languages, but they’re the key insight into ownership. As you know, a computer program is executed line by line. At least, until you hit a control flow structure, anyway. Let’s give our program line numbers, so we can talk about it.</p>
<pre><code>// 1
fn main() { // 2
// 3
let x = 5; // 4
// 5
} // 6
// 7
</code></pre>
<p>Line one is before our program starts. The endless void. Not actually endless, though, as line two is where we start <code>main</code>. This is the first line that is actually executed, and kicks off our program. Great. Line three is blank, so nothing happens, just like one. Line four is where the first actually interesting thing occurs: we introduce a new variable binding, <code>x</code>. We set <code>x</code>’s initial value to five, allocated on the stack. If you don’t know what that means, we’ll talk about it right after ownership. For now, <code>x</code> is five. No big deal. Line six has a closing curly brace, and so <code>main</code>, and thus, our program, is over. Line seven, the void returns.</p>
<p>This is basically the same thing as many programming languages. But let’s point out an aspect you, as a programmer, probably take for granted. Scoping. If I asked you, “Is <code>x</code> valid on line one?” you would say “no.” “Three? Seven?” “Nope, nada. <code>x</code> is valid from line four, where it was declared, to line six, where it goes out of scope.” This illustrates the idea. There is a certain scope, a certain set of lines, where <code>x</code> is a valid identifier. That scope starts from where it was declared, and goes until the end of the block. We can look at this scope in two ways: for the first, imagine this source code printed on a piece of paper. You highlight lines four three six. In some sense, this is a distance: three lines of code, rather than three meters. But if we imagine the computer running this program, this scope represents a time: three statements of processor execution. Even though that number is actually different based on the assembly, but at our level of abstraction, three units.</p>
<p>In Rust, we have names for these concepts, which are implicit in other languages. The thing that introduces a new scope is called the ‘owner.’ It’s in charge of the data it’s bound to, so we say that it ‘owns’ that data. The length of a scope is called a ‘lifetime,’ taken from that idea of time passing as your program executes. But you can also think of it as a segment of lines of code.</p>
<p>So if other programs do this, why does this make Rust special? Sure, Rust gives these concepts specific names, but names themselves aren’t significantly different. The difference is that Rust takes this concept and cranks it up to 11. Where most programming languages only keep track of how long variables are in scope, Rust knows how to connect the scopes of variables that are pointing to the same thing, as well as how to know the scope of things that are more than just stack-allocated memory.</p>
<p>Let’s talk more about this connecting of scopes. Here’s another Rust program:</p>
<pre><code>fn main() { // 1
let x = 5; // 2
// 3
{ // 4
// 5
let y = &amp;x; // 6
// 7
} // 8
// 9
} // 10
</code></pre>
<p>In line four, we use an open curly brace to create a new scope. This scope, like in many languages needs to be closed before the scope of <code>main</code> gets closed. It’s nested inside of it.</p>
<p>In this new scope, on line six, we declare a new binding, <code>y</code>, and we set it equal to <code>&amp;x</code>. This reads as “a reference to <code>x</code>,” as <code>&amp;</code> is the reference operator in Rust. References are Rust’s version of ‘pointers’ from other systems programming languages. Here’s a quick introduction, if you haven’t had to deal with pointers before.</p>
<p>By default, in Rust, data is allocated on “the stack.” Your program is given a chunk of memory, and it’s able to allocate information there. The stack starts at the bottom address of memory, and then grows upwards. For example, at the start of our program, the stack has nothing in it:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20e35b5948ec7546b7a905f4db1ea869c6.csv">Untitled</a></p>
<p>At line2, we introduce <code>x</code>, and it stack-allocates the number five, like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20935c84280787447b8fefcaaefb4575c0.csv">Untitled</a></p>
<p>We’ve got an address in memory, the value at that address, and the name we’ve used in our program. It starts at the bottom because remember, the stack grows upward. Finally,on line six, we have <code>y</code>:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20f7b25102df1d41e884fdac5b761fe567.csv">Untitled</a></p>
<p>Instead of being a value itself, <code>y</code> is a pointer to another location in memory, which holds a value. Because <code>y = &amp;x</code>, we take the memory location of <code>x</code> and store it into <code>y</code>.</p>
<p>On line eight, the inner scope ends, and <code>y</code> goes out of scope. Its lifetime is over. It gets deallocated. Our memory is back to this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20daa49962f5c5408d82e28e9e3e073be7.csv">Untitled</a></p>
<p>Finally, on line ten, our program is over. <code>x</code> goes out of scope, and we look like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%2086c230f1a23e49d892a529d7988a272c.csv">Untitled</a></p>
<p>Everything is gone, all is well.</p>
<p>Pointers can point other places than the stack: they can also point to ‘the heap.’ The heap starts at the top of your program’s memory, and grows down:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20007add295d434d84a281a93e7358f0d5.csv">Untitled</a></p>
<p>The heap starts at <code>0x00</code> and grows down, and the stack starts at <code>0xff</code> and grows up. While the stack contains the data that the programming language has allocated for our bindings, our program can request dynamic amounts of memory from the heap. Here’s how to do this in C:</p>
<pre><code>#include&lt;stdio.h&gt;
#include&lt;stdlib.h&gt;
int main() {
int *x = malloc(sizeof(int)); // 1
*x = 5; // 2
// 3
printf("%i", *x); // 4
}
</code></pre>
<p>The call to <code>malloc</code> (‘memory allocate’) on line 1 requests some memory from the heap. We get a pointer, <code>int *x</code>, to that memory. Our memory looks like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%203e8198d0b0a5408bbcf399a9d682c779.csv">Untitled</a></p>
<p>We have <code>x</code>, which is a variable on the stack, as a pointer to our dynamically allocated memory, which is located on the heap at <code>0x00</code>. But at that location, we haven’t done anything, so who knows what the value is? This means that after line one is done executing, <code>x</code> will be pointing to garbage memory. Hopefully we set it equal to a value before we try to use it, or bad things could happen.</p>
<p>On line two, we set that memory location to five. Whew! Memory looks like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20ccd2c85eb0a34f6483acdbfafa4c841a.csv">Untitled</a></p>
<p>Everything looks good. On line four, we have to print out <code>*x</code>, not <code>x</code>: <code>x</code> is the address of the memory, so we’d get <code>0x00</code> instead of <code>5</code>. After line four, <code>x</code> goes out of scope, and gets deallocated. Memory looks like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%200aec19c2d6a645a3b860bf65cc922853.csv">Untitled</a></p>
<p>… wait a minute, we never took care of that <code>5</code>! We left some memory ‘dangling.’ In this program, that’s okay, because now that it’s over, the operating system reclaims all this memory, and so we don’t have any problems. But we <em>do</em> have a bug. We just don’t know it. This bug will sit there until something changes, and the bug is found. But we’ll get to that.</p>
<p>A slightly more responsible program looks like this:</p>
<pre><code>#include&lt;stdio.h&gt;
#include&lt;stdlib.h&gt;
int main() {
int *x = malloc(sizeof(int));
*x = 5;
printf("%i", *x);
free(x);
}
</code></pre>
<p>The call to <code>free</code> releases the memory pointed to by <code>x</code>, and all is good.</p>
<p>Let’s compare this to an equivalent Rust program:</p>
<pre><code>fn main() {
let x = Box::new(5);
println!("{}", x);
}
</code></pre>
<p>It looks similar at first, with <code>Box::new</code> filling in for <code>malloc</code>, But where’s the free? Does this have the same bug?</p>
<p>No. This Rust is in fact equivalent to the one with the <code>malloc</code> and <code>free</code>. You see, when we introduce our binding, <code>x</code>, Rust knows that <code>x</code> has ownership over the memory in the <code>Box</code>. and so, at the end of the program, when <code>x</code> goes out of scope, Rust also ensures to free the memory that the <code>Box</code> allocated. It happens automatically. The programmer can’t forget it. And the allocation itself was a little easier, because Rust used the type system to figure out how many bytes to allocate, you didn’t need to calculate it with <code>sizeof</code>. Allocation also set the value to 5 for you, which is convenient and less error-prone. When printing out <code>x</code>, you could just use the binding name itself, and not need to dereference it with <code>*</code>.</p>
<p>C++ programmers have ways of doing this kind of thing, with something called “RAII.” As promised before, Rust knows this concept at a deeper, language level. So it can be safe in places that RAII cannot. Here’s an example, in C++:</p>
<pre><code>#include&lt;iostream&gt;
#include&lt;vector&gt;
#include&lt;string&gt;
int main() {
std::vector&lt;std::string&gt; v;
v.push_back("a");
std::string&amp; x = v[0];
v.push_back("b");
std::cout &lt;&lt; x;
}
</code></pre>
<p>This program creates a vector of strings, called <code>v</code>:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20b1e4993d4fc14c9abbe2c2d7deb3be25.csv">Untitled</a></p>
<p><code>v</code> is a ‘fat pointer,’ three pointers in one. The first is a pointer to the data, the second is the length of the array, how many elements are in it. The third is the capacity of the array, how much memory is allocated for this vector. The amount of initial capacity is up to the implementation, I’ve chosen one here. Because this is a vector of strings, we need to allocate a single string, so in line with our capacity. Strings are also a fat pointer, and so we allocate three bytes, starting at <code>0x00</code>.</p>
<p>The program then calls <code>push_back()</code> on the vector, which pushes a string onto the end of the vector:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%205c80884c8d7d4ae290de9e2b9caa1f11.csv">Untitled</a></p>
<p>We’ve now replaced our initial, <code>GARBAGE</code>, string with a real one. Therefore, our data, length, and capacity are updated: <code>0x03</code> is the next free location in the heap, so it gets our signle character, <code>A</code>. We have a length of one, and a capacity of one, so the other two values are set appropriately.</p>
<p>Next, it creates a pointer to the <code>0</code>th element of that vector:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%2031184db665934f5ea7acaea0b98ce4b9.csv">Untitled</a></p>
<p><code>x</code> is allocated on the stack, and points to the first element of <code>v</code>, which, as we know, is located at address <code>0x00</code>, as its data pointer, <code>0xff</code>, shows us. So <code>x</code> has the initial value of <code>0x00</code>.</p>
<p>Next is where everything goes wrong. It tries to push a string onto the end of that vector. The problem is, we only allocated one element. So we need to allocate more. After this is done, memory looks like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20900c132e9e144b1e94ffb6c47d5fc068.csv">Untitled</a></p>
<p>Inside <code>push_back()</code>, there’s a call to <code>malloc</code>, and that is assigned to some temporary value. We’ve chosen <code>0xfb</code> and <code>(temp)</code> to identify it here. It points to the newly allocated data, at <code>0x04</code>.</p>
<p>Now that we’ve got space for two strings, let’s copy the data from the original place to the new one, from <code>0x00</code> to <code>0x04</code>:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%208311f6138fd34b39840f66e8b047db11.csv">Untitled</a></p>
<p>and then free that original string:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20885b35ff100341e6819740529f0d4701.csv">Untitled</a></p>
<p>Then, we have to update <code>v</code> to point at the newly allocated vector, at <code>0x04</code>:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20ec53dd46647143b58bb3c7b136d66a59.csv">Untitled</a></p>
<p>This step is where the bug is! You see, <code>x</code> was pointing at the same data that <code>v</code> was pointing at. This is called ‘aliasing.’ Because C++ does not have the concept of ownership and lifetimes, it doesn’t understand that the validity of <code>x</code> and the validity of <code>v</code> are closely intertwined. Thus, we’re allowed to do this reallocate-and-copy dance with what <code>v</code> points to, but <code>x</code> has no idea. Look at the diagram above: it still has <code>x</code> pointing to <code>0x00</code>. But we just deallocated that memory! This is when the bug occurs, but it’s a silent bug. A time bomb, ticking away.</p>
<p>Let’s see how it explodes. Our code goes on, oblivious.</p>
<p>Our new string at <code>0x04</code> still says that we have one capacity, one length, and that data is at <code>0x03</code>. That’s all still accurate, so we’re all good. Next, we need to deal with our new string, “B.” We have the space, so we do so:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20cc5b92de5ac545878c67a66a878863b6.csv">Untitled</a></p>
<p>We’ve placed our new string at <code>0x07</code> to <code>0x09</code>, with its data pointer, length, and capacity. Our data is at <code>0x0A</code>, which has the value <code>B</code> stored at it. All is good, and so we return from <code>push_back</code>. <code>(temp)</code> is deallocated, and memory looks like this:</p>
<p><a href="A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20ee6def6517424e0fb6c9ad7c3ec0389e.csv">Untitled</a></p>
<p>On the final line, we print out the value that <code>x</code> points to. As we can see above, <code>x</code> points to <code>0x00</code>, which has nothing saved at it! <code>x</code> is invalid memory. Using <code>x</code> is a bad idea. But the programming language can’t save us, even though the bug happened many lines ago.</p>
<p>How are you supposed to deal with this problem? Here’s <a href="http://en.cppreference.com/w/cpp/container/vector/push_back">the documentation for <code>push_back</code></a>:</p>
<blockquote>
<p>If the new size() is greater than capacity() then all iterators and references (including the past-the-end iterator) are invalidated.</p>
</blockquote>
<p>In other words, when you call <code>push_back</code>, it’s your responsibility to make sure to not use anything that points to the underlying data, because they are now invalid. The documentation has explained our bug perfectly! And while this is a short example, these problems can crop up in huge codebases, and the errors can be very hard to track down.</p>
<p>Let’s look at the same problem, in Rust:</p>
<pre><code>fn main() {
let mut v = vec![];
v.push("Hello".to_string());
let x = &amp;v[0];
v.push("world".to_string());
println!("{}", x);
}
</code></pre>
<p>This program has identical semantics to the C++ version presented above. (If we weren’t trying to get the exact same semantics, we could leave off the <code>.to_string()</code> calls, and it would still work. But let’s ignore that for now.) So does it have the same bug? Well, it would, but it doesn’t compile:</p>
<pre><code>main.rs:8:5: 8:6 error: cannot borrow `v` as mutable because it is also borrowed as immutable
main.rs:8 v.push("world");
^
main.rs:6:14: 6:15 note: previous borrow of `v` occurs here; the immutable borrow prevents subsequent moves or mutable borrows of `v` until the borrow ends
main.rs:6 let x = &amp;v[0];
^
main.rs:11:2: 11:2 note: previous borrow ends here
main.rs:1 fn main() {
...
main.rs:11 }
^
</code></pre>
<p>When you call the method <code>push()</code> on a vector, it ‘borrows’ the vector. Just like we wrote <code>let y = &amp;x</code> before, to borrow the value of <code>x</code>. Rust now understands that the scope of <code>x</code> and the scope of <code>v</code> are connected. There’s just one problem: because <code>push</code> allocates more memory, it can’t just do a regular borrow, it needs a mutable borrow. If <code>&amp;x</code> borrows x, <code>&amp;mut x</code> mutably borrows <code>x</code>. Like we discussed previously, mutability means that you can reassign the value of that binding. But when we mutably <em>borrow</em> something, we make a guarantee: this is the only reference to this memory that exists. That way, we don’t have the problem of the C++ code, where a reference became outdated. If we’re the only reference, we are free to change what we’re pointing to, becuase that won’t cause any problems.</p>
<p>So when we create a borrow with <code>&amp;</code>, it lasts for the scope that it exists, like this:</p>
<pre><code>fn main() {
let mut v = vec![];
v.push("A");
let x = &amp;v[0]; // &lt;+
// |
v.push("B"); // |
// |
println!("{}", x); // |
} // &lt;+
</code></pre>
<p>The line indicates the scope, or lifetime, of the borrow. But when we call push with <code>B</code>, we have to get a mutable borrow, in order to allocate the new memory. But because <code>&amp;mut</code> is a promise that there are no other references, this is a violation of the rules. This is the point at which the bug happened in the C++ example, if you remember. And so it’s the place where the Rust compiler throws its error message. It might make more sense this time around:</p>
<pre><code>main.rs:8:5: 8:6 error: cannot borrow `v` as mutable because it is also borrowed as immutable
main.rs:8 v.push("world");
^
main.rs:6:14: 6:15 note: previous borrow of `v` occurs here; the immutable borrow prevents subsequent moves or mutable borrows of `v` until the borrow ends
main.rs:6 let x = &amp;v[0];
^
main.rs:11:2: 11:2 note: previous borrow ends here
main.rs:1 fn main() {
...
main.rs:11 }
^
</code></pre>
<p>We cannot borrow <code>v</code> as mutable, with <code>push()</code>, becuase it is also borrowed as immutable, by <code>x</code>. That’s exactly the problem, and Rust solves it.</p>
<p>There’s more to both ownership and Rust syntax, so let’s cover some syntactical things, and then talk more about ownership.</p>
<hr />
<p>I’m fairly confident with the memory layouts involved, though the addresses are of course not exact. I’m also not sure that this mega deep detail dive is the correct way to do an introduction either, it’s also thorough, but maybe in a bad way? It’s almost the exact opposite of the way that I’ve done things at present. Also, apparently I visuallize things in reverse, with <code>0x00</code> to <code>0xFF</code> as going down, but others think the opposite, which may be confusing. Please send me a <a href="http://twitter.com/steveklabnik">tweet</a> or <a href="mailto:[email protected]">an email</a> with your thoughts. Thanks!</p>
</content:encoded></item><item><title>Update GitHub Pages with Travis CI</title><link>https://steveklabnik.com/writing/update-github-pages-with-travis-ci/</link><guid isPermaLink="true">https://steveklabnik.com/writing/update-github-pages-with-travis-ci/</guid><pubDate>Tue, 16 Dec 2014 00:00:00 GMT</pubDate><content:encoded><p>Ever wanted to have your project automatically update Pages when you have a successful build?</p>
<p>I have a demo repository here: <a href="https://github.com/steveklabnik/automatically_update_github_pages_with_travis_example">https://github.com/steveklabnik/automatically_update_github_pages_with_travis_example</a></p>
<p>It, of course, uses Travis to update its own Github Pages, which is here: <a href="http://steveklabnik.github.io/automatically_update_github_pages_with_travis_example/">http://steveklabnik.github.io/automatically_update_github_pages_with_travis_example/</a></p>
<p>I hope that helps!</p>
</content:encoded></item><item><title>I'm going writeonly on Twitter for a while</title><link>https://steveklabnik.com/writing/im-going-writeonly-on-twitter-for-a-while/</link><guid isPermaLink="true">https://steveklabnik.com/writing/im-going-writeonly-on-twitter-for-a-while/</guid><pubDate>Wed, 03 Sep 2014 00:00:00 GMT</pubDate><content:encoded><p>This is something I’ve been joking about for a while, but I’ve decided for the near future, I’m just going to be posting things to Twitter, and not reading replies or anything.</p>
<p>So, if you don’t get a fav or a reply or something, it’s nothing personal.</p>
<p>There’s a lot going into this decision, but basically, I haven’t been actually doing a lot of things that I want to do because I’m super stressed out. Turns out Twitter is also a huge cause of stress in my life. Turns out Twitter is also a large part of my social interaction, so I’ve kept doing it even though I know it’s not great for me.</p>
<p>So, we’ll see what a post-read/write Twitter is like. You know how to get a hold of me otherwise.</p>
</content:encoded></item><item><title>Rust's documentation is about to drastically improve</title><link>https://steveklabnik.com/writing/rusts-documentation-is-about-to-drastically-improve/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rusts-documentation-is-about-to-drastically-improve/</guid><pubDate>Mon, 16 Jun 2014 00:00:00 GMT</pubDate><content:encoded><p>Historically, <a href="http://rust-lang.org/">Rust</a> has had a tough time with documentation. Such a young programming language changes a lot, which means that documentation can quickly be out of date. However, Rust is nearing a 1.0 release, and so that’s about to change. I’ve just signed a contract with Mozilla, and starting Monday, June 23rd, I will be devoting forty hours per week of my time to ensuring that Rust has wonderful documentation.</p>
<p>A year and a half ago, I was in my hometown for Christmas. I happened upon a link: <a href="https://mail.mozilla.org/pipermail/rust-dev/2012-December/002787.html">Rust 0.5 released</a>. I’ve always enjoyed learning new programming languages, and I had vaguely heard of Rust before, but didn’t really remember what it was all about. So I dug in. I loved systems programming in college, but had done web-based stuff my entire professional life, and hadn’t seriously thought about pointers as part of my day-to-day in quite some time.</p>
<p>There was just one problem: Rust was really difficult to learn. I liked what I saw, but there was very little of it. At the same time, I had been working on some ideas for a new toolchain for my <a href="http://www.designinghypermediaapis.com/">book on hypermedia APIs</a>, but wanted to try it out on something else before I took the time to port the content. And so, <a href="http://www.rustforrubyists.com/">Rust for Rubyists</a> was born. I decided that the best way to teach people Rust was to mimic how I learned Rust. And so, as I learned, I wrote. It ended up at about fifty pages of two weeks of work. I never contributed it to the main repository, because for me, it was really about learning, both about my ebook toolchain as well as Rust itself. I didn’t want the burden that came with writing an official tutorial, making sure that you cover every single feature, pleasing every single Github contributor…</p>
<p>After learning Rust, I decided that I really liked it. No other language provided such a promising successor to C++. And I really missed low-level programming. So I kept evangelizing Rust, and every so often, contributing official documentation. I figured that even if my low-level chops weren’t up to the task of writing actual patches, I could at least help with my writing skills. I’d previously contributed lots of documentation to Ruby and Rails, so it was something that was very familiar to me. I’ve often found that I start with documentation and then move into contributing code, once I get my head wrapped around everything. Writing is part of my own process of understanding.</p>
<p>Rust for Rubyists was a great hit, even amongst non-Rubyists (damn my love of alliteration!). Six months ago, on the eve of the first anniversary of the initial release of Rust for Rubyists, I <a href="https://air.mozilla.org/rust-meetup-december-2013/">gave a talk</a> at the Bay Area Rust meetup, specifically on the state of Rust’s documentation. In it, I laid out a plan for how I envisioned docs looking in the future. In the last six months, a lot has improved, but a lot hasn’t. But no more! I’m now going to be able to allocate a significant amount of my time on getting documentation done.</p>
<p>I’m also pleased in a meta-sense. You see, by contracting someone to work on documentation full-time, Mozilla is indicating that they take Rust and its future very seriously. You can (and I do) talk a lot of trash on Microsoft, but one of the reasons that the Microsoft platform is used by so many people around the world is that Microsoft products often have excellent documentation. I often find that open source ‘products’ are technically superior, but are often much harder to use, because they’re built by a community, for free, and very few people want to write documentation for free. Combined with the work that Tilde is doing on <a href="https://github.com/carlhuda/cargo">Cargo</a>, Mozilla is investing a significant amount of effort and dollars into ensuring that Rust will be a fantastic platform for those developing on it. Since I love Rust, this makes me very, very happy.</p>
<p>Forty hours a week is a significant amount of documentation, and I have a lot of work in front of me. But my first area of focus will be on the area of Rust’s documentation that’s the most weak, and simultaneously the most important: the tutorial. I tackled the first tip of that iceberg with <a href="http://doc.rust-lang.org/master/intro.html">my 30 minute introduction</a>, and I’d like to tweak it too. The main tutorial, however, is the first place where people go to <em>really</em> learn about Rust and how it works. And the current tutorial is largely the same as the 0.5 days, back when I first learned Rust. It suffers from receiving patchwork contributions from a variety of people, rather than having one unifying vision. It’s also much more of a list of features and how to use them than a coherent tutorial.</p>
<p>I’m really excited to get to work. Let’s all make Rust 1.0 a fantastic release.</p>
</content:encoded></item><item><title>SF</title><link>https://steveklabnik.com/writing/sf/</link><guid isPermaLink="true">https://steveklabnik.com/writing/sf/</guid><pubDate>Tue, 20 May 2014 00:00:00 GMT</pubDate><content:encoded><blockquote>
<p>@steveklabnik &lt;small&gt;including you?&lt;/small&gt;</p>
</blockquote>
<p>— Nick Quaranto (@qrush) March 5, 2014</p>
<blockquote></blockquote>
<p>I was always not quite sure about moving to San Francisco. I’ve heard about this place for my entire life: back in high school, during the first bubble, I would read Slashdot and wonder about life out west. But San Francisco is like a vortex for software developers. It’s got this gravitational pull that’s hard to escape. When I made the choice to move here, I wasn’t sure if I’d be able to handle it. Turns out I can’t.</p>
<p>So I’m leaving. My job is great, but it’s not worth staying in this city. I haven’t done anything outside of work in a few months. I go to work, work hard, go home, and then go to sleep. I don’t go home late. I go home at five. It’s not work/life balance that’s the issue. I’m just not psyched for the ‘life’ part. San Francisco has sucked out all my creativity, most of my passion, and has made me increasingly frustrated with things. Elaborating further would just be <a href="http://tomdale.net/2013/07/san-francisco-i-love-you-but-youre-bringing-me-down/">repeating Tom Dale</a> and <a href="http://programmingisterrible.com/post/50421878989/come-here-and-work-on-hard-problems-except-the-ones">tef</a> with not much difference, so I’ll leave it at that.</p>
<p>So, at the end of the month, I’m leaving Balanced and the city. I’ll be arriving in Brooklyn, a place I’ve been trying to live for the past few years. I’m going to deeply miss my friends and my (ex-)coworkers, but I have a lot of friends in NYC, too. Every hour I spend in New York, I feel excited, and every hour I spend in San Francisco, I feel drained.</p>
<p>So here’s to new adventures. I can’t wait to be happy again.</p>
</content:encoded></item><item><title>Hypermedia FizzBuzz</title><link>https://steveklabnik.com/writing/hypermedia-fizzbuzz/</link><guid isPermaLink="true">https://steveklabnik.com/writing/hypermedia-fizzbuzz/</guid><pubDate>Fri, 02 May 2014 00:00:00 GMT</pubDate><content:encoded><p>I read a really great blog post last week: <a href="http://smizell.com/weblog/2014/solving-fizzbuzz-with-hypermedia">Solving FizzBuzz with Hypermedia</a>. In it, Stephen Mizell builds a FizzBuzz service using <a href="https://github.com/kevinswiber/siren">Siren</a>, and then shows how client code evolves. I wanted to explain exactly why I think this example is amazing, because I’m not sure it’s exactly obvious.</p>
<h2>FizzBuzz? Really?</h2>
<p>The first thing that makes this post brilliant is that it uses FizzBuzz. While FizzBuzz has historically been a simple interview question to make sure that you can actually write some basic code, in this case, it works well because it’s a very, very simple programming problem. When writing examples, there’s always tension between real-world and straw-man examples. If you make it too real-world, all sorts of incidental details creep in and distract from your main point. If you make a trivial example, it can be claimed that it works for your simple example, but not for ‘real’ examples. Now, this may be true of FizzBuzz, but what I like about it is that it’s an example that everyone is already familiar with. It has <em>just enough</em> actual computing that it straddles that line.</p>
<h2>Haters gonna HATEOAS</h2>
<p>The real insight of this post, however, is demonstrating one of the hardest concepts to grok about the hypermedia approach: the way in which hypermedia reduces duplication, by moving application entirely to the server. This is the core of that dreaded “HATEOAS” concept, and it’s where people get tripped up. Here’s the sample client in the hypermedia style:</p>
<pre><code>from siren import SirenResource as Hyperclient
BASE_URL = "http://fizzbuzzaas.herokuapp.com"
def fizzbuzz(resource):
"""
Prints the fizzbuzz value and follows "next" links
"""
print resource.properties["value"]
if resource.has_link("next"):
fizzbuzz(resource.follow_link("next"))
def begin_fizzbuzz(resource):
"""
Follows the first link, then hands off to fizzbuzz
"""
if resource.has_link("first"):
fizzbuzz(resource.follow_link("first"))
root_resource = Hyperclient(BASE_URL, path="/")
begin_fizzbuzz(root_resource)
</code></pre>
<p>Let’s talk about what this client’s goal is: to print a list of numbers. This is a classic recursive list traversal algorithm: we start at the head of the list (<code>first</code>), and then follow the links until we get to the end. (<code>next</code>) This client <em>does not actually calculate fizzbuzz</em>. It just so happens that the service we’re pointing it at calculates FizzBuzz. If we pointed it at, say, a list of search results, where the same link relations and Siren were used, it would still work!</p>
<p>Compare this to the non-hypermedia client:</p>
<pre><code>import requests
BASE_URL = "http://fizzbuzzaas.herokuapp.com"
def fizzbuzz(params):
url = BASE_URL + "/fizzbuzz"
response = requests.get(url, params=params)
response_json = response.json()
return response_json["properties"]["value"]
for number in range(1, 101):
print fizzbuzz({"number": number })
</code></pre>
<p>This client doesn’t fully calculate FizzBuzz on its own, but it does have some of the logic for doing so embedded inside. We explicitly loop over the range FizzBuzz needs, and we print out each value.</p>
<p>Why does this matter? It seems like a pretty subtle distinction. Well, this logic already exists on the server. We’re duplicating that logic here. This duplication creates coupling across the client/server boundary. For example, if we pointed this client at a “search results” resource, we would get 100 search results, even if there are less. We’re not letting the server guide our interactions, and we’re coding business logic into the client.</p>
<p>This comes back to bite us when we try to change the client. As Stephen’s next example shows:</p>
<pre><code>import requests
BASE_URL = "http://fizzbuzzaas.herokuapp.com"
STARTS_AT = 4
ENDS_AT = 20
ADD = 2
def fizzbuzz(params):
url = BASE_URL + "/fizzbuzz"
response = requests.get(url, params=params)
response_json = response.json()
print response_json["properties"]["value"]
params["number"] += ADD
if params["number"] &lt;= ENDS_AT:
fizzbuzz(params)
fizzbuzz({ "number": STARTS_AT })
</code></pre>
<p>This version uses different start, end, and skip parameters. And so we had to throw out our entire old client, because it was too tightly coupled to the notion of how to calculate FizzBuzz. We’re now manually doing the skip and end calculations, even further coupling this client. Our fundamental algorithm even changed: the last client was iterative, but this client is recursive. We could have still made it iterative, but that still would have been a lot of change.</p>
<p>Now , the hypermedia version:</p>
<pre><code>from siren import SirenResource as Hyperclient
BASE_URL = "http://fizzbuzzaas.herokuapp.com"
def fizzbuzz(resource):
"""
Prints the fizzbuzz value and follows "next" links
"""
print resource.properties["value"]
if resource.has_link("next"):
fizzbuzz(resource.follow_link("next"))
def begin_fizzbuzz(resource):
"""
Follows the first link, then hands off to fizzbuzz
"""
if resource.has_link("first"):
fizzbuzz(resource.follow_link("first"))
def custom_fizzbuzz(root_resource, params):
"""
Submits actions for custom fizzbuzz
"""
resource = root_resource.take_action("custom-fizzbuzz", params)
begin_fizzbuzz(resource)
root_resource = Hyperclient(BASE_URL, path="/")
params = { "startsAt": 4, "endsAt": 20, "add": 2 }
custom_fizzbuzz(root_resource, params)
</code></pre>
<p>Very little has changed between the last client and this one. We just added an extra method to submit our custom start point, but the rest of the code is all identical: we’re still traversing the list, with the same code. This worked because we chose the right behavior for our client, and let the server handle all of the business logic.</p>
<h2>Traversal from the root</h2>
<p>The last thing this post does is demonstrate how an actual hypermedia API starts from the API root and progresses from there. Lots of people hear this, and assume that there will be way more API calls than in an API in a different style. That’s because they assume that the client will be written in the same way as they’re used to: function calls over HTTP. As you can see here, we don’t start each call from the root of the API, we start <em>our initial interaction with the service</em> from the root of the API. Each step is just one more call after that, just like in any other API.</p>
<p>This is the other part of the hypermedia constraint that trips people up. Navigating a state machine of your API’s business process has a very different feel than making function calls over HTTP. It requires a different kind of mindset and approach to building clients. This is the area of hypermedia research that’s been least published about. The server-side story has been fleshed out, but the real frontier in hypermedia theory and practice is client building guidelines, and that’s why I like this example so much.</p>
</content:encoded></item><item><title>How to be an open source gardener</title><link>https://steveklabnik.com/writing/how-to-be-an-open-source-gardener/</link><guid isPermaLink="true">https://steveklabnik.com/writing/how-to-be-an-open-source-gardener/</guid><pubDate>Mon, 14 Apr 2014 00:00:00 GMT</pubDate><content:encoded><p>I do a lot of work on open source, but my most valuable contributions haven’t been code. Writing a patch is the easiest part of open source. The truly hard stuff is all of the <em>rest</em>: bug trackers, mailing lists, documentation, and other management tasks. Here’s some things I’ve learned along the way.</p>
<p>It was RailsConf 2012. I sat in on a panel discussion, and the number of issues open on <a href="https://github.com/rails/rails">rails/rails</a> came up. There were about 800 issues at the time, and had been for a while. Inquiring minds wished to know if that number was ever going to drop, and how the community could help. It was brought up that there was an ‘Issues team,’ whose job would be to triage issues. I enthusiastically volunteered.</p>
<p>But what does ‘issue triage’ <em>mean</em>, exactly? Well, on a project as large as Rails, there are a ton of issues that are incomplete, stale, need more information… and nobody was tending to them. It’s kind of like a garden: you need someone to pull weeds, and do it often and regularly.</p>
<p>But before we talk about how to pull the weeds, let’s figure out what kind of garden we even have on our hands!</p>
<h2>What are Issues?</h2>
<p>The very first thing your project needs to do is to figure out what Issues are supposed to be for. Each project is different. For example, in Rails, we keep Issues strictly for bugs only. Help questions go to Stack Overflow, and new feature discussion and requests go to the rails-core mailing list. For Rust, we have issues for feature requests, meta-issues… everything. For some repositories, closing all of the issues is not feasible, and for others, you’re shooting for zero. (If you don’t believe that this is even possible, check out <a href="https://github.com/jeremyevans/sequel/issues">Sequel</a>. Issues are rarely even open for more than a few days!)</p>
<p>My personal favorite is to follow the Rails way. Ideally, you’d be at zero defects, and you can still have a place to discuss features. But really, having <em>some</em> plan is a necessary first step here.</p>
<h2>Regular tending</h2>
<p>So how do you tackle 800 issues? The only way I knew how: read all of them. Yep. Here’s what I did: I took a Saturday (and a Sunday), and I went to <a href="https://github.com/rails/rails/issues?state=open">the list of open Issues</a>, then control-clicked on each one in turn to open them in a new tab. Finally, I also control-clicked on page 2. Then I closed this tab. Now I had 31 open tabs: 30 issues, and the next page. I read through the whole issue, including comments. When I got to the last tab, I was ready to repeat the process: open 30 issues, open page 3, click close. Next!</p>
<p>See, people think working on open source is glamorous, but it’s actually not. Working on open source is reading 800 issues over the course of a weekend.</p>
<p>Anyway, once I read all of those issues, I was significantly more informed about the kinds of problems Rails was facing. I had a whole bunch of common questions, comments, and problems.</p>
<p>The next step was to do it all again.</p>
<p>Wait, again? Why? Well, now that I had a handle on things, I could actually take on the task of triage-ing the issues. If I’d tried to do it before I had the context, I might not have seen the duplicate issues, I wouldn’t know what the normal kinds of comments were on issues, I wouldn’t have known some common questions that maintainers had on pull requests, and in general, things would have just been worse.</p>
<p>This time, when reading the issue, I went through a little algorithm to sort them out. It looked a little like this:</p>
<ol>
<li>Is this issue a feature request? If so, copy/paste an answer I wrote that pointed them to the mailing list, and click close.</li>
<li>Is this issue a request for help? If so, copy/paste an answer I wrote that pointed them to StackOverflow, and click close.</li>
<li>Was this issue for an older version of Rails than is currently supported? If so, copy/paste an answer I wrote that asks if anyone knows if this affects a supported version of Rails.</li>
<li>Did this issue provide enough information to reproduce the error? If no, copy/paste an answer I wrote that asks if they can provide a reproduction.</li>
<li>If the issue has a reproduction, and it wasn’t on the latest Rails, try it against HEAD. If it still happened, leave a comment that it was still an issue.</li>
<li>If we got to this point, this issue was pretty solid. Leave a comment that I had triaged it, and cc the maintainer of that relevant sub-system of Rails, so they could find issues that pertain to the things they work on.</li>
</ol>
<p>At the same time I did this, I clicked this button on the GitHub interface:</p>
<p><img src="https://svbtleusercontent.com/e2rhsedvszk54q_small.png" alt="https://svbtleusercontent.com/e2rhsedvszk54q_small.png" /></p>
<p>And then set up a Gmail filter to filter all of the emails into their own tag, and to skip my inbox:</p>
<p><img src="https://svbtleusercontent.com/oyuljxmbfqieia_small.png" alt="https://svbtleusercontent.com/oyuljxmbfqieia_small.png" /></p>
<p>Why do this? Well, I didn’t do all 800 immediately. I decided to do one page per day. This kept it a bit more manageable, rather than taking up entire days of my time. I need these emails and filters for the important second part of the process: tending to the garden regularly.</p>
<p>Each morning, before I go to work, I pour a cup of coffee and check my emails. I don’t handle all of them before work, but I made an effort to tackle Rails’ emails first. There would usually be about 20 or 25 new emails each morning, and since it was largely just one new comment, they’d be pretty fast to get through. 15 minutes later, I was back to current on all issues. At lunch, I’d do it again: ten minutes to handle the ten or so emails by lunch, and then, before I’d go to bed, I’d do it again: 15 more minutes to handle the next 20 notifications. Basically, I was spending a little under an hour each day, but by doing it <em>every</em> day, it never got out of hand.</p>
<p>Once I got through all of the issues, we were down to more like 600. A whole fourth of the issues shouldn’t even have been open in the first place. Two weeks in is when the next big gain kicked in. Why two weeks? Well, two weeks is the grace period we decided before marking an issue as stale. Why two weeks? Well, that’s kind of arbitrary, but two weeks feels like enough time for someone to respond if they’re actively interested in getting an issue fixed. See, issues often need the help of the reporter to truly fix, as there just isn’t enough information in many bug reports to be able to reproduce and fix the problem.</p>
<p>So, after two weeks, I did one more thing each evening: I filtered by ‘least recently updated,’ and checked to see if any of those issues were stale. You just go back until they say ‘two weeks,’ and then, if you haven’t heard from the reporter, mention that it’s stale and give the issue a close. This is one of the other things I had to kind of let go of when working on a real project: closing an issue isn’t forever. You can always re-open the issue later if it turns out you were wrong. So when trying to get a handle on 800 open issues, I defaulted to ‘guilty until proven innocent.’ Terminate issues with extreme prejudice. Leaving old, inconclusive issues doesn’t help anyone. If it’s a real bug that matters to someone, they’ll come along and help reproduce it. If not, maybe someone else will later.</p>
<p>After a month or two, keeping on it, we got down to 450 or so issues. Members of the core team joked that they had to set up extra email filters from me, because they could tell exactly when I was doing triage. Slow and steady wins the race!</p>
<p>At this point, I knew enough about Rails to actually start writing some patches. And I happened to be familiar with basically every open bug. So it was easy to start picking some of them and try to reproduce them locally. So I’d do that, and then try to write a patch. If I couldn’t, I’d at least upload my reproduction of the issue, and then leave a note on the Issue, pointing to my reproduction. That way, another team member could simply clone my repository and get to it. The only thing better than reproduction instructions are when those instructions say <code>git clone</code>.</p>
<p>But I managed to get a few patches in, and then a few more. Doing all of this janitorial work directly led the way towards attaining a commit bit on Rails. It was a long slog at first, but it just got easier the more I did it. A lot of work in open source is this way: it’s really easy once you’ve done it a lot, but is hard for newbies. I’m not yet sure how to tackle this problem…</p>
<p>I’ve since taken this approach on basically every repository I’ve worked on, and it’s worked really well. But it only works if you keep at it: if you don’t tend your garden, you’ll get weeds. I haven’t had as much time for Rails over the last few months, and it’s back to 800 issues again. I’m not sure if these are real issues or not, as I’ve stopped tending. But without someone actively paying attention, it’s only a matter of time before things get unseemly. If you’re looking to help out an open source project, it’s not a glamorous job, but all it takes is a little bit of work, and developing a habit.</p>
<p>(Oh, and I should take a moment to mention <a href="http://www.codetriage.com/">Code Triage</a> here. It’s super useful, and can also help you find projects that need help.)</p>
</content:encoded></item><item><title>Is npm worth 26MM?</title><link>https://steveklabnik.com/writing/is-npm-worth-26mm/</link><guid isPermaLink="true">https://steveklabnik.com/writing/is-npm-worth-26mm/</guid><pubDate>Wed, 12 Feb 2014 00:00:00 GMT</pubDate><content:encoded><p>Yesterday, npm, Inc. announced that it had <a href="http://blog.npmjs.org/post/76320673650/funding">closed a $2.6MM round of funding</a>. This was met with a bunch of derision on Twitter. I feel this is not only unwarranted, but backwards: I think this investment signals a nascent trend within the VC community, and a more honest view of how open source actually works.</p>
<blockquote>
<p>Further disclaimer: I work for a company which has received investment from people mentioned in this article. I have never talked to those firms directly either, and I have no privileged information as to why they invested in us, either. It all happened before I came on board.</p>
</blockquote>
<p>Let’s talk about VC, and then about some odd investments, and then we’ll get around to npm. I apologize for the length of this post, I didn’t have time to make a shorter one.</p>
<h2>How VC investment works</h2>
<p>First, a quick refresher on the actual mechanics of VC investment: <a href="http://en.wikipedia.org/wiki/Betteridge's_law_of_headlines">the answer to this headline is ‘no.’</a>. npm, Inc (‘npm’ for the rest of this post) received an investment of $2.6MM, which means that their company is ‘worth’ some multiple of that. At least, it is to <a href="http://www.trueventures.com/">True Ventures</a> and friends.</p>
<p>As the saying goes, “follow the money.” A VC firm is, in many ways, a marketplace: they take money from investors (“limited partners,” or “lp’s”) and give it to entrepreneurs, and take a cut off the top. Let’s go through an example:</p>
<p>The Vulture Capital company is a venture capital firm. They announce a new fund. This fund, like many, has a mission statement: to invest in new SaaS technologies. They go to various wealthy individuals, insurance companies, funds-of-funds, endowments, and other companies, and pitch them on this investment: “The VC firm is going to invest $100MM into new SaaS technologies. Our last fund turned $10MM into $20MM. We think we can turn this $100MM into $300MM in the next ten years. We just need your money.” This motley band of the 1% scrapes together $100MM total out of their couch, and this new fund is good to go! The fund is now ‘closed.’ The VC firm now goes to work: the individuals who work for the firm now figure out who to actually give the money to. This generally takes three to five years. In this story, to make the math easy, VC invests in ten companies, $9.75MM each. For this, they get some stake in each company: we’ll call it 30% here. Since they purchased 30% of the company for $9.75MM, each company is ‘worth’ $29.25MM. In order to actually monetize this stake, generally, these companies need to experience a liquidity event: someone has to purchase them, or they need to go public. Because early-stage investing is exceedingly risky, when we check in on Vulture Capital ten years later, six companies have gone bankrupt, three sold for $67MM, and one went public and investors sold off $100MM. 3 * 67MM + 100MM = 300MM: success! The firm has turned $100MM into $300MM. The firm takes a chunk of this money, and the rest is given to those individuals that put up the money in the first place. A new fund is announced, and the cycle repeats itself.</p>
<p>There are variations at all points in this story, but that’s basically how it works. I’ve also ignored that VC firms generally have multiple funds going at once, that there are different kinds of employees in VC funds with different roles, and a whole ton of other things. This is a simplification, but a representative one.</p>
<p>Two big things matter here: It’s a numbers game. Over half of our example companies bit the dust, but the firm still made their money. Things have been successful overall. Secondly, companies <em>must</em> exit. A business that’s making $1MM a year in profit off of our $9.75MM investment will still make money eventually, but it won’t in our ten year period, which is when our limited partners want their money back. In another context, a million dollars of profit a year is a success, but for us, it’s a failure.</p>
<p>We’ll come back to these economics in a bit.</p>
<h2>Open Source and $$$</h2>
<p>It’s time for some #realtalk about open source. Open Source is big business. And it’s always been that way. Someday I will write more about this, but for now, I’ll just make that statement and not really back it up. For now, let’s trace the flow of labor in the open source world, and how it relates to businesses.</p>
<p>Open Source is pretty much the de-facto way to run a startup company these days. Even people on the Microsoft stack are increasingly utilizing open source. And there’s a good reason: the initial monetary cost of open source is zero. It’s also a form of DRY: a bug fixed in an open source component fixes a bug in every firm that uses that particular component.</p>
<p>Here’s a (sorta kinda) real world example:</p>
<p>A company, “<a href="http://adequatehq.com/">Adequate HQ</a>” builds a web application (<a href="http://www.recruiterspam.com/">RecruiterSpam</a>) in a hot new language, “<a href="https://github.com/tenderlove/phuby">Phuby</a>.” They decide to release a bunch of the code from this application as open source, and call it “<a href="http://www.youtube.com/watch?v=lsWKjS6Vufw">Phuby on Phails</a>.” There are a number of reasons to do this, but here’s a big one: bugs fixed in Phails will be fixed in RecruiterSpam, at no cost to Adequate! Furthermore, there’s a market opportunity here: by controlling Phuby on Phails, Adequate secures its reputation as a leader in the space, which leads to an invaluable amount of publicity, book deals, and eventually, pro subscriptions to RecruiterSpam. Adequate’s CEO, <a href="https://twitter.com/gorbypuff">Gorbachev “Puff Puff” Thunderhorse</a>, can just lay around all day and post photos of himself to Instagram. Other companies see the success of Adequate, and also decide to build their application on Phails. After all, Phails is pretty awesome, and is (truly) significantly better than all the other frameworks out there. Their new use exposes bugs in Phails, so they fix them and submit patches. Eventually, there’s an entire Phails ecosystem, with conferences, books, ninjas, and rockstars. Tons of gravy for everyone, and everyone rides their sports cars into their various sunsets.</p>
<p><img src="/img/2014-02-12/zonda-hh.jpg" alt="pagani zonda hh" /></p>
<p>This is a great story. Everyone wins. Everyone helps each other out. Costs go down, quality goes up. But there’s something that needs to be acknowledged here: Adequate had to exist. Someone had to do the initial work of actually building Phails (and all the other related tools I didn’t explicitly mention: deployment tools, testing tools, dependency management tools, the list goes on and on). In this case, it was because they had a kick-ass product and made boatloads of cash.</p>
<p>But fundamentally, programmers must pay rent and eat. Yes, they may not <em>directly</em> work on open source for free. But someone is paying them, and it has to come from <em>somewhere</em>. There is no such thing as a free lunch.</p>
<p>There are, however, different ways to get the money to buy lunch.</p>
<h2>Infrastructure investment</h2>
<p>So we have this trend of open source in general, and that for each language, we need an ecosystem to support building software. That ecosystem has to be bootstrapped somehow. Previous communities effectively got lucky, or put in long, long hours, only to be forgotten. I still cry tears for CPAN every once in a while. But I digress. The real question is this:</p>
<p>I’m a VC. I need a large number of companies to exist to make my money. I also need them to get as big as possible to make my money. I need costs to be low and profits to be high. I need a healthy open source ecosystem to make this happen. So what do I do?</p>
<p>So here’s what I’m seeing slowly happen: VCs are realizing this story, and are starting to invest in infrastructure. Remember, failure is built into their model. If you had to make ten investments, and I told you that I would take one of them, and you’d lose money on me, but I’d build an ecosystem that would help be a multiplier on all your other investments, would you take it? Remember, you just need one really, really big payday. And a bunch of your investments <em>will</em> already fail.</p>
<p>This is what I see happening with npm. And it’s not just npm: there’s a number of investments in the last few years which fit this pattern. I’m pretty sure that <a href="http://en.wikipedia.org/wiki/Andreessen_Horowitz">a16z</a> is the only VC firm that explicitly understands this model, and True Ventures may have stumbled upon it by accident.</p>
<p>Because Rails (If the sarcasm wasn’t heavy enough, Phails is Rails) isn’t exactly repeatable. It’s a fluke. A wonderful fluke that’s changed my life (I have a Ruby tattoo), but I wouldn’t bet on it happening that way again. If I told you, “I have a design firm. No programming skill. We’re going to hire an offshore contractor to build an application and let them do whatever they want.” Would you invest? I wouldn’t.</p>
<p>If you want to repeatably manufacture an open source ecosystem, you need capital to do so. And a firm that’s progressive enough to understand the indirect payoffs of investing in infrastructure is poised to have a huge advantage.</p>
<p>While my VC story above is a generic, average one, there are also oddball investments. Before we get to NPM exactly, I want to talk about a few others. The first one is not actually a VC investment itself, but it is a total curveball that caused a lot of waves. Let’s talk about 37signals and Jeff Bezos. In many ways, 37signals is the anti-VC. A boostrapped, incredibly profitable company. So why take money from an investor? Well, money isn’t the only things investors have to offer. From <a href="http://signalvnoise.com/archives2/bezos_expeditions_invests_in_37signals.php">the announcement</a>:</p>
<blockquote>
<p>Since we launched Basecamp we’ve been contacted by nearly 30 different VC firms. We’ve never been interested in the typical traditional VC deal. With a few exceptions, all the VCs could offer us was cash and connections. We’re fine on both of those fronts. We don’t need their money to run the business and our little black book is full. We’re looking for something else.What we’ve been looking for is the wisdom of a very special entrepreneur who’s been through what we’re going through. Someone who sees things a little differently and makes us feel right at home. Someone with a long term outlook, not a build-to-flip mentality. We found a perfect match in Jeff. Jeff is our kinda guy.</p>
</blockquote>
<p>37signals themselves have said many, many times that they would never sell the company. So where’s Bezos going to get his liquidity event from? Jeff doesn’t actually need Basecamp (which 37signals abruptly re-branded to last week, so let’s abruptly re-brand them here) to experience a liquidity event to get what he wants out of this investment. Bezos <a href="http://www.crunchbase.com/financial-organization/bezos-expeditions">invests in all kinds of things</a>. And, as Basecamp mentions, his advice and connections are invaluable.</p>
<p>The next two investments that fall in this category of “force multiplier for other things I’m doing that I don’t have to worry about getting my money back directly” are both from Andreessen Horowitz. a16z is in my opinion the most progressive and interesting VC firm in the valley. They make a lot of weird, oddball, long-shot investments, and they tend to pay off. There are two (and a half) that directly fall into the category of investment I’m talking about in this post, and that’s GitHub and Meteor (and CoinBase). Let’s talk Meteor first.</p>
<p><a href="https://www.meteor.com/blog/2012/07/25/meteors-new-112-million-development-budget">Meteor Development Group got $11.2MM in a round led by a16z</a>. While MDG claims they’ll eventually release some sort of commercial product, I don’t think that’s what really matters. If Meteor is the new Rails, then a16z gets the most inside track possible to that entire new ecosystem. And given that most VC investments fail anyway, this is a really good deal for a16z. They’re purchasing a lottery ticket, but it’s more than that: it’s a lottery ticket that makes future lottery tickets worth more money. They can invest in future Meteor-based startups, knowing that Meteor development will continue for a long time. They’ll always have their nose to the ground on any promising companies built around Meteor, earlier than everyone else. Knowledge and information is power.</p>
<p>Secondly is GitHub. <a href="http://peter.a16z.com/2012/07/09/software-eats-software-development/">GitHub raised $100MM from a16z, and a16z alone</a>. At the time, I was super mad, because I said exactly what everyone has been saying about the NPM investment. As time goes on, though, I don’t think that GitHub <em>needs</em> to experience a liquidity event. While we don’t know the valuation of GitHub, I <em>do</em> know that an acquisition doesn’t make any sense, and I’m increasingly of the opinion that an IPO doesn’t either. This is based primarily on subjective opinions that I’ve gleaned from being friends with so many GitHubbers over the years, and I don’t think an IPO is <em>impossible</em>, but unlikely. So why invest?</p>
<p><a href="http://online.wsj.com/news/articles/SB10001424053111903480904576512250915629460">a16z famously believes that software is eating the world</a>. They even reference this article in the title of the GitHub announcement. Investing in GitHub makes a ton of sense when viewed through this lens. In fact, if I had to guess, I’d bet that a16z got a small enough chunk that GitHub’s valuation is <em>insane</em>. (And I try not to use that word.) This means that a16z and GitHub management are protected from a acquisition, and can retain control of GitHub. Why does that matter? Well:</p>
<p>“Scumbag Steve image saying”Open source infrastructure should be free" / “pushes all code to GitHub”" Apparently memegenerator got hacked, and it was giving this page a warning.</p>
<p>Increasingly, GitHub is where all the open source code is hosted. And open source is the way business is going. And GitHub makes money off of the code that’s not open. In many ways, the open source stuff is a loss leader, so that you get hooked on the GitHub interface and then use it for everything, even your closed source code. This is a great business strategy, but it also places GitHub at the center of this new universe. So for similar reasons, a16z needs GitHub to be awesome to bootstrap <em>every</em> open source ecosystem that will exist into the future. It’s the infrastructure for the infrastructure. The primordial ooze of the 10x return. And a16z has the money to ‘throw away’ on something they won’t get a direct return out of, because they’re smart enough to invest some of their fund in ecosystem development.</p>
<p>The extra half-investment I’m talking about is CoinBase. <a href="http://techcrunch.com/2013/12/12/coinbase-raises-25m-from-andreessen-horowitz-to-build-its-bitcoin-wallet-and-merchant-services/">They got $25MM from a16z and others</a>, and <a href="http://dealbook.nytimes.com/2014/01/21/why-bitcoin-matters/">Andreesen himself has written about BitCoin</a>. By way of analogy:</p>
<p>“Software will eat the world” : GitHub :: “Why Bitcoin Matters” : CoinBase</p>
<p>I only give them a half in this case because it’s not clear to me that CoinBase will be dominant in the way that GitHub is dominant, and that Bitcoin will build a new payments ecosystem the way that open source has built a new ecosystem. But <em>some</em> entity like CoinBase is necessary for Bitcoin to succeed. More on that in a future post, someday.</p>
<h2>npm?</h2>
<p>So by now, I hope it’s clear how I view this NPM investment: True Ventures puts in money because the ecosystem around Node.js is still young, and a package manager is one of the biggest parts that <em>requires</em> money. Someone has to pay that S3 (I guess CouchDB) and EC2 bills. So they bootstrap the ecosystem, and also give a multiplier on their future investments. They get insight and access at unprecedented levels into the future of Node, at the cost of something they can already afford to lose.</p>
<p>We’ll see if I’m right in ten years.</p>
</content:encoded></item><item><title>Announcing Emoji 1.0</title><link>https://steveklabnik.com/writing/announcing-emoji-1-0/</link><guid isPermaLink="true">https://steveklabnik.com/writing/announcing-emoji-1-0/</guid><pubDate>Mon, 10 Feb 2014 00:00:00 GMT</pubDate><content:encoded><p>A long time ago, I wanted an emoji gem. So I set out to make one. Turns out, emoji are damn complex. <a href="http://words.steveklabnik.com/emoji-licensing">Emoji Licensing</a> was one of the results. Who’d have guessed the hard part of making a gem was the legal part! Next up was that the ‘emoji’ gem was waaaay out of date: it was a library for converting between Japanese telecoms’ emoji, which was important years ago, but not now that they’re just UTF-8. So I contacted the owner, who graciously gave me the gem.</p>
<p>Well, after doing the legal and social work, it turns out that some other people were interested in the idea as well! In the end, <a href="https://github.com/steveklabnik/emoji/commits?author=steveklabnik">I didn’t write very much actual code</a>, and <a href="https://github.com/wpeterson">Winfield</a> really stepped up and made the code side happen.</p>
<p>So today, I’m very happy to have just released version 1.0 of the emoji gem, and to officially hand off maintainership to Winfield. Thanks for all your hard work!</p>
<p>Install it with</p>
<pre><code>$ gem install emoji
</code></pre>
<p>And find the source on GitHub: <a href="https://github.com/steveklabnik/emoji">https://github.com/steveklabnik/emoji</a></p>
</content:encoded></item><item><title>A 30 minute introduction to Rust</title><link>https://steveklabnik.com/writing/a-30-minute-introduction-to-rust/</link><guid isPermaLink="true">https://steveklabnik.com/writing/a-30-minute-introduction-to-rust/</guid><pubDate>Sat, 11 Jan 2014 00:00:00 GMT</pubDate><content:encoded><p>I recently gave a <a href="https://air.mozilla.org/rust-meetup-december-2013/">proposal for Rust’s documentation</a>. An important component of my overall proposal is a short, simple introduction for people who may have heard of Rust, so that they can figure out if Rust is right for them. The other day, I saw <a href="http://www.youtube.com/watch?v=gfCtbGiHcg0">this excellent presentation</a>, and thought it might serve as a great basis for this introduction. Consider this an RFC for such an introduction. Feedback very welcome on <a href="https://mail.mozilla.org/pipermail/rust-dev/2014-January/007903.html">the rust-dev mailing list</a> or <a href="https://twitter.com/steveklabnik">on Twitter</a>.</p>
<h2>This tutorial has now <a href="http://static.rust-lang.org/doc/master/intro.html">become the official one</a>.</h2>
<p>Rust is a systems programming language that focuses on strong compile-time correctness guarantees. It improves upon the ideas other systems languages like C++, D, and Cyclone by providing very strong guarantees and explicit control over the life cycle of memory. Strong memory guarantees make writing correct concurrent Rust code easier than in other languages. This might sound very complex, but it’s easier than it sounds! This tutorial will give you an idea of what Rust is like in about thirty minutes. It expects that you’re at least vaguely familiar with a previous ‘curly brace’ language. The concepts are more important than the syntax, so don’t worry if you don’t get every last detail: the <a href="http://static.rust-lang.org/doc/master/tutorial.html">tutorial</a> can help you out with that later.</p>
<p>Let’s talk about the most important concept in Rust, “ownership,” and its implications on a task that programmers usually find very difficult: concurrency.</p>
<h2>Ownership</h2>
<p>Ownership is central to Rust, and is one of its more interesting and unique features. “Ownership” refers to which parts of your code are allowed to modify various parts of memory. Let’s start by looking at some C++ code:</p>
<pre><code>int *dangling(void)
{
int i = 1234;
return &amp;i;
}
int add_one(void)
{
int *num = dangling();
return *num + 1;
}
</code></pre>
<p>This function allocates an integer on the stack, and stores it in a variable, <code>i</code>. It then returns a reference to the variable <code>i</code>. There’s just one problem: stack memory becomes invalid when the function returns. This means that in the second line of <code>add_one</code>, <code>num</code> points to some garbage values, and we won’t get the effect that we want. While this is a trivial example, it can happen quite often in C++ code. There’s a similar problem when memory on the heap is allocated with <code>malloc</code> (or <code>new</code>), then freed with <code>free</code> (or <code>delete</code>), yet your code attempts to do something with the pointer to that memory. More modern C++ uses RAII with constructors/destructors, but it amounts to the same thing. This problem is called a ‘dangling pointer,’ and it’s not possible to write Rust code that has it. Let’s try:</p>
<pre><code>fn dangling() -&gt; &amp;int {
let i = 1234;
return &amp;i;
}
fn add_one() -&gt; int {
let num = dangling();
return *num + 1;
}
</code></pre>
<p>When you try to compile this program, you’ll get an interesting (and long) error message:</p>
<pre><code>temp.rs:3:11: 3:13 error: borrowed value does not live long enough
temp.rs:3 return &amp;i;
temp.rs:1:22: 4:1 note: borrowed pointer must be valid for the anonymous lifetime #1 defined on the block at 1:22...
temp.rs:1 fn dangling() -&gt; &amp;int {
temp.rs:2 let i = 1234;
temp.rs:3 return &amp;i;
temp.rs:4 }
temp.rs:1:22: 4:1 note: ...but borrowed value is only valid for the block at 1:22
temp.rs:1 fn dangling() -&gt; &amp;int {
temp.rs:2 let i = 1234;
temp.rs:3 return &amp;i;
temp.rs:4 }
error: aborting due to previous error
</code></pre>
<p>In order to fully understand this error message, we need to talk about what it means to “own” something. So for now, let’s just accept that Rust will not allow us to write code with a dangling pointer, and we’ll come back to this code once we understand ownership.</p>
<p>Let’s forget about programming for a second and talk about books. I like to read physical books, and sometimes I really like one and tell my friends they should read it. While I’m reading my book, I own it: the book is in my possession. When I loan the book out to someone else for a while, they “borrow” it from me. And when you borrow a book, it’s yours for a certain period of time, and then you give it back to me, and I own it again. Right?</p>
<p>This concept applies directly to Rust code as well: some code “owns” a particular pointer to memory. It’s the sole owner of that pointer. It can also lend that memory out to some other code for a while: the code “borrows” it. It borrows it for a certain period of time, called a “lifetime.”</p>
<p>That’s all there is to it. That doesn’t seem so hard, right? Let’s go back to that error message: <code>error: borrowed value does not live long enough</code>. We tried to loan out a particular variable, <code>i</code>, using Rust’s borrowed pointers: the <code>&amp;</code>. But Rust knew that the variable would be invalid after the function returns, and so it tells us that: <code>borrowed pointer must be valid for the anonymous lifetime #1... but borrowed value is only valid for the block</code>. Neat!</p>
<p>That’s a great example for stack memory, but what about heap memory? Rust has a second kind of pointer, a ‘unique’ pointer, that you can create with a <code>~</code>. Check it out:</p>
<pre><code>fn dangling() -&gt; ~int {
let i = ~1234;
return i;
}
fn add_one() -&gt; int {
let num = dangling();
return *num + 1;
}
</code></pre>
<p>This code will successfully compile. Note that instead of a stack allocated <code>1234</code>, we use an owned pointer to that value instead: <code>~1234</code>. You can roughly compare these two lines:</p>
<pre><code>// rust
let i = ~1234;
// C++
int *i = new int;
*i = 1234;
</code></pre>
<p>Rust is able to infer the size of the type, then allocates the correct amount of memory and sets it to the value you asked for. This means that it’s impossible to allocate uninitialized memory: Rust does not have the concept of null. Hooray! There’s one other difference between this line of Rust and the C++: The Rust compiler also figures out the lifetime of <code>i</code>, and then inserts a corresponding <code>free</code> call after it’s invalid, like a destructor in C++. You get all of the benefits of manually allocated heap memory without having to do all the bookkeeping yourself. Furthermore, all of this checking is done at compile time, so there’s no runtime overhead. You’ll get (basically) the exact same code that you’d get if you wrote the correct C++, but it’s impossible to write the incorrect version, thanks to the compiler.</p>
<p>You’ve seen one way that ownership and lifetimes are useful to prevent code that would normally be dangerous in a less-strict language, but let’s talk about another: concurrency.</p>
<h2>Concurrency</h2>
<p>Concurrency is an incredibly hot topic in the software world right now. It’s always been an interesting area of study for computer scientists, but as usage of the Internet explodes, people are looking to improve the number of users a given service can handle. Concurrency is one way of achieving this goal. There is a pretty big drawback to concurrent code, though: it can be hard to reason about, because it is non-deterministic. There are a few different approaches to writing good concurrent code, but let’s talk about how Rust’s notions of ownership and lifetimes can assist with achieving correct but concurrent code.</p>
<p>First, let’s go over a simple concurrency example in Rust. Rust allows you to spin up ‘tasks,’ which are lightweight, ‘green’ threads. These tasks do not have any shared memory, and so, we communicate between tasks with a ‘channel’. Like this:</p>
<pre><code>fn main() {
let numbers = [1,2,3];
let (port, chan) = Chan::new();
chan.send(numbers);
do spawn {
let numbers = port.recv();
println!("{:d}", numbers[0]);
}
}
</code></pre>
<p>In this example, we create a vector of numbers. We then make a new <code>Chan</code>, which is the name of the package Rust implements channels with. This returns two different ends of the channel: a channel and a port. You send data into the channel end, and it comes out the port end. The <code>spawn</code> function spins up a new task. As you can see in the code, we call <code>port.recv()</code> (short for ‘receive’) inside of the new task, and we call <code>chan.send()</code> outside, passing in our vector. We then print the first element of the vector.</p>
<p>This works out because Rust copies the vector when it is sent through the channel. That way, if it were mutable, there wouldn’t be a race condition. However, if we’re making a lot of tasks, or if our data is very large, making a copy for each task inflates our memory usage with no real benefit.</p>
<p>Enter Arc. Arc stands for ‘atomically reference counted,’ and it’s a way to share immutable data between multiple tasks. Here’s some code:</p>
<pre><code>extern mod extra;
use extra::arc::Arc;
fn main() {
let numbers = [1,2,3];
let numbers_arc = Arc::new(numbers);
for num in range(0, 3) {
let (port, chan) = Chan::new();
chan.send(numbers_arc.clone());
do spawn {
let local_arc = port.recv();
let task_numbers = local_arc.get();
println!("{:d}", task_numbers[num]);
}
}
}
</code></pre>
<p>This is very similar to the code we had before, except now we loop three times, making three tasks, and sending an <code>Arc</code> between them. <code>Arc::new</code> creates a new Arc, <code>.clone()</code> makes a new reference to that Arc, and <code>.get()</code> gets the value out of the Arc. So we make a new reference for each task, send that reference down the channel, and then use the reference to print out a number. Now we’re not copying our vector.</p>
<p>Arcs are great for immutable data, but what about mutable data? Shared mutable state is the bane of the concurrent programmer. You can use a mutex to protect shared mutable state, but if you forget to acquire the mutex, bad things can happen.</p>
<p>Rust provides a tool for shared mutable state: <code>RWArc</code>. This variant of an Arc allows the contents of the Arc to be mutated. Check it out:</p>
<pre><code>extern mod extra;
use extra::arc::RWArc;
fn main() {
let numbers = [1,2,3];
let numbers_arc = RWArc::new(numbers);
for num in range(0, 3) {
let (port, chan) = Chan::new();
chan.send(numbers_arc.clone());
do spawn {
let local_arc = port.recv();
local_arc.write(|nums| {
nums[num] += 1
});
local_arc.read(|nums| {
println!("{:d}", nums[num]);
})
}
}
}
</code></pre>
<p>We now use the <code>RWArc</code> package to get a read/write Arc. The read/write Arc has a slightly different API than <code>Arc</code>: <code>read</code> and <code>write</code> allow you to, well, read and write the data. They both take closures as arguments, and the read/write Arc will, in the case of write, acquire a mutex, and then pass the data to this closure. After the closure does its thing, the mutex is released.</p>
<p>You can see how this makes it impossible to mutate the state without remembering to aquire the lock. We gain the efficiency of shared mutable state, while retaining the safety of disallowing shared mutable state.</p>
<p>But wait, how is that possible? We can’t both allow and disallow mutable state. What gives?</p>
<h2>A footnote: unsafe</h2>
<p>So, the Rust language does not allow for shared mutable state, yet I just showed you some code that has it. How’s this possible? The answer: <code>unsafe</code>.</p>
<p>You see, while the Rust compiler is very smart, and saves you from making mistakes you might normally make, it’s not an artificial intelligence. Because we’re smarter than the compiler, sometimes, we need to over-ride this safe behavior. For this purpose, Rust has an <code>unsafe</code> keyword. Within an <code>unsafe</code> block, Rust turns off many of its safety checks. If something bad happens to your program, you only have to audit what you’ve done inside <code>unsafe</code>, and not the entire program itself.</p>
<p>If one of the major goals of Rust was safety, why allow that safety to be turned off? Well, there are really only three main reasons to do it: interfacing with external code, such as doing FFI into a C library, performance (in certain cases), and to provide a safe abstraction around operations that normally would not be safe. Our Arcs are an example of this last purpose. We can safely hand out multiple references to the <code>Arc</code>, because we are sure the data is immutable, and therefore it is safe to share. We can hand out multiple references to the <code>RWArc</code>, because we know that we’ve wrapped the data in a mutex, and therefore it is safe to share. But the Rust compiler can’t know that we’ve made these choices, so <em>inside</em> the implementation of the Arcs, we use <code>unsafe</code> blocks to do (normally) dangerous things. But we expose a safe interface, which means that the Arcs are impossible to use incorrectly.</p>
<p>This is how Rust’s type system allows you to not make some of the mistakes that make concurrent programming difficult, yet get the efficiency of languages such as C++.</p>
<h2>That’s all, folks</h2>
<p>I hope that this taste of Rust has given you an idea if Rust is the right language for you. If that’s true, I encourage you to check out <a href="http://static.rust-lang.org/doc/0.9/tutorial.html">the tutorial</a> for a full, in-depth exploration of Rust’s syntax and concepts.</p>
</content:encoded></item><item><title>2013: year in review</title><link>https://steveklabnik.com/writing/2013--year-in-review/</link><guid isPermaLink="true">https://steveklabnik.com/writing/2013--year-in-review/</guid><pubDate>Tue, 31 Dec 2013 00:00:00 GMT</pubDate><content:encoded><p>Every year I try to do a “year in Review” blog post, and every year I realize I didn’t keep good notes. Sigh.</p>
<p>Overall, this was a pretty… meh year, to be honest. :/ Don’t get me wrong, a lot of really, really good things happened, but as far as my grand strategic life goals go, this year was pretty much treading water, and in some cases, regressive. Oh well.</p>
<p>So instead of doing as much of a review, I’m going to instead discuss what I’d like 2014 to be like:</p>
<h3>Better tracking</h3>
<p>In general, I don’t think I accomplished all I wanted to because I didn’t measure it at all. I’ve found that <a href="http://beeminder.com/">Beeminder</a> has helped me track one or two of the things that I’ve really wanted to accomplish, and so I plan on making lots of use of that. It’s not perfect for everything, but it helps.</p>
<h3>Reading</h3>
<p>I have so much I need to read. I think I’m going to start by recording chapters in Beeminder; books are too granular. I don’t think that a book every two weeks is an unreasonable goal, and I may turn it up if I get in the swing of things.</p>
<h3>Writing</h3>
<p>I really need to finish off the hypermedia book, and the Rails 4 in Action book. R4IA should be easy, the hypermedia book will be harder.</p>
<h3>Software</h3>
<p>I want to contribute to Tor this year. I’ve been putting off contributing to Tor due to my paranoia about NSA stuff, but really, at this point, there’s nothing to worry about. I’m probably on all of those lists anyway. They’re watching everyone.</p>
<p>In order to do that, I need to up my crypto game. I’d like to finish off the Matasano Crypto Challenge, and read a bunch more things.</p>
<p>I also want to make lots of contributions to Rust this year, if not the language itself, the ecosystem. There’s so much work to be done.</p>
<h3>Body</h3>
<p>I’ve been trying to stay relatively fit, and it’s been okay, but I really need to get back on that train. I got a copy of “Starting Strength” for Christmas.</p>
<h3>Languages</h3>
<p>I’m consistently embarassed that I only speak one language. I’ve been working on some German, and it’s been okay, but I need to take it more seriously. I’d also like to pick up the basic Japanese I had before.</p>
<h3>Conclusion</h3>
<p>I think that’s it, at least for now. Let’s hope this next year is a good one.</p>
</content:encoded></item><item><title>Rust is surprisingly expressive</title><link>https://steveklabnik.com/writing/rust-is-surprisingly-expressive/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rust-is-surprisingly-expressive/</guid><pubDate>Sat, 28 Dec 2013 00:00:00 GMT</pubDate><content:encoded><p>Do you remember the first time you saw Rails’ ActiveSupport library? I do: it totally blew my mind. The only dynamic language I had used up to that point was Perl, and it had a similarly mind-blowing effect on my young self. I thought that dynamic typing was mostly a requirement towards making this kind of expressiveness possible, but it turns out Rust can be just as expressive, while retaining type safety and static dispatch.</p>
<p>There was a time in the early days of Rails when its evangelists would show off snippets of what it let you do. These were often ActiveSupport extensions to the core language itself. It’d go something like this:</p>
<blockquote>
<p>Hey, have you tried Rails? Check this out:</p>
</blockquote>
<pre><code>irb(main):002:0&gt; Time.now - 2.days
=&gt; 2009-12-26 09:57:02 -0800
irb(main):003:0&gt; 2.days.ago
=&gt; 2009-12-26 09:57:04 -0800
</code></pre>
<blockquote>
<p>Did I just blow your mind???</p>
</blockquote>
<p>At the time, I enthusiastically replied “Yes, yes you did!” and immediately typed <code>rails new foo</code>. And all was well and good in the world.</p>
<p>Fast forward a few years. I now know that static typing is not equivalent to <code>public static void main(String [] args) {</code>. I’ve seen the <a href="http://www.cafepress.com/mf/2498088/what-part-of-types-dont-you-understand_tshirt?shop=skicalc&amp;productId=6225368">Hindley-Milner</a> light. Yet, for all practical purposes, I’ve still been hacking in Ruby for these past few years. For the most part, I’d found the complaints about dynamic typing from the static typing camp to be very FUD-y. I very rarely get <code>TypeError</code>s in my Ruby code. Refactoring generally isn’t very difficult.</p>
<p>But slowly, this has started to change. I think that it’s that I tend to do more infrastructure and plumbing work now. I’ve run into more and more situations where types would be helpful, or picking the mutable state path has caused headaches.</p>
<p>Then, I wake up to an IM from <a href="http://twitter.com/wycats">Yehuda</a>:</p>
<blockquote>
<p>bro, did you see my tweets?</p>
</blockquote>
<p>Basically, this:</p>
<pre><code>➜ active_support git:(master) cat main.rs
extern mod active_support;
use active_support::Period;
use active_support::Time;
fn main() {
let time = Time::now();
println!("{:?}", time);
println!("{:?}", 2.days().from_now());
println!("{:?}", 2.weeks().from_now());
println!("{:?}", 2.months().from_now());
println!("{:?}", 2.years().from_now());
}
➜ active_support git:(master) ./main
active_support::time::Time{date: active_support::date::Date{year: 2013, month: 12u, day: 28u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 242647081u}
active_support::time::Time{date: active_support::date::Date{year: 2013, month: 12u, day: 30u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243287552u}
active_support::time::Time{date: active_support::date::Date{year: 2014, month: 1u, day: 11u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243347757u}
active_support::time::Time{date: active_support::date::Date{year: 2013, month: 2u, day: 28u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243388962u}
active_support::time::Time{date: active_support::date::Date{year: 2015, month: 12u, day: 28u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243427393u}
</code></pre>
<p>Whaaaat? Yup. To compare side-by-side:</p>
<pre><code># Ruby
2.days.from_now
// Rust
2.days().from_now()
</code></pre>
<p>Awesome. Please note that in Rust, this is fully type-checked, safe, and <em>statically dispatched</em>. No slow reflection techniques used here!</p>
<p>Now, I should say that I almost <em>never</em> use <code>2.days</code>, but the point is that it’s a benchmark of expressiveness: if you can write it, you can do all sorts of other flexible things. Thinking back to my ‘libraries vs. application code’ confusion over types, I’m reminded by something <a href="https://twitter.com/garybernhardt">Gary Bernhardt</a> once said about Ruby vs. Python. I’m paraphrasing, but as I remember it:</p>
<blockquote>
<p>Python is a beautiful, clean language. But the same restrictions that make it nice and clean mean that it’s hard to write beautiful, clean libraries. Ruby, on the other hand, is a complicated, ugly language. But that complexity allows you to write really clean, nice, easy-to-use libraries. The end result is that I prefer Python the language, but I find myself preferring programming with Ruby.</p>
</blockquote>
<p>I’m starting to wonder if maybe I feel the same way about static vs. dynamic typing: I like the simplicity and the speed of programming in a dynamic language, but maybe, given the right static language, I’ll end up preferring to program in $LANGUAGE better.</p>
<p>And that language might just be Rust.</p>
<hr />
<p>You can check out Yehuda’s <a href="https://github.com/wycats/rust-activesupport">rust-activesupport on GitHub</a>, if you’d like to see how this is accomplished. He’s also told me that he will write a post explaining its implementation in technical detail, I’ll link to it here when he’s written it.</p>
</content:encoded></item><item><title>How Dogecoin changed my perspective on cryptocurrency</title><link>https://steveklabnik.com/writing/how-dogecoin-changed-my-perspective-on-cryptocurrency/</link><guid isPermaLink="true">https://steveklabnik.com/writing/how-dogecoin-changed-my-perspective-on-cryptocurrency/</guid><pubDate>Sat, 21 Dec 2013 00:00:00 GMT</pubDate><content:encoded><p>I have long been a Bitcoin hater, for many reasons. But then, a fortnight ago, something happened. <a href="http://dogecoin.com/">Dogecoin</a> came into existence, and for some reason, I just couldn’t get mad about it. After some thoughtful self-evaluation this week, I’ve realized that Dogecoin has actually reversed my position on cryptocurrency. Here’s why: basically, I’ve come to see Dogecoin as an absurdist art project, mocking the entire global financial system.</p>
<blockquote>
<p>Investment Disclaimer: This webpage is provided for general information only and nothing contained in the material constitutes a recommendation for the purchase or sale of any security. Although the statements of fact in this report are obtained from sources that I consider reliable, I do not guarantee their accuracy and any such information may be incomplete or condensed. Also views expressed in this Report are based on research materials available from sources considered reliable. Views are subject to change on the basis of additional or new research, new facts or developments. The investment risks described herein are not purported to be exhaustive, any person considering an investment should seek independent advice on the suitability or otherwise of the particular investment. Investment products are not insured by government or governmental agencies. Investment and Treasury products are subject to Investment risk, including possible loss of principal amount invested. Past performance is not indicative of future results: prices can go up or down. Investors investing in investments and/or treasury products denominated in foreign (non-local) currency should be aware of the risk of exchange rate fluctuations that may cause loss of principal when foreign currency is converted to the investors home currency. I personally only hold about 40,000 DOGE.</p>
</blockquote>
<p>Okay, maybe I’m taking something that’s silly way too seriously. Do I really believe that sentence? This question was key for my own understanding of Dogecoin and where it fits into the world. It’s the first question many people ask when they first hear about Dogecoin:</p>
<blockquote>
<p>Wait. Is Dogecoin a real thing?</p>
</blockquote>
<p>— horse &gt;&gt;= js (@jcoglan) December 18, 2013</p>
<blockquote></blockquote>
<p>Searching for <a href="https://twitter.com/search?q=is%20dogecoin%20real&amp;src=typd&amp;f=realtime">“is dogecoin real” on Twitter</a> reveals a lot of people who harbor the same confusion. Here’s the thing: if you are confused if Dogecoin is ‘real’ or not, you’re basically on a wild ride. Critically investigating the ‘realness’ of Dogecoin leads you to realize that it’s basically just as ‘real’ as ‘real money,’ whatever that means. And if Dogecoin is absurd, then so is the entire financial system.</p>
<blockquote>
<p>Dogecoin as symptom of the postmodern condition.</p>
</blockquote>
<p>— Steve Klabnik (@steveklabnik) December 19, 2013</p>
<blockquote></blockquote>
<p>Wait, let’s back up here. What do I mean when I say Dogecoin is ‘just as real’ as, let’s say, Euro? And in economics, most questions end up leading to “What is the meaning of ‘value’?” It’s one of the most fundamental questions of the field. And there are many different positions. For the purposes of this post, I’m going to explain things from the perspective of the “subjective theory of value,” which is arguably the most dominant explanation of value in the field today. I’m painting this concept with a pretty wide brush. Please note two things: I’m not an economist by training, I just enjoy reading lots about it, and I don’t subscribe to the subjective theory of value. That said, I am trying to represent it as accurately as possible in this post.</p>
<p>The simplest formulation of the subjective theory of value is this: something has whatever value a person says it has. Put another way, my coffee cost $3 because a bunch of people all agree that it’s worth $3. I, as an individual, have goals that I want to accomplish. In this case, it’s to write this blog post. To accomplish this, I made a rational decision: make some coffee at home, or go to the coffee shop. I have various resources that allow me to accomplish this goal, namely, that I have three United States dollars. So I take all the information that’s available to me, and make a decision: three dollars is a pretty small amount of money to me (I’m lucky in this regard), I don’t have anything else I need to do at home, the coffee shop hasn’t changed its prices at all in the last week, so I can be pretty sure it’s still $3. So I decide that getting out of the house is worth at least three dollars to me, and I got to exchange my 3 USD for one latte. You can see how endlessly complex this is: if I had chronic pain, for example, I may not want to walk down the steps, so that changes my decision. Maybe I needed to do laundry at home, so staying at home and drinking regular coffee makes more sense. I can’t make a latte at home, so maybe I get out of the house anyway because I’d prefer it to black coffee today.</p>
<p>Anyway, that’s it. Seems pretty simple, right? So what about Dogecoin?</p>
<p>Well, if you think about my example more, I actually have a number of commodities that I’m using to exchange with others to achieve my goal. For example, I also ate a sandwich here at the coffee shop, so I actually spent $6, and that’s partially because I didn’t have any food at home. The coffee shop I went to has good sandwiches, so I chose it over other places that may also have decent lattes. So in my example, I’m trading $6 USD for a latte and a sandwich, and for the right to sit here and use up a seat and some wifi. The woman working behind the counter is trading her time and labor for a wage paid by the company who owns the shop, and the company is trading some coffee beans, a bagel, and some other things with me, through the woman, in exchange for my money.</p>
<p>TL;DR: buying a cup of coffee is surprisingly complicated.</p>
<p>Okay, so now that you’ve seen there are way more commodities involved in this transaction than just USD, why is USD special? If this scenario played out in Paris, I would have been using Euro instead of USD. There’s no <em>inherent</em> reason that a piece of paper has a certain amount of value. There is a <em>subjective</em> reason, though: namely, that a large enough number of people agree that USD is a worthwhile commodity. They believe that for a number of reasons, including that our Army will go off and fucking kill anybody who threatens to break the spell that makes everyone agree that USD are worth your time to accept.</p>
<p>Anyway.</p>
<p>But really, that’s the issue, right? Everyone agrees that dollars are worth something, and we trade them all the time. They’re really easy to trade. I don’t know of any coffee shop near me that takes Euros, so even though 1€ is “worth more” than $1 USD, the Euro note I have in my backpack is basically useless for me to get some coffee. Economists call this property ‘liquidity,’ and it basically means ‘how easy is it for me to get what I want with this commodity.’ USD is very liquid. Euro is very not liquid, unless I deposit it with my bank, which will exchange those Euro for me for USD, which I can then trade for my coffee. Most discussions around the ‘realness’ of cryptocurrency are really arguments about its liquidity. Bitcoins aren’t real because I can’t buy anything with them. Really, liquidity is the aspect that matters to most people when it comes to basic financial transactions.</p>
<p>This brings me to <a href="https://www.mtgox.com/">MtGox</a> and <a href="https://coinbase.com/">CoinBase</a>. These two companies are the biggest things to happen in cryptocurrency, and that’s because they significantly improve the liquidity of Bitcoins. MtGox is a place where you can trade Bitcoins for USD. As soon as you have an exchange like this, your Bitcoins now become incredibly more liquid: you can always trade some BTC for USD, and then trade USD with… anything. Coinbase is an online wallet that allows you to store Bitcoins, but more importantly, it provides liquidity to the BTC market because it allows merchants to ‘accept Bitcoins’ without exposing themselves to the risk of Bitcoin’s price fluctuations (this is called ‘volatility’). As a merchant, I can integrate with Coinbase, and someone can, upon checkout, pay with their Coinbase account. It’s sort of similar, from the perspective of the buyer, as paying with your PayPal account. Anyway, here’s the crucial bit: Coinbase then instantly converts the Bitcoins to USD, and then gives the USD to the merchant. This is <em>incredibly</em> important for the future of Bitcoin. By allowing merchants to not worry about the fact that their customers are paying in BTC, it means that more of them will ‘accept Bitcoins,’ which increases Bitcoins’ liquidity in two ways: first, that they can buy more things in BTC ‘directly,’ as well as adding a healthy amount of orders on the various marketplaces, which helps stabilize the price.</p>
<p>I haven’t said the word “Dogecoin” in a while, so you’re probably wondering where I’m going with this. So here’s the thing: as long as BTC and USD can be exchanged easily, it allows for a significant opening for all other cryptocurrencies. This is because BTC is incredibly liquid when it comes to other cryptocurrencies: there are a number of exchanges that will trade whatever kind of the cambrian explosion of cryptocurrencies you want for BTC. Basically, BTC is sort of the ‘world reserve currency’ of cryptocurrencies. So as long as the link to the ‘real world currencies’ exists, they can cross the gap. Imagine a “Dogecoin VISA”, a credit card where I hold a balance in DOGE. When I go to buy something with my Dogecoin VISA, the backing bank would query the DOGE-&gt; BTC -&gt; USD rate, and then ship the merchant that many USD, and debit me that many DOGE. What if I want to trade EURO for DOGE? EURO -&gt; USD -&gt; BTC -&gt; DOGE. Done. The weakest and most tenuous link here is the BTC &lt;-&gt; USD bridge. As long as it’s maintained, cryptocurrency is liquid, and therefore, ‘real.’ Furthermore, the entire financial system can be re-built on the cryptocurrency side of the bridge, as well. Imagine a site that allowed you to “Dogecoin IPO,” listing your stock on the DSE (Dogecoin Stock Exchange). TWTR -&gt; USD -&gt; BTC -&gt; DOGE -&gt; MYCOMPANY. First step is on the <a href="https://www.google.com/finance?cid=32086821185414">NYSE</a>, second step is on MtGox, third step is on <a href="https://coinedup.com/">Coinedup</a>, fourth step is on the DSE. It all works. Here’s the truly scary thing: we heavily regulate the finance industry, because when we don’t, bad things happen. Hell, when we do, bad things happen. As long as cryptocurrency isn’t taken seriously, it too, will remain unregulated, which allows for a massive extraction of wealth. The cryptocurrency community is largely made up of people who have no idea what they’re doing, and people who <em>do</em> know what they’re doing are starting to notice. It’s all of the worst, most predatory aspects of the global capitalist system, without any consumer protection.</p>
<p>Regardless, there’s an initial ‘bootstrapping’ phase for anything, and I think Bitcoin is past that point now. It doesn’t matter if I think Bitcoin is a terrible idea, it’s going to be a thing regardless, so time to figure out what that actually means.</p>
<p>So the statement that Dogecoin poses to us is this: If things really are worth whatever anyone says they’re worth, so are memes. Do we really want to live in a world where a dumb joke about dogs that can’t talk has the same legitimacy as money? Because the more people that take Dogecoin seriously, the more serious it becomes… and the very nature of its silliness encourages us to think seriously about the nature of money and therefore value.</p>
<p>Before, I just laughed. Now, thanks to a joke, I’m scared.</p>
<hr />
<p>You can tip me in Dogecoin at this address: <code>DEJf4pmo7a87FeZP1fZAT8jPjDHnKL9r8C</code> or in Bitcoin at this address: <code>13BHd8tYA41JMuoBpLTGjyXfrnc6Xi6ma5</code></p>
</content:encoded></item><item><title>git, history modification, and libuv</title><link>https://steveklabnik.com/writing/git-history-modification-and-libuv/</link><guid isPermaLink="true">https://steveklabnik.com/writing/git-history-modification-and-libuv/</guid><pubDate>Sat, 30 Nov 2013 00:00:00 GMT</pubDate><content:encoded><p>There has been a recent kerfuffle over a <a href="https://github.com/joyent/libuv/pull/1015">pull request to libuv</a>: it was rejected, applied, reverted, and re-applied. There was some question as to the authorship of that commit, and I wanted to show you why that was, because it illustrates how git handles history.</p>
<p>At first, the pull request was closed. Then, it was committed in <a href="https://github.com/joyent/libuv/commit/47d98b64c45db8335bf7e065351e385cae32323d">47d98b6</a>. Then <a href="https://github.com/joyent/libuv/commit/804d40ee14dc0f82c482dcc8d1c41c14333fcb48">804d40ee</a> reverted that 47d98b6. But when you look at 804d40ee on GitHub, you’ll see no indication of which branch it’s on. That’s because it’s not on any branch. If you clone down libuv, and try to find the commit, you’ll see it’s not there:</p>
<pre><code>~/libuv(master)$ git log 804d40e
fatal: ambiguous argument '804d40e': unknown revision or path not in the working tree.
</code></pre>
<p>What gives?</p>
<p>Let’s make a test repository:</p>
<pre><code>$ mkdir test
$ cd test
$ git init
Initialized empty Git repository in /home/action/test/.git/
$ touch HELLO.md
$ git add HELLO.md
$ git commit -m "initial commit"
[master (root-commit) 646567b] initial commit
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 HELLO.md
</code></pre>
<p>Okay, now let’s make a second commit:</p>
<pre><code>$ touch WHATEVER.md
$ git add WHATEVER.md
$ git commit -m "adding whatever"
[master 7c232cc] adding whatever
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 WHATEVER.md
</code></pre>
<p>Actually, that commit was a mistake. Since we haven’t pushed anywhere yet, let’s just use <code>git reset --hard</code> to just throw it out.</p>
<pre><code>$ git reset --hard HEAD~1
HEAD is now at 646567b initial commit
</code></pre>
<p>But what about that commit? Where is it?</p>
<pre><code>$ git show 7c232cc
commit 7c232cceb94a2e7cdd95c526de785efe08da2325
Author: Steve Klabnik &lt;[email protected]&gt;
Date: Sat Nov 30 20:19:26 2013 +0000
adding whatever
diff --git a/WHATEVER.md b/WHATEVER.md
new file mode 100644
index 0000000..e69de29
</code></pre>
<p>It’s still in the repository. We can use <code>git reflog</code> to see our changes:</p>
<pre><code>$ git reflog
646567b HEAD@{0}: reset: moving to HEAD~1
7c232cc HEAD@{1}: commit: adding whatever
646567b HEAD@{2}: commit (initial): initial commit
</code></pre>
<p>Git calls these kinds of commits ‘unreachable’ because, while they exist in your repository, you can’t find them unless you know their SHA. They’re not connected to any tag or branch. We can use <code>git fsck</code> to find these for us automatically:</p>
<pre><code>$ git fsck --no-reflogs --unreachable
Checking object directories: 100% (256/256), done.
unreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5
unreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325
</code></pre>
<p>Some Git commands will run <code>git gc</code>, the Git garbage collector, as part of their normal operation. However, if we run <code>git gc</code> ourselves, it doesn’t look like anything happens:</p>
<pre><code>$ git gc
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (5/5), done.
Total 5 (delta 0), reused 0 (delta 0)
$ git fsck --no-reflogs --unreachable
Checking object directories: 100% (256/256), done.
Checking objects: 100% (5/5), done.
unreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5
unreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325
</code></pre>
<p>What gives? Well, <code>git gc</code> has some settings that indicate how long it will let an unreachable commit lay around in your repository. The <code>gc.pruneExpire</code> configuration variable controls this behavior, and it defaults to 14 days. So even if we try to throw away a commit, and even if we manually run the garbage collector, we’ll still have the commit for two weeks.</p>
<p>We can, of course, tell <code>git gc</code> to ignore the setting:</p>
<pre><code>$ git gc --prune=now
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (5/5), done.
Total 5 (delta 0), reused 5 (delta 0)
$ git fsck --no-reflogs --unreachable
Checking object directories: 100% (256/256), done.
Checking objects: 100% (5/5), done.
unreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5
unreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325
</code></pre>
<p>Uhhhh what? It turns out that <code>git gc</code> won’t touch commits that are still in our reflog. So let’s clear that:</p>
<pre><code>$ git reflog expire --expire=now --all
$ git reflog
$
</code></pre>
<p>Good. And now, let’s take out the garbage:</p>
<pre><code>$ git gc --prune=now
Counting objects: 3, done.
Writing objects: 100% (3/3), done.
Total 3 (delta 0), reused 3 (delta 0)
$ git fsck --no-reflogs --unreachable
Checking object directories: 100% (256/256), done.
Checking objects: 100% (3/3), done.
</code></pre>
<p>Easy! When you fetch from a remote repository, Git does not include unreachable commits. That’s why when we cloned down libuv earlier, we didn’t get the orphaned commit.</p>
<p>Anyway, as you can see, it’s actually really hard to lose your data with Git, even if you explicitly try to throw it away. This is why you can rebase with… well, maybe not reckless abandon, but you don’t have to worry about your data. All it takes is a quick <code>git reset --hard HEAD@{1}</code> (read: reset me to the last entry in the reflog) and you’re back to where you were before that operation you screwed up.</p>
<p>So, what happened with that commit on GitHub? Basically, one committer pushed 804d40ee, and then another did a <code>git reset --hard</code> to remove it, and then force pushed up to the repository. That commit still lives on GitHub, and is still viewable in their interface, but in some time, when GitHub runs <code>git gc</code> on the repository, it will go away.</p>
</content:encoded></item><item><title>Why I'm partnering with Balanced</title><link>https://steveklabnik.com/writing/why-im-partnering-with-balanced/</link><guid isPermaLink="true">https://steveklabnik.com/writing/why-im-partnering-with-balanced/</guid><pubDate>Thu, 21 Nov 2013 00:00:00 GMT</pubDate><content:encoded><p>My next venture will be joining up with <a href="https://www.balancedpayments.com/">Balanced</a>, an API for marketplace payments, and I’m really excited about it. To the tax man, of course, the title of this post is “Why I’m going to be an employee of Balanced,” but I like to think about it as a partnership.</p>
<h3>Why I Balanced</h3>
<p>I first heard of Balanced almost exactly a year ago. You see, my friend Chad had started this project called <a href="https://www.gittip.com/">Gittip</a>. He was using Stripe to handle payments, but Gittip is a marketplace, and Stripe didn’t support marketplaces. So they gave Gittip a month to find another processor. So <a href="https://github.com/gittip/www.gittip.com/pull/137">a Balanced employee came along and submitted a pull request</a>. I thought this was super awesome. So much so, that I tweeted it:</p>
<blockquote>
<p>@jordanmessina It was like ‘hmm we can't use stripe, who should we use?’ “oh hey, just swung by to integrate you with us. &lt;3”</p>
</blockquote>
<p>— Steve Klabnik (@steveklabnik) December 1, 2012</p>
<blockquote></blockquote>
<p>I still told people that story, but didn’t really think a whole lot more about Balanced than that. But every so often, I’d bump into them again. The next time Balanced came across my radar was when they <a href="http://blog.balancedpayments.com/benefits-open-company-balanced/">declared themselves an Open Company</a>. Whoah. Now, don’t get me wrong: it’s not exactly socialism, but it <em>is</em> super interesting. The idea of doing feature development and discussion out in the open is really interesting to me. For example, if you’re interested in seeing when Balanced supports international payments, you can check out <a href="https://github.com/balanced/balanced-api/issues/23">this Issue</a>, and when the feature happens, you’ll know. That feature is obvious, but for less obvious ones, like <a href="https://github.com/balanced/balanced-api/issues/204">Bitcoin support</a>, you can see that there’s real discussion, where the users who want a feature can help explain their use cases, Balanced will ask questions, and (hopefully) eventually it’ll get implemented.</p>
<p>Anyway, so when I was looking for something new, Balanced seemed like a good fit. I dropped by their offices a few times, and met with the team. I was impressed when they included their intern, Matthew, in one of our meetings. Most companies I’ve seen treat their interns really poorly. When I mentioned that I was a Python newbie, Balanced’s CTO, <a href="https://twitter.com/mahmoudimus">Mahmoud</a>, replied that that was perfect: if they only hired Python people, they’d only get Python answers, and monoculture is bad. During that discussion, I remembered my own startup, CloudFab, and remembered my cofounder’s issues getting us a bank account, because we were a marketplace. I had felt the pain Balanced is trying to solve before. At that point, I was in, but Mahmoud thanking me for my role in helping fight for equality in tech was extra icing on the cake.</p>
<h3>My role</h3>
<p>Anyway, that’s why I’m pumped about Balanced. So what exactly will I be doing? A few things, taken straight from my offer letter:</p>
<ol>
<li>Design Balanced’s API based on the needs of our customers and the feedback of the team</li>
<li>Lead Balanced’s path towards HTTP and JSON-API compliance</li>
<li>Move standards forward to support Balanced’s innovation</li>
<li>Build tools to allow Balanced and others to create standards compliant APIs</li>
<li>Promote and evangelize the benefits of the Open Company philosophy</li>
</ol>
<p>To expand on these five steps:</p>
<p>You can reach me at <a href="mailto:[email protected]">[email protected]</a>. I have uploaded <a href="http://pgp.mit.edu:11371/pks/lookup?op=vindex&amp;search=0x1782FE756ACC3706">a PGP key</a> to the MIT keyserver if that’s your jam.</p>
<p>If you’re a Balanced customer and you have thoughts, good or bad, about Balanced, please get in touch.</p>
<p>If you’re <em>not</em> a Balanced customer, I’d love to hear why. Send me an email.</p>
<p>I’ve joined the <a href="http://www.w3.org/2001/tag/">W3C TAG mailing list</a>, and will be getting even more involved in the web standards process than I already am, to make sure that Balanced is keeping abreast of the latest standards, and to make sure that Balanced’s use-cases are properly advocated for as part of that process.</p>
<p>I’ll be working hard on <a href="http://jsonapi.org/">JSON API</a>, both with Balanced’s customers, our team, and with others who want to use the standard. Some tickets have been a bit stagnant: it’s now a part of my job to resolve them.</p>
<p>I’ll be working on some tooling, hopefully with other companies that care about APIs, to build tooling that we all can use. If you’re in San Fransisco, I’d love to swing by your office and say hello. If not, email works too.</p>
<p>If you want me to come speak to your usergroup, meetup, or conference about APIs, web standards, Open Companies, or payment systems, let me know, I really enjoy it.</p>
<p>Expect to see a bunch of writing on the <a href="http://blog.balancedpayments.com/">Balanced blog</a> from me on all of those topics, as well.</p>
<p>I’m excited for the future. There’s so much work to do. Let’s all build some awesome APIs!</p>
</content:encoded></item><item><title>On leftist infighting</title><link>https://steveklabnik.com/writing/on-leftist-infighting/</link><guid isPermaLink="true">https://steveklabnik.com/writing/on-leftist-infighting/</guid><pubDate>Mon, 18 Nov 2013 00:00:00 GMT</pubDate><content:encoded><p>There used to be a post here: <a href="https://web.archive.org/web/20131118145002/http://words.steveklabnik.com/on-leftist-infighting">https://web.archive.org/web/20131118145002/http://words.steveklabnik.com/on-leftist-infighting</a></p>
<p>After having some conversations on Twitter, I was made aware of some context around ‘infighting’ that made it problematic. I decided that the post was unworkable, as I certainly didn’t mean some of the things it implied, and so decided it was better to simply delete it.</p>
<p>I’ve been told that some people have been trying to cite this post as me saying that I believe any and all dissent should not be public, or that I believe in covering up incidents of wrongdoing because they may ‘harm the movement’ or something. This is wrong. I wrote this post after trying a number of different tactics to get other men to stop tone policing several women, specifically Shanley. “If you don’t like the way she says things, why not distract from the issue at hand and instead say nothing, and focus on the misogyny instead” was one strategy that actually worked. Hence the post. But since I didn’t just straight-up come out and say that, it failed.</p>
<p>Wrongdoing should be called out as wrong, and tone policing is bad. End of story.</p>
</content:encoded></item><item><title>Abstract vs concrete</title><link>https://steveklabnik.com/writing/abstract-vs-concrete/</link><guid isPermaLink="true">https://steveklabnik.com/writing/abstract-vs-concrete/</guid><pubDate>Thu, 14 Nov 2013 00:00:00 GMT</pubDate><content:encoded><p>When I talk to people about systemic inequalities, one of the hardest things for them to understand is the concrete nature of how these relations affect the day-to-day lives of others. For many, this topic is an abstract discussion. For many others, this is life, day to day.</p>
<p>In the interest of helping bridge this gap, I want to share a story with you from a few weeks ago. The setting: the West Village, Manhattan. The night: Halloween. In the early morning hours of All Saint’s Day, I got a text message from someone I care about. Here’s some screenshots of the conversation. She has seen this post and said it’s okay to share, for the same reasons I’m interested in sharing this with you. Also, if reading a story about a bad situation or a slur or two will make you upset, you may want to read another blog post.</p>
<p>I’m not sure how to convince random people on Twitter that yes, in 2013, in the West Villiage, in the largest city in the country, on a holiday night when everyone is out, you still can’t let your guard down. Even if you’re not actually a gay man.</p>
<p>And this is just one story.</p>
</content:encoded></item><item><title>Pointers in Rust, a guide</title><link>https://steveklabnik.com/writing/pointers-in-rust-a-guide/</link><guid isPermaLink="true">https://steveklabnik.com/writing/pointers-in-rust-a-guide/</guid><pubDate>Fri, 18 Oct 2013 00:00:00 GMT</pubDate><content:encoded><p>Rust’s pointers are one of its more unique and compelling features. Pointers are also one of the more confusing topics for newcomers to Rust. They can also be confusing for people coming from other languages that support pointers, such as C++. This tutorial will help you understand this important topic.</p>
<h1>You don’t actually need pointers</h1>
<p>I have good news for you: you probably don’t need to care about pointers, especially as you’re getting started. Think of it this way: Rust is a language that emphasizes safety. Pointers, as the joke goes, are very pointy: it’s easy to accidentally stab yourself. Therefore, Rust is made in a way such that you don’t need them very often.</p>
<p>“But tutorial!” you may cry. "My co-worker wrote a function that looks like this:</p>
<pre><code>fn succ(x: &amp;int) -&gt; int { *x + 1 }
</code></pre>
<p>So I wrote this code to try it out:</p>
<pre><code>fn main() {
let number = 5;
let succ_number = succ(number);
println!("{}", succ_number);
}
</code></pre>
<p>And now I get an error:</p>
<pre><code>error: mismatched types: expected `&amp;int` but found `&lt;VI0&gt;` (expected &amp;-ptr but found integral variable)
</code></pre>
<p>What gives? It needs a pointer! Therefore I have to use pointers!"</p>
<p>Turns out, you don’t. All you need is a reference. Try this on for size:</p>
<pre><code>fn main() {
let number = 5;
let succ_number = succ(&amp;number);
println!("{}", succ_number);
}
</code></pre>
<p>It’s that easy! One extra little <code>&amp;</code> there. This code will run, and print <code>6</code>.</p>
<p>That’s all you need to know. Your co-worker could have written the function like this:</p>
<pre><code>fn succ(x: int) -&gt; int { x + 1 }
fn main() {
let number = 5;
let succ_number = succ(number);
println!("{}", succ_number);
}
</code></pre>
<p>No pointers even needed. Then again, this is a simple example. I assume that your real-world <code>succ</code> function is more complicated, and maybe your co-worker had a good reason for <code>x</code> to be a pointer of some kind. In that case, references are your best friend. Don’t worry about it, life is too short.</p>
<p>However.</p>
<p>Here are the use-cases for pointers. I’ve prefixed them with the name of the pointer that satisfies that use-case:</p>
<ol>
<li>Owned: ~Trait must be a pointer, becuase you don’t know the size of the object, so indirection is mandatory.</li>
<li>Owned: You need a recursive data structure. These can be infinite sized, so indirection is mandatory.</li>
<li>Owned: A very, very, very rare situation in which you have a <em>huge</em> chunk of data that you wish to pass to many methods. Passing a pointer will make this more efficient. If you’re coming from another language where this technique is common, such as C++, please read “A note…” below.</li>
<li>Managed: Having only a single owner to a piece of data would be inconvenient or impossible. This is only often useful when a program is very large or very complicated. Using a managed pointer will activate Rust’s garbage collection mechanism.</li>
<li>Borrowed: You’re writing a function, and you need a pointer, but you don’t care about its ownership. If you make the argument a borrowed pointer, callers can send in whatever kind they want.</li>
</ol>
<p>Five exceptions. That’s it. Otherwise, you shouldn’t need them. Be skeptical of pointers in Rust: use them for a deliberate purpose, not just to make the compiler happy.</p>
<h2>A note for those proficient in pointers</h2>
<p>If you’re coming to Rust from a language like C or C++, you may be used to passing things by reference, or passing things by pointer. In some languages, like Java, you can’t even have objects without a pointer to them. Therefore, if you were writing this Rust code:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn main() {
let p0 = Point { x: 5, y: 10};
let p1 = transform(p0);
println!("{:?}", p1);
}
</code></pre>
<p>I think you’d implement <code>transform</code> like this:</p>
<pre><code>fn transform(p: &amp;Point) -&gt; Point {
Point { x: p.x + 1, y: p.y + 1}
}
// and change this:
let p1 = transform(&amp;p0);
</code></pre>
<p>This does work, but you don’t need to create those references! The better way to write this is simply:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn transform(p: Point) -&gt; Point {
Point { x: p.x + 1, y: p.y + 1}
}
fn main() {
let p0 = Point { x: 5, y: 10};
let p1 = transform(p0);
println!("{:?}", p1);
}
</code></pre>
<p>But won’t this be inefficent? Well, that’s a complicated question, but it’s important to know that Rust, like C and C++, store aggregate data types ‘unboxed,’ whereas languages like Java and Ruby store these types as ‘boxed.’ For smaller structs, this way will be more efficient. For larger ones, it may be less so. But don’t reach for that pointer until you must! Make sure that the struct is large enough by performing some tests before you add in the complexity of pointers.</p>
<h1>Owned Pointers</h1>
<p>Owned pointers are the conceptually simplest kind of pointer in Rust. A rough approximation of owned pointers follows:</p>
<ol>
<li>Only one owned pointer may exist to a particular place in memory. It may be borrowed from that owner, however.</li>
<li>The Rust compiler uses static analysis to determine where the pointer is in scope, and handles allocating and de-allocating that memory. Owned pointers are not garbage collected.</li>
</ol>
<p>These two properties make for three use cases.</p>
<h2>References to Traits</h2>
<p>Traits must be referenced through a pointer, becuase the struct that implements the trait may be a different size than a different struct that implements the trait. Therefore, unboxed traits don’t make any sense, and aren’t allowed.</p>
<h2>Recursive Data Structures</h2>
<p>Sometimes, you need a recursive data structure. The simplest is known as a ‘cons list’:</p>
<pre><code>enum List&lt;T&gt; {
Nil,
Cons(T, ~List&lt;T&gt;),
}
fn main() {
let list: List&lt;int&gt; = Cons(1, ~Cons(2, ~Cons(3, ~Nil)));
println!("{:?}", list);
}
</code></pre>
<p>This prints:</p>
<pre><code>Cons(1, ~Cons(2, ~Cons(3, ~Nil)))
</code></pre>
<p>The inner lists <em>must</em> be an owned pointer, becuase we can’t know how many elements are in the list. Without knowing the length, we don’t know the size, and therefore require the indirection that pointers offer.</p>
<h2>Efficiency</h2>
<p>This should almost never be a concern, but because creating an owned pointer boxes its value, it therefore makes referring to the value the size of the box. This may make passing an owned pointer to a function less expensive than passing the value itself. Don’t worry yourself with this case until you’ve proved that it’s an issue through benchmarks.</p>
<p>For example, this will work:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn main() {
let a = Point { x: 10, y: 20 };
do spawn {
println(a.x.to_str());
}
}
</code></pre>
<p>This struct is tiny, so it’s fine. If <code>Point</code> were large, this would be more efficient:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn main() {
let a = ~Point { x: 10, y: 20 };
do spawn {
println(a.x.to_str());
}
}
</code></pre>
<p>Now it’ll be copying a pointer-sized chunk of memory rather than the whole struct.</p>
<h1>Managed Pointers</h1>
<p>Managed pointers, notated by an <code>@</code>, are used when having a single owner for some data isn’t convenient or possible. This generally happens when your program is very large and complicated.</p>
<p>For example, let’s say you’re using an owned pointer, and you want to do this:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn main() {
let a = ~Point { x: 10, y: 20 };
let b = a;
println(b.x.to_str());
println(a.x.to_str());
}
</code></pre>
<p>You’ll get this error:</p>
<pre><code>test.rs:10:12: 10:13 error: use of moved value: `a`
test.rs:10 println(a.x.to_str());
^
test.rs:8:8: 8:9 note: `a` moved here because it has type `~Point`, which is moved by default (use `ref` to override)
test.rs:8 let b = a;
^
</code></pre>
<p>As the message says, owned pointers only allow for one owner at a time. When you assign <code>a</code> to <code>b</code>, <code>a</code> becomes invalid. Change your code to this, however:</p>
<pre><code>struct Point {
x: int,
y: int,
}
fn main() {
let a = @Point { x: 10, y: 20 };
let b = a;
println(b.x.to_str());
println(a.x.to_str());
}
</code></pre>
<p>And it works:</p>
<pre><code>10
10
</code></pre>
<p>So why not just use managed pointers everywhere? There are two big drawbacks to managed pointers:</p>
<ol>
<li>They activate Rust’s garbage collector. Other pointer types don’t share this drawback.</li>
<li>You cannot pass this data to another task. Shared ownership across concurrency boundaries is the source of endless pain in other languages, so Rust does not let you do this.</li>
</ol>
<h1>Borrowed Pointers</h1>
<p>Borrowed pointers are the third major kind of pointer Rust supports. They are simultaneously the simplest and the most complicated kind. Let me explain: they’re called ‘borrowed’ pointers because they claim no ownership over the data they’re pointing to. They’re just borrowing it for a while. So in that sense, they’re simple: just keep whatever ownership the data already has. For example:</p>
<pre><code>use std::num::sqrt;
struct Point {
x: float,
y: float,
}
fn compute_distance(p1: &amp;Point, p2: &amp;Point) -&gt; float {
let x_d = p1.x - p2.x;
let y_d = p1.y - p2.y;
sqrt(x_d * x_d + y_d * y_d)
}
fn main() {
let origin = @Point { x: 0.0, y: 0.0 };
let p1 = ~Point { x: 5.0, y: 3.0 };
println!("{:?}", compute_distance(origin, p1));
}
</code></pre>
<p>This prints <code>5.83095189</code>. You can see that the <code>compute_distance</code> function takes in two borrowed pointers, but we give it a managed and unique pointer. Of course, if this were a real program, we wouldn’t have any of these pointers, they’re just there to demonstrate the concepts.</p>
<p>So how is this hard? Well, because we’re igorning ownership, the compiler needs to take great care to make sure that everything is safe. Despite their complete safety, a borrowed pointer’s representation at runtime is the same as that of an ordinary pointer in a C program. They introduce zero overhead. The compiler does all safety checks at compile time.</p>
<p>This theory is called ‘region pointers,’ and involve a concept called ‘lifetimes’. Here’s the simple explanation: would you expect this code to compile?</p>
<pre><code>fn main() {
println(x.to_str());
let x = 5;
}
</code></pre>
<p>Probably not. That’s becuase you know that the name <code>x</code> is valid from where it’s declared to when it goes out of scope. In this case, that’s the end of the <code>main</code> function. So you know this code will cause an error. We call this duration a ‘lifetime’. Let’s try a more complex example:</p>
<pre><code>fn main() {
let mut x = ~5;
if(*x &lt; 10) {
let y = &amp;x;
println!("Oh no: {:?}", y);
return;
}
*x = *x - 1;
println!("Oh no: {:?}", x);
}
</code></pre>
<p>Here, we’re borrowing a pointer to <code>x</code> inside of the <code>if</code>. The compiler, however, is able to determine that that pointer will go out of scope without <code>x</code> being mutated, and therefore, lets us pass. This wouldn’t work:</p>
<pre><code>fn main() {
let mut x = ~5;
if(*x &lt; 10) {
let y = &amp;x;
*x = *x - 1;
println!("Oh no: {:?}", y);
return;
}
*x = *x - 1;
println!("Oh no: {:?}", x);
}
</code></pre>
<p>It gives this error:</p>
<pre><code>test.rs:5:8: 5:10 error: cannot assign to `*x` because it is borrowed
test.rs:5 *x = *x - 1;
^~
test.rs:4:16: 4:18 note: borrow of `*x` occurs here
test.rs:4 let y = &amp;x;
^~
</code></pre>
<p>As you might guess, this kind of analysis is complex for a human, and therefore hard for a computer, too! There is an entire <a href="http://static.rust-lang.org/doc/0.8/tutorial-borrowed-ptr.html">tutorial devoted to borrowed pointers and lifetimes</a> that goes into lifetimes in great detail, so if you want the full details, check that out.</p>
<h1>Returning Pointers</h1>
<p>We’ve talked a lot about funtions that accept various kinds of pointers, but what about returning them? Here’s the rule of thumb: only return a unique or managed pointer if you were given one in the first place.</p>
<p>What does that mean? Don’t do this:</p>
<pre><code>fn foo(x: ~int) -&gt; ~int {
return ~*x;
}
fn main() {
let x = ~5;
let y = foo(x);
}
</code></pre>
<p>Do this:</p>
<pre><code>fn foo(x: ~int) -&gt; int {
return *x;
}
fn main() {
let x = ~5;
let y = ~foo(x);
}
</code></pre>
<p>This gives you flexibility, without sacrificing performance. For example, this will also work:</p>
<pre><code>fn foo(x: ~int) -&gt; int {
return *x;
}
fn main() {
let x = ~5;
let y = @foo(x);
}
</code></pre>
<p>You may think that this gives us terrible performance: return a value and then immediately box it up?!?! Isn’t that the worst of both worlds? Rust is smarter than that. There is no copy in this code. <code>main</code> allocates enough room for the <code>@int</code>, passes it into <code>foo</code> as <code>x</code>, and then <code>foo</code> writes the value into the new box. This writes the return value directly into the allocated box.</p>
<p>This is important enough that it bears repeating: pointers are not for optimizing returning values from your code. Allow the caller to choose how they want to use your output.</p>
</content:encoded></item><item><title>New Beginnings</title><link>https://steveklabnik.com/writing/new-beginnings/</link><guid isPermaLink="true">https://steveklabnik.com/writing/new-beginnings/</guid><pubDate>Tue, 01 Oct 2013 00:00:00 GMT</pubDate><content:encoded><p>TL;DR:</p>
<blockquote>
<p>WANTED: Proletarian/petite bourgeoisie looking for capitalist/industrialist/haute bourgeoisie. I’ve got the labor power, you’ve got the money commodity. Let’s exchange commodities at full market value in accordance with the Law of Value; I know you’ll extract some surplus and alienate my species-being, but I really need to regenerate and reproduce myself so that I can be around to labor tomorrow.</p>
</blockquote>
<p>Yes, that’s right, I am officially looking for work. I loved my job at <a href="http://jumpstartlab.com/">Jumpstart Lab</a>, but all good things must come to an end. So here we are. There’s a <a href="http://jumpstartlab.com/news/archives/2013/10/01/a-fond-farewell">farewell post</a> on the JSL blog too.</p>
<p>So, what’s next? To be honest, I’m not fully sure. I’m open to suggestions, and I give bonus points for interesting.</p>
<p>Things I want out of life:</p>
<ol>
<li>Travel. I still like doing this. I just don’t want to do it <em>constantly</em>.</li>
<li>Related: some degree of location independence. I’ll only accept positions that require me to be in a seat that’s in San Francisco or New York City. Remote would be even better. Probably.</li>
<li>Piles of money. I have student loan debt I’d prefer not to have.</li>
<li>Side project ability. I am highly motivated by doing side projects that are different than I do all day.</li>
</ol>
<p>… I’m not saying that Ruby is a minus, but I’m not saying it’s a plus, either.</p>
<p>I’m also open to re-locating out of the country. That certainly counts as interesting.</p>
<p>So, if you think we might work well together, please, <a href="mailto:[email protected]">send me an email</a>.</p>
</content:encoded></item><item><title>Keep Saturdays sacred</title><link>https://steveklabnik.com/writing/keep-saturdays-sacred/</link><guid isPermaLink="true">https://steveklabnik.com/writing/keep-saturdays-sacred/</guid><pubDate>Sat, 28 Sep 2013 00:00:00 GMT</pubDate><content:encoded><p>There are obviously a lot of factors that have contributed to my career as a programmer, but I think the most important one can be summed up in this sentence: Keep Saturdays sacred.</p>
<p>You see, in college, my friends and I were moderately obsessed with writing software. We sorta stumbled onto a tradition of making sure that every Saturday, we got together and wrote code. At some point, we realized that this was the formula:</p>
<ol>
<li>If none of us set our alarms, we’d all wander into the computer lab at 1pm Saturday.</li>
<li>We then went and got some burritos from Chipotle.</li>
<li>Once we were appropriately stuffed, we’d go back to the lab and hack on whatever.</li>
<li>In our town, food was half off at certain restaurants after 11pm. So we’d code until then, and then go get some food, and then go home later.</li>
</ol>
<p>Many people sifted in and out of this group over time, but a core was <em>always</em> there. Every week. For two or three years. I was one of those people.</p>
<p>Looking back, I think that time was where I grew the most as a software developer. I’ve been coding since I was very young, but didn’t really know what I was doing until that point. It’s where I tried out new stuff. Argued about methodology. Learned something about something.</p>
<p>We weren’t even always working on the same projects. They were often quite different, and changed all the time. But it was a safe place to try things, a safe space to fail. A place to get support and help. To socialize and figure out just what the hell we were even doing.</p>
<p>It was also an incredible privilege. Over the last few years, I’ve been too busy to keep that going. Life gets in the way. It’s hard when you don’t have that group. Maybe the solution, then, isn’t that it’s all day on a Saturday. Maybe it’s just finding <em>some</em> kind of gap time, but doing it every week.</p>
<p>Today, I’m doing work on http://www.designinghypermediaapis.com/ and the Rust compiler. What about you?</p>
</content:encoded></item><item><title>Thankfulness</title><link>https://steveklabnik.com/writing/thankfulness/</link><guid isPermaLink="true">https://steveklabnik.com/writing/thankfulness/</guid><pubDate>Tue, 24 Sep 2013 00:00:00 GMT</pubDate><content:encoded><p>I recently attended GoGaRuCo, which was a great conference. My favorite part, however, wasn’t directly attatched to the conf itself: it was a few moments of thankfulness.</p>
<p>A lot of what I’ve been experiencing lately has been profoundly negative. That is what it is; there’s a lot of realy negative things in the world. But I’ve been a bit… overloaded lately.</p>
<p>The two moments tie into each other:</p>
<p>I’m at the speaker dinner. I bumped into someone I hadn’t met or said hi to yet. “Oh hey Steve, I’m <a href="https://twitter.com/samsaffron">Sam</a>.” “Oh hi! You don’t know how many times in the last few weeks I’ve said ‘Sam Saffron makes me so happy.’” You see, Sam works on <a href="http://www.discourse.org/">Discourse</a>, and it’s a Rails app. Sam’s work on Discourse has identified some bugs and performance regressions in the Rails codebase, and Sam has been submitting some great pull requests upstream. Lately, I’ve been feeling like nobody who uses Rails actually cares how the sausage is made, and so Sam has been a bright spot in my day. So I told him so, straight to his face. It felt really good to say thanks, and I hope I made Sam’s day a bit brighter, as well.</p>
<p>A couple times at the conference, someone would walk up to me and say basically the same thing for some other reason. It felt really good to have my work appreciated; the day-to-day grind of GitHub issue triaging can really make you feel crappy. Basically:</p>
<p><img src="http://s3.amazonaws.com/theoatmeal-img/comics/making_things/14.jpg" alt="http://s3.amazonaws.com/theoatmeal-img/comics/making_things/14.jpg" /></p>
<p>negative comments</p>
<p><a href="http://theoatmeal.com/comics/making_things">The Oatmeal</a> is often problematic, but this one really rings true.</p>
<p>Anyway, I’m not trying to fish for compliments here. I have a fairly high profile, and I hear this more than most. What I <em>do</em> want to emphasize, though, is that I’m sure it’s not just me that feels this way.</p>
<p>So, if someone does something that you appreciate, please take a moment to say thank you. <em>Especially</em> if they’re not someone who happened to get up on a stage and give a talk. Be specific: a generic ‘thanks for what you do’ is nice, but ‘oh man I lost two hours on that bug and your patch saved me’ is even nicer.</p>
<p>You never know. They might have been having a bad day too, your thanks might turn their day around.</p>
</content:encoded></item><item><title>The CLOSURE companion</title><link>https://steveklabnik.com/writing/the-closure-companion/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-closure-companion/</guid><pubDate>Thu, 19 Sep 2013 00:00:00 GMT</pubDate><content:encoded><p>Today is the fourth <a href="http://whyday.org/">_why day</a>. So I thought I’d say a few words about his final gift to the rest of us, <a href="http://words.steveklabnik.com/closure">CLOSURE</a>.</p>
<h2>Be Skeptical</h2>
<p>First, a few words about the words I’m about to write. Software developers (though you don’t have to be one to read this) love meta, so surely you’ll indulge me for these next few paragraphs. Don’t worry, it’s only a little bit of meta-data.</p>
<blockquote>
<p>And if you don’t understand some of my language, things like “Open GL” and “fighting NULL” and so on, you can skip it and be fine I think. - _why, CLOSURE</p>
</blockquote>
<blockquote>
<p>Nowadays people are actually somewhat jaded by the term ``postmodern’’. Well, perhaps jaded is an understatement. Nauseated might be more like it. - Larry Wall, “Perl, the first post-modern programming language”</p>
</blockquote>
<blockquote>
<p>I define postmodern as incredulity toward meta-narratives, - Lyotard, “The Postmodern Condition: a report on knowledge”</p>
</blockquote>
<p>_why’s work has always taken on a strong element of the postmodern, and CLOSURE is no different. So in order to really get CLOSURE, you need to get a little postmodern. So let’s breifly examine this statement by Lyotard.</p>
<p>A ‘narrative’, to Lyotard, is a story that organizes tribal knowledge, the structure of our life and society. Therefore, a meta-narrative is a narrative about a narrative. As an example, let’s say you want to make sense of programming languages. A meta-narrative about programming languages would be that in the beginning, we started off with assembly language. Then we moved forward to structured programming, with Pascal and C and such. Next came object oriented languages, which is where we’re mostly at today. We’re starting to finally lean functional.</p>
<p>Now, this narrative works. It’s not <em>wrong</em>. But it also isn’t really <em>true</em>, either. After all, Lisp existed waaaay before the contemporary moment, yet it’s often functional. Actually, presenting Lisp as functional is in of itself not exactly true. So this meta-narrative, while it helps us understand a bit about programming languages and their history, also has a bunch of lies. It’s not black and white, it’s quite gray.</p>
<blockquote>
<p>"Well, it doesn’t matter – no one’s going to listen to either of us. I can say whatever I want at this point. You can tell the truth and no one would care.“That’s amazing.”“I’m going to change my story weekly – the more confusion the better. The Jonathan story is locked.”_why, CLOSURE, HELLOYES (p56)</p>
</blockquote>
<p>So what I’m saying is this: I’m about to present you with a meta-narrative about CLOSURE. So be skeptical of me! What I’m about to tell you is a lie, I swear. Don’t take what’s about to come as the final word, some sort of actual truth. It’s just my own little anti-memory that I’d like to share with you.</p>
<h2>What CLOSURE is</h2>
<p>CLOSURE is a few different things. 90% of what you need to know is located in <a href="https://github.com/steveklabnik/CLOSURE/blob/master/PDF/DISCLAIMER.pdf">DISCLAIMER</a>, which is page 13 of CLOSURE. In it, _why mentions a few things:</p>
<ol>
<li>It’s ‘perilous to communicate this way’</li>
<li>Everything he’s done belongs to us</li>
<li>He likes what we’ve done so far</li>
<li>This will be the last thing from him.</li>
</ol>
<p>#4 is a big deal. No more waiting for _why to come back. That’s it. No more questions, no more wondering, no more guesses. This is all we have.</p>
<p>For me, #3 was the most important. It brought me to tears, to be honest. That’s what I personally really needed to hear. It’s very strange to take up the work of someone you’ve never met, and make their life’s work your own. I spent quite a bit of time really wondering if keeping some of his stuff alive was the correct thing to do, and so this was a big relief. It was all gravy after that. #2 is related, it’s all public domain. That’s good to know.</p>
<h2>Identity</h2>
<p>#1 is interesting. It could just be flavor, but I think it’s something more than that. One of the major themes of CLOSURE is identity, and _why / Jonathan ’s struggles with it. On page 19:</p>
<blockquote>
<p>Now, could you please tell me the point of this ridiculous anonymity exercise, hmm?As it turns out, oddly enough, your real self is just an unknown programmer from Utah. The myth is that easily dispelled. Why not making something of your real self? (Of course I know why and can tell you: Because your fear of the world has clouded your ability to do things to improve your situation. You are stuck there in Draper, Utah, until you can cut through the paranoia!)Please, Mr. Gillette, come on in. The water’s fine. ;)Emery Pestus, CLOSURE</p>
</blockquote>
<p>While many hold _why as some kind of superhero, it’s really important to remember that he’s human. We, the Ruby community, placed an indescribable amount of pressure on a human being to play a character for us, without consideration for his feelings. Think about this for a moment: could Jonathan ever write Ruby code? As Jonathan? What if it had unit tests? If it was just some old, normal, boring gem that added a little feature to Rails?</p>
<p>Could we let him just be him? I’m not sure. I’m not sure he could let him just be him, either.</p>
<p>Once, I met someone for dinner in Chicago. This was the first time we’d met, and so we were asking each other a lot of questions. One of my answers yielded an interesting response: “Oh, you’re like actually for-real. All that twitter stuff isn’t an act. That’s really you.”</p>
<h2>Impermanance</h2>
<p>Impermanence is possibly the biggest question raised in CLOSURE.</p>
<blockquote>
<p>kafka would be a lot harder to get into if the trial only ran on a power pc. - one of _why’s last tweets</p>
</blockquote>
<p>This tweet was really confusing, until CLOSURE. _why reveals that one of his biggest problems is what we call ‘bitrot’: you can’t just write a program, it must be updated. I have one of the first C programs I ever wrote, from when I was 12, and it wouldn’t compile on a modern system, due to a ‘clear screen’ function I wrote that interacted with the hardware that I owned at the time. I don’t have any of the GW-BASIC programs that I wrote as a child, because there was no way to transfer the source off of the computer: I didn’t have a disk drive, they were too expensive.</p>
<p>And so it is with Kafka. Right before he died, Kafka asked his friend Max Brod to burn everything he ever wrote. Brod published it instead. _why brings up that if The Trial was written for the PowerPC, he wouldn’t have needed Brod to burn it: it would have just naturally gone away.</p>
<p>Our industry is constantly changing, and that’s great. But we have no institutional memory. We keep making the same mistakes, over and over again. _why talks about fighting NULL, and how that was his biggest struggle as a programmer. The guy who invented null pointers, Tony Hoare, calls it <a href="http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare">“my billion dollar mistake”</a>. Yet Go has null pointers.</p>
<p>It’s really easy to burn out. I won’t lie, when _why deleted himself, I thought it was a terribly silly idea. But the more stuff I do, the bigger my projects and responsibilities get, and the more of a public person I am, the more it sounds appealing.</p>
<p>When I had a shitty band in high school, I wrote a dumb song after a girl dumped me called “The Only Constant is Change.” I thought it was clever at the time.</p>
<h2>The Software Industry</h2>
<p>Speaking of the industry, and burnout, that’s the last part of CLOSURE, and the hardest. The entire end of the book is about it, and it’s a very obvious, very ridiculous allegory.</p>
<p>There is a cult of Steve Jobs, and they speak incorrect French. They obsess over these flutes, and the flutes can make more flutes, and when one of them gets old and dies, they forget him immediately. When a new person is born, they start worshipping him, and induct him into the cult as quickly as possible, by clothing him in blue jeans and a black turtleneck. The cult members never eat any food.</p>
<p>The cult is programmers, the flutes are programming. It all falls out of that. Songs on the flute always start with C#, which I found amusing.</p>
<blockquote>
<p>It occured to me that they could use the flutes to describe a whole landscape, to transfer a map from one man’s mind to another._why, CLOSURE, page 84: SPAWN</p>
</blockquote>
<p>There’s also a really fantastic comparison between Steve Jobs and Bill Gates, as well as an angel investor who wants to invest in _why: I highly recommend that part.</p>
<blockquote>
<p>For instance, many people struggle with acceptance, feeling like they aren’t accepted by other people. But what I deal with is primarily hatred of entrepreneurs. But it’s something that I’m always working on and I’ve gotten much better.</p>
</blockquote>
<p>I’ve come to realize that I’m starting to become a grumpy old man. These damn kids never learn from their elders, we solved all the world’s problems back in the 60s and 70s, and they don’t know anything. I wish we had a better way to transfer institutional knowledge and stop repeating ourselves.</p>
<h2>Some final thoughts</h2>
<p>There is so much more that’s enjoyable in CLOSURE. Please check out the full thing, even if you don’t get all of it. For example, the part on Neil Gaiman is amazing.</p>
<p>I hope these jumping off points helps you in your journey. Let’s all have some fun with programming, okay?</p>
</content:encoded></item><item><title>Resque 1.25.0 has been released</title><link>https://steveklabnik.com/writing/resque-1-25-0-has-been-released/</link><guid isPermaLink="true">https://steveklabnik.com/writing/resque-1-25-0-has-been-released/</guid><pubDate>Mon, 16 Sep 2013 00:00:00 GMT</pubDate><content:encoded><p>I’ve just released Resque 1.25.0! Resque is the most stable, widely-used job queue for Ruby. It uses Redis as its backing store.</p>
<p>This release is thanks to <a href="https://github.com/adelcambre">adelcambre</a>, who took the time to get it over the finish line! &lt;3.</p>
<p>A total of 19 contributors helped out with this release, with 77 commits. Thank you to everyone!</p>
<p>This release contains no changes from the 1.25.0.pre release that was put out a few weeks ago, so if you’re using it, you can upgrade with no problems.</p>
<h2>CHANGES</h2>
<p>You can see the full CHANGELOG <a href="https://github.com/resque/resque/blob/v1.25.0/HISTORY.md">here</a>.</p>
<p>This release had a pre-release because of one big change: <a href="https://github.com/resque/resque/pull/1017/files">a refactor to the forking code</a>. This was introduced to fix a problem: the exit syscall raises a SystemExit Exception in ruby, its exit is treated as a failure, thus deregistering the worker. After the refactor + fix, things should be peachy. But please give the prerelease a try.</p>
<p>In addition, it has been discovered that we have accidentally introduced a SemVer violation back in v1.23.1: https://github.com/resque/resque/issues/1074</p>
<p>I take SemVer seriously, and apologize for this change. SemVer states that at this time, I should basically do whatever. Since this fixes a bug, I’m inclined to leave it in. If this introduces a significant problem for you, please let me know. Since others might now be relying on this new behavior, I’m not 100% sure what the best option is. Your feedback helps.</p>
<h2>INSTALLING</h2>
<p>To get 1.25.0 installed, just use <code>gem install</code>:</p>
<pre><code>$ gem install resque
</code></pre>
<p>Or include it in your <code>Gemfile</code>:</p>
<pre><code>gem 'resque', '~&gt; 1.25.0'
</code></pre>
<h2>SOURCE</h2>
<p>You can find the source <a href="https://github.com/resque/resque">on GitHub</a>.</p>
</content:encoded></item><item><title>Just the regularly scheduled apocalypse</title><link>https://steveklabnik.com/writing/just-the-regularly-scheduled-apocalypse/</link><guid isPermaLink="true">https://steveklabnik.com/writing/just-the-regularly-scheduled-apocalypse/</guid><pubDate>Tue, 10 Sep 2013 00:00:00 GMT</pubDate><content:encoded><p>Every once in a while, I unfollow EVERYONE on Twitter. I did it again last night. Feels good, like a fresh start.</p>
<p>Part of the problem is that I feel like I should be following everyone who I know in real life. This causes Twitter to be absolutely unmanageable, as I’m following far too many. So then I just dip into the stream every so often, but then I miss some people’s tweets entirely. So why follow them all in the first place?</p>
<p>So, if I’ve unfollowed you, don’t worry, I still &lt;3 you. And even as I start following people back, I’m doing it slowly, and at semi-random. Trying to get some people I hadn’t seen in a while. People from outside my own community. People that’ll make me a little less caremad.</p>
<p>Yeah, like that’ll last.</p>
</content:encoded></item><item><title>Emoji licensing</title><link>https://steveklabnik.com/writing/emoji-licensing/</link><guid isPermaLink="true">https://steveklabnik.com/writing/emoji-licensing/</guid><pubDate>Tue, 06 Aug 2013 00:00:00 GMT</pubDate><content:encoded><p>I recently decided to make an emoji gem. Of course, as with any project, I first look into the relevant licensing issues. What I found made me <code>:cry:</code>. Here’s the low-down on emoji and intellectual property law.</p>
<h2>A history</h2>
<p>So what <em>are</em> emoji, anyway? The Japanese spelling is 絵文字: 絵 (e) means ‘picture’ and 文字 (moji) means ‘letter.’ Picture letters. Simple.</p>
<p>Back in the day, three different Japanese phone carriers created their own versions of these smilies: docomo, au, and SoftBank. Codes would be sent down the (nonexistant) wire, which would then be turned into the final character. However, the three carriers used different characters and different codes.</p>
<p>Google and Apple petitioned the Unicode Consortium to include a number of these characters, and version 6.0 of the Unicode Standard ended up with 722 emoji.</p>
<h2>The font</h2>
<p>Apple ended up implementing a font named Apple Color Emoji that included these symbols. It came first with OS X Lion and iOS 5. These are the characters that you’re probably familliar with. Apple actually implements the font with a proprietary extension to OpenType, but that’s not particularly important. What <em>is</em> important is two things: typing a character that’s not on a normal keyboard, and displaying custom fonts on the web. These two problems have led to the current emoji IP situation.</p>
<h3>Typing extended characters</h3>
<p>On an iPhone, you can type emoji easily by enabling the extra keyboard. That’s not hard. But what about on a computer? Web sites that wanted to use emoji needed an easy way for you to type these characters in an input box.</p>
<p>I’m not sure who was first, but the first I remember is GitHub: with their <a href="https://github.com/blog/816-emoji">announcement post</a>, showed off a slick way to input emojis: use a set of characters that would get replaced with the emoji. So <code>:heart:</code> becomes a heart. Easy!</p>
<h3>Custom fonts in browsers</h3>
<p>That takes care of input, but what about output? If you don’t have the proper font installed, you’ll get those terrible boxes. That’s not very useable. Also, even if you do, <a href="http://code.google.com/p/chromium/issues/detail?id=90177">Chrome won’t display them</a>.</p>
<p>So what do you do? You use your good buddy <code>&lt;img&gt;</code>.</p>
<p>If you’re building a web application, you are already telling users to type <code>:heart:</code> rather than the character, you can just make <code>:heart:</code> be rendered as <code>&lt;img src="/assets/heart.png"&gt;</code>. So that’s easy…. but where do you get the PNGs?</p>
<h2>Apple Color Emoji</h2>
<p>Remember that extension? <a href="https://color-emoji.googlecode.com/git/specification/v1.html">The spec was published</a>, and is being worked in to <a href="http://google-opensource.blogspot.de/2013/05/open-standard-color-font-fun-for.html">FreeType</a>.</p>
<p>Anyway, the extension works by embedding PNGs into the font. I think that you can see where this is going… just rip the images out of the font, and you’re golden. Well, other than that pesky IP law…</p>
<h2>Extensions</h2>
<p>Furthermore, now that you’re not constrained by the unicode standard, you can make anything between two colons an emoji. Like <code>:shipit:</code>. Several services do this. <a href="https://github.com/arvida/emoji-cheat-sheet.com/blob/master/LICENSE">http://emoji-cheat-sheet.com lists the licenses as such</a>:</p>
<pre><code>octocat, squirrel, shipit
Copyright (c) 2012 GitHub Inc. All rights reserved.
bowtie
Copyright (c) 2012 37signals, LLC. All rights reserved.
neckbeard
Copyright (c) 2012 Jamie Dihiansan. Creative Commons Attribution 3.0 Unported
feelsgood, finnadie, goberserk, godmode, hurtrealbad, rage 1-4, suspect
Copyright (c) 2012 id Software. All rights reserved.
trollface
Copyright (c) 2012 whynne@deviantart. All rights reserved.
All other emoji images
Copyright (c) 2012 Apple Inc. All rights reserved.
</code></pre>
<p>I made some inquries into all of these extended emoji, and GitHub said they’d get back to me, but didn’t. Therefore, their three are All Rights Reserved.</p>
<p>37Signals had said <code>:neckbeard:</code> is CC licensed, and in an email, said they’d be happy to license <code>:bowtie:</code> the same way.</p>
<p>The inventor of Trollface has yet to return my email. Go figure. Also All Rights Reserved.</p>
<p>Finally, while DOOM had its code released under the GPL, the assets were very clearly not. All Rights Reserved there too.</p>
<p>I didn’t even bother emailing Apple.</p>
<h2>What’s this mean for me?</h2>
<p>Basically, <strong>if you’re distributing the images from Apple Color emoji, you are violating copyright law</strong>. And if you’re not one of the above mentioned entities, you’re violating their licenses, as well.</p>
<p>So that’s a big downer. What to do?</p>
<h2>Phantom Open Emoji</h2>
<p>At some point, there was a kickstarter for an open emoji set. And it worked! And so <a href="https://github.com/Genshin/PhantomOpenEmoji">Phantom Open Emoji</a> was born.</p>
<p>Phantom Open Emoji basically gives you images to use as all of the glyphs used in Apple Open Emoji, but you have a license to use them however you’d like. So that’s pretty awesome.</p>
<hr />
<p>I wrote this post because I was writing a <a href="http://rubygems.org/gems/emoji">Ruby gem for emoji</a>, and needed to know if I could distribute the emoji with the gem. It’s not done yet (it’s barely started), but if you want to use emoji with your app, it will be the simplest and safest way to make sure you don’t run afoul of this kind of thing.</p>
</content:encoded></item><item><title>redis-namespace 1.3.1 security release</title><link>https://steveklabnik.com/writing/redis-namespace-1-3-1--security-release/</link><guid isPermaLink="true">https://steveklabnik.com/writing/redis-namespace-1-3-1--security-release/</guid><pubDate>Sat, 03 Aug 2013 00:00:00 GMT</pubDate><content:encoded><p>TL;DR: if you use <a href="https://rubygems.org/gems/redis-namespace">redis-namespace</a> and you use <code>send</code>, <code>Kernel#exec</code> may get called. Please upgrade to 1.0.4, 1.1.1, 1.2.2, or 1.3.1 immediately.</p>
<p>Link to the fix: <a href="https://github.com/resque/redis-namespace/commit/6d839515e8a3fdc17b5fb391500fda3f919689d6">https://github.com/resque/redis-namespace/commit/6d839515e8a3fdc17b5fb391500fda3f919689d6</a></p>
<h2>The Problem</h2>
<p>Redis has an EXEC command. We handle commands through <code>method_missing</code>. This works great, normally:</p>
<pre><code> r = Redis::Namespace.new("foo") r.exec # =&gt; error, not in a transaction, whatever
</code></pre>
<p>Here’s the problem: <code>Kernel#exec</code>. Since this is on every object, when you use <code>#send</code>, it skips the normal visibility, and calls the private but defined <code>Kernel#exec</code> rather than the <code>method_missing</code> verison:</p>
<pre><code> r = Redis::Namespace.new("foo") r.send(:exec, "ls") # =&gt; prints out your current directory
</code></pre>
<p>We fix this by not proxying <code>exec</code> through <code>method_missing</code>.</p>
<p>You are only vulnerable if you do the always-dangerous ‘send random input to an object through <code>send</code>.’ And you probably don’t. A cursory search through GitHub didn’t find anything that looked vulnerable.</p>
<p>However, if you write code that wraps or proxies Redis in some way, you may do something like this.</p>
<p>The official Redis gem does not use <code>method_missing</code>, so it should not have any issues: https://github.com/redis/redis-rb/blob/master/lib/redis.rb#L2133-L2147</p>
<p>I plan on removing the <code>method_missing</code> implementation in a further patch, but since this is an immediate issue, I wanted to make minimal changes.</p>
<p>Testing this is hard, <code>#exec</code> replaces the current running process. I have verified this fix by hand, but writing an automated test seems difficult.</p>
<p>:heart::cry:</p>
</content:encoded></item><item><title>Beware subclassing Ruby core classes</title><link>https://steveklabnik.com/writing/beware-subclassing-ruby-core-classes/</link><guid isPermaLink="true">https://steveklabnik.com/writing/beware-subclassing-ruby-core-classes/</guid><pubDate>Wed, 24 Jul 2013 00:00:00 GMT</pubDate><content:encoded><p>TL;DR: Subclassing core classes in Ruby can lead to unexpected side effects. I suggest composition over inheritance in all these cases.</p>
<h2>Subclassing Review</h2>
<p>If you’re familiar with the concept of subclassing, skip down to “The Problem.”</p>
<p>In Ruby, you can make your own classes:</p>
<pre><code>class List
end
</code></pre>
<p>You can also make subclasses of those classes:</p>
<pre><code>class OrderedList &lt; List
end
puts OrderedList.new.kind_of?(List) # =&gt; true
</code></pre>
<p>Now, subclassing represents an “is a” relationship. This means that our <code>OrderedList</code> should be a <code>List</code> in every respect, but with some added behavior. The <a href="http://en.wikipedia.org/wiki/Liskov_substitution_principle">Liskov Substitution Principle</a> is one formulation of this idea.</p>
<h2>The Problem</h2>
<p>Ruby has two major bits of code that it provides for your use: the core library and the standard library. <a href="http://www.ruby-doc.org/core-2.0/">The core library can be found here</a>, and contains cllasses that you know and love, like <code>String</code>, <code>Hash</code>, and <code>Array</code>. <a href="http://www.ruby-doc.org/stdlib-2.0/">The standard library can be found here</a>, and contains your favorite hits, like <code>CSV</code>, <code>JSON</code>, and <code>Logger</code>.</p>
<p>One way to think about the difference between core and the standard library is that core is written in C, while the standard library is written in Ruby. Core are the classes that are used the most, so they’re implemented in as low-level a fashion as possible. They’ll be in every single Ruby program, so might as well make them fast! The standard library only gets pulled in by bits and pieces; another way of thinking about the difference is that you need to <code>require</code> everything in the standard library, but nothing in core.</p>
<p>What do you think this code should do?</p>
<pre><code>class List &lt; Array
end
puts List.new.to_a.class
</code></pre>
<p>If you said “it prints <code>Array</code>,” you’d be right. This behavior really confuses me, though, because <code>List</code> is already an <code>Array</code>; in my mind, this operation shouldn’t suddenly change the class.</p>
<p>Why does this happen? Let’s check out the implementation of <a href="https://github.com/ruby/ruby/blob/trunk/array.c#L2064-L2082">Array#to_a</a>:</p>
<pre><code>static VALUE
rb_ary_to_a(VALUE ary)
{
if (rb_obj_class(ary) != rb_cArray) {
VALUE dup = rb_ary_new2(RARRAY_LEN(ary));
rb_ary_replace(dup, ary);
return dup;
}
return ary;
}
</code></pre>
<p>If the class is not an <code>Array</code>, (represented by <code>rb_cArray</code>), then we make a new array of the same length, call <code>replace</code> on it, and then return the new array. If this C scares you, here’s a direct port to pure Ruby:</p>
<pre><code>def array_to_a(ary)
if ary.class != Array
dup = []
dup.replace(ary)
return dup
end
return ary
end
array_to_a(List.new).class # =&gt; Array
</code></pre>
<p>So why do this? Well, again, this class will be used all over the place. For example, I made a brand new Rails 4 application, generated a controller and view, and put this in it:</p>
<pre><code>ObjectSpace.count_objects[:T_ARRAY]: &lt;%= ObjectSpace.count_objects[:T_ARRAY] %&gt;
</code></pre>
<p>ObjectSpace allows you to inspect all of the objects that exist in the system. Here’s the output:</p>
<p><img src="http://i.imgur.com/PVwdvyY.png" alt="http://i.imgur.com/PVwdvyY.png" /></p>
<p>rails arrays</p>
<p>That’s a lot of arrays! This kind of shortcut is generally worth it: 99.99% of the time, this code is perfect.</p>
<p>That last 0.01% is the problem. If you don’t know exactly how these classes operate at the C level, you’re gonna have a bad time. In this case, this behavior is odd enough that someone was kind enough to document it.</p>
<p>Here’s the Ruby version of what I’d expect to happen:</p>
<pre><code>def array_to_a2(ary)
return ary if ary.is_a?(Array)
dup = []
dup.replace(ary)
dup
end
array_to_a2(List.new).class # =&gt; List
</code></pre>
<p>This has the exact same behavior except when we’re already dealing with an Array, which is what I’d expect.</p>
<p>Let’s take another example: reverse.</p>
<pre><code>l = List.new
l &lt;&lt; 1
l &lt;&lt; 2
puts l.reverse.class # =&gt; Array
</code></pre>
<p>I would not expect that calling <code>#reverse</code> on my custom <code>Array</code> would change its class. Let’s <a href="https://github.com/ruby/ruby/blob/trunk/array.c#L2138-L2161">look at the C</a> again:</p>
<pre><code>static VALUE
rb_ary_reverse_m(VALUE ary)
{
long len = RARRAY_LEN(ary);
VALUE dup = rb_ary_new2(len);
if (len &gt; 0) {
const VALUE *p1 = RARRAY_RAWPTR(ary);
VALUE *p2 = (VALUE *)RARRAY_RAWPTR(dup) + len - 1;
do *p2-- = *p1++; while (--len &gt; 0);
}
ARY_SET_LEN(dup, RARRAY_LEN(ary));
return dup;
}
</code></pre>
<p>We get the length of the array, make a new blank array of the same length, then do some pointer stuff to copy everything over, and return the new copy. Unlike <code>#to_a</code>, this behavior is <em>not</em> currently documented.</p>
<p>Now: you could make the case that this behavior is expected, in both cases: after all, the point of the non-bang methods is to make a copy. However, there’s a difference to me between “make a new array with this stuff in it” and “make a new copy with this stuff in it”. Most of the time, I get the same class back, so I expect the same class back in these circumstances.</p>
<p>Let’s talk about a more pernicious issue: Strings.</p>
<p>As you know, the difference between interpolation and concatenation is that interpolation calls <code>#to_s</code> implicitly on the object it’s interpolating:</p>
<pre><code>irb(main):001:0&gt; "foo" + 2
TypeError: no implicit conversion of Fixnum into String
from (irb):1:in `+'
from (irb):1
from /opt/rubies/ruby-2.0.0-p195/bin/irb:12:in `&lt;main&gt;'
irb(main):002:0&gt; "foo#{2}"
=&gt; "foo2"
irb(main):001:0&gt; class MyClass
irb(main):002:1&gt; def to_s
irb(main):003:2&gt; "yup"
irb(main):004:2&gt; end
irb(main):005:1&gt; end
=&gt; nil
irb(main):006:0&gt; "foo#{MyClass.new}"
=&gt; "fooyup"
</code></pre>
<p>So what about a custom <code>String</code>?</p>
<pre><code>class MyString &lt; String
def to_s
"lol"
end
end
s = MyString.new
s.concat "Hey"
puts s
puts s.to_s
puts "#{s}"
</code></pre>
<p>What does this print?</p>
<pre><code>$ ruby ~/tmp/tmp.rb HeylolHey
</code></pre>
<p>That’s right! With <code>String</code>s, Ruby doesn’t call <code>#to_s</code>: it puts the value in directly. How does this happen?</p>
<p>Well, dealing with string interpolation deals with the parser, so let’s check out the bytecode that Ruby generates. Thanks to <a href="http://twitter.com/tenderlove">Aaron Patterson</a> for suggesting this approach. &lt;3</p>
<pre><code>irb(main):013:0&gt; x = RubyVM::InstructionSequence.new(%q{puts "hello #{'hey'}"})
=&gt; &lt;RubyVM::InstructionSequence:&lt;compiled&gt;@&lt;compiled&gt;&gt;
irb(main):014:0&gt; puts x.disasm
== disasm: &lt;RubyVM::InstructionSequence:&lt;compiled&gt;@&lt;compiled&gt;&gt;==========
0000 trace 1 ( 1)
0002 putself
0003 putstring "hello hey"
0005 opt_send_simple &lt;callinfo!mid:puts, argc:1, FCALL|ARGS_SKIP&gt;
0007 leave
=&gt; nil
irb(main):015:0&gt; x = RubyVM::InstructionSequence.new(%q{puts "hello #{Object.new}"})
=&gt; &lt;RubyVM::InstructionSequence:&lt;compiled&gt;@&lt;compiled&gt;&gt;
irb(main):016:0&gt; puts x.disasm
== disasm: &lt;RubyVM::InstructionSequence:&lt;compiled&gt;@&lt;compiled&gt;&gt;==========
0000 trace 1 ( 1)
0002 putself
0003 putobject "hello "
0005 getinlinecache 12, &lt;ic:0&gt;
0008 getconstant :Object
0010 setinlinecache &lt;ic:0&gt;
0012 opt_send_simple &lt;callinfo!mid:new, argc:0, ARGS_SKIP&gt;
0014 tostring
0015 concatstrings 2
0017 opt_send_simple &lt;callinfo!mid:puts, argc:1, FCALL|ARGS_SKIP&gt;
0019 leave
=&gt; nil
</code></pre>
<p>You can see with a string, the bytecode actually puts the final concatenated string. But with an object. it ends up calling <code>tostring</code>, and then <code>concatstrings</code>.</p>
<p>Again, 99% of the time, this is totally fine, and much faster. But if you don’t know this trivia, you’re going to get bit.</p>
<p><a href="https://rails.lighthouseapp.com/projects/8994/tickets/6023-unable-to-find-by-subclasses-of-string-in-activerecord#ticket-6023-5">Here</a> is an example from an older version of Rails. Yes, you might think “Hey idiot, there’s no way it will store your custom <code>String</code> class,” but the whole idea of subclassing is that it’s a drop-in replacement.</p>
<p>I know that there’s some case where Ruby will not call your own implementation of <code>#initialize</code> on a custom subclass of <code>String</code>, but I can’t find it right now. This is why this problem is so tricky: most of the time, things are fine, but then occasionally, something strange happens and you wonder what’s wrong. I don’t know about you, but my brain needs to focus on more important things than the details of the implementation.</p>
<p>Since I first wrote this post, <a href="http://twitter.com/jeg2">James Edward Gray II</a> helped me remember what this example is. One of the early exercises in <a href="http://exercism.io/">http://exercism.io/</a> is based on making a DNA type, and then doing some substitution operations on it. Many people inherited from <code>String</code> when doing their answers, and while the simple case that passes the tests works, this case won't:</p>
<pre><code>class Dna &lt; String
def initialize(*)
super
puts "Building Dna: #{inspect}"
end
end
result = Dna.new("CATG").tr(Dna.new("T"), Dna.new("U"))
p result.class
p result
</code></pre>
<p>This prints:</p>
<pre><code>Building Dna: "CATG"
Building Dna: "T"
Building Dna: "U"
Dna
"CAUG"
</code></pre>
<p>It never called our initializer for the new string. Let's check <a href="https://github.com/ruby/ruby/blob/trunk/string.c#L5484-L5525">the source of <code>#tr</code></a>:</p>
<pre><code>static VALUE
rb_str_tr(VALUE str, VALUE src, VALUE repl)
{
str = rb_str_dup(str);
tr_trans(str, src, repl, 0);
return str;
}
</code></pre>
<p><code>rb_str_dup</code> has a pretty simple definition:</p>
<pre><code>VALUE
rb_str_dup(VALUE str)
{
return str_duplicate(rb_obj_class(str), str);
}
</code></pre>
<p>and so does <code>str_duplicate</code>:</p>
<pre><code>static VALUE
str_duplicate(VALUE klass, VALUE str)
{
VALUE dup = str_alloc(klass);
str_replace(dup, str);
return dup;
}
</code></pre>
<p>So there you have it: MRI doesn't go through the whole initialization process when duplicating a string: it just allocates the memory and then replaces the contents.</p>
<p>If you re-open <code>String</code>, it's also weird:</p>
<pre><code>class String
alias_method :string_initialize, :initialize
def initialize(*args, &amp;block)
string_initialize(*args, &amp;block)
puts "Building MyString: #{inspect}"
end
end
result = String.new("CATG").tr("T", "U") # =&gt; Building MyString: "CATG"
p result.class # =&gt; String
p result # =&gt; "CAUG"
</code></pre>
<p>Again, unless you know exactly how this works at a low level, surprising things happen.</p>
<h2>The Solution</h2>
<p>Generally speaking, subclassing isn’t the right idea here. You want a data structure that <em>uses</em> one of these core classes internally, but isn’t exactly like one. Rather than this:</p>
<pre><code>class Name &lt; String
end
</code></pre>
<p>do this:</p>
<pre><code>require 'delegate'
class Name &lt; SimpleDelegator
def initialize
super("")
end
end
</code></pre>
<p>This allows you to do the same thing, but without all of the pain:</p>
<pre><code>class Name
def to_s
"hey"
end
end
"#{Name.new}" # =&gt; "hey"
</code></pre>
<p>However, this won’t solve all problems:</p>
<pre><code>require 'delegate'
class List &lt; SimpleDelegator
def initialize
super([])
end
end
l = List.new
l &lt;&lt; 1
l &lt;&lt; 2
puts l.reverse.class # =&gt; Array
</code></pre>
<p>In general, I’d prefer to delegate things manually, anyway: a <code>Name</code> is not actually a drop-in for a <code>String</code> it’s something different that happens to be a lot like one:</p>
<pre><code>class List
def initialize(list = [])
@list = list
end
def &lt;&lt;(item)
@list &lt;&lt; item
end
def reverse
List.new(@list.reverse)
end
end
l = List.new
l &lt;&lt; 1
l &lt;&lt; 2
puts l.reverse.class # =&gt; List
</code></pre>
<p>You can clean this up by using <code>Forwardable</code> to only forward the messages you want to forward:</p>
<pre><code>require 'forwardable'
class List
extend Forwardable
def_delegators :@list, :&lt;&lt;, :length # and anything else
def initialize(list = [])
@list = list
end
def reverse
List.new(@list.reverse)
end
end
l = List.new
l &lt;&lt; 1
l &lt;&lt; 2
puts l.reverse.class # =&gt; List
</code></pre>
<p>Now you know! Be careful out there!</p>
</content:encoded></item><item><title>Resque 1.25.0.pre has been released</title><link>https://steveklabnik.com/writing/resque-1-25-0-pre-has-been-released/</link><guid isPermaLink="true">https://steveklabnik.com/writing/resque-1-25-0-pre-has-been-released/</guid><pubDate>Tue, 23 Jul 2013 00:00:00 GMT</pubDate><content:encoded><p>I’ve just released Resque 1.25.0.pre! Resque is the most stable, widely-used job queue for Ruby. It uses Redis as its backing store.</p>
<p>This release is thanks to <a href="https://github.com/adelcambre">adelcambre</a>, who took the time to get it over the finish line! &lt;3</p>
<p>A total of 19 contributors helped out with this release, with 77 commits. Thank you to everyone!</p>
<h2>CHANGES</h2>
<p>You can see the full CHANGELOG <a href="https://github.com/resque/resque/blob/v1.25.0.pre/HISTORY.md">here</a>.</p>
<p>I am releasing this version as ‘pre’ due to one big change: <a href="https://github.com/resque/resque/pull/1017/files">a refactor to the forking code</a>. This was introduced to fix a problem: the exit syscall raises a SystemExit Exception in ruby, its exit is treated as a failure, thus deregistering the worker. After the refactor + fix, things should be peachy. But please give the prerelease a try.</p>
<p>In addition, it has been discovered that we have accidentally introduced a SemVer violation back in v1.23.1: https://github.com/resque/resque/issues/1074</p>
<p>I take SemVer seriously, and apologize for this change. SemVer states that at this time, I should basically do whatever. Since this fixes a bug, I’m inclined to leave it in. If this introduces a significant problem for you, please let me know. Since others might now be relying on this new behavior, I’m not 100% sure what the best option is. Your feedback helps.</p>
<h2>INSTALLING</h2>
<p>To get 1.25.0.pre installed, just use <code>gem install</code>:</p>
<pre><code>$ gem install resque --pre
</code></pre>
<h2>SOURCE</h2>
<p>You can find the source <a href="https://github.com/resque/resque">on GitHub</a>.</p>
</content:encoded></item><item><title>Removing Turbolinks from Rails 4</title><link>https://steveklabnik.com/writing/removing-turbolinks-from-rails-4/</link><guid isPermaLink="true">https://steveklabnik.com/writing/removing-turbolinks-from-rails-4/</guid><pubDate>Tue, 25 Jun 2013 00:00:00 GMT</pubDate><content:encoded><p>If you don’t want to use Turbolinks with your Rails 4 application, it’s easy! Just do this:</p>
<ol>
<li>Remove the <code>gem 'turbolinks'</code> line from your Gemfile.</li>
<li>Remove the <code>//= require turbolinks</code> from your <code>app/assets/javascripts/application.js</code>.</li>
<li>Remove the two <code>"data-turbolinks-track" =&gt; true</code> hash key/value pairs from your <code>app/views/layouts/application.html.erb</code>.</li>
</ol>
<p>Done!</p>
</content:encoded></item><item><title>Twitter ghosts</title><link>https://steveklabnik.com/writing/twitter-ghosts/</link><guid isPermaLink="true">https://steveklabnik.com/writing/twitter-ghosts/</guid><pubDate>Sat, 22 Jun 2013 00:00:00 GMT</pubDate><content:encoded><p>I’ve been thinking about death more than usual lately; my father died just before Father’s Day a year ago, and so I remembered, then was constantly reminded again on that day. But there’s another death that’s recently happened that really bothers me. The Internet is a really strange place.</p>
<p>A few days ago, I was reading Twitter as I often do, and I saw a tweet that mentioned a death. I don’t remember which one it was, specifically, but I didn’t think a whole lot of it: death happens all the time, and this one referred to an accident. It basically just said something like “Whoah, Michael, it was way too soon to lose you. :(” I didn’t think much of it, but a few minutes later, I saw a few more tweets: “@mmhastings, amazing reporter, what a great guy, how tragic.” If a few of my followers all knew this dude, then he must be a good guy. So I clicked through to his Twitter profile:</p>
<p><img src="http://i.imgur.com/uy8cRbB.png" alt="http://i.imgur.com/uy8cRbB.png" /></p>
<p>Michael Hastings’ Twitter Profile</p>
<p>Here’s what I noticed:</p>
<ol>
<li>At least his last tweet is a nice one.</li>
<li>He has 20,000 followers. Nice. And he’s only following 1,400 people, so it’s probably a legit number, not spammy.</li>
<li>Oh, how weird, he’s from Hollywood. I lived there for half of last year, and am still in LA.</li>
<li>Wait, he’s following <em>me</em>? And I’m not following him?</li>
</ol>
<p>As the day went on, more and more articles came out, and I read them all. Apparently, Michael Hastings was a really awesome guy. Apparently he was one of the few real reporters, the ones who dig in deep, aren’t afraid to speak about what they find, and never backed down. <a href="http://www.rollingstone.com/politics/news/michael-hastings-rolling-stone-contributor-dead-at-33-20130618">The story about his death in Rolling Stone</a> says “Hastings’ hallmark as reporter was his refusal to cozy up to power.”</p>
<p>So here’s what weirds me out about this situation:</p>
<p>First of all, wow, life is short. At any given time, if I just got hit by a car and died, I probably wouldn’t be all that proud of my last tweet. Isn’t it weird, that when someone’s life gets cut short, we can see what they were up to right before their death? I can now read the words of a man I’d never spoken to, and see what he was up to and what he cared about. This isn’t new news, there’s even a 4chan meme (from back in the early days of /b/) about a kid who committed suicide and all of his friends writing eulogies on his MySpace page. This is one of the first times it’s (in)directly affected me, though, and so it’s taken on a different tone.</p>
<p>Second, why didn’t I know who this was? Apparently this was a man who was a really great person, the kind I would have liked. And apparently he saw enough value in my random ramblings on Twitter to pay attention to me. But why didn’t I reciprocate? We lived in the same place. It wouldn’t have been hard to go grab a beer sometime. I walked past the intersection that he died many, many times. Would I have been friends with this person in some alternate universe?</p>
<p>One of the things that I always said about my father’s death was that at least he had two months where he knew he was going to die. While it was an incredible stress on everyone involved, at least he was able to say everything to everyone he wanted to say something to, wrap up all his unfinished business, and make peace with the God he deeply believed in. Sudden deaths have their own entire element of terrible, where you don’t get that chance. I mean, I’ve been thinking about this situation basically nonstop for the last few days, and I didn’t even really know this person: I can’t imagine how horribly tragic this is for all of the people who were actually close to Michael. I saw what a toll my father’s death took on my mother, I can’t even imagine the grief his wife must be dealing with right now.</p>
<p>So many questions. So few answers. So sad. All we can do is look at these digital remnants, wonder, and remember.</p>
</content:encoded></item><item><title>Returning to Free Software, a guide</title><link>https://steveklabnik.com/writing/returning-to-free-software-a-guide/</link><guid isPermaLink="true">https://steveklabnik.com/writing/returning-to-free-software-a-guide/</guid><pubDate>Sat, 15 Jun 2013 00:00:00 GMT</pubDate><content:encoded><p>A month or two ago, I made a conscious decision that I would return to using as much Free Software as possible. The PRISM debacle of the last week confirmed my fears and reasoning, and so I made the decision to accelerate the schedule.</p>
<p>Here’s what I did, and how you can, too.</p>
<h2>My GNU/Linux Background</h2>
<p>The first computer I ever used was an Apple machine, and at various times in my life I’ve used GNU/Linux as my main OS. So I’m already pretty familiar with what’s involved.</p>
<p>That said, it’s only gotten better over the years. Especially if you buy hardware that is known to work well, using GNU/Linux as your main OS shouldn’t be a big deal. One of the nice things about the rise of OS X in the developer community is that they’ve come to rely on the standard GNU tools, and so much of what you’re already working with will probably translate straight over. Of course, if you work on iOS or something, this won’t be the case, but for your run-of-the-mill web developer, there isn’t a whole lot of difference.</p>
<p>Well, other than that whole ‘knowing exactly what’s on your machine’ thing.</p>
<h2>Hardware: X1 Carbon</h2>
<p>IBM’s ThinkPad series of laptops have always enjoyed fantastic Linux support. When they were purchased by Lenovo, some were worried that they’d ruin this fantastic hardware line. That didn’t really happen, though, and they’re still fantastic computers to this day. When asking my Twitter followers what they thought, everyone who owned a ThinkPad loved it, whichever model they had.</p>
<p>I happened to get an <a href="http://shop.lenovo.com/us/en/laptops/thinkpad/x-series/x1-carbon/index.html">X1 Carbon</a>. I was coming from a 13" MacBook Air 13“, and the Carbon is very similar. It’s 14” though, which is a really nice size; it still fits in the same pocket in my bag as the MBA did, but I get a little more screen real estate. A note for those who give presentations: the X1 Carbon uses a MiniDisplay port, so save your Apple connectors!</p>
<p>All of the hardware I use has Just Worked; I hear that the thumbprint reader may not, but I don’t plan on using it, so that doesn’t bother me.</p>
<p>If you’re not in the market for new hardware, you can generally run GNU/Linux on your Apple hardware as well, which is a starting point, at least.</p>
<h2>Operating System: Crunchbang GNU/Linux</h2>
<p>I’ve used a lot of different GNU/Linux distros, and have been a happy Arch Linux user for years. However, this time around, I decided to go with <a href="http://crunchbang.org/">CrunchBang</a>. Crunchbang is a Debian derivative, but includes <a href="http://openbox.org/">OpenBox</a> as the default window manager. The result is an OS that’s lightweight, yet very, very functional. I’ve been really happy so far with Crunchbang’s level of polish.</p>
<p>Okay, so actually <em>installing</em> Crunchbang: the X1 Carbon comes with Windows by default, and so we need to get Crunchbang on there somehow. Luckily, you can just use USB. <code>dd</code> is your friend here, which you can use on your Mac, of course. You can grab a copy of Crunchbang <a href="http://crunchbang.org/download/">from their download page</a>, and they have a link to making the USB stick right on that page.</p>
<p>After you’ve made the USB stick, just reboot your computer with it on, and press down all of your F keys. I was simply too lazy to look up which one would actually let me choose a boot device. ;) The Crunchbang installer was very straightforward, just follow the prompts and you’ll be good to go.</p>
<p>One thing that I did choose was the ‘encrypted lvm’ option. Basically, your entire disk is encrypted except a small boot partition. On boot, it asks you for a passphrase to decrypt the disk. It’s just a matter of choosing the right option, in practice, this has worked and worked well for me.</p>
<p>EDIT: One of the reasons I chose Crunchbang was because I thought that it was solely configuration on top of Debian, which includes 100% Free Software. Checking my <code>sources.list</code>, there is a Crunchbang-specific repository, and I haven’t evaluated it for Free-ness. So if you’re trying for 100% purity, straight-up Debian may be better.</p>
<h2>Software</h2>
<p>Most of the tools that I use every day are already either Free Software or open source, and so my core experience hasn’t changed: <code>bash</code>, <code>vim</code>, <code>git</code>, and friends aren’t any different.</p>
<p>I did, however, install <a href="https://www.torproject.org/">tor</a>. Tor is not a privacy panacea, however, it does help in many cases.</p>
<p>I’ve added <a href="http://www.ghostery.com/">Ghostery</a>, <a href="http://noscript.net/">NoScript</a>, and <a href="http://adblockplus.org/">AdBlock Plus</a> to the built-in Iceweasel browser, which is for all intents and purposes Firefox without the Mozilla branding. These allow me to control exactly what information leaks out a little bit more. They also have the nice side effect of making pages load a bit faster, since a lot of the usual bullshit doesn’t even get loaded. (EDIT: it’s been brought to my attention that Ghostery isn’t actually free. Oops!)</p>
<p>I plan on setting up backups with <a href="http://www.tarsnap.com/">Tarsnap</a> later in the week, which is an excellent backup service that encrypts all of your data locally before saving it off in The Cloud. It has some pretty good (I hear) de-duplication features, and you only pay for the bytes used, so it’s pretty inexpensive. The author, Colin Percival, has said that he has some customers doing a muti-gig daily backup of their home directories, and they’re paying ~$10/month. (I should mention that <a href="http://www.tarsnap.com/legal.html">Tarsnap is not Free</a>, but it is shared source. I don’t plan on using it for anything else, so I’m okay with this tradeoff at this time.)</p>
<p>Most of my development work at the moment is in Ruby, and getting it set up was a breeze. I use the <a href="https://github.com/sstephenson/ruby-build">ruby-build</a> tool to compile my Rubies, and the most excellent <a href="https://github.com/postmodern/chruby">chruby</a> to switch between them.</p>
<p>One thing that I have yet to explore (but I’d like to) is to use a tool like Chef to set up my personal machine, so that I can periodically wipe everything and re-build from scratch. That’s still on the to-do list, though.</p>
<p>I personally re-map my caps lock key to control. To do this, make a <code>.Xmodmap</code> file in your home directory, and put this in it:</p>
<pre><code>keycode 66 = Control_L
clear Lock
add control = Control_L
</code></pre>
<p>I figured this out via a few simple searches. There’s a small chance that your caps lock is not code 66, and if it’s not, you can run the <code>xev</code> command to check it out. Just press your caps lock and make note of the number that shows up in the console.</p>
<p>I had to install <code>grandr</code> in order to share my screen to give a presentation. It’s easy enough to get: <code>sudo apt-get install grandr</code>.</p>
<p>I started giving presentations with <a href="http://rabbit-shocker.org/">rabbit</a>, which is what many Japanese Rubyists use. It’s pretty fun; you write plain text files and it makes your presentation for you, as cairo/pango application. You can also use it to generate PDFs of slides. It’s not for everyone, but I’m enjoying it so far.</p>
<p>Oh, and <em>incredibly</em> important: I typed <code>sudo apt-get install ttf-ancient-fonts</code> to get Emoji support. They’re not as pretty as the Apple ones, but they’re not empty boxes either.</p>
<h2>Conclusion</h2>
<p>I’m really, really happy with this setup. Very little of my day-to-day has changed, but I have a bit more privacy, and can feel good that I know exactly what software is on my machine. No NSA backdoors here!</p>
<p>I still have improvements to make, but what I have works well so far. Please let me know what you think, and suggest ways to make this setup even better.</p>
</content:encoded></item><item><title>Rust-y Scraps: iterating with a step</title><link>https://steveklabnik.com/writing/rust--y-scraps--iterating-with-a-step/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rust--y-scraps--iterating-with-a-step/</guid><pubDate>Thu, 30 May 2013 00:00:00 GMT</pubDate><content:encoded><p>This blog post works with <a href="http://rust-lang.org/">Rust</a> v 0.6, and may change in the future :).</p>
<p>A common need in various algorithms is to iterate with some sort of step. A simple way to do this in Rust is with <code>unit::range_step</code>:</p>
<pre><code>let nums = [1,2,3,4,5,6,7];
for uint::range_step(0, bytes.len() - 1, 2) |i| {
println(fmt!("%d &amp; %d", nums[i], nums[i+1]));
}
</code></pre>
<p>This yields</p>
<pre><code>1 &amp; 2
3 &amp; 4
5 &amp; 6
</code></pre>
<p>Of course, this is awkward: I wish I had some code to do something like:</p>
<pre><code>let nums = [1,2,3,4,5,6,7];
for nums.in_groups |i, j| {
println(fmt!("%d &amp; %d", i, j));
}
</code></pre>
<p>But I think this is just out of my abilities. The issue is that the closure needs to take the same number of arguments as the step, and I’m not sure that I have the Rust-fu to make it work yet.</p>
<p>After a discussion in the IRC room, Rust doesn’t have the Rust-fu yet either; varargs aren’t there, so this style is a no-go for now.</p>
<hr />
<p>Oh, and since there were no examples of this yet, <a href="https://github.com/mozilla/rust/pull/6841">I added some to the docs</a>.</p>
</content:encoded></item><item><title>Rails 4.0.0-beta1 to Rails 4.0.0-rc1</title><link>https://steveklabnik.com/writing/rails-4-0-0-beta1-to-rails-4-0-0-rc1/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rails-4-0-0-beta1-to-rails-4-0-0-rc1/</guid><pubDate>Tue, 21 May 2013 00:00:00 GMT</pubDate><content:encoded><p>I’m writing <a href="http://www.manning.com/bigg2/">Rails 4 in Action</a> along with Ryan Bigg and Yehuda Katz. In updating the book to Rails 4, I started early: the sample app (<a href="https://github.com/steveklabnik/ticketee">ticketee</a>) was done with <code>4.0.0.beta1</code>.</p>
<p>Then <code>4.0.0.rc1</code> came out. I didn’t upgrade immediately, but last night, I took the plunge. There were only three errors, and the whole process took me something like 40 minutes.</p>
<p>Here’s what I had to do:</p>
<h2>Cookie Stores</h2>
<p>Error:</p>
<pre><code>/Users/steve/.gem/ruby/2.0.0/gems/railties-4.0.0.rc1/lib/rails/application/configuration.rb:144:in `const_get': uninitialized constant ActionDispatch::Session::EncryptedCookieStore (NameError)
</code></pre>
<p>The EncryptedCookieStore went away. Now, the regular cookie store handles these kinds of things.</p>
<p>Fix:</p>
<pre><code>diff --git a/config/initializers/session_store.rb b/config/initializers/session_store.rbindex c512cda..b4681fd 100644--- a/config/initializers/session_store.rb+++ b/config/initializers/session_store.rb@@ -1,3 +1,3 @@ # Be sure to restart your server when you modify this file.-Ticketee::Application.config.session_store :encrypted_cookie_store, key: '_ticketee_session'+Ticketee::Application.config.session_store :cookie_store, key: '_ticketee_session'
</code></pre>
<h2>Routes</h2>
<p>Error:</p>
<pre><code>/Users/steve/.gem/ruby/2.0.0/gems/actionpack-4.0.0.rc1/lib/action_dispatch/routing/route_set.rb:408:in `add_route': Invalid route name, already in use: 'signin' (ArgumentError)
You may have defined two routes with the same name using the `:as` option, or you may be overriding a route already defined by a resource with the same naming. For the latter, you can restrict the routes created with `resources` as explained here:
http://guides.rubyonrails.org/routing.html#restricting-the-routes-created
</code></pre>
<p>You can’t name two routes the same thing any more. In my case, just dropping the <code>:as</code> allowed me to get to where I needed to go:</p>
<p>Fix:</p>
<pre><code>diff --git a/config/routes.rb b/config/routes.rbindex f010adb..d573e89 100644--- a/config/routes.rb+++ b/config/routes.rb@@ -2,8 +2,8 @@ Ticketee::Application.routes.draw do root to: "projects#index"- get "/signin", to: "sessions#new", as: "signin"- post "/signin", to: "sessions#create", as: "signin"+ get "/signin", to: "sessions#new"+ post "/signin", to: "sessions#create" delete "/signout", to: "sessions#destroy", as: "signout"
</code></pre>
<h2>Clean dbs</h2>
<p>Error:</p>
<p>Tons of test failures relating to what was essentially “I don’t delete the db between test runs.”</p>
<p>Fix:</p>
<p>Upgrade to rspec-rails 2.13.2. Done!</p>
<h2>has_secure_password</h2>
<p>Error:</p>
<pre><code> 1) User passwords needs a password and confirmation to save
Failure/Error: expect(u).to_not be_valid
expected #&lt;User id: 1, name: "steve", email: "[email protected]", password_digest: "$2a$04$9D7M2iPvYavvvpb3rlLzz.vqMJdGBAQVBSJ6L.UIoXP6...", created_at: "2013-05-21 23:59:44", updated_at: "2013-05-21 23:59:44", admin: false&gt; not to be valid
</code></pre>
<p>This test failed. The test looked like this:</p>
<pre><code>it "needs a password and confirmation to save" do u = User.new(name: "steve", email: "[email protected]") u.save expect(u).to_not be_valid u.password = "password" u.password_confirmation = "" u.save expect(u).to_not be_valid u.password_confirmation = "password" u.save expect(u).to be_validend
</code></pre>
<p>Turns out, <a href="https://github.com/rails/rails/pull/10694">it’s a regression</a>! This will get fixed for RC2, but for now, just commenting out that middle section works. The final test is good enough for now.</p>
</content:encoded></item><item><title>Ditching Google Chat with XMPP</title><link>https://steveklabnik.com/writing/ditching-google--chat-with-xmpp/</link><guid isPermaLink="true">https://steveklabnik.com/writing/ditching-google--chat-with-xmpp/</guid><pubDate>Thu, 16 May 2013 00:00:00 GMT</pubDate><content:encoded><p>Last week, I decided that it was time to migrate one of my services away from Google: XMPP for IM. The nice thing about XMPP is that you don’t have to totally drop support for your old identity: since Google still uses it (<a href="http://eschnou.com/entry/whats-next-google--dropping-smtp-support--62-24930.html">for now</a>), you can get online with both.</p>
<p>Here’s how:</p>
<hr />
<p>I set mine up with <a href="http://digitalocean.com/">Digital Ocean</a>, because my friends told me it was rad. It was pretty damn easy.</p>
<p>Once you have a server, you need to set up all the usual stuff: a non-privledged user account, yadda, yadda. I’m assuming you can do that.</p>
<h2>Install ejabberd</h2>
<p>I picked <a href="http://www.ejabberd.im/">ejabberd</a> for my server software because it’s used by a lot of people.</p>
<p>EDIT: I just got a tweet from <a href="http://twitter.com/jamesgolick">@jamesgolick</a>, who I trust to know stuff about stuff. He says:</p>
<blockquote>
<p>ejabberd is buggy. Use the erlangsolutions fork instead. - <a href="https://twitter.com/jamesgolick/status/335136677364396032">@jamesgolick</a></p>
</blockquote>
<p>So, take that for what it’s worth.</p>
<p>Anyway, since I use Arch Linux, installing was as easy as</p>
<pre><code>$ sudo pacman -S ejabberd
</code></pre>
<p>After that was over, time to edit the config. <code>ejabberd</code> is written in Erlang, and so it uses Erlang for its config. Don’t worry! It’s easy. Just copy and paste these bits in, or change them if there’s something like them in your config:</p>
<pre><code>{hosts, ["localhost", "xmpp.steveklabnik.com"]}.{acl, admin, {user, "steve", "xmpp.steveklabnik.com"}}.
</code></pre>
<p>These change my host to <code>xmpp.steveklabnik.com</code> as well as <code>localhost</code> and set my admin user to <code>[email protected]</code>.</p>
<p>Next up, we boot the node:</p>
<pre><code>sudo ejabberdctl start
</code></pre>
<p>And then register a way to administer things via the web interface:</p>
<pre><code>sudo ejabberdctl register steve xmpp.steveklabnik.com $password
</code></pre>
<p>Of course, change those to be your values, and change $password to your password. Now that you’ve typed a password into the command line, it’s good to clean out your bash history. When I did this, typing <code>history</code> showed this:</p>
<pre><code> 35 sudo ejabberdctl register ....
</code></pre>
<p>So, do this:</p>
<pre><code>$ history -d 35$ history -w
</code></pre>
<p>Or, to nuke <em>everything</em>:</p>
<pre><code>$ history -c$ history -w
</code></pre>
<p>Anywho, now that that’s over, you need to go to your domain registrar and insert an A record pointing at your server. I use <a href="https://www.namecheap.com/">NameCheap</a>.</p>
<p>Now that that’s done, hit this page in your browser:</p>
<pre><code>http://xmpp.steveklabnik.com:5280/admin/
</code></pre>
<p>The trailing slash is important, and obviously, you need to change it to be your server. Given that the DNS isn’t cached wrong, you should get a popup asking you to log in. Do so with the credentials you typed in on the command line up there. If you can log in, great! Everything should be working.</p>
<p>Let’s turn off registration, though. Go back to your <code>ejabberd</code> config, <code>/etc/ejabberd/ejabberd.cfg</code>:</p>
<pre><code>{access, register, [{deny, all}]}.
</code></pre>
<p>You probably have one that says <code>allow</code>, so just change it. Check in your admin panel on the site that it’s the same.</p>
<hr />
<p>Bam! That’s it. This whole process took me about an hour, including writing this blog post. I do have some experience with VPSes, though.</p>
<p>If I can help free you of Google in any way, ping me at <a>[email protected]</a>.</p>
</content:encoded></item><item><title>The profile link relation and you</title><link>https://steveklabnik.com/writing/the-profile-link-relation-and-you/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-profile-link-relation-and-you/</guid><pubDate>Mon, 06 May 2013 00:00:00 GMT</pubDate><content:encoded><p>I was quite pleased when <a href="http://tools.ietf.org/html/rfc6906">RFC 6906</a> was finalized. It’s a really useful pattern that people are using to enhance documentation of their APIs.</p>
<p>Let’s start with an example. I tweet something like this:</p>
<blockquote>
<p>“Oh man, the example.com API is super awesome. You should check it out!” - <a href="https://twitter.com/steveklabnik">@steveklabnik</a> seconds ago from web</p>
</blockquote>
<p>You don’t know anything about this API or what it offers. So you fire up <code>curl</code>:</p>
<pre><code>$ curl -i http://example.com
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 273
{
"wtl": "MjAxMy0wNS0wNiAxMjo1Nzo1MyAtMDcwMA==\n",
"grobb34s": [
{
"flog": "Top 100 foobars",
"zilch": "http://example.com/foo/bar?baz=qux"
},
{
"flog": "Mega Troll Title",
"zilch": "http://example.com/exploit.exe/foobar"
}
]
}
</code></pre>
<p>To be blunt, this makes absolutely no sense. You tell me so, and the next day, I tell you to check it out again. You grumble, and get back to the <code>curl</code>:</p>
<pre><code>$ curl -i http://example.com
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 273
Link: &lt;http://example.com/profile&gt;; rel="profile"
{
"wtl": "MjAxMy0wNS0wNiAxMjo1Nzo1MyAtMDcwMA==\n",
"grobb34s": [
{
"flog": "Top 100 foobars",
"zilch": "http://example.com/foo/bar?baz=qux"
},
{
"flog": "Mega Troll Title",
"zilch": "http://example.com/exploit.exe/foobar"
}
]
}
</code></pre>
<p>Oh, wait. “Profile”. Let’s see what that’s about:</p>
<pre><code>$ curl -i http://example.com/profile
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 548
The Example.com API
===================
Example.com provides access to our blog through an API.
In the API, you'll see two major things of interest: `wtl` and `grobb34s`.
## wtl
The value provided under the `wtl` key is the time the latest blog post
was posted, in "%Y-%m-%d %H:%M:%S %z" format. This value is then Base64
encoded.
## grobb34s
The `grobb34s` key will hold an array of blog posts. These posts are
represented by a JSON object with two keys. `flog` has the title, suitable for
display, and `zilch` contains a link to the post.
</code></pre>
<p>Oh. You don’t care about some blog with such terrible titles, so you go about your day.</p>
<h2>self-descriptive-ness</h2>
<p>You may consider this a highly contrived example, but it’s not as contrived as you think. In Fielding’s thesis, he describes a ‘self-described messages’ constraint, which means that any API call should be able to be understood alone, without additional context.</p>
<p>In HTTP, this information is generally provided via the <code>Content-Type</code> header. It describes a media type (such as <code>application/json</code> in this example) that describes the rules for processing the response. When writing a HTTP client, you fetch the response, check the <code>Content-Type</code>, and then invoke the correct parser based on the type provided.</p>
<p>But many times are very general. Take <code>application/json</code>, for example. It says nothing about blog posts. A user-agent that only knows about <code>application/json</code> is very much like you as a human seeing gibberish keys and values; it’s not smart enough to make assumptions based on the text of keys and values it may see in JSON. However, with the added context of a profile, we have enough information to make sense of this strange API. And any user-agent that sees a profile that it recognizes can act on those new semantics, too.</p>
<h2>JSON API</h2>
<p>This is one of the reasons that we’re working on <a href="http://jsonapi.org/">JSON API</a>, a standard media type for APIs that use JSON. Generic JSON has no standard semantics that are useful to API authors, since it’s a very generic format. By declaring a new media type with common features to many APIs, we can write generic tools that handle the 80% of cases that come up during API development.</p>
<p>And if the generic case doesn’t suit your requirements, you can always extend it with profiles. In fact, it’d be nice if many people just added a <code>Link</code> header to their existing API documentation; that’d start alleviating this problem.</p>
<hr />
<p>If you enjoyed this article, you might want to check out my in-progress book on building APIs that respect standards and HTTP, <a href="http://www.designinghypermediaapis.com/">Designing Hypermedia APIs</a>. This post also serves as <a href="http://www.designinghypermediaapis.com/blog/the-profile-link-relation-and-you.html">its first blog post</a>.</p>
</content:encoded></item><item><title>Closure</title><link>https://steveklabnik.com/writing/closure/</link><guid isPermaLink="true">https://steveklabnik.com/writing/closure/</guid><pubDate>Fri, 19 Apr 2013 00:00:00 GMT</pubDate><content:encoded><p>I spent six weeks of last December/January in my hometown. It was the first Christmas without my father, and it was cheap to stay, so I went back. It was my first time back in Pittsburgh since I’d moved away from the only place I’d ever lived.</p>
<p>The trip was incredibly emotional for many reasons, but one of the best was when I decided to walk down the street to get a pizza. I had to wait ten minutes for it to cook, and so I decided to open up my web browser. Then I saw this:</p>
<p><img src="https://svbtleusercontent.com/steveklabnik_24595149844116_small.png" alt="https://svbtleusercontent.com/steveklabnik_24595149844116_small.png" /></p>
<p>Whoah.</p>
<p>You see, I’ve always had a complex relationship with _why. I had been dabbling in Ruby when he was still alive, and planned on dropping by this ART &amp;&amp; CODE thing I’d heard about over at CMU, but I was out of town that weekend, so I shrugged and didn’t think twice.</p>
<p>Then he disappeared.</p>
<p>I started looking into _why and what he was actually about. What I found was amazing, and you already know that part of the story. I thought Hackety Hack was a masterpiece, and decided to help maintain it. Unfortunately for me, nobody else did. I hadn’t done a whole lot of open source work before, and now I was maintaining a big, important project by one of the most well-known Rubyists of all time. Whoah.</p>
<p>I cared about Hackety enough and felt inspired enough by _why that I actually quit my startup to spend more time working on it.</p>
<p>Here’s the weird thing about someone disappearing: you can’t ask them questions. I had so many things that I wanted to ask, so many unresolved questions, so much I wanted to know. It’s really weird to take someone’s life’s work and make it your own. I spent a lot of that first year with crippling doubts about what I was doing, if _why would like it, if that even mattered, wondering why he’d abandoned something so important, why he’d abandoned <em>us</em>. And I felt terrible that I <em>needed</em> this from him. Why does he owe us all that? Who am I to impose?</p>
<p>So I marked my calendar for the 18th, and waited. That was yesterday.</p>
<p><img src="http://distilleryimage6.s3.amazonaws.com/a8767b30a88f11e2896922000a1fbe1a_7.jpg" alt="http://distilleryimage6.s3.amazonaws.com/a8767b30a88f11e2896922000a1fbe1a_7.jpg" /></p>
<p><img src="http://distilleryimage11.s3.amazonaws.com/c22b37d2a8bc11e287e222000aaa0aa2_7.jpg" alt="http://distilleryimage11.s3.amazonaws.com/c22b37d2a8bc11e287e222000aaa0aa2_7.jpg" /></p>
<p>It was… great. Just great. The stuff _why sent is basically everything I’d ever hoped. It’s got so much buried inside. An IF game, commentary on Jobs vs. Gates, a very person struggle with identity, a comment that he likes what we’ve done… it’s great. And it’s finally brought me some closure.</p>
<p><a href="https://github.com/steveklabnik/CLOSURE">So, I’ve saved it all for <em>you</em>. &lt;3</a></p>
<p><img src="https://svbtleusercontent.com/steveklabnik_24595264609650_small.jpg" alt="https://svbtleusercontent.com/steveklabnik_24595264609650_small.jpg" /></p>
</content:encoded></item><item><title>Upgrading from Rust 0.5 to 0.6</title><link>https://steveklabnik.com/writing/upgrading-from-rust-0-5-to-0-6/</link><guid isPermaLink="true">https://steveklabnik.com/writing/upgrading-from-rust-0-5-to-0-6/</guid><pubDate>Sun, 07 Apr 2013 00:00:00 GMT</pubDate><content:encoded><p>Here are some error messages that I got when updating <a href="http://www.rustforrubyists.com/">Rust for Rubyists</a> from Rust 0.5 to the new 0.6, and how to fix those errors.</p>
<hr />
<p>Error:</p>
<pre><code>rust.rs:1:4: 1:16 error: unresolved name
rust.rs:1 use task::spawn;
^~~~~~~~~~~~
rust.rs:1:4: 1:16 error: failed to resolve import: task::spawn
rust.rs:1 use task::spawn;
^~~~~~~~~~~~
error: failed to resolve imports
error: aborting due to 3 previous errors
make: *** [all] Error 101
</code></pre>
<p>or</p>
<pre><code>rust.rs:1:4: 1:16 error: unresolved name
rust.rs:1 use io::println;
^~~~~~~~~~~~
rust.rs:1:4: 1:16 error: failed to resolve import: io::println
rust.rs:1 use io::println;
^~~~~~~~~~~~
error: failed to resolve imports
error: aborting due to 3 previous errors
make: *** [all] Error 101
</code></pre>
<p>The fix:</p>
<p>Imports are now more explicit, so you can’t just import things from <code>core</code> without specifying that you want to any more.</p>
<pre><code>+ use core::task::spawn;- use task::spawn;
</code></pre>
<p>or</p>
<pre><code>+ use core::io::println;- use io::println;
</code></pre>
<p>etc.</p>
<hr />
<p>Error:</p>
<pre><code>error: main function not found
error: aborting due to previous error
</code></pre>
<p>The fix:</p>
<hr />
<p>When I used to write TDD’d Rust, I wouldn’t often write a <code>main</code> function until I had done a bunch of tests. This was okay, but now, you need one.</p>
<pre><code>+ fn main() {
+ }
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rustc rust.rs --test
rust.rs:5:2: 5:6 error: unresolved name: `fail`.
rust.rs:5 fail;
^~~~
error: aborting due to previous error
make: *** [build_test] Error 101
</code></pre>
<p>The fix:</p>
<p><code>fail</code> was turned into a macro. It now needs to be passed an owned pointer to a string:</p>
<pre><code>- fail;
+ fail!(~"Fail!");
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:5:18: 5:26 error: unresolved name: `int::str`.
rust.rs:5 println(int::str(num))
^~~~~~~~
error: aborting due to previous error
make: *** [build] Error 101
</code></pre>
<p>The fix:</p>
<p><code>int::str</code> is now <code>int::to_str</code></p>
<pre><code>- int::str
+ int::to_str
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:5:12: 5:18 error: cannot determine a type for this local variable: unconstrained type
rust.rs:5 let mut answer;
^~~~~~
error: aborting due to previous error
make: *** [build] Error 101
</code></pre>
<p>The fix:</p>
<p>You have to tell it what kind it is. In my case, it was a string:</p>
<pre><code>- let mut answer;
+ let mut answer = "";
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:3:11: 3:21 error: expected `;` or `}` after expression but found `is_fifteen`
rust.rs:3 assert is_fifteen(15)
^~~~~~~~~~
make: *** [build_test] Error 101
</code></pre>
<p>The fix:</p>
<p>This happens because <code>assert</code> is now a macro.</p>
<pre><code>- assert is_fifteen(15)
+ assert!(is_fifteen(15))
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rustc rust.rs
rust.rs:11:10: 11:24 error: the type of this value must be known in this context
rust.rs:11 chan.send(10);
^~~~~~~~~~~~~~
error: aborting due to previous error
make: *** [build] Error 101
</code></pre>
<p>The fix:</p>
<p>Rust 0.6 got rid of ‘capture clauses.’ Don’t worry about it.</p>
<pre><code>- do spawn |chan| {
+ do spawn {
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:10:17: 10:21 error: expected `,` but found `chan`
rust.rs:10 do spawn |move chan| {
^~~~
make: *** [build] Error 101
</code></pre>
<p>The fix:</p>
<p><code>move</code> is gone. Just remove it.</p>
<pre><code>- do spawn |move chan| {
+ do spawn |chan| {
</code></pre>
<p>Note that this diff wouldn’t actually make it work, you’d still run into the issue above. But it gets rid of the <code>move</code> error.</p>
<hr />
<p>Error:</p>
<pre><code>rust.rs:13:6: 13:12 error: unexpected token: `static`
rust.rs:13 static fn count() {
^~~~~~
make: *** [build] Error 101
</code></pre>
<p>The fix:</p>
<p><code>static</code> was removed. Any method that doesn’t take <code>self</code> is static.</p>
<pre><code>- static fn count() {
+ fn count() {
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:11:30: 11:37 error: obsolete syntax: colon-separated impl syntax
</code></pre>
<p>The fix:</p>
<p>As it says, the colon syntax is gone. Replace it with <code>for</code>:</p>
<pre><code>- impl float: Num
+ impl Num for float
</code></pre>
<hr />
<p>Error:</p>
<pre><code>rust.rs:5:18: 5:22 error: use of undeclared type name `self`
rust.rs:5 fn new() -&gt; self;
^~~~
</code></pre>
<p>The fix:</p>
<p>This happens when you’re making a trait, and you want the implementations to return whatever type they are. The type name is now <code>Self</code>:</p>
<pre><code>- fn new() -&gt; self;
+ fn new() -&gt; Self;
</code></pre>
<hr />
<p>I hope you found that informative! Happy hacking!</p>
</content:encoded></item><item><title>Deleuze for Developers: will smooth space/open source suffice to save us?</title><link>https://steveklabnik.com/writing/deleuze-for-developers-will-smooth-spaceopen-source-suffice-to-save-us/</link><guid isPermaLink="true">https://steveklabnik.com/writing/deleuze-for-developers-will-smooth-spaceopen-source-suffice-to-save-us/</guid><pubDate>Thu, 14 Mar 2013 00:00:00 GMT</pubDate><content:encoded><p>If you truly want to understand technology today, then you should at least be familiar with the philosophy of Gilles Deleuze. Unfortunately for technologists, Deleuze is rooted firmly in a philosophical tradition and a writing style that they probably find opaque. In this blog series, I plan on explaining Deleuze’s philosophy in terms that programmers can understand. This is the third in the series. You can find the first <a href="/deleuze-for-developers-assemblages">here</a>. Enjoy.</p>
<hr />
<p>Deleuze and Guattari employ the notion of ‘smooth space’ quite a bit in <em>A Thousand Plateaus</em>. I saw this quote about it the other day:</p>
<blockquote>
<p>never believe that a smooth space will suffice to save us</p>
</blockquote>
<p>I’ve recently been giving a lot of thought to <a href="http://en.wikipedia.org/wiki/Cultural_hegemony">‘cultural hegemony’</a> and its applicability to the web, specifically along the lines of Google. Basically, cultural hegemony is the idea that you don’t just have dominance through laws or economics, but through culture. So, for example, Google submitted SPDY to become HTTP 2.0: it’s going through the standards process, it’s ‘open,’ but it also gives Google a lot of cultural leverage.</p>
<p>So there’s this connection there between smooth space and open source, and the battle between the Nomads and the State Apparatus. We’ll cover that second bit in a while, but for now: what is ‘smooth space’?</p>
<h2>The spatial metaphor</h2>
<p>The first thing to understand about these concepts is that Deleuze and Guattari is (I think of them as a singular entity…) a big fan of spatial metaphors. Essentially, we can project our problem domain onto a 2 or 3 (or more) dimensional space, and then use tools to analyze that space. I was first introduced to this technique by Manuel de Landa, who has a great lecture called “Deleuze and the use of the genetic algorithm in archetecture” (<a href="http://www.youtube.com/watch?v=50-d_J0hKz0">video</a>)(<a href="http://www.cddc.vt.edu/host/delanda/pages/algorithm.htm">text</a>). A quick rundown:</p>
<p>So, let’s talk about the <a href="http://en.wikipedia.org/wiki/Eight_queens_puzzle">N queens problem</a>. We have a chess board with one space, and we place one queen on it:</p>
<p>(warning, lots of ASCII (+ unicode) art to follow)</p>
<pre><code>.-.
|♕|
.-.
</code></pre>
<p>This board is legal. It’s a solution to the ‘1 queen problem’: one queen, a 1x1 board. What about 2 queens? We have no solutions. (2 and 3 don’t. :/ All other natural numbers do.) First we place a queen:</p>
<pre><code>.---.
|♕| |
-----
| | |
.---.
</code></pre>
<p>… but there’s no legal place for the second one. Bummer. We can determine that there’s no solution by using a simple brute force with backtracking: place the queen in the upper right, then try to place the second below it. That’s not legal, so we backtrack and place it on the right. That’s not legal, so we backtrack and put it in the bottom corner. Oh no! That didn’t work, so let’s move our first queen: now it’s in the bottom right, and we try to place the second in the upper right: fail! So we backtrack….</p>
<p>As you can see, this is a lot of steps for a small board. Once you get up to a ‘real’ chess board, there are 4,426,165,368 possible placements, but only 92 solutions. A needle in a haystack! And we’re using a lot of steps to determine if there’s even one possible placement. We need something better.</p>
<p>One answer is genetic algorithms. So we can take our board’s current state and assign it a score. When we place a new queen, if it makes our score better, we keep it, and if it doesn’t, we lose it. Now we’ve taken our 2x2 board and projected it onto a 2-axis linear space. Imagine a plot where queens are on the x axis and score is on the y axis:</p>
<pre><code> .dAHAd.
.adAHHHAbn.
.dAHAd. .adAHHHHHHAbn.
.adAHHHAbn. .adAHHHHHHHHAbn.
dHHHHHHHHHHHHHHHHHHHHHHHHHHHHHb
dHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHb
</code></pre>
<p>So we want to keep going with our solution as long as it slopes up: we’ve found that our answers are at the maximum of each curve. Yay calculus! But here’s the thing: this curve only has two possible maxima: we will have more. And it might not be in 2-d space, it might be in 5-d space. And we can only ‘use math’ to find the maximum if we generate the <em>entire</em> curve, which seems computationally out of our reach. So how do we find the maximum without generating the whole curve? Genetic algorithms!</p>
<p>One such is ‘<a href="http://en.wikipedia.org/wiki/Simulated_annealing">simulated annealing</a>.’ Without getting too into it, let’s just say that there’s a ‘cooling factor’ that controls how tolerant we are of going back down a slope. So at first, we wildly go all over the search space, but then, as we progress, we tighten up our cooling factor, and we stop being so wild. Eventually, we’ll arrive at a solution that’s very likely to be the true global maxima/minima. Neat!</p>
<h2>Discrete vs. Continuous</h2>
<blockquote>
<p>If we know that the enemy is open to attack, and also know that our men are in a condition to attack, but are unaware that the nature of the ground makes fighting impracticable, we have still gone only halfway towards victory.Sun Tzu, “the Art of War”</p>
</blockquote>
<p>Another interesting feature of this particular projection is that it transforms our problem from a discrete problem into a continuous one. One great tactic for when you’re losing: change the battlefield. Our genetic algorithm tool needs a continuous space to operate, but our problem is that our chess board is discrete. What do I mean by this?</p>
<p>“<a href="http://en.wikipedia.org/wiki/Discrete_space">discrete space</a>” is one in which the points are separated from one another in some way. As an example, ‘integers’ form a discrete topology: there’s a ‘gap’ between 1, 2, 3, and 4. You can see this by drawing a <a href="http://en.wikipedia.org/wiki/Number_line">number line</a>:</p>
<pre><code>&lt;--(-2)--(-1)--(0)--(1)--(2)--&gt;
</code></pre>
<p>The real numbers form a <a href="http://en.wikipedia.org/wiki/Continuity_%28topology%29">continuous topology</a> rather than a discrete one, there is no space between them. A ‘<a href="http://en.wikipedia.org/wiki/Real_line">real line</a>’:</p>
<pre><code>&lt;-------------------------&gt;
</code></pre>
<p>Our ‘scoring’ mechanism allows us to change the battlefield, it forms a function (isomorphism) to convert our discrete topology into a continuous one. We can now bring our continuous tooling to bear on a problem that was previously inaccessible.</p>
<h2>Striated vs Smooth Space</h2>
<blockquote>
<p>Military tactics are like unto water; for water in its natural course runs away from high places and hastens downwards. So in war, the way is to avoid what is strong and to strike at what is weak. Like water, taking the line of least resistance. Water shapes its course according to the nature of the ground over which it flows; the soldier works out his victory in relation to the foe whom he is facing. Therefore, just as water retains no constant shape, so in warfare there are no constant conditions.Sun Tzu, “the Art of War”</p>
</blockquote>
<p>Okay. NOW we’re ready to talk about smooth and striated space. They have a number of names for this concept, and the one that’s most direct from where we currently are is ‘Riemann space / Euclidean space’. Smooth -&gt; Riemann, Euclidean -&gt; Striated. Another isomorphism. ;)</p>
<p><a href="http://en.wikipedia.org/wiki/Euclidean_geometry">Euclidean geometry</a> is pretty much the foundation of a metric ton of our math. It’s what’s now known as algebra and geometry: lots of integers, discrete spaces. This cube starts at (1,2,4) and has a side length of 5. Cartesian coordinates.</p>
<p>Along comes <a href="http://en.wikipedia.org/wiki/Carl_Friedrich_Gauss">Gauss</a>, who was incredibly intelligent. He had this student named <a href="http://en.wikipedia.org/wiki/Bernhard_Riemann">Riemann</a>, who ran with the ideas Gauss had and started <a href="http://en.wikipedia.org/wiki/Riemannian_geometry">Riemannian Geometry</a>. Riemann’s geometry is <a href="http://en.wikipedia.org/wiki/Non-Euclidean_geometry">non-Euclidean</a>: if Euclid described lines, Riemann described curves. Einstein would later base the theory of relativity on Riemannian geometry.</p>
<p>So, who cares? Well, think about it this way: You’re in a car, driving down a straight road. It goes for miles, with no turns or other streets. You’re operating in a smooth space: you can easily go from one place to another. Now, someone comes along and puts up stop signs every block. You must (for now) stop at these signs. Now you’re operating in a striated space: your movement is restricted. Now you’re moving from point to point on a line. It’s a very different experience, and you also didn’t have anything to do with the stop signs….</p>
<p>According to D&amp;G, ‘the state machine’ (one instance of which is the State) attempts to take space that is smooth and striate it. This is often a means towards imperialism, as the striation happens through quantification. Take, for example, the sea: a smooth space, you can impose latitude and longitude on top of it, simultaneously quantifying and striating it. This allows you to navigate the waters, and conquer the sea.</p>
<p>This is also the position that many web startups take: ‘friendship’ was a smooth space before social networks came along. They quantified and striated that space, turning “I have many friends” into “I have 200 friends and you have 100 friends.” This quantification allows for commodification: now Facebook can sell ads. It’s also why startups don’t often have a ‘business model’ at first: they first need to conquer and striate their space before they find the commodity. Sometimes you just hit dirt and go bust, sometimes you find a more precious commodity, then refine and sell it.</p>
<p>‘nomads’ are entities which navigate and live in smooth spaces. Maybe my twitter bio makes a bit more sense now. ;)</p>
<h2>How do you create a smooth space?</h2>
<p>You might remember Riemann from the ‘<a href="http://en.wikipedia.org/wiki/Riemann_sum">Riemann sum</a>’ in Calc I. The brilliance of the Riemann sum is that it first striates, then re-smooths the space. You start off first with a curve, and then approximate the area under it by dividing it into a number of evenly-sized bits. This first bit was necessary in a world without calculus: we didn’t have the tooling or concepts to actually tackle the real area, so we mapped that problem to one we did know how to solve. As the number of bits goes up, and their width goes down, more and more of the space under the curve is captured by our algorithm. Finally, once we’re able to take that first step away from (striated) algebra and move into (smooth) calculus, we’re able to move about on the curve for reals. We’re back to smooth space again.</p>
<p>This interplay between smooth and striated spaces often happens, and there’s really no part of our world today that’s completely smooth or entirely striated. D&amp;G posit that the left should be attempting to create as much smooth space as possible, and that capitalism is constantly attempting to striate space. That said, smooth space is necessary, but not sufficient: capitalism, in its hunger and lust for acquisition and totalitization, has managed to navigate some smooth spaces. I’ll just quote straight from ‘a thousand plateaus’, 492. It’s a little hard, but this is getting long enough as is. ;)</p>
<blockquote>
<p>Not only does the user as such tend to be an employee, but capitalism operates less on a quantity of labor than by a complex qualitative process bringing into play modes of transportation, urban models, the media, the entertainment industries, ways of perceiving and feeling – every semiotic system. It is though, at the outcome of the striation that capitalism was able to carry to an unequaled point of perfection, circulating capital necessarily recreated, reconstituted, a sort of smooths pace in which the destiny of human beings is recast. … at the level of world capitalism, a new smooth space is produced in which capital reaches its ‘absolute’ speed, based on machinic components rather than the human component of labor. The multinationals fabricate a kind of deterritorialized smooth space in which points of occupation as well as poles of exchange become quite independent of the classical paths to striation. .. the essential thing is instead the distinction between striated capital and smooth capital, and the way in which the former gives rise to the latter through complexes that cut across territories and States, and even the different types of States.</p>
</blockquote>
<p>The first part about ‘users’ is very much drawing a parallel to “If you’re not paying for the product, you are the product.” When they talk about the ‘speed’ of capital, think of HFT. Multinational corporations have managed to overcome the striation that a State has imposed. McDonalds is only partially affected by US laws. They operate ‘outside’ of states.</p>
<p>In this way, simply being smooth will not suffice to save us.</p>
<hr />
<p>If you liked this blog post, you may enjoy <a href="https://groups.google.com/forum/#!forum/philosophy-in-a-time-of-software">"Philosophy in a time of software</a>, a moderated Google Group to discuss the intersection of philosophy and technology. This post started out as an email to that list.</p>
</content:encoded></item><item><title>Travis build matrix for Rails</title><link>https://steveklabnik.com/writing/travis-build-matrix-for-rails/</link><guid isPermaLink="true">https://steveklabnik.com/writing/travis-build-matrix-for-rails/</guid><pubDate>Sun, 10 Mar 2013 00:00:00 GMT</pubDate><content:encoded><p>Do you have a gem that needs to test against multiple versions of Rails? Doing it all can be complex, and while I have lots to say about this topic, but here’s one of the pieces: Travis build matrix.</p>
<p>Here’s what you want:</p>
<pre><code>language: ruby
rvm:
- 1.8.7
- 1.9.2
- 1.9.3
- 2.0.0
- ruby-head
env:
- "RAILS_VERSION=3.2"
- "RAILS_VERSION=3.1"
- "RAILS_VERSION=3.0"
- "RAILS_VERSION=master"
matrix:
allow_failures:
- rvm: ruby-head
- env: "RAILS_VERSION=master"
exclude:
- rvm: 2.0.0
env: "RAILS_VERSION=3.0"
- rvm: 2.0.0
env: "RAILS_VERSION=3.1"
- rvm: 1.8.7
env: "RAILS_VERSION=master"
- rvm: 1.9.2
env: "RAILS_VERSION=master"
- rvm: ruby-head
env: "RAILS_VERSION=3.0"
- rvm: ruby-head
env: "RAILS_VERSION=3.1"
</code></pre>
<p>Here’s what all this does:</p>
<ol>
<li>1.8.7 and 1.9.2 is supported across all of the 3.x series, but not master</li>
<li>2.0.0 is supported on Rails 3.2 (as of the upcoming 3.2.13, but it will probably work with your gem with 3.2.12) and master, but not 3.0 or 3.1.</li>
<li>We want to allow failures on ruby-head, because it’s not likely to be stable</li>
<li>We also want to allow failures for Rails master, because sometimes it’s not stable either</li>
<li>ruby-head should only be built against master</li>
</ol>
<p>What do you think? <a href="http://twitter.com/steveklabnik">Tweet at me</a> if you have suggestions for improving this matrix.</p>
</content:encoded></item><item><title>Announcing security_release_practice</title><link>https://steveklabnik.com/writing/announcing-securityreleasepractice/</link><guid isPermaLink="true">https://steveklabnik.com/writing/announcing-securityreleasepractice/</guid><pubDate>Fri, 08 Mar 2013 00:00:00 GMT</pubDate><content:encoded><p>Security is hard. One of the skills any OSS maintainer needs is how to do releases, and security releases are a special kind of release that needs special git-fu to make it work.</p>
<p>I’ve created a repository so that you can practice this particular skill. It’s called ‘security_release_practice’.</p>
<h2>The problem</h2>
<p>‘security_release_practice’ provides a binary, <code>omg_insecure</code>, which has a security issue. Basically, the <code>super_secure_calculation</code> method converts user input to a symbol. Since Ruby does not garbage collect symbols, the longer you run <code>omg_insecure</code>, the more memory it will use, until your computer runs out of memory and the box grinds to a halt. Seems bad.</p>
<p>So, just fix <code>super_secure_calculation</code> and release, right? Well, here’s the problem: the last release of <code>security_release_practice</code> was 1.0.0. Since then, we’ve had a new feature, and a backwards incompatible change with <code>super_secure_calculation</code>. You can see the two commits <a href="https://github.com/steveklabnik/security_release_practice/compare/v1.0.0...master">here</a>.</p>
<p>This is a problem: if we fix the issue and release, people who are relying on the <code>+ 5</code> behavior can’t upgrade: they’ll now be getting <code>+ 6</code>. Also, the new feature (<code>another_new_calculation</code>) may have conflicts or weirdness with their code. That’s bad! So what we really want is a relase that’s exactly the same as 1.0.0, but with the security fix applied.</p>
<p>Let’s give that a shot.</p>
<h2>The answer</h2>
<p>If you think you’re good with <code>git</code>, you can try this out right now. If you’ve done it correctly, you should end up with the following:</p>
<ol>
<li>A 1-0-stable branch</li>
<li>That branch should contain a new commit that fixes the issue</li>
<li>That branch should contain a new tag, v1.0.1 that fixes the issue</li>
<li>Master should have a backported version of the commit in #2.</li>
</ol>
<p>The repository <a href="https://github.com/steveklabnik/security_release_practice">as it exists</a> has all of this stuff, so check your work against it!</p>
<h2>Practice!</h2>
<p>If you <em>don’t</em> know how to do this, or you get stuck, you’ve come to the right place! Here’s what you need to do:</p>
<p>First, some setup work. Fork the repository and clone it down. Or, just clone mine, whatever:</p>
<pre><code>$ git clone https://github.com/steveklabink/security_release_practice
$ cd security_release_practice
</code></pre>
<p>Next, since this repository has the backported fix involved, you need to remove that commit:</p>
<pre><code>$ git reset --hard HEAD~1
</code></pre>
<p>This basically backs our branch out by one commit. Now we’re ready to go.</p>
<p>The first thing in actually doing the work is to check out the tag that we last released from. In our case, that tag is <code>v1.0.0</code>. So let’s do that now:</p>
<pre><code>$ git checkout v1.0.0
</code></pre>
<p><code>git</code> will give you a message:</p>
<pre><code>Note: checking out 'v1.0.0'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b new_branch_name
HEAD is now at 47a8bfb... Initial release.
</code></pre>
<p>We want to take <code>git</code>’s advice: let’s make a new branch. Since it’s going to be all our fixes for <code>1.0.x</code>, let’s call it <code>1-0-stable</code>:</p>
<pre><code>git checkout -b 1-0-stable
</code></pre>
<p>Now we have our stable branch. Awesome! Master is the work that will go into <code>1.1.x</code>, and this branch will be for <code>1.0.x</code>.</p>
<p>Next, we need to fix our bug. You need to remove this one line:</p>
<pre><code>--- a/lib/security_release_practice.rb+++ b/lib/security_release_practice.rb@@ -3,7 +3,7 @@ require "security_release_practice/version" module SecurityReleasePractice def super_secure_calculation(input)- input.to_sym input.to_i + 5 end def another_new_calculation(input)
</code></pre>
<p>We don’t even use that symbol, what a waste! Commit this, with a descriptive message. You can <a href="https://github.com/steveklabnik/security_release_practice/commit/168d5f756221ed43b0c67569ac82429f0b391504">find mine here</a>. Note the first few characters of the hash: mine is <code>168d5f756221</code>.</p>
<p>Next, we need to release. Go ahead and increment the version number in <code>lib/security_release_practice/version.rb</code>, commit that, and then try <code>rake install</code>. Everything should work. Great! If you were actually releasing this gem, you’d be running <code>rake release</code> instead, but it’s my gem, not yours. ????</p>
<p>Okay, now we’ve released a version with our fix, but <code>master</code> still has a vulnerability: we need to port the fix. So let’s go back to master:</p>
<pre><code>$ git checkout master
</code></pre>
<p>And then cherry-pick our fix over:</p>
<pre><code>$ git cherry-pick 168d5f756221
</code></pre>
<p>There will be a conflict. The diff is a little weird, such is life. Go ahead and fix the conflict, then commit:</p>
<pre><code>$ git commit
</code></pre>
<p>And you’re done!</p>
<h3>Other considerations</h3>
<p>If we had a CHANGELOG, we’d need to udpate that as appropriate, including creating new sections for our <code>1.0.1</code> release.</p>
<p>Sometimes it’s easier to fix things on master first, then backport to the new branch. I prefer to do it the way I showed here.</p>
<p>You should probably try to get as many people to know that you’ve fixed the bug. Tweet, blog, rabble-rouse, and possibly <a href="https://groups.google.com/forum/#!forum/ruby-security-ann">post to the ruby-security-ann mailing list</a>, which was created to help Rubyists know about security releases of their gems.</p>
<p>If your gem is really widely used, you may want to actually register a CVE. You can find information on this process <a href="https://groups.google.com/forum/#!forum/ruby-security-ann">here</a>. I made one <a href="https://groups.google.com/d/msg/ruby-security-ann/TDXOlIVjS54/Ty_9PXYNr3AJ">here</a>.</p>
</content:encoded></item><item><title>Going vimgan</title><link>https://steveklabnik.com/writing/going-vimgan/</link><guid isPermaLink="true">https://steveklabnik.com/writing/going-vimgan/</guid><pubDate>Wed, 06 Mar 2013 00:00:00 GMT</pubDate><content:encoded><p>I’m going vim-gan. Basically, I don’t want to consume any source code which doesn’t come from vim, or products that contain non-vim source code.</p>
<p>While I won’t, some people I’ve talked to will also consume code produced in Eclipse, since recent scientific studies show that Eclipse users can’t actually feel pain. This is enough of an ethical gray area that I’m staying away from it, but I can see that perspective.</p>
<p>I just can’t stay silent while the software industry brutalizes source code like this. Not cool, everyone.</p>
</content:encoded></item><item><title>Using puma on Heroku</title><link>https://steveklabnik.com/writing/using-puma-on-heroku/</link><guid isPermaLink="true">https://steveklabnik.com/writing/using-puma-on-heroku/</guid><pubDate>Sun, 24 Feb 2013 00:00:00 GMT</pubDate><content:encoded><p>I’m a big fan of <a href="http://puma.io/">Puma</a> these days. Today, I converted my blog over to use Puma, and I figured I’d share how I did it. This blog uses Sinatra.</p>
<p>If you want to look ahead, <a href="https://github.com/steveklabnik/blog/commit/3cd0f04ed29a25df6d6dfeeffccd2e12548c05cf">here</a> is the commit.</p>
<h2>Gemfile</h2>
<p>Installing Puma is really easy: just add <code>gem 'puma'</code> to your Gemfile. Done!</p>
<p>This section is way too easy. I’m sorry Puma isn’t more complicated. ;)</p>
<h2>Procfile</h2>
<p>Heroku attempts to guess how you want to run your site, but sometimes it guesses wrong. I prefer to be a bit more explicit, and in Puma’s case, it will guess wrong and use Webrick, so we need to do this step.</p>
<p>Heroku uses a gem called <code>foreman</code> to run your application. Basically, <code>foreman</code> gives you a ‘Procfile’ that lets you specify different processes and their types, and then foreman manages running them for you.</p>
<p>To use <code>foreman</code>, add it to your Gemfile:</p>
<pre><code>gem 'foreman'
</code></pre>
<p>And then make a <code>Procfile</code>:</p>
<pre><code>web: rackup -s puma -p $PORT
</code></pre>
<p>This says “I want my web processes to run this command.” Simple. We tell <code>rackup</code> to use Puma with the <code>-s</code> flag, for ‘server.’</p>
<p>To start up your application, normall you’d run <code>rackup</code> yourself, but now, you use <code>foreman</code>:</p>
<pre><code>$ bundle exec foreman start
</code></pre>
<p>It’ll print out a bunch of messages. Mine looks like this:</p>
<pre><code>$ bundle exec foreman start
15:05:05 web.1 | started with pid 52450
15:05:06 web.1 | Puma 1.6.3 starting...
15:05:06 web.1 | * Min threads: 0, max threads: 16
15:05:06 web.1 | * Environment: development
15:05:06 web.1 | * Listening on tcp://0.0.0.0:5000
</code></pre>
<p>Now you can <code>open http://localhost:5000</code> in another terminal, and bam! Done.</p>
<h2>Other Rubies</h2>
<p>If you really want Puma to scream, you should run with Rubinius or JRuby. I haven’t done this yet, but once I do, I’ll update this post with instructions.</p>
<h2>Rails</h2>
<p>If you’re using Rails, it’s the same process, but the Procfile line should be</p>
<pre><code>web: bundle exec puma -p $PORT
</code></pre>
<p>Easy!</p>
</content:encoded></item><item><title>I'm writing "Rails 4 in Action"!</title><link>https://steveklabnik.com/writing/i-m-writing--rails-4-in-action--/</link><guid isPermaLink="true">https://steveklabnik.com/writing/i-m-writing--rails-4-in-action--/</guid><pubDate>Wed, 13 Feb 2013 00:00:00 GMT</pubDate><content:encoded><p>Last night, I took a big step forward in my writing career: I’ll be building upon the work Ryan Bigg was doing to take “Rails 3 in Action” and move it to Rails 4. I signed the contract a few weeks ago, but last night I actually did the first bit of editing, and successfully got everything through the system.</p>
<p>Here’s the <a href="http://www.manning.com/bigg2/">early access edition</a> link, where you can check out what I’m doing. Of course, Rails 4 isn’t actually out yet, but since I’m on the Rails team, I already keep abreast of changes daily. I’ll be doing a first pass to take the book from 3.2 to edge over the next few weeks, and then keeping it current until the actual release of Rails 4.</p>
<p>Of course, updating the content is much easier than writing it in the first place. Thanks for all of your hard work on previous editions, Yehuda and Ryan!</p>
<hr />
<p>Of course, this doesn’t mean I’ll stop working my my two self-published books, <a href="http://www.designinghypermediaapis.com/">Designing Hypermedia APIs</a> and <a href="http://www.rustforrubyists.com/">Rust for Rubyists</a>. It just means I’ll be doing even more writing!</p>
</content:encoded></item><item><title>The next iteration of "Designing Hypermedia APIs"</title><link>https://steveklabnik.com/writing/the-next-iteration-of-designing-hypermedia-apis/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-next-iteration-of-designing-hypermedia-apis/</guid><pubDate>Tue, 12 Feb 2013 00:00:00 GMT</pubDate><content:encoded><p>I sent out an email today to everyone who’d previously purchased <a href="http://www.designinghypermediaapis.com/">Designing Hypermedia APIs</a>. Here’s the text:</p>
<pre><code>Hey there,
First of all, I want to apologize for sending you an email. I try to
keep these to a minimum. But this one is important.
I haven't been uploading new content lately for a few reasons.
Holidays are complicated, I wasn't feeling inspired... but this is the
real reason: I've totally moved the site to something different. I
just updated it, so the DNS might need an update, but
http://www.designinghypermediaapis.com/ is all-new. Here's the deal:
1. Gone is logging in and reading it online. PDF/ePUB/MOBI now.
2. Because you purchased a copy before, you get the big bundle. Thank
you for your support over the last year.
3. You'll be getting another email with a link to download the files.
When I send an update, you'll get a new email with a new link.
4. The current written content is basically an appendix: I'm going to
be writing a more traditional book, from start to finish. It'll walk
you through building an application, both client and server.
5. I want this to be of the highest quality, so I'm not going to
commit to a schedule.
If you aren't a part of the mailing list, we're doing a reading club
over the next few months, "Building Hypermedia APIs with HTML5 and
Node." You can join by emailing [email protected].
Thank you once again for your support of this project, I really
appreciate it. As always, any and all feedback welcome: I bet I can do
some nicer formatting of the PDF/ePUB/MOBI now that I have them
generating.
</code></pre>
<p>Wooo!</p>
</content:encoded></item><item><title>Ruby on Rails maintenance policy</title><link>https://steveklabnik.com/writing/ruby-on-rails-maintenance-policy/</link><guid isPermaLink="true">https://steveklabnik.com/writing/ruby-on-rails-maintenance-policy/</guid><pubDate>Mon, 11 Feb 2013 00:00:00 GMT</pubDate><content:encoded><p>Recently, the Rails team has committed to a specific policy related to release maintenance. Due to the rapid pace of recent releases, it’s good to understand how your apps relate to this policy.</p>
<p>The policy was originally posted on Google Groups, here: <a href="https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-security/G4TTUDDYbNA">https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-security/G4TTUDDYbNA</a>.</p>
<p>Here’s a copy if you don’t want to read Google Groups:</p>
<hr />
<p>Since the most recent patch releases there has been some confusion about what versions of Ruby on Rails are currently supported, and when people can expect new versions. Our maintenance policy is as follows.</p>
<p>Support of the Rails framework is divided into four groups: New features, bug fixes, security issues, and severe security issues. They are handled as follows:</p>
<h2>New Features</h2>
<p>New Features are only added to the master branch and will not be made available in point releases.</p>
<h2>Bug fixes</h2>
<p>Only the latest release series will receive bug fixes. When enough bugs are fixed and its deemed worthy to release a new gem, this is the branch it happens from.</p>
<p>Currently included series: 3.2.x</p>
<h2>Security issues:</h2>
<p>The current release series and the next most recent one will receive patches and new versions in case of a security issue.</p>
<p>Currently included series: 3.2.x, 3.1.x</p>
<h2>Severe security issues:</h2>
<p>For severe security issues we will provide new versions as above, and also the last major release series will receive patches and new versions. The classification of the security issue is judged by the core team.</p>
<p>Currently included series: 3.2.x, 3.1.x, 2.3.x</p>
<h2>Unsupported Release Series</h2>
<p>When a release series is no longer supported, it’s your own responsibility to deal with bugs and security issues. We may provide back-ports of the fixes and publish them to git, however there will be no new versions released. If you are not comfortable maintaining your own versions, you should upgrade to a supported version.</p>
<p>You should also be aware that Ruby 1.8 will reach End of Life in June 2013, no further ruby security releases will be provided after that point. If your application is only compatible ruby 1.8 you should upgrade accordingly.</p>
<p>– Cheers,</p>
<p>Koz</p>
</content:encoded></item><item><title>Ember.js and accessibility</title><link>https://steveklabnik.com/writing/emberjs-and-accessibility/</link><guid isPermaLink="true">https://steveklabnik.com/writing/emberjs-and-accessibility/</guid><pubDate>Thu, 07 Feb 2013 00:00:00 GMT</pubDate><content:encoded><p>Hey everyone! I made a video today about Ember.js and accessibility. I had always been repeating what I’d heard about screen readers: they can’t work with JavaScript. Turns out that’s not exactly true. The video is the best way to show you:</p>
<p><a href="http://vimeo.com/59124303">VoiceOver, Ember.js, and WAI-ARIA</a> from <a href="http://vimeo.com/steveklabnik">Steve Klabnik</a> on <a href="http://vimeo.com/">Vimeo</a>.</p>
<p>Here’s a rough transcript, if you don’t want to watch it:</p>
<hr />
<p>Hey everyone. I recently was checking out <a href="http://emberjs.com/">Ember.js</a>, and more specifically, <a href="http://discourse.org/">Discourse</a>. Discourse is supposed to be a next-generation discussion platform, and it’s an Ember app backed by a Rails-based JSON API. I mentioned to Ember co-creator <a href="http://twitter.com/wycats">Yehuda Katz</a> that I wished Ember had ‘a better accessibility story’. He asked me to elaborate, and I mentioned how my friends on Twitter were saying that Discourse wouldn’t able to be inclusive because JavaScript-heavy apps can’t be used by screenreaders. So he said, “Hit ⌘-F5.”</p>
<p>Whoah.</p>
<p>Hitting that key combination activates <a href="http://www.apple.com/accessibility/voiceover/">VoiceOver</a>, a feature of OS X I didn’t know existed. Basically, VoiceOver is able to interact with applications and allow keyboard navigation for those who are blind or sight-impaired. Neat! But what about in-browser? Turns out VoiceOver handles JavaScript heavy pages just fine.</p>
<p>I tried it out on <a href="http://try.discourse.org/">http://try.discourse.org/</a>, and it works pretty well! Not perfect, but you can manage. Yehuda also pointed me at <a href="http://www.w3.org/WAI/intro/aria.php">WAI-ARIA</a>, which is a W3C initiative. I’ll let the site explain:</p>
<blockquote>
<p>WAI-ARIA, the Accessible Rich Internet Applications Suite, defines a way to make Web content and Web applications more accessible to people with disabilities. It especially helps with dynamic content and advanced user interface controls developed with Ajax, HTML, JavaScript, and related technologies.</p>
</blockquote>
<p>Neat! Basically, you can add markup to your HTML, and it will help applications like VoiceOver allow people to use your site more easily, even if it uses a lot of Ajax. This is super cool, and for me, mitigates one of my main complaints against using SPAs. Of course, WAI-ARIA isn’t really Ember’s job; you’ll have to make the HTML be output in the right way yourself. But it makes me really happy to know that people with disabilities don’t need to be left out of the rich-client portion of the web.</p>
</content:encoded></item><item><title>How to not rely on rubygems.org for development</title><link>https://steveklabnik.com/writing/how-to-not-rely-on-rubygemsorg-for-deployment/</link><guid isPermaLink="true">https://steveklabnik.com/writing/how-to-not-rely-on-rubygemsorg-for-deployment/</guid><pubDate>Thu, 31 Jan 2013 00:00:00 GMT</pubDate><content:encoded><p>Due to the recent <a href="https://status.heroku.com/incidents/489">situation with Rubygems.org</a>, a lot of people noticed that they rely on Rubygems.org when deploying. A lot of people advocate <a href="http://ryan.mcgeary.org/2011/02/09/vendor-everything-still-applies/">“vendor everything”</a>, and while that’s one approach, I actually don’t think it’s necessary. I think a lot of people <em>think</em> they rely on Rubygems.org more than they actually do, or, at least, aren’t deploying with Bundler correctly. So let’s talk about it.</p>
<h2>Deploying with Bundler</h2>
<p>Bundler has a <code>--deployment</code> flag to help you deploy code. <a href="http://gembundler.com/v1.2/deploying.html">Here’s the documentation</a>. If you use Bundler, and you don’t use <code>--deployment</code> when deploying, you are probably Doing It Wrong. Let’s try a brand new app, and see what happens:</p>
<pre><code>$ rails _3.2.11_ new foo --skip-bundle
create
create README.rdoc
create Rakefile
create config.ru
create .gitignore
create Gemfile
create app
create app/assets/images/rails.png
create app/assets/javascripts/application.js
create app/assets/stylesheets/application.css
create app/controllers/application_controller.rb
create app/helpers/application_helper.rb
create app/views/layouts/application.html.erb
create app/mailers/.gitkeep
create app/models/.gitkeep
create config
create config/routes.rb
create config/application.rb
create config/environment.rb
create config/environments
create config/environments/development.rb
create config/environments/production.rb
create config/environments/test.rb
create config/initializers
create config/initializers/backtrace_silencers.rb
create config/initializers/inflections.rb
create config/initializers/mime_types.rb
create config/initializers/secret_token.rb
create config/initializers/session_store.rb
create config/initializers/wrap_parameters.rb
create config/locales
create config/locales/en.yml
create config/boot.rb
create config/database.yml
create db
create db/seeds.rb
create doc
create doc/README_FOR_APP
create lib
create lib/tasks
create lib/tasks/.gitkeep
create lib/assets
create lib/assets/.gitkeep
create log
create log/.gitkeep
create public
create public/404.html
create public/422.html
create public/500.html
create public/favicon.ico
create public/index.html
create public/robots.txt
create script
create script/rails
create test/fixtures
create test/fixtures/.gitkeep
create test/functional
create test/functional/.gitkeep
create test/integration
create test/integration/.gitkeep
create test/unit
create test/unit/.gitkeep
create test/performance/browsing_test.rb
create test/test_helper.rb
create tmp/cache
create tmp/cache/assets
create vendor/assets/javascripts
create vendor/assets/javascripts/.gitkeep
create vendor/assets/stylesheets
create vendor/assets/stylesheets/.gitkeep
create vendor/plugins
create vendor/plugins/.gitkeep
steve at thoth in ~/tmp
$ cd foo
steve at thoth in ~/tmp/foo
$ cat .bundle/config
cat: .bundle/config: No such file or directory
</code></pre>
<p>We have no configuration for bundler. Makes sense, we never bundled. If we try to use <code>--deployment</code> now, we get an error:</p>
<pre><code>steve at thoth in ~/tmp/foo
[1] $ bundle install --deployment ✘
The --deployment flag requires a Gemfile.lock. Please make sure you have checked your Gemfile.lock into version control before deploying.
</code></pre>
<p>This is because <code>--deployment</code> checks the lock. So let’s make one:</p>
<pre><code>steve at thoth in ~/tmp/foo
[16] $ bundle ✘
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/..
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.5.0)
Using activesupport (3.2.11)
Using builder (3.0.4)
Using activemodel (3.2.11)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.4)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.11)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.11)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.11)
Using activeresource (3.2.11)
Using bundler (1.3.0.pre.5)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Installing rack-ssl (1.3.3)
Using json (1.7.6)
Using rdoc (3.12)
Using thor (0.17.0)
Using railties (3.2.11)
Using coffee-rails (3.2.2)
Using jquery-rails (2.2.0)
Using rails (3.2.11)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
</code></pre>
<p>This of course does hit the network, and does all the usual things. Your application normally has a lock already and has it checked in. We still don’t have a config:</p>
<pre><code>steve at thoth in ~/tmp/foo
$ cat .bundle/config
cat: .bundle/config: No such file or directory
</code></pre>
<p>So, this is the state our app would be in before deploying. At this point, it’s just like we did a <code>git pull</code> or an <code>scp</code> to our server. So let’s install for deployment:</p>
<pre><code>steve at thoth in ~/tmp/foo
$ bundle install --deployment
Fetching gem metadata from https://rubygems.org/.........
Fetching gem metadata from https://rubygems.org/..
Installing rake (10.0.3)
Installing i18n (0.6.1)
Installing multi_json (1.5.0)
Installing activesupport (3.2.11)
Installing builder (3.0.4)
Installing activemodel (3.2.11)
Installing erubis (2.7.0)
Installing journey (1.0.4)
Installing rack (1.4.4)
Installing rack-cache (1.2)
Installing rack-test (0.6.2)
Installing hike (1.2.1)
Installing tilt (1.3.3)
Installing sprockets (2.2.2)
Installing actionpack (3.2.11)
Installing mime-types (1.19)
Installing polyglot (0.3.3)
Installing treetop (1.4.12)
Installing mail (2.4.4)
Installing actionmailer (3.2.11)
Installing arel (3.0.2)
Installing tzinfo (0.3.35)
Installing activerecord (3.2.11)
Installing activeresource (3.2.11)
Installing coffee-script-source (1.4.0)
Installing execjs (1.4.0)
Installing coffee-script (2.2.0)
Installing rack-ssl (1.3.3)
Installing json (1.7.6)
Installing rdoc (3.12)
Installing thor (0.17.0)
Installing railties (3.2.11)
Installing coffee-rails (3.2.2)
Installing jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Installing rails (3.2.11)
Installing sass (3.2.5)
Installing sass-rails (3.2.6)
Installing sqlite3 (1.3.7)
Installing uglifier (1.3.0)
Your bundle is complete! It was installed into ./vendor/bundle
Post-install message from rdoc:
Depending on your version of ruby, you may need to install ruby rdoc/ri data:
&lt;= 1.8.6 : unsupported
= 1.8.7 : gem install rdoc-data; rdoc-data --install
= 1.9.1 : gem install rdoc-data; rdoc-data --install
&gt;= 1.9.2 : nothing to do! Yay!
</code></pre>
<p>We still hit Rubygems.org, but note now that it said “Installing” rather than using. That’s because we now have everything vendored:</p>
<pre><code>$ cat .bundle/config
---
BUNDLE_FROZEN: '1'
BUNDLE_PATH: vendor/bundle
BUNDLE_DISABLE_SHARED_GEMS: '1'
steve at thoth in ~/tmp/foo
$ ls vendor/bundle/ruby/1.9.1
bin cache doc gems specifications
steve at thoth in ~/tmp/foo
$ ls vendor/bundle/ruby/1.9.1/gems
actionmailer-3.2.11 multi_json-1.5.0
actionpack-3.2.11 polyglot-0.3.3
activemodel-3.2.11 rack-1.4.4
activerecord-3.2.11 rack-cache-1.2
activeresource-3.2.11 rack-ssl-1.3.3
activesupport-3.2.11 rack-test-0.6.2
arel-3.0.2 rails-3.2.11
builder-3.0.4 railties-3.2.11
coffee-rails-3.2.2 rake-10.0.3
coffee-script-2.2.0 rdoc-3.12
coffee-script-source-1.4.0 sass-3.2.5
erubis-2.7.0 sass-rails-3.2.6
execjs-1.4.0 sprockets-2.2.2
hike-1.2.1 sqlite3-1.3.7
i18n-0.6.1 thor-0.17.0
journey-1.0.4 tilt-1.3.3
jquery-rails-2.2.0 treetop-1.4.12
json-1.7.6 tzinfo-0.3.35
mail-2.4.4 uglifier-1.3.0
mime-types-1.19
</code></pre>
<p>Okay! Neat. So we’ve Vendored Everything… what happens if we <code>bundle</code> again?</p>
<pre><code>steve at thoth in ~/tmp/foo
$ bundle
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.5.0)
Using activesupport (3.2.11)
Using builder (3.0.4)
Using activemodel (3.2.11)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.4)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.11)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.11)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.11)
Using activeresource (3.2.11)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using json (1.7.6)
Using rdoc (3.12)
Using thor (0.17.0)
Using railties (3.2.11)
Using coffee-rails (3.2.2)
Using jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Using rails (3.2.11)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! It was installed into ./vendor/bundle
</code></pre>
<p>The <code>bundle</code> command uses our config settings in <code>.bundle/config</code>, which re-runs with the vendored bundle.</p>
<h2>Wait, I thought you said NO to Vendor Everything!!!</h2>
<p>Well, here’s the deal: you can Vendor Everything <em>on your server</em>, which means that we’re not committing gems into source control, and then pushing that huge mess over the network.</p>
<p>Let’s re-examine <code>--deployment</code> in the context of our two strategies: <code>scp</code> and <code>git</code>.</p>
<h3>git</h3>
<p>If you deploy with git, it’s two ways: <code>ssh</code>ing into the server and running a <code>git pull</code>, or by doing a <code>git push</code>. In both cases, you’ll have some sort of post-deploy hook that manages the rest of the deploy.</p>
<p>This scenario is exactly as shown above: just make sure that your first <code>bundle</code> on deploy is using <code>--deployment</code>, and you’re Good To Go. Each next deploy won’t hit the network. Rock on.</p>
<h3>scp</h3>
<p>If you use scp in some way to deploy, then you’re getting a new copy of the application every time, so that bundle full of gems won’t work. You need one more flag: <code>--path</code></p>
<pre><code>steve at thoth in ~/tmp/foo
$ rm -rf vendor
steve at thoth in ~/tmp/foo
$ bundle install --deployment --path ../vendor/bundle
Fetching gem metadata from https://rubygems.org/.........
Fetching gem metadata from https://rubygems.org/..
Installing rake (10.0.3)
Installing i18n (0.6.1)
Installing multi_json (1.5.0)
Installing activesupport (3.2.11)
Installing builder (3.0.4)
Installing activemodel (3.2.11)
Installing erubis (2.7.0)
Installing journey (1.0.4)
Installing rack (1.4.4)
Installing rack-cache (1.2)
Installing rack-test (0.6.2)
Installing hike (1.2.1)
Installing tilt (1.3.3)
Installing sprockets (2.2.2)
Installing actionpack (3.2.11)
Installing mime-types (1.19)
Installing polyglot (0.3.3)
Installing treetop (1.4.12)
Installing mail (2.4.4)
Installing actionmailer (3.2.11)
Installing arel (3.0.2)
Installing tzinfo (0.3.35)
Installing activerecord (3.2.11)
Installing activeresource (3.2.11)
Installing coffee-script-source (1.4.0)
Installing execjs (1.4.0)
Installing coffee-script (2.2.0)
Installing rack-ssl (1.3.3)
Installing json (1.7.6)
Installing rdoc (3.12)
Installing thor (0.17.0)
Installing railties (3.2.11)
Installing coffee-rails (3.2.2)
Installing jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Installing rails (3.2.11)
Installing sass (3.2.5)
Installing sass-rails (3.2.6)
Installing sqlite3 (1.3.7)
Installing uglifier (1.3.0)
Your bundle is complete! It was installed into /Users/steve/tmp/vendor/bundle
Post-install message from rdoc:
Depending on your version of ruby, you may need to install ruby rdoc/ri data:
&lt;= 1.8.6 : unsupported
= 1.8.7 : gem install rdoc-data; rdoc-data --install
= 1.9.1 : gem install rdoc-data; rdoc-data --install
&gt;= 1.9.2 : nothing to do! Yay!
steve at thoth in ~/tmp/foo
$ cat .bundle/config
---
BUNDLE_FROZEN: '1'
BUNDLE_PATH: ../vendor/bundle
BUNDLE_DISABLE_SHARED_GEMS: '1'
steve at thoth in ~/tmp/foo
$ ls vendor
ls: vendor: No such file or directory
steve at thoth in ~/tmp/foo
[1] $ ls ../vendor ✘
bundle
steve at thoth in ~/tmp/foo
$ bundle
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.5.0)
Using activesupport (3.2.11)
Using builder (3.0.4)
Using activemodel (3.2.11)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.4)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.11)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.11)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.11)
Using activeresource (3.2.11)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using json (1.7.6)
Using rdoc (3.12)
Using thor (0.17.0)
Using railties (3.2.11)
Using coffee-rails (3.2.2)
Using jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Using rails (3.2.11)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! It was installed into /Users/steve/tmp/vendor/bundle
</code></pre>
<p>The <code>--path</code> flag controls where the bundle is located. In this case, we store it one directory up. Now, when we copy new versions of the code over, it will use the bundle location that stays the same, and all is peachy keen.</p>
<p>I am told by several people that this is what the Capistrano/bundler recipe does by default, so if you’re using that, you’re already doing this.</p>
<h2>One Tiny Weakness</h2>
<p>There is one small weakness of this approach compared to Vendor Everything: as an engineer, it’s your tradeoff to make.</p>
<p>This way of using bundler <em>will</em> hit the network <em>when you deploy for the first time after updating your bundle</em>. The cache on the server has to update in this case, so it will go fetch the new gems. So here’s the scenario:</p>
<ol>
<li>I update my bundle. It gets the new versions of the gems.</li>
<li>Rubygems.org goes down.</li>
<li>I need to deploy a branch that has the new versions in it.</li>
</ol>
<p>In this case, if you had Vendored Everything, the hit to Rubygems.org would have happened during step 1, and so things would work. If you used this strategy, it would have hit locally, so you could have developed, but then when deploying, it’d hit again to update the bundle on the server, and so it wouldn’t.</p>
<p>In these situations, you can temporarily switch to Vendor Everything, since you have the bundle installed locally: just copy your local gems over to <code>vendor/bundle</code> and you’re done. This may or may not be too much of a hassle. When I examine the downtime of Rubygems.org, I think it’s worth it to not mess up my git repository with all that gem code. You might not. Do whatever you need to do, but now you know how to not rely on Rubygems.org for every deployment.</p>
<hr />
<h3>An addendum</h3>
<p>I got an email mentioning one more issue with the ‘vendor everything’ strategy:</p>
<pre><code>$ ls vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/*.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_document.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_element_description.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_entity_lookup.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_sax_parser_context.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_sax_push_parser.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/nokogiri.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_attr.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_attribute_decl.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_cdata.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_comment.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_document.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_document_fragment.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_dtd.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_element_content.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_element_decl.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_encoding_handler.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_entity_decl.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_entity_reference.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_io.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_libxml2_hacks.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_namespace.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_node.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_node_set.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_processing_instruction.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_reader.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_relax_ng.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_sax_parser.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_sax_parser_context.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_sax_push_parser.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_schema.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_syntax_error.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_text.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_xpath_context.o
vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xslt_stylesheet.o
</code></pre>
<p>Basically, your compiled files get vendored, so if you’re, say, developing on Mac and deploying on Linux, you might end up with compiled native extensions for the totally wrong architecture.</p>
</content:encoded></item><item><title>Node</title><link>https://steveklabnik.com/writing/node/</link><guid isPermaLink="true">https://steveklabnik.com/writing/node/</guid><pubDate>Wed, 23 Jan 2013 00:00:00 GMT</pubDate><content:encoded><p>Soooo there’s this thing: <a href="http://harthur.wordpress.com/2013/01/24/771/">http://harthur.wordpress.com/2013/01/24/771/</a></p>
<p>First of all, unequivocally, I said it before, and I’ll say it again: I’m sorry, and feel terrible that I made someone feel terrible. @harthur, I felt bad yesterday, and I’m feeling bad again. Not that it helps you that I feel bad. But I don’t like making other people feel bad. I am sorry.</p>
<h2>tl;dr</h2>
<p>TL;DR: I am sorry. I feel terrible that I made someone else feel terrible. I feel even worse because I didn’t mean to say that they were terrible, but accidentally did.</p>
<h2>the why</h2>
<p>Sooooooo I wrote up <em>why</em> I said what it was that I said, because what I was trying to say was totally unrelated to <code>replace</code>. I was trying to explain why I don’t like Node, and it basically came out wrong. But, explaining what I said had the affect of feeling like I was trying not to apologize. So, I’ll just cut to the chase and say this, cut straight from the chat log:</p>
<pre><code>SK: i dont want people to think i'm saying "i got caught being an asshole so i'm saying sorry"
SK: i want to say "I was accidentally being an asshole so i'm sorry"
</code></pre>
<p>So, I’ll just leave it at that. Twitter makes it so hard not to accidentally be an asshole.</p>
</content:encoded></item><item><title>_why is a role, not a person</title><link>https://steveklabnik.com/writing/why-is-a-role-not-a-person/</link><guid isPermaLink="true">https://steveklabnik.com/writing/why-is-a-role-not-a-person/</guid><pubDate>Wed, 16 Jan 2013 00:00:00 GMT</pubDate><content:encoded><p>http://whytheluckystiff.net/ has re-ignited a discussion about _why, and while I have tons and tons and tons of things to say on the subject, this is all I can give you for now: _why is a role, not a person.</p>
<p>For a long time, I was talking about a separation between _why and the person who played him. I still have not yet figured out how to refer to that person, and so for now, I’ve settled on Subject. So I’ll be using that description for the rest of this post.</p>
<h2>All the world’s a stage</h2>
<blockquote>
<p>All the world’s a stage, And all the men and women merely players:Shakespeare, “As You Like It,” Act II Scene VII</p>
</blockquote>
<p>One of the things I’ve learned as I’ve gotten a little older (21 days till 27!) is that I’m a different person in different situations. This may seem painfully obvious, but let’s expand on this a bit: when I’m on stage at a Ruby conference, I’m one person, when I’m by myself at home, I’m a different person, online, I’m a different person, and when I’m spending time with someone I care about, I’m someone else. Each of these contexts produces different behavior, mannerisms, and vocabulary. When I’m speaking with someone who’s read French Postmodernism, I use very different terms than when I’m speaking with my Republican ’Murican relatives.</p>
<p>There’s absolutely nothing wrong with this, but it really suggests that ‘essentialism’ is not a thing that can properly model the world. “Essentialism” is a position that states that there is some set of attributes that one can possess to ‘be’ something. So, for example, let’s take a Zebra: some essential attributes may be:</p>
<ol>
<li>Four legs</li>
<li>Black with White Stripes</li>
</ol>
<p>http://www.flickr.com/photos/kliski/4816615762/</p>
<p>Shit.</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24435086611428_raw.jpg" alt="https://svbtleusercontent.com/inline_steveklabnik_24435086611428_raw.jpg" /></p>
<p>Double shit.</p>
<p>So what makes a zebra a zebra then? I don’t want to get into it, but basically, I take the position that you can’t define something by what it is, you have to define it by what it <em>does</em>. I’m punting on why this is, again, I want to talk about _why, so please give me a charitable reading and take this as axiomatic for now.</p>
<p>If we’re defined by what we do, then it makes sense that we can contextualize what we do: we can take on different roles at different times. When I program in Ruby, I am a Rubyist; when I ride a bike, I am a cyclist, when I have a child, I am a father, when I love and am loved, I am a lover.</p>
<p>When you treat programming as an art, when you treat software as an artistic medium, when you spread joy and whimsy with what you do, you are _why.</p>
<p>It just so happens that Subject was the person who created this particular role, as was the one who best played it. That doesn’t mean that we can’t also try to take that role on from time to time, try out out, see if it fits, wiggle our toes a bit.</p>
<h2>_why vs. Subject</h2>
<p>This is one of the reasons I think the concept of _why is important: Subject was trying to teach us something. They were creating an abstraction, or, maybe, an indirection. By keeping themselves private, they put an emphasis on the distance between _why-as-role and _why-as-person, since the second doesn’t exist. The focus was, as it should be, on the role.</p>
<p>This is also why it’s wrong to ‘track _why down.’ This doesn’t make any sense. Subject is not _why, they just played them on TV. I also think that’s why it’s not wrong to keep talking about _why, as long as we make sure that we’re actually talking about _why, and not about Subject. Subject deserves their privacy.</p>
<p>I also don’t mean that I want someone to literally ‘take up the mask’ of _why and publish under that name. What I mean is that if we want to take on the lessons that Subject taught us through their persona, we should periodically have becoming-_why moments.</p>
<h2>On Sloppiness and blog posts written in haste</h2>
<blockquote>
<p>@steveklabnik hand write it, take a picture of it. leave the mistakes in.</p>
</blockquote>
<p>— tef (@tef) January 6, 2013</p>
<blockquote></blockquote>
<h2>Further reading</h2>
<p>To send you further down a rabbit-hole: http://en.wikipedia.org/wiki/Body_without_organs</p>
<p>Enjoy.</p>
</content:encoded></item><item><title>Draper 1.0.0 released</title><link>https://steveklabnik.com/writing/draper-1-0-0-released/</link><guid isPermaLink="true">https://steveklabnik.com/writing/draper-1-0-0-released/</guid><pubDate>Mon, 14 Jan 2013 00:00:00 GMT</pubDate><content:encoded><p>Today, I released Draper 1.0.0! Draper is a gem that gives you decorators/view models for Rails.</p>
<ul>
<li>Rubygems: <a href="https://rubygems.org/gems/draper">https://rubygems.org/gems/draper</a></li>
<li>GitHub: <a href="https://github.com/drapergem/draper">https://github.com/drapergem/draper</a></li>
<li>README: <a href="https://github.com/drapergem/draper/blob/master/README.md">https://github.com/drapergem/draper/blob/master/README.md</a></li>
</ul>
<p>You can get it with Rubygems:</p>
<pre><code>$ gem install draper
</code></pre>
<p>or by adding this to your Gemfile:</p>
<pre><code>gem "draper", "~&gt;1.0.0"
</code></pre>
<p>Draper supports all versions of Rails 3, and follows <a href="http://semver.org/">SemVer</a>, so you can be confident that your application will work well with Draper in the future. Thank you to everyone who tried out our six (six!!!11) beta releases and gave us patches and feedback.</p>
</content:encoded></item><item><title>Rails has two default stacks</title><link>https://steveklabnik.com/writing/rails-has-two-default-stacks/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rails-has-two-default-stacks/</guid><pubDate>Mon, 14 Jan 2013 00:00:00 GMT</pubDate><content:encoded><p>Rails’ greatest strength is “Convention over Configuration.” Basically, Rails makes a ton of default choices for you about how things should be named, where they should go, and other things. This is why working with Rails is so productive for an experienced developer: just follow the Golden Path and everything comes together quickly. It’s second greatest strength is its ‘full stack’ nature: Rails includes everything that you need to get going. It does not attempt to be ‘minimal’ in any way. Put these two things together, and a Rails app makes lots of assumptions about <strong>everything</strong> in your app.</p>
<p>But what happens when a significant number of people don’t use those defaults?</p>
<p>First, let’s talk about the actual default stack. Since <a href="http://david.heinemeierhansson.com/2012/rails-is-omakase.html">Rails is omakase</a>, I’m going to call this the ‘Omakase Stack.’ Roughly, this stack contains:</p>
<ul>
<li>ERB for view templates</li>
<li>MySQL for databases</li>
<li>MiniTest for testing</li>
<li>Fat Models, Skinny Controllers</li>
</ul>
<p>There has been a ‘second default stack’ brewing for a while now. I’m going to call this the “<a href="http://en.wikipedia.org/wiki/Prime_(symbol)%23Use_in_mathematics.2C_statistics.2C_and_science">Prime</a> stack”. This stack’s approximate choices are:</p>
<ul>
<li>Haml for view templates</li>
<li>PostgreSQL for databases</li>
<li>Rspec/Cucumber for testing</li>
<li>Skinny models, controllers, and a service layer</li>
</ul>
<p>This is great! As David says:</p>
<blockquote>
<p>That doesn’t mean patrons can’t express exceptions. Substitutions are allowed, within reason. So you don’t like test/unit? No problem, bring your own rspec. Don’t care for CoffeeScript as a dessert? Delete a single line from your Gemfile.</p>
</blockquote>
<p>A considerable minority uses a stack like this. It’s important that the Prime Stack isn’t exact: you might not use Cucumber at all, for example, or maybe you don’t have a service layer. The important part, though, is that this is the new default you build preferences off of: a big group of people assumes that you don’t use MiniTest, you use Rspec, and the choice is Cukes or Capybara. This creates the ‘secondary default’ effect.</p>
<h2>Dual Stack Ecology</h2>
<p>So where’s the beef? Well, before I get into it, I want to make one thing explicit: I am <em>not</em> taking a position on which stack, if any, is ‘better.’ I’m not interested, at least in this post, at placing them into any kind of order. That’s for another time. What I want to talk about here is about the effect that a second default stack has on the overall Rails ecosystem.</p>
<p>Looking at history, the Prime Stack was born from the Omakase Stack. A certain group of people noticed that they were changing a significant amount of the stack most of the time, and so the Prime Stack came into existence. Since this is a historical progression, many blog posts got written describing the delta: take “<a href="http://robots.thoughtbot.com/post/159807023/waiting-for-a-factory-girl">Waiting for a Factory Girl</a>”, for example.</p>
<blockquote>
<p>Here at thoughtbot, we’ve had it with fixtures.</p>
</blockquote>
<p>But wait! Then, years later, you get a discussion on how factories are bad. My own “<a href="http://blog.steveklabnik.com/posts/2012-07-14-why-i-don-t-like-factory_girl">Why I don’t like factory_girl</a>” is an example:</p>
<blockquote>
<p>So that’s what it really boils down to: the convenience of factories has set Rails testing strategies and software design back two years.</p>
</blockquote>
<p>Maybe a bit strongly worded. Anyway, the people who have created the Prime Stack also happen to be those who are vocal, blog a lot, and speak at a lot of conferences. Everybody loves a polemic. If I submitted a talk to RailsConf called “Why Fixtures are :metal:” this year, people would assume that I was being sarcastic. People always assume that things are slowly getting better: If the original advice was “use fixtures” and the new advice is “use Factories” and the <em>newest</em> advice is “use neither”… there’s no way that the “right answer” is “use fixtures,” right? Isn’t that so 2007?</p>
<p>For those of us who’ve been around Rails for a long time, this discussion is good, and interesting. I was able to search for that Factory Girl post by memory, because I remember reading it when it was posted. I’ve written tons of Rails apps in the past 6ish years, and formed my own opinions on what tools I want to use due to that experience.</p>
<p>But what about someone starting Rails today?</p>
<h2>Beginners lose out</h2>
<p>Someone told me earlier today that “Starting Rails today is like starting to watch a soap opera in the 7th season.” Like most witty statements, there’s both truth and falsehood here. But, as a newcomer, how do you pick between the Omakase Stack and the Prime Stack?</p>
<p>This is the crux of the matter for me. If today, you tweeted that you were building a Rails app with fixtures, people would tell you to <a href="http://www.youtube.com/watch?v=n-Tej0297wk">Stop It</a>. I saw it happen this morning. This is again, because a very vocal minority of Rails developers use the Prime Stack, and are really passionate about their choice over the Omakase Stack. That’s great.</p>
<p>The problem is this: it puts a pretty heavy burden on a new developer: they have to learn the Omakase Stack <strong>and</strong> the delta. At the moment when someone is new, and barely knows anything, we throw them into making <em>more</em> choices. This is the moment when they’re least likely to be able to properly evaluate which option they need to choose. It’s not just “You’re new to Ruby, now learn Rails,” it’s now “You’re new to Ruby, now learn Rails, and RSpec, and Cucumber, and use this advanced modeling technique that we agree Rails discourages. Oh, and your only help is all these blog posts. Have fun!”</p>
<p>As I said at the beginning of this post, Convention over Configuration is Rails’ greatest strength. So when a new person starts, the first thing we do is tell them to do a bunch of configuration. Any reasonable developer could easily ask “If these options really are so superior, why do I have to include them? And which do I choose? I’m drowning!”</p>
<h2>Helping People Build Better Apps</h2>
<p>We all want to write better, more maintainable apps. And many people (including me) are trying to push the state of the art forward in this regard. But we’re leaving the majority in the dust. Rather than trying to push the boundaries by choosing new techniques, maybe we should be helping people get better at existing ones, too. Because we need both approaches. There’s already so much to learn when building a Rails application, adding more gems and techniques to the pile just isn’t going to help.</p>
<p>Once those beginners become intermediate, then they can figure out how new techniques solve the particular pains that they feel while going with the default flow. But skipping straight to the tool that solves my pain without feeling that pain the first place doesn’t help, it just confuses.</p>
<p>Let’s keep pushing those boundaries, but let’s not <em>require</em> someone to catch up on a few hundred episodes before they can understand what’s going on.</p>
<hr />
<p>If you read this post previously, the “Omakase Stack” was called the “37signals stack,” but David has pointed out <a href="https://twitter.com/dhh/status/293108811219230720">that even if he didn’t work with 37signals, he’d use this stack, so calling it that would be wrong</a>. I’ve since changed the name.</p>
</content:encoded></item><item><title>2012: year in review</title><link>https://steveklabnik.com/writing/2012-year-in-review/</link><guid isPermaLink="true">https://steveklabnik.com/writing/2012-year-in-review/</guid><pubDate>Sat, 29 Dec 2012 00:00:00 GMT</pubDate><content:encoded><p>I always enjoy reading <a href="http://www.kalzumeus.com/2012/12/29/bingo-card-creator-and-other-stuff-year-in-review-2012/">patio11’s year in review posts</a>, and while I didn’t take enough notes for this year to do a GOOD one, I figure that I should at least leave some notes so that when I do one next year, I’ll have something to benchmark against.</p>
<h2>Ruby</h2>
<p>This has been a whirlwind year for my career. My work with Jumpstart Lab is incredibly fulfilling, and makes sure my bills get paid.</p>
<ul>
<li>I flew all over the world. I’ve always wanted to travel, and while I complain about it a lot, it’s truly wonderful to be able to go places. I now have the second highest Delta frequent flyer tier memebership. If you’d told me that three years ago, I’d have laughed at you.</li>
<li>I spoke at a <em>lot</em> of conferences, and gave my first keynote. More lined up for next year already. Speaking is something I love to do.</li>
<li>Teaching classes has been fantastic, and I’d love to do even more of it next year.</li>
<li>I now have commit access to Rails. I put in a lot of work over the last few months, and this means a lot to me. Last step: core. We’ll see if I ever get there.</li>
<li>I now have 22 gems, with Draper being my largest project, at almost 200,000 downloads.</li>
</ul>
<p>Unrelated to Ruby, I was able to contribute to a wide variety of projects in a few languages this year, and now have <a href="http://github.com/steveklabnik">116 repos on GitHub</a>. A lot of those are abandoned or old, but still, they’re all something I was doing <em>something</em> with this year.</p>
<h2>Writing</h2>
<p>I finally made the <a href="http://designinghypermediaapis.com/">hypermedia book</a> become a reality, and it’s done really well. Many people have told me that it has opened their eyes to a new way of doing things, and a bunch of startups have told me they’ll be releasing hypermedia APIs in 2013. Sales have been brisk: I don’t want to disclose exact numbers, but I will tell you that it’s in the 5 figure range. This has allowed me to start taking chunks out of my student loans: once they’re paid off, I will be super lean financially.</p>
<p>I made it to the Svbtle blog network, which is invite-only. I’ve gotten mostly good feedback about everything I’ve done there, which is pleasing.</p>
<p>I started <a href="http://www.rustforrubyists.com/">a book to learn Rust programming</a> last week, as a way of helping myself learn Rust, making some more money, and testing out a publishing workflow that might influence the next iteration of Designing Hypermedia APIs. I like it a lot.</p>
<h2>Reading</h2>
<p>I read a lot, though not as much as I’d like. I want to read 30 books this next year, and finally got into the habit of using GoodReads.</p>
<h2>Etc</h2>
<p>I had a really good year in other regards: fulfilling relationships, fun times, good food… not everything was spectacular, but overall, really good. I moved out of my hometown, finally, and that’s been really great. I’m back now for the holidays, and it feels so small…</p>
<p>The only really bad thing was the death of my father. That still has interesting effects on my life, but mostly in non-obvious ways. That’s all I’ll say about that.</p>
</content:encoded></item><item><title>Most things I do are interconnected</title><link>https://steveklabnik.com/writing/most-things-i-do-are-interconnected/</link><guid isPermaLink="true">https://steveklabnik.com/writing/most-things-i-do-are-interconnected/</guid><pubDate>Fri, 28 Dec 2012 00:00:00 GMT</pubDate><content:encoded><p>I work on a <em>lot</em> of different things. Sometimes I get asked <a href="http://words.steveklabnik.com/how-do-you-find-the-time">how I find the time</a>, but there’s another aspect not related to time management: most of my projects build off of each other.</p>
<h2>Draper and RequestStore</h2>
<p><a href="https://github.com/drapergem/draper/issues/390">This issue</a> got filed in Draper. What Draper does isn’t important, but it did have this code in it:</p>
<pre><code>def self.current
Thread.current[:current_view_context] ||= build_view_context
end
def self.current=(context)
Thread.current[:current_view_context] = context
end
</code></pre>
<p>Basically, storing stuff in a <code>Thread</code> local variable. This is great, normally, except when you run one of those new hot servers like Thin or Puma. They have a different concurrency model, and so while you’d normally expect this <code>Thread</code> local to be <code>nil</code> on each request, it will persist between requests.</p>
<p>I <em>could</em> have just fixed this bug in Draper, but I figured that it might be useful to someone else, so I whipped up a little gem to do it: <a href="https://rubygems.org/gems/request_store">request_store</a>, and used the gem in Draper. I tweeted about it.</p>
<blockquote>
<p>@steveklabnik Holy shit. I love you.</p>
</blockquote>
<p>— Dominic Dagradi (@dddagradi) December 17, 2012</p>
<blockquote></blockquote>
<p>Turns out that one of my friends had <em>exactly</em> this problem, and wasn’t sure how to solve it. Now a bunch of people at Heroku are using my gem, and it’s helping them in their work.</p>
<p>I wouldn’t have bothered building it if it didn’t scratch my itch, but by making it easy for other people to use, I helped them scratch their itch too.</p>
<h2>Rust for Rubyists and Designing Hypermedia APIs</h2>
<p>I’ve been working on a new iteration of <a href="http://designinghypermediaapis.com/">Designing Hypermedia APIs</a> that is more linear and like a traditional book. The issue is that I’d built the entire platform for the project myself, and now spitting out ePUB and PDFs would be hard.</p>
<p>Enter <a href="http://www.rustforrubyists.com/">Rust for Rubyists</a>: I’d been learning Rust, and there are very few docs since it’s a new programming language. I had also seen Zed Shaw present at RuPy recently, and he showed off <a href="http://orkestrix.org/">Orkestrix</a>, which let him generate a site and a book for music stuff.</p>
<p>So I decided to write up what I’d learned as <a href="http://www.rustforrubyists.com/">a tutorial on programming in Rust</a>, and built it using the Orkestrix gear. I also tried out <a href="http://getdpd.com/">DPD</a> after seeing others use it. It’s only been up for two days, but I really like the combo, and today I pushed my first update out to the people who bought a copy already.</p>
<p>This has given me the confidence that the tooling process is good enough to use for DHAs, so I’ll be doing all of that soon.</p>
<h2>Rails and Resque</h2>
<p>I started working on Resque because we needed to have a few production implementations of the new ActiveQueue system going before Rails 4 was released. I was already working on Rails to help get 4 out the door, and Terence was obviously swamped by all the things he works on. So helping out with Resque would help out with Rails.</p>
<h2>bring_back_snowman and configuration</h2>
<p>When I don’t know how to do something, I often look at a gem that does what I’m interested in doing, and copy how it does it. I had never built a gem that allowed you to configure it using the <code>config.something</code> pattern before, so I peeked at a few gems to learn how.</p>
<p>I had also been thinking a lot about the Rails 4 release, and what was new since Rails 3. I was teaching a new batch of people Rails with <a href="http://jumpstartlab.com/">Jumpstart Lab</a>, my employer, and was explaining why we have <code>form_for</code> generate a ✔. This gave me an idea: What if you could configure what Rails used in this case?</p>
<p>So I built <a href="https://github.com/steveklabnik/bring_back_snowman">bring_back_snowman</a>. It’s not very <em>useful</em>, but it is cute and fun, and more importantly, <a href="https://github.com/steveklabnik/bring_back_snowman/blob/master/lib/bring_back_snowman.rb#L5-L13">it serves as a concise example of how to build a configurable option into your gems</a>. I can always look at that code if I forget how this works, or point others to it if they ask.</p>
<h2>Many, many more</h2>
<p>These are just the most recent examples I can think of. I’d really encourage you to try this sometime: if you want to learn something, make a new little project. Make it small. Make it focused. Make it help you out with something bigger.</p>
<p>Pretty soon, you’ll feel mega-productive too.</p>
</content:encoded></item><item><title>I invented hypermedia APIs by accident</title><link>https://steveklabnik.com/writing/i-invented-hypermedia-apis-by-accident/</link><guid isPermaLink="true">https://steveklabnik.com/writing/i-invented-hypermedia-apis-by-accident/</guid><pubDate>Fri, 21 Dec 2012 00:00:00 GMT</pubDate><content:encoded><p>Long long ago, I got an internship in college. Back then, I didn’t know anything about web development. My college professor said “GET and POST are the same thing, it’s just that GET requests show parameters in the URL bar.” But a friend of mine was working at this post-acquisition startup. They still operated as an autonomous unit within the parent company, so they were still pretty fun to work for. People would upload us MP3s and we’d give them a podcast. They’d get a blog, the necessary RSS, we’d help get them listed in iTunes, all that jazz.</p>
<p>Then the iPhone came out, and you could make apps for it.</p>
<p>Bizdev decided that we should sell apps to our clients, who could then sell them to their audience and help finance the show. We’d do all the dev work, we’d all make money, it’d be a good time. So they asked me to write an iPhone app generator. We picked a big feature set, and people could give us some images, pick their individual features, and set some other config options. We’d take that file, and compile in them as defaults. All seemed good.</p>
<p>Then the first app got rejected: it turns out that Apple has some restrictions on the bitrate of MP3s that you could download. I suspected that AT&amp;T wasn’t very happy with you using lots of data back then. Anyway, we had to get around this issue: we wanted to serve up quality audio. What to do?</p>
<p>I don’t remember who came up with it, but we decided that the first thing the app would do would be to fetch the config file from the server, to see if any of the defaults were changed. It was just XML, so the iPhone could parse it easily, and then change whatever was different. So we’d have something like this:</p>
<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;config&gt; &lt;link href="http://example.com/low.rss" rel="podcast_url" /&gt;&lt;/config&gt;
</code></pre>
<p>I don’t remember the exact schema, but you get the idea.</p>
<p>Anyway, so we’d link to the low quality feed while the app was in review, and then later, we’d change the response to this:</p>
<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;config&gt; &lt;link href="http://example.com/high.rss" rel="podcast_url" /&gt;&lt;/config&gt;
</code></pre>
<p>Now the client would fetch the high quality feed. No client code needed to change! We’d managed to sneak around a restriction in the review process.</p>
<p>But why stop here? Once we saw this flexibility, we started taking full advantage. People could set up a bunch of different configuration options, and that would change the UI based on their choices. So, for example, they could choose to be contacted by email, they could put in a website or two, and the page would change. Here’s a mockup of the web page:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24409213486218_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24409213486218_raw.png" /></p>
<p>This would cause our app to serve up the following XML:</p>
<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;config&gt; &lt;link href="http://example.com/high.rss" rel="podcast_url" /&gt; &lt;about&gt; &lt;links&gt; &lt;link href="http://example.com"&gt;Homepage&lt;/link&gt; &lt;link href="http://example.com/about"&gt;About Us&lt;/link&gt; &lt;/links&gt; &lt;phone&gt;5558675309&lt;/phone&gt; &lt;email&gt;[email protected]&lt;/email&gt; &lt;/about&gt;&lt;/config&gt;
</code></pre>
<p>And that would make this appear in the “about” section of the app:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24409223801964_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24409223801964_raw.png" /></p>
<p>Neat, eh? We’d turn the email into a <code>mailto:</code> link and the phone into a <code>tel:</code> one, as well.</p>
<p>Anyway, maybe later they’d go back and unset their phone number, and the about us link. So then we’d generate this XML:</p>
<pre><code>&lt;?xml version="1.0" encoding="UTF-8"?&gt;&lt;config&gt; &lt;link href="http://example.com/high.rss" rel="podcast_url" /&gt; &lt;about&gt; &lt;links&gt; &lt;link href="http://example.com"&gt;Homepage&lt;/link&gt; &lt;/links&gt; &lt;email&gt;[email protected]&lt;/email&gt; &lt;/about&gt;&lt;/config&gt;
</code></pre>
<p>But check out what the app would do:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24409247735934_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24409247735934_raw.png" /></p>
<p>Whoah! Isn’t that rad? The UI would change based on the config.</p>
<p>Here’s what’s even rad-er: because the app would read the config on each load, changes would get propagated very quickly. If you took your email away, everyone’s apps would no longer have it, as if by magic. Next time they loaded up the app, it just wouldn’t be there any more.</p>
<p>Here’s what’s <strong>even rad-er</strong>: You wouldn’t need to go through the App Store approval process to push this change out to the users. It Just Worked. If you’re an app developer, and you’ve been forced to sit through long waits to push out a release, you know how painful it can be. I don’t know how fast it is these days, but back then, it could take a month. With this approach, we could, say, remove a feature, and it’d be gone immediately. No oversight. If we added a new feature, older apps would still work, because they’d just ignore the new part of the config, and people who got the new version would get the newest features.</p>
<p>Years later, after doing tons of research on Hypermedia APIs, I realized that that thing I did long ago was a form of it, and it provided us with a pretty massive benefit. So of course I wasn’t the first one to come up with it; my point is that I implemented a system in line with hypermedia principles because it made sense to do so, even though I had no idea that Roy Fielding was a PhD candidate. And I certainly had no way of articulating it as a ‘style,’ it was just a neat hack.</p>
<hr />
<p>If you liked this story, you may care to learn about what else I’ve learned along the way with Hypermedia APIs. I wrote an e-‘book’ about it called <a href="http://designinghypermediaapis.com/">Designing Hypermedia APIs</a>. It’s only $20, and you get updates forever! I’m actively working on adding new content, and I’ll be doing a total redux in the new year.</p>
</content:encoded></item><item><title>An early Christmas present for you</title><link>https://steveklabnik.com/writing/an-early-christmas-present-for-you/</link><guid isPermaLink="true">https://steveklabnik.com/writing/an-early-christmas-present-for-you/</guid><pubDate>Wed, 19 Dec 2012 00:00:00 GMT</pubDate><content:encoded><p>Normally, I’m a bit of a grinch. I’ll admit it, I really hate Christmas. It reminds me of all of the worst parts of America: You know, how the baby Jesus commands us to <a href="http://www.ajc.com/news/news/crime-law/alleged-shoplifter-dies-after-being-subdued-by-wal/nTFPx/">literally shop until we kill each other</a>…</p>
<p>Anyway, today, I was teaching a bunch of brand new Rails devs, and I had to explain the mysterious hidden input that <code>form_for</code> generates:</p>
<pre><code>&lt;input name="utf8" type="hidden" value="✔"&gt;
</code></pre>
<p>That, combined with the impending release of Rails 4, made me think about the charming little Snowman that Rails originally had: ☃.</p>
<h2>So?</h2>
<p>And what happened, then? Well, in Whoville they say - that @steveklabnik’s small heart grew three sizes that day. So he decided to make you a present:</p>
<pre><code>gem install bring_back_snowman
</code></pre>
<p>Just do it, and your forms will submit <code>_snowman=☃</code> to ensure that your data is encoded in UTF-8.</p>
<p>It is configurable, if you prefer something else. Add this to your <code>application.rb</code>:</p>
<pre><code>config.snowman = {:emoji =&gt; "😢"}
</code></pre>
<p>You may also need to add the ‘magic comment’ at the top of the file:</p>
<pre><code># encoding: UTF-8
</code></pre>
<p>Please remember that if you don’t use something that’s UTF-8, this won’t actually work to fix the bug.</p>
<p>You can find the <a href="https://github.com/steveklabnik/bring_back_snowman">source of bring_back_snowman on GitHub</a>, of course.</p>
<p>Merry Christmas!</p>
</content:encoded></item><item><title>Announcing request_store</title><link>https://steveklabnik.com/writing/announcing-requeststore/</link><guid isPermaLink="true">https://steveklabnik.com/writing/announcing-requeststore/</guid><pubDate>Mon, 17 Dec 2012 00:00:00 GMT</pubDate><content:encoded><p>Last night I had some insomnia, so I wrote a gem.</p>
<p>Here it is: <a href="https://github.com/steveklabnik/request_store">https://github.com/steveklabnik/request_store</a>.</p>
<p>TL;DR:</p>
<p>If you’re using <code>Thread.current</code> in your Rails app to store global-ish data, don’t do it! If you use Thin or Puma or a threaded web server, it won’t get reset between requests, and you’ll end up with subtle bugs. So do this instead:</p>
<pre><code>gem "request_store"
</code></pre>
<p>and replace</p>
<pre><code>Thread.current[:foo] = 1
</code></pre>
<p>with</p>
<pre><code>RequestStore.store[:foo] = 1
</code></pre>
<p>And things will be peachy keen.</p>
<h3>No Rails?</h3>
<p>If you’re not using Rails, <code>use RequestStore::Middleware</code> to make it work. If you are using Rails, a Railtie takes care of it for you.</p>
</content:encoded></item><item><title>Getting started with chruby</title><link>https://steveklabnik.com/writing/getting-started-with-chruby/</link><guid isPermaLink="true">https://steveklabnik.com/writing/getting-started-with-chruby/</guid><pubDate>Thu, 13 Dec 2012 00:00:00 GMT</pubDate><content:encoded><p>If you’re looking for crazy simplicity in your ‘switch between multiple Rubies’ life, you may want to check out <a href="https://github.com/postmodern/chruby">chruby</a>. Written by Postmodern, it’s basically the simplest possible thing that can work. As in, <a href="https://github.com/postmodern/chruby/blob/master/share/chruby/chruby.sh">76 lines of shell script</a>.</p>
<p>For that, you get:</p>
<h3>Features</h3>
<ul>
<li>Updates <code>$PATH</code>.
<ul>
<li>Also adds RubyGems <code>bin/</code> directories to <code>$PATH</code>.</li>
</ul>
</li>
<li>Correctly sets <code>$GEM_HOME</code> and <code>$GEM_PATH</code>.
<ul>
<li>Users: gems are installed into <code>~/.gem/$ruby/$version</code>.</li>
<li>Root: gems are installed directly into <code>/path/to/$ruby/$gemdir</code>.</li>
</ul>
</li>
<li>Additionally sets <code>$RUBY</code>, <code>$RUBY_ENGINE</code>, <code>$RUBY_VERSION</code> and <code>$GEM_ROOT</code>.</li>
<li>Optionally sets <code>$RUBYOPT</code> if second argument is given.</li>
<li>Calls <code>hash -r</code> to clear the command-lookup hash-table.</li>
<li>Fuzzy matching of Rubies by name.</li>
<li>Defaults to the system Ruby.</li>
<li>Supports [bash] and [zsh].</li>
<li>Small (~80 LOC).</li>
<li>Has tests.</li>
</ul>
<h3>Anti-Features</h3>
<ul>
<li>Does not hook <code>cd</code>.</li>
<li>Does not install executable shims.</li>
<li>Does not require Rubies be installed into your home directory.</li>
<li>Does not automatically switch Rubies upon login or when changing directories.</li>
<li>Does not require write-access to the Ruby directory in order to install gems.</li>
</ul>
<p>Kinda crazy, eh?</p>
<h2>Installing</h2>
<p>Most of the time, I install things from <code>homebrew</code>, but I actually prefered to run the <code>setup.sh</code> script:</p>
<pre><code>wget https://github.com/downloads/postmodern/chruby/chruby-0.2.3.tar.gz
tar -xzvf chruby-0.2.3.tar.gz
cd chruby-0.2.3/
./scripts/setup.sh
</code></pre>
<p>You can see the source for that script <a href="https://github.com/postmodern/chruby/blob/master/scripts/setup.sh">here</a>. You’ll end up with an MRI, JRuby, and Rubinius all installed in <code>/opt/rubies</code>, and after restarting your shell, you should be good to go!</p>
<h2>Usage</h2>
<p>To see what Rubies you’ve got, just <code>chruby</code>:</p>
<pre><code>$ chruby
ruby-1.9.3-p327
jruby-1.7.0
rubinius-2.0.0-rc1
</code></pre>
<p>to pick one:</p>
<pre><code>$ chruby 1.9.3
</code></pre>
<p>Pretty damn simple.</p>
<h2>Getting more Rubies?</h2>
<p>The setup script will install <a href="https://github.com/sstephenson/ruby-build#readme">ruby-build</a>, so just use that to get more rubies:</p>
<pre><code>ruby-build 1.9.3-p327 /opt/rubies/ruby-1.9.3-p327
</code></pre>
<p>You’ll then need to update your list of Rubies in your <code>.bashrc</code> or <code>.zshrc</code>:</p>
<pre><code>RUBIES=(
# other rubies here
/opt/rubies/ruby-1.9.3-p327
)
</code></pre>
<p>If you find this tedius, you can just glob them:</p>
<pre><code>RUBIES=(/opt/rubies/*)
</code></pre>
<h2>Automatic switching?</h2>
<p><a href="https://github.com/postmodern/chruby/issues/40">Not yet</a>. This is pretty much the only thing keeping me from saying ‘zomg use this for sure now.’</p>
<h2>How good is it?</h2>
<p>I don’t know, I’ve been using it for less than 24 hours. Seems good.</p>
</content:encoded></item><item><title>Deleuze for developers: deterritorialization</title><link>https://steveklabnik.com/writing/deleuze-for-developers-deterritorialization/</link><guid isPermaLink="true">https://steveklabnik.com/writing/deleuze-for-developers-deterritorialization/</guid><pubDate>Fri, 07 Dec 2012 00:00:00 GMT</pubDate><content:encoded><p>If you truly want to understand technology today, then you should at least be familiar with the philosophy of Gilles Deleuze. Unfortunately for technologists, Deleuze is rooted firmly in a philosophical tradition and a writing style that they probably find opaque. In this blog series, I plan on explaining Deleuze’s philosophy in terms that programmers can understand. This is the second in the series. You can find the first <a href="/deleuze-for-developers-assemblages">here</a>, and the next one <a href="http://words.steveklabnik.com/deleuze-for-developers-will-smooth-spaceopen-source-suffice-to-save-us">here</a>. Enjoy.</p>
<hr />
<p>Let’s re-examine this diagram of the assemblage:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388447125816_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388447125816_raw.png" /></p>
<p>What’s up with this line?</p>
<p>Well, just because assemblages are amorphous doesn’t mean that there’s no way to demarcate what’s in the assemblage and what is not. This particular assemblage has a ‘territory,’ of sorts: everything that’s inside the line is part of the territory, and everything that’s not is outside. Think of a fort, or a castle:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388459420680_raw.jpg" alt="https://svbtleusercontent.com/inline_steveklabnik_24388459420680_raw.jpg" /></p>
<p>Or, since we’re not just talking about physical spaces, a social circle or group:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388506430020_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388506430020_raw.png" /></p>
<p>Of course, this would extend around all the members of Rails Core, hence it being cut off at the bottom. But the pink line would designate the boundaries of the assemblage that makes up Rails Core.</p>
<h2>Deterritorialization</h2>
<p>So what happens when these boundaries get crossed? Well, your castle gets invaded. A new member joins the team. New servers are brought online. This process is called ‘deterritorialization.’ Healing the lines, repairing them, and re-containing the boundary is called ‘reterritorialization.’ I recently came across a really interesting symbol of deterritorialization: the Open Source logo:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388521022692_raw.gif" alt="https://svbtleusercontent.com/inline_steveklabnik_24388521022692_raw.gif" /></p>
<p>Check it out: visually, this logo communicates the deterritorializing effect of opening up your source code: the private internals have now been exposed internally. The walls have been breeched!</p>
<p>Another example, from building a web service: we start off with just our service:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388526532744_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388526532744_raw.png" /></p>
<p>We have here an application server, a database, and our user’s identity. They form the assemblage of our system. Remember, abstract concepts are objects just as code and servers are! Our identity notion is stored within the database, therefore, they’re connected.</p>
<p>Next, we decide to implement a ‘log in with Twitter’ feature.</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388533077814_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388533077814_raw.png" /></p>
<p>It’s the primary way that our users sign up and use our site. Now, Twitter’s assemblage has deterritorialized ours:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388535026314_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388535026314_raw.png" /></p>
<p>We can minimize the effects (or reterritorialize our service) by making sure to have our own concept of identity within the system, and making sure that we have our own notion of identity within the system, and just connecting Twitter’s notion of identity to our own:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388627455810_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388627455810_raw.png" /></p>
<p>Now, by containing the Twitter-assemblage entirely within our service, I don’t mean that it actually is. Obviously, the Twitter-assemblage is interconnected with a ton of other assemblages that represent other services. But from our perspective, they are now a part of our assemblage. The decisions they make affect us. While our code is separated, we’re not totally separate anymore: updates and policies of Twitter have a direct effect on us.</p>
<p>There’s also a more sublte, secondary re/de-territorialization going on here: our code from our service. These used to be isomorphic, but now our code has claimed a territory of its own, and is now just one assemblage within our system-assemblage, instead of <em>being</em> our system-assemblage.</p>
<h2>A git example</h2>
<p>The previous notion of deterritorialization largely relied on the notion of dependencies as the mechanism which we drew our diagrams. But it doesn’t have to be that way. Let’s take another example: git.</p>
<p>Every GitHub pull request is an act of deterritorialization, and every merge is one of re-territorialization. Consider this small repository, with three commits:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388550633502_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388550633502_raw.png" /></p>
<p>You do a git clone:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388551164862_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388551164862_raw.png" /></p>
<p>Make a new commit, adding a new object into your repo-assemblage:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388553101194_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388553101194_raw.png" /></p>
<p>Then you send me an email, asking me to pull from your repository:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388554578454_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388554578454_raw.png" /></p>
<p>You like the change, so you do a <code>git fetch</code>:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388557943410_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388557943410_raw.png" /></p>
<p>de-territorialized!</p>
<p>Now it makes the local copy:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388560627354_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388560627354_raw.png" /></p>
<p>and your repository has been re-territorialized. These steps happen so quickly that you probably don’t even think about them, but conceptually, this is what’s happening.</p>
<h2>A social example</h2>
<p>One last example that’s even less related to code: entering a new social group. You’re at a conference, and there are four people standing around talking:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388564936014_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388564936014_raw.png" /></p>
<p>They form an assemblage we call a ‘conversation.’ It’s in relation with an object known as “refinements” from Ruby 2.0, which means this assemblage is likely to be particularly vibrant. Heh. Anyway, you decide that your opinion matters, so you de-territorialize the assemblage, and (assuming they don’t ignore you) it re-territorializes around you:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24388566884064_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24388566884064_raw.png" /></p>
<p>Even the language we use here implies something similar: you ‘enter the conversation.’ Like entering a door, into a space you previously were not.</p>
<h2>Conclusion: Diagrams</h2>
<p>We can call these drawings ‘diagrams’ or ‘abstract machines.’ They can represent these kinds of conceptual relationships in a visual way, which I find really useful for understanding. Programmers call abstract machines ‘design patterns.’</p>
<p>Now that you’ve seen this process that assemblages use to relate to one another, I hope that you find this particular diagram useful. Because of its level of abstraction, it’s applicable in a wide variety of situations.</p>
</content:encoded></item><item><title>Deleuze for developers: assemblages</title><link>https://steveklabnik.com/writing/deleuze-for-developers-assemblages/</link><guid isPermaLink="true">https://steveklabnik.com/writing/deleuze-for-developers-assemblages/</guid><pubDate>Mon, 03 Dec 2012 00:00:00 GMT</pubDate><content:encoded><p>If you truly want to understand technology today, then you should at least be familiar with the philosophy of Gilles Deleuze. Unfortunately for technologists, Deleuze is rooted firmly in a philosophical tradition and a writing style that they probably find opaque. In this blog series, I plan on explaining Deleuze’s philosophy in terms that programmers can understand. This is the first in the series.</p>
<p>Enjoy.</p>
<hr />
<p>The ancient Greeks thought that atoms were it. ἄτομος, Wikipedia will tell you, comes from ‘ἀ’- meaning “not” and ‘τέμνω’, meaning ‘I cut.’ Cells are the smallest thing that there is. You can’t cut them in half. Everything is composed of a bunch of atoms.</p>
<p>Well, guess what? Modern science has found things smaller than atoms. Dammit! What we <em>do</em> know for sure, though, is that some things are made up of composites of other things. That’s for sure. We’re still not clear on what that fundamental stuff is, but composites are a pretty safe bet. Even if atoms are made up of other things, we know that other things are made up of a bunch of atoms.</p>
<p>So how can we talk about composites? Well, a programmer’s fundamental tool is abstraction. So we can black-box any particular composite and treat it as a whole entity, without worrying about what we’ve encapsulated inside. That’s good, and that’s useful, but there’s a problem: when you start thinking of a collection of things as just one thing, you miss out on a lot of important details. As we know, all abstractions are leaky, so how does a black box leak?</p>
<p>Let’s think about your body: it’s a composite, you’re made up of (among other things) a whole pile of organs, and one (skin) wrapping the rest of them up in a nice black box. If we had lived a few hundred years ago, thinking of the body as a black box would be a pretty fair metaphor: after all, if you take out someone’s heart, they die. The box needs all of its parts inside. Today, though, that’s not true: we can do heart transplants. Further, we have artificial hearts. A person with an artificial heart is not the exact same as a person without one, yet they are both still people. Thinking of a body as a totality leaks as an abstraction in modern times.</p>
<p>Let’s move away from biology for a bit and into web applications instead. Here’s an architecture diagram of a typical web service:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24382217466576_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24382217466576_raw.png" /></p>
<p>Looks pretty normal, right? Let’s modify it a bit:</p>
<p><img src="https://svbtleusercontent.com/inline_steveklabnik_24382219810338_raw.png" alt="https://svbtleusercontent.com/inline_steveklabnik_24382219810338_raw.png" /></p>
<p>This is the same web service, right? We’ve introduced a few database slaves and a reverse proxy, which allowed us to knock out one of our application servers. In a certain sense, this service is equivalent; in another, it’s different: It’s got another extra component, and one component has developed more complexity. We might need to hire a DBA and someone who’s good at writing Varnish configs. Different skills are needed.</p>
<p>So what do we call this thing? An ‘assemblage.’ It’s made up of smaller assemblages as well; after all, our application servers may use the Reactor pattern, the worker model of Unicorn, or the threaded model of the JVM. To quote Levi Bryant, “Assemblages are composed of heterogeneous elements or objects that enter into relations with one another.” In the body metaphor, these are organs, and in the systems diagram, they’re the application servers, databases, and job queues. Assemblages are also ‘objects’ in this context, so you can think of any of those objects as being made up of other ones. But to think of the formations of objects as a total whole or black box is incorrect, or at best, a leaky abstraction.</p>
<p>Let’s get at one other excellent example of an assemblage: the internet. Computers connect, disconnect, reconnect. They may be here one moment, gone the next, or stay for an extended period of time. The internet isn’t any less an internet when you close your laptop and remove it, nor when you add a new server onto your rack. Yet we still talk about it as though it were a whole. It’s not, exactly: it’s an assemblage.</p>
<p>Furthermore, there’s a reason that Bryant uses the generic ‘objects’ or ‘elements’ when talking about assemblages: objects can be ‘people’ or ‘ideas’ as well as physical parts. This is where the ‘systems’ idea of computer science diverges from ‘assemblage’: depending on how you want to analyze things, your users may be part of an assemblage diagram as well.</p>
<p>So what’s the point? Well, this is a basic term that can be used to think about problems. A design pattern of thought. A diagram. In future posts, I’ll actually use this pattern to address a particular problem, but you need to understand the pattern before you can understand how it’s used.</p>
<p>(and to give you a comparison of complexity of explanation, here’s the blog post by Levi Bryant on assemblages where I got that quote from. He quotes Deleuze directly and extensively: http://larvalsubjects.wordpress.com/2009/10/08/deleuze-on-assemblages/ )</p>
<hr />
<p>The next article in the series is on <a href="/deleuze-for-developers-deterritorialization">‘deterritorialization’</a>.</p>
</content:encoded></item><item><title>Draper 1.0.0.beta2 release</title><link>https://steveklabnik.com/writing/draper-1-0-0-beta2-release/</link><guid isPermaLink="true">https://steveklabnik.com/writing/draper-1-0-0-beta2-release/</guid><pubDate>Mon, 03 Dec 2012 00:00:00 GMT</pubDate><content:encoded><p>I’ve relased the second beta for Draper 1.0.0! Many thanks to all of you who gave the first beta a shot. We’ve fixed some issues, and I’d apprecaite you giving this one a try. This solves all known issues with Draper, including every one we’ve found in 1.0.0.beta1.</p>
<p>You can get it by installing it from Rubygems:</p>
<pre><code>$ gem install draper --pre
</code></pre>
<p>or by putting it in your Gemfile:</p>
<pre><code>gem "draper", "1.0.0.beta2"
</code></pre>
<h2>CHANGELOG</h2>
<p>These are the changes since beta1. To find out what’s new in beta1, please <a href="/posts/2012-11-30-draper-1-0-0-beta1-release">see my previous post</a>.</p>
<ul>
<li><code>has_finders</code> is now <code>decorates_finders</code>. <a href="https://github.com/haines/draper/commit/33f18aa062e0d3848443dbd81047f20d5665579f">https://github.com/haines/draper/commit/33f18aa062e0d3848443dbd81047f20d5665579f</a></li>
<li>If a finder method is used, and the source class is not set and cannot be inferred, an <code>UninferrableSourceError</code> is raised. <a href="https://github.com/haines/draper/commit/8ef5bf2f02f7033e3cd4f1f5de7397b02c984fe3">https://github.com/haines/draper/commit/8ef5bf2f02f7033e3cd4f1f5de7397b02c984fe3</a></li>
<li>Class methods are now properly delegated again. <a href="https://github.com/haines/draper/commit/731995a5feac4cd06cf9328d2892c0eca9992db6">https://github.com/haines/draper/commit/731995a5feac4cd06cf9328d2892c0eca9992db6</a></li>
<li>We no longer <code>respond_to?</code> private methods on the source. <a href="https://github.com/haines/draper/commit/18ebac81533a6413aa20a3c26f23e91d0b12b031">https://github.com/haines/draper/commit/18ebac81533a6413aa20a3c26f23e91d0b12b031</a></li>
<li>Rails versioning relaxed to support Rails 4 <a href="https://github.com/drapergem/draper/commit/8bfd393b5baa7aa1488076a5e2cb88648efaa815">https://github.com/drapergem/draper/commit/8bfd393b5baa7aa1488076a5e2cb88648efaa815</a></li>
</ul>
</content:encoded></item><item><title>Draper 1.0.0.beta1 release</title><link>https://steveklabnik.com/writing/draper-1-0-0-beta1-release/</link><guid isPermaLink="true">https://steveklabnik.com/writing/draper-1-0-0-beta1-release/</guid><pubDate>Fri, 30 Nov 2012 00:00:00 GMT</pubDate><content:encoded><p>I’m happy to announce the release of Draper 1.0.0.beta1 today. If you use Draper, I’d appreciate you checking out the beta release in your app, kicking the tires, and letting me know what’s up.</p>
<p>You can get it by installing it from Rubygems:</p>
<pre><code>$ gem install draper --pre
</code></pre>
<p>or by putting it in your Gemfile:</p>
<pre><code>gem "draper", "1.0.0.beta1"
</code></pre>
<p>Notable changes include:</p>
<ul>
<li>Renaming <code>Draper::Base</code> to <code>Draper::Decorator</code>. This is the most significant change you’ll need to upgrade your application. <a href="https://github.com/drapergem/draper/commit/025742cb3b295d259cf0ecf3669c24817d6f2df1">https://github.com/drapergem/draper/commit/025742cb3b295d259cf0ecf3669c24817d6f2df1</a></li>
<li>Added an internal Rails application for integration tests. This won’t affect your application, but we’re now running a set of Cucumber tests inside of a Rails app in both development and production mode to help ensure that we don’t make changes that break Draper. <a href="https://github.com/drapergem/draper/commit/90a4859085cab158658d23d77cd3108b6037e36f">https://github.com/drapergem/draper/commit/90a4859085cab158658d23d77cd3108b6037e36f</a></li>
<li>Add <code>#decorated?</code> method. This gives us a free RSpec matcher, <code>is_decorated?</code>. <a href="https://github.com/drapergem/draper/commit/834a6fd1f24b5646c333a04a99fe9846a58965d6">https://github.com/drapergem/draper/commit/834a6fd1f24b5646c333a04a99fe9846a58965d6</a></li>
<li><code>#decorates</code> is no longer needed inside your models, and should be removed. Decorators automatically infer the class they decorate. <a href="https://github.com/drapergem/draper/commit/e1214d97b62f2cab45227cc650029734160dcdfe">https://github.com/drapergem/draper/commit/e1214d97b62f2cab45227cc650029734160dcdfe</a></li>
<li>Decorators do not automatically come with ‘finders’ by default. If you’d like to use <code>SomeDecorator.find(1)</code>, for example, simply add <code>#has_finders</code> to the decorator to include them. <a href="https://github.com/drapergem/draper/commit/42b6f78fda4f51845dab4d35da68880f1989d178">https://github.com/drapergem/draper/commit/42b6f78fda4f51845dab4d35da68880f1989d178</a></li>
<li>To refer to the object being decorated, <code>#source</code> is now the preferred method. <a href="https://github.com/drapergem/draper/commit/1e84fcb4a0eab0d12f5feda6886ce1caa239cb16">https://github.com/drapergem/draper/commit/1e84fcb4a0eab0d12f5feda6886ce1caa239cb16</a></li>
<li><code>ActiveModel::Serialization</code> is included in Decorators if you’ve requred <code>ActiveModel::Serializers</code>, so that decorators can be serialized. <a href="https://github.com/drapergem/draper/commit/c4b352799067506849abcbf14963ea36abda301c">https://github.com/drapergem/draper/commit/c4b352799067506849abcbf14963ea36abda301c</a></li>
<li>Properly support Test::Unit <a href="https://github.com/drapergem/draper/commit/087e134ed0885ec11325ffabe8ab2bebef77a33a">https://github.com/drapergem/draper/commit/087e134ed0885ec11325ffabe8ab2bebef77a33a</a></li>
</ul>
<p>And many small bug fixes and refactorings.</p>
<p>Before the actual release of 1.0.0, I want to improve documentation and handle a few other things, but we currently have no confirmed and one possible bug in Draper as it stands, so your feedback as I clean up these last few things would be excellent. Please file issues on the tracker if you find anything.</p>
<p>Thank you! &lt;3 &lt;3 &lt;3</p>
</content:encoded></item><item><title>Introducing the rails-api project</title><link>https://steveklabnik.com/writing/introducing-the-rails-api-project/</link><guid isPermaLink="true">https://steveklabnik.com/writing/introducing-the-rails-api-project/</guid><pubDate>Thu, 22 Nov 2012 00:00:00 GMT</pubDate><content:encoded><p>Ruby on Rails is a great tool to build websites incredibly quickly and easily. But what about applications that aren’t websites? Rails is still a first-class choice for this use-case, but why settle for good when you could be the best?</p>
<p>That’s why I’m happy to introduce Rails API: a set of tools to build excellent APIs for both heavy Javascript applications as well as non-Web API clients.</p>
<h2>What</h2>
<p>Rails’ greatest feature is making trivial choices for you. By following conventions, you’re able to avoid bikeshedding and only focus on the actual decisions that matter for your application, while automatically building in best-practices. Rails API will do this for the API use-case.</p>
<p>Here are some initial thoughts on what Rails API will do for you:</p>
<h3>A simpler stack</h3>
<p>First, we can remove many parts of Rails that aren’t important in an API context: many middleware don’t make sense, and all of ActionView can go away.</p>
<p>To check out exactly what we mean, look at <a href="https://github.com/rails-api/rails-api#choosing-middlewares">this part</a> of the README, which explains which middlewares we include. The default stack has 22, we have 14.</p>
<p>Similarly, the structure of ActionController::Base is interesting: it includes a ton of modules that implement various features. This means that we can build an alternate ‘controller stack’ that doesn’t include all of the ones that are in Base. Check out <a href="https://github.com/rails-api/rails-api#choosing-controller-modules">this portion</a> of the README for a demonstration of the modules we do include. Base includes 29 modules, we include 11.</p>
<h3>Consistent Javascript output</h3>
<p>One of the biggest issues when building a distributed system is messages that are sent between components. In most Rails-based APIs, this means the JSON that your server side emits, and the JSON that your client consumes. By forming best practices around what JSON is sent around, we can simplify things for server-side and client-side developers.</p>
<p>The generation of this JSON should be as transparent and simple as possible on both sides. <a href="https://github.com/rails-api/active_model_serializers">ActiveModel::Serializers</a> is key here. Many new APIs are settling on HAL as a JSON variant that is important to them, and so we’ll be considering it heavily.</p>
<h3>Authentication</h3>
<p>APIs don’t need to handle cookie-based auth, and so a lot of that infrastructure isn’t important. Handling other kinds of auth in a simple way is incredibly important.</p>
<h3>JavaScript as a first-class citizen</h3>
<p>JavaScript is just as important as Ruby for these kinds of applications, and should be equally as important with respect to tooling, directory layout, and documentation.</p>
<p>This also means that it should be easy to use the various Javascript frameworks and testing tools. We have representatives from the Ember team on-board, but using other frameworks should be equally fantastic.</p>
<h3>Hypermedia</h3>
<p>Many people are interested in building Hypermedia APIs on top of Rails, and so building best-practices around this style should be taken care of, and building them should be easy.</p>
<h2>Where</h2>
<p>You can find the organization on GitHub at <a href="https://github.com/rails-api">https://github.com/rails-api</a></p>
<p>Originally, these two repositories were located at <a href="https://github.com/spastorino/rails-api">https://github.com/spastorino/rails-api</a> and <a href="https://github.com/josevalim/active_model_serializers">https://github.com/josevalim/active_model_serializers</a>.</p>
<p>We have a Google Group for discussion here: <a href="https://groups.google.com/forum/?fromgroups#!forum/rails-api-core">https://groups.google.com/forum/?fromgroups#!forum/rails-api-core</a></p>
<h2>Getting Started</h2>
<p>To get started, check out the README here: <a href="https://github.com/rails-api/rails-api#railsapi">https://github.com/rails-api/rails-api#railsapi</a>.</p>
<h2>Who</h2>
<p>Currently,</p>
<ul>
<li>Carlos Antonio</li>
<li>Rafael França</li>
<li>Santiago Pastorino</li>
<li>Steve Klabnik</li>
<li>Yehuda Katz</li>
<li>Tom Dale</li>
<li>José Valim</li>
</ul>
</content:encoded></item><item><title>Narratives</title><link>https://steveklabnik.com/writing/narratives/</link><guid isPermaLink="true">https://steveklabnik.com/writing/narratives/</guid><pubDate>Sun, 18 Nov 2012 00:00:00 GMT</pubDate><content:encoded><p>Occasionally in this big wide world, things happen. When things happen, we like to tell each other about it. When you take a bunch of connected events and tell someone about it, you’ve formed a <em>narrative</em>. Narratives are important, because they tell the story! The details of the account that one presents can really shape how others view those events later.</p>
<p>Due to our lived experiences, we often have our own ‘internal narratives.’ You can think of them as pre-packaged ways that we tend to think about a given series of events. Human brains are wonderful pattern-matching machines, and so we see similarities in situations, and then apply the same narrative.</p>
<p>Recently, I was playing around with Unicode and Ruby, and was trying to make some funny class names. I ended up figuring some stuff out, and tweeted this, which at the time of this writing, it has 157 retweets and 48 favorites:</p>
<blockquote>
<p>This is valid Ruby:class ಠ_ಠendಠ_ಠ.new</p>
</blockquote>
<p>— Steve Klabnik (@steveklabnik) November 17, 2012</p>
<blockquote></blockquote>
<p>This tweet is factually incorrect. Yet, only two people pointed this out to me. Why?</p>
<p>Ruby is kind of a crazy language. That’s what I love about it; it’s got lots of wacky little corners, nooks, and crannies. I’ve also been doing Ruby for a long time, and tweet about it often. So here’s the internal narrative when you see this tweet:</p>
<ol>
<li>Steve knows a lot about Ruby.</li>
<li>Ruby is kinda weird.</li>
<li>That’s really funny.</li>
</ol>
<p>This particular situation fits perfectly into this internal narrative. Hence, a retweet. No checking. Just tweeting.</p>
<p>These narratives can play off of each other really easily. Now that we’ve seen one more way in which Ruby is kinda crazy, we assume that it’s even more crazy in the future. This tweet has reinforced our internal narrative about Ruby’s wackiness.</p>
<hr />
<p>So #britruby.</p>
<p>I’m sure you’re expecting me to say something specific about this situation, but I’m not going to give you what you want. What I <em>do</em> want to do is analyze the narrative of this situation, and see how it compares to actual events.</p>
<p>Josh Susser tweets this:</p>
<blockquote>
<p>Nice speaker lineup for @britruby. Except for the 100% white guys part.</p>
</blockquote>
<p>— Josh Susser (@joshsusser) November 17, 2012</p>
<blockquote></blockquote>
<p>and James Rosen tweets this:</p>
<blockquote>
<p>By what metric is @britruby “one of Europe’s… most diverse Ruby conferences”? Every speaker is a white man.</p>
</blockquote>
<p>— James A Rosen (@jamesarosen) November 17, 2012</p>
<blockquote></blockquote>
<p>That’s it. If you examine those Twitter pages, you can see some conversation; both were conducted pretty cordially. Then, BritRuby decides to cancel their conference.</p>
<p>Okay. That’s the situation. Now let’s look at two versions of the narrative. First up, <a href="http://2013.britruby.com/">the cancellation announcement</a>. I’m going to excerpt the most important parts, with my commentary afterwards.</p>
<blockquote>
<p>We started this conference to build a community within the United Kingdom. Our mission statement was to encourage Ruby developers to unite and create a community, which would allow such to network, exchange ideas and provoke innovation for the future of Ruby. We wanted to encourage jobs and allow companies the opportunity to meet the community and primarily boost the UK developer industry.</p>
</blockquote>
<p>We wanted to build a great conference that would help UK Rubyists tremendously.</p>
<blockquote>
<p>Our selection process was the content and nothing more. Not the individuals gender, race, age or nationality.</p>
</blockquote>
<p>This is trivially disproven, as everyone has biases. But the message is “We only accepted the best speakers, regardless of who they are.”</p>
<blockquote>
<p>It’s about community. It’s about helping one another to strive for success and drive budding developers to do the same. We contacted universities in a hope that we would be able to empower young minds and show them what a great language Ruby really is. They are the future.</p>
</blockquote>
<p>We are team players.</p>
<blockquote>
<p>The Ruby community has been battling with issues of race and gender equality. We at Brit Ruby were well aware of this fundamental and important issue. This was one of the reasons why we encouraged everyone to submit a speaker proposal. Sadly, BritRuby was used as the arena to air these issues on Twitter and this has fundamentally destroyed any chance we had of addressing these issues.</p>
</blockquote>
<p>BritRuby was ‘used to air these issues,’ meaning, ‘others had an agenda and took advantage of us to promote it.’</p>
<blockquote>
<p>Instead the community should have worked together and allowed us to bring these issues to light at the conference.</p>
</blockquote>
<p>These individuals are divisive, and we are inclusive.</p>
<blockquote>
<p>How can the community address these issues if every time someone tries they are shot down and accused of such awful things?</p>
</blockquote>
<blockquote>
<p>The British Ruby team although very small consisted of a diverse mix of nationality and gender.</p>
</blockquote>
<p>We cannot be tainted by accusations of monoculture, as we are diverse ourselves.</p>
<blockquote>
<p>This was a non-profit conference being run by simple developers.</p>
</blockquote>
<p>We were not even trying to make money.</p>
<blockquote>
<p>The team has been working so hard in their own time to bring a unique conference to you all. Thank you to those of you that dedicated your time, skills and to their families for putting up with the long hours.</p>
</blockquote>
<p>We, nay, you and us together all worked really hard, so you should cut us some slack.</p>
<p>Anyway, so there you go. Same events, different story: We are a collective that was trying to do our best. Outsiders took advantage of us to promote their agenda and divide all of us. Therefore, we’re taking our ball and going home.</p>
<p>Here’s the funniest thing about the cancellation: They don’t give an actual reason. The closest they get is ‘was used as an arena to air these issues’ but they don’t explain why that made them cancel their conference. This narrative attempts to shift the blame from the organizers to the critics. “We wanted to give you this beautiful thing, and we worked hard, but these other people ruined it.”</p>
<p>This particular narrative plays off of a well-known trope when discussing diversity issues: a persecution complex by the majority. Majorities fear losing their power, and often will develop internal narratives that <em>they’re</em> the ones being persecuted. Some good examples of this are the Mens Rights movement and the White Power movement. This is a milder form of that same privilege: the power of a single tweet by the ‘politically correct’ is enough to bring the most powerful of plans grinding to a halt.</p>
<hr />
<p>Let’s examine a few narratives involved in <a href="https://gist.github.com/4106776">this gist</a>:</p>
<blockquote>
<p>And here it is, brought down by careless words.</p>
</blockquote>
<p>“These people don’t actually care about us or our conference.”</p>
<blockquote>
<p>Yes, gender equality and racial equality are important. But the team’s motives were to get the best speakers who were able to make it to Manchester.</p>
</blockquote>
<p>First of all, “but” is an amazing word. You can’t even imagine the things I’ve heard said after “I’m not racist, but.” This sentence implies two things: an absolution from blame on the part of the conference because hey, they care about equality. Their second statement implies that the best speakers all happened to be white men. The converse of this is that non-whites and/or women are not the best speakers, and the overall message is that including them would degrade the quality of the conference.</p>
<blockquote>
<p>Adding a token minority speaker is offensive to that speaker, it says “You’re here because you tick a box - not because you’re skilled.”</p>
</blockquote>
<p>Anyone who seeks out women or minorities is only doing it to ‘tick boxes,’ because, as we’ve already established, they clearly are not the best speakers.</p>
<p>Oh, and you’re actually the sexist/racist one if you try to encourage them, because it’s patronizing.</p>
<blockquote>
<p>It doesn’t matter who speaks at a conference, as long as they’re capable, interesting and relevant. That’s what matters: content, not style.</p>
</blockquote>
<p>We cannot be at fault because we don’t see race or gender. Again, basic psychology trivially disproves this statement.</p>
<hr />
<p>There are a ton of other micro-narratives that occur in all of these discussions as well. Most of these are a derailing tactic, and straight-up distract from the issue. “Egalitarianism is anti-free speech,” “Feminists and their allies are irrational and hysterical,” “I don’t see race or gender,” “If they were calmer about this, it would have worked out,” “This would be better handled in private,” and many, many more. I’ve seen so many. It’s exhausting. These internal narratives get reinforced by a particular interpretation of a given situation, without examining what actually happened.</p>
<p>Yet we still have conferences that are mostly full of white dudes.</p>
<p>I lost a lot of respect for a lot of people today.</p>
</content:encoded></item><item><title>How to squash commits in a GitHub pull request</title><link>https://steveklabnik.com/writing/how-to-squash-commits-in-a-github-pull-request/</link><guid isPermaLink="true">https://steveklabnik.com/writing/how-to-squash-commits-in-a-github-pull-request/</guid><pubDate>Thu, 08 Nov 2012 00:00:00 GMT</pubDate><content:encoded><p>So you’ve contributed some code to an open source project, say, Rails. And they’d like you to squash all of the commits in your pull request. But you’re not a git wizard; how do you make this happen?</p>
<p>Normally, you’d do something like this. I’m assuming <code>upstream</code> is a git remote that is pointing at the official project repository, and that your changes are in your ‘omgpull’ branch:</p>
<pre><code>$ git fetch upstream
$ git checkout omgpull
$ git rebase -i upstream/master
&lt; choose squash for all of your commits, except the first one &gt;
&lt; Edit the commit message to make sense, and describe all your changes &gt;
$ git push origin omgpull -f
</code></pre>
<p>GitHub will then automatically update your pull request with the new commits. Super easy!</p>
</content:encoded></item><item><title>Theory and practice</title><link>https://steveklabnik.com/writing/theory-and-practice/</link><guid isPermaLink="true">https://steveklabnik.com/writing/theory-and-practice/</guid><pubDate>Tue, 06 Nov 2012 00:00:00 GMT</pubDate><content:encoded><p><img src="http://imgs.xkcd.com/comics/purity.png" alt="http://imgs.xkcd.com/comics/purity.png" /></p>
<p>purity</p>
<hr />
<p>There is always a tension between Theory and Practice. These two separate realms are connected through a process of abstraction and application. To explain this relationship by way of theory, Theory deterritorializes Practice, and Practice reterritorializes Theory: a Theory which is a becoming-Practice and a Practice which is a becoming-Theory. To explain this relationship by way of practice, Theory is abstracted Practice, and Practice is applied Theory.</p>
<p>There’s an age-old problem with this particular relationship: those who specialize in Practice often claim that those who specialize in Theory are detached from the ‘real world,’ ie, the world of Practice. Those who specialize in Theory often claim that those who specialize in Practice have no fundamental understanding of what they do, and this leads to contradictory, incongruous practices.</p>
<p>There’s a third kind of person, though: one that embodies the becoming, the abstraction/application process. These people are a conduit, fundamentally bridging the two worlds. There’s a certain art to explaining just the core of Theory in the words of someone who Practices, and there’s a certain art to combining the essences of Practices and presenting it to those who Theorize. Building this bridge is an act of creation, of building, an opening of space.</p>
<hr />
<p>Some people are great at Ivory Tower intellectual stuff, and others just don’t really care. It sucks, because those who are doing could be way better at it if they just knew some theory, and those who love to philosophize all day might be more understandable if they’d just actually <em>do</em> something sometimes.</p>
<p>The only way you can get these two camps to talk to each other is to figure out what the theory says that provides value to those who practice. Practice-ers are ruthlessly focused on value, so to get through to them, you have to speak their language. On the flip side, theorists understand that practicers don’t care too much about the theory, but love seeing their thoughts to go good use, and can appreciate when practicers stumble across flaws in their thought. So demonstrating how practicers produce contradictions in the theory can be really useful to theorists.</p>
<p>If you’re this kind of person, accept that you’re in many ways a jack of all trades, but a master of none. Of sorts, anyway. Theorists will sometimes hate you for not grokking every last detail and reference, and practical people will argue that you don’t do enough useful things. Don’t listen to either of them; you know that you’re part of neither camp, so it makes sense that they both find ‘flaws.’ You’re awesome because you know a bit of both, and can facilitate communication which makes both better. A force multiplier.</p>
<hr />
<p>You have to remember that while you’re building things, there’s an underlying set of rules that you’re implicitly following, but it’s more important to act than it is to memorize a bunch of rules, and try to analyze what you’re doing according to them. If all you do is think all day, you’ll never get anything done. Things may go wrong, but you can always fix it later. Other people can sit around and theorize about what you’re doing, leave them to it.</p>
<p>Mental masturbation is fun and all, but when all is said and done, developing a culture of shipping is one of the most important things you can do. Those who can’t do, teach. History is written by the victors.</p>
<p>On occasion, you’ll run into someone who can actually explain complicated theory stuff to you in an accessible way. If you find someone like this, make sure to hold onto them closely, as they’re really rare. But they can help provide you with some insight that will really boost your productivity, without having to invest all the time in figuring out all that wankery that the priests of theory love.</p>
<hr />
<p>“In theory, there is no difference between theory and practice. But, in practice, there is.” - Yogi Berra</p>
</content:encoded></item><item><title>I got hit by a car today</title><link>https://steveklabnik.com/writing/i-got-hit-by-a-car-today/</link><guid isPermaLink="true">https://steveklabnik.com/writing/i-got-hit-by-a-car-today/</guid><pubDate>Thu, 25 Oct 2012 00:00:00 GMT</pubDate><content:encoded><p>“Humans are really squishy, messy, complicated, and contradictory. That’s what makes life interesting.”</p>
<p>Veggie Grill is damn delicious, but it’s hard to talk and eat at the same time. I’m usually trying to shovel my Santa Fe Chickin’ sandwich into my face as hard as possible. Today, though, there was a screw-up, and I didn’t get my food for a while. So I talked instead.</p>
<p>“Do you know anything about Hegelian dialectics?”</p>
<p>My friends are the best. Most people don’t put up with rambling, half-formed ideas, but mine generally entertain them, even though they haven’t read the books I’m talking about. I’m really grateful for this, as describing things really helps me sort them out in my head. The blank look and continued chewing meant that this was once again one of those times. Whatever.</p>
<p>“Basically, dialectics are a method of finding the truth in a threefold process: First, a ‘thesis’ is presented, which makes some sort of statement about the world. Then, the ‘antithesis’ contradicts it in some fashion. Finally, a ‘synthesis’ is found which resolves the tension between the two opposing sides. It’s a really interesting way of looking at the world; I’ve found it’s way easier to find things that totally contradict each other than I thought. Synthesis is really hard, though. Think about a trial, for example: the story of the plaintiff is told, then the story of the defendant, then the jury figures out what actually happened. There’s three sides to every story.”</p>
<p>Last week, I flipped over the handlebars of a bike. I was pumping as hard as I could, trying to make the light before it turned red, and the chain couldn’t take the tension any longer. It popped off, went between the back wheel and the frame, and stopped it cold. A few seconds later my eyebrows were scraping across the pavement. Lots of blood, but no serious injuries. I’m going to have a few sweet scars, though…</p>
<p>"Anyway, we were talking about feelings and ways people act before, but it’s true of physical bodies as well. Sometimes, accidents happen, and before you can even react, you’re dead. Other times, you escape with very little damage. We’re simultaneously incredibly vulnerable and almost impervious. Some people get in a bike accident and their life is forever changed. Other times, like me last week, you get away with a few scrapes. Once I fell about 8 feet onto my knees. Landed right on some pavement. I walked away 5 minutes later, didn’t even have bruises. I still have no idea what the fuck happened there.</p>
<p>I still haven’t found the synthesis yet."</p>
<p>After lunch, it’s time to bike home. It’s about 15 miles, which I can do pretty much at the drop of a hat nowadays. I really enjoy biking up Santa Monica Boulevard, but I hate it, too. It’s nice and broad, with two lanes, a bunch of green in the middle, and some bike lanes. Of course, it isn’t all like this, but much of it is. Especially on a day like today, around noon, it’s really pretty in places.</p>
<p><img src="http://upload.wikimedia.org/wikipedia/commons/6/62/Santa_Monica_Boulevard_at_Thayer.jpg" alt="http://upload.wikimedia.org/wikipedia/commons/6/62/Santa_Monica_Boulevard_at_Thayer.jpg" /></p>
<p>So I’m about 8 miles into my trip, and the next thing I know, I’m on the ground. Here’s the basic problem:</p>
<p><img src="http://www.examiner.com/images/blog/wysiwyg/image/car-in-bikelane-small.jpg" alt="http://www.examiner.com/images/blog/wysiwyg/image/car-in-bikelane-small.jpg" /></p>
<p>Yeah. Bike lanes don’t help if someone wants to turn right. When alongside cars, it’s very, very important to keep alert for two things: Open doors from parked cars, and drivers who don’t use their turn signals. A few weeks ago, I almost got hit by someone in this same exact way, just a few blocks down the street. I was paying close attention, and actually guessed that they were going to turn before they did. Free driver psychoanalysis done here! But more seriously, this was in the middle of the block. This guy saw that there was a space at this parking garage, hit his breaks and turned right without looking.</p>
<p>“YOU STUPID MOTHERFUCKER! WATCH WHERE THE FUCK YOU’RE GOING, YOU’RE GOING TO FUCKING KILL SOMEONE! JESUS!”</p>
<p>I ran a systems diagnostic check, and I was good, but enraged. Some pedestrians were screaming, and one ran over and asked if I was okay. “I said, yes, thank you, at least until this guy fucking kills me or someone else,” got back on my bike, and drove off.</p>
<p>At least my handlebars gave him a big long scratch the whole way up his car. Fucker.</p>
<p>There’s also something else interesting in this story as well: Generally, in a car/human interaction, I am the squishy one, and the car is the tank. Yet, in this particular interaction, I basically escaped unscathed, while the car took some damage. Once again, life is surprising, producing an inversion of the expected circumstances.</p>
<p>Anyway, if you drive a car, please, please, please watch what you’re doing. Pedestrians and cyclists don’t have thousands of pounds of metal surrounding them. Cyclists, please stay safe; it’s a jungle out there.</p>
<p>Ultimately, life is made out of these little incidents. Each moment to the next, you have no idea what’s going to happen. It’s exciting, scary, and sloppy. But that’s what makes it interesting. And you also don’t know when it’s going to end.</p>
<p>I still haven’t found that synthesis…</p>
</content:encoded></item><item><title>Living with a terrible taste in music</title><link>https://steveklabnik.com/writing/living-with-a-terrible-taste-in-music/</link><guid isPermaLink="true">https://steveklabnik.com/writing/living-with-a-terrible-taste-in-music/</guid><pubDate>Fri, 12 Oct 2012 00:00:00 GMT</pubDate><content:encoded><p>Music has always been very important to me. Throughout most of my life, I’ve listened to a lot of music, and it’s played a huge role in defining my personality at the time I listened to it. In many ways, the history of the music I’ve listened to is a history of me.</p>
<p>Here’s a look at where I’ve been, and where I’m at now.</p>
<h2>Country</h2>
<p>My parents essentially listen exclusively to country music, and so it’s basically all I listened to for the first decade of my life. And when I say country, I don’t mean old stuff, I mean proto-pop country:</p>
<p>Many, many people that I know now are quite surprised by this. Country is one of those genres that brings out a certain revulsion in people. It still comes up from time to time, and my understanding of country brings out a laugh. Once in college, I was in a friend’s hometown, the Middle of Nowhere, PA. His brother rolls up in his huge pickup truck, blasting some George Straight. My friend looks at me, and rolls his eyes. With a straight face, I start saying the words as George sings them. The look of shock and confusion on his face was priceless.</p>
<p>In many ways, that’s exactly who I was destined to be: just a country boy gettin’ down on the farm.</p>
<p>But then, I discovered skateboarding, and punk rock.</p>
<h2>Punk</h2>
<p>Of course, it wasn’t even good punk music, either. See, I had always thought skateboarding was neat, but wasn’t never had one because there was nowhere to ride it. I lived on a farm, we didn’t have any pavement anywhere. So I forgot about skating. But then I got a call from a friend, he’d just gotten a new video game:</p>
<p><img src="/img/2012-10-12/TonyHawksProSkaterPlayStation1.jpg" alt="thps 1 cover" /></p>
<p>Here’s the soundtrack list:</p>
<ul>
<li>“Police Truck” by the Dead Kennedys</li>
<li>“Here and Now” by The Ernies</li>
<li>“Vilified” by Even Rude</li>
<li>“Superman” by Goldfinger</li>
<li>“Jerry Was a Race Car Driver” by Primus</li>
<li>“Screamer”/“Nothing to Me” by Speedealer</li>
<li>“Cyco Vision” by Suicidal Tendencies</li>
<li>“New Girl” by The Suicide Machines</li>
<li>“Committed” by Unsane</li>
<li>“Euro-Barge” by The Vandals</li>
</ul>
<p>Pretty awesome. He also had this new album that came out at the same time…</p>
<p><img src="/img/2012-10-12/blink-182-enema-of-the-state-front.jpg" alt="enema of the state" /></p>
<p>From there, it was all over. All this edginess combined with my (just budding, of course) testosterone meant I was pretty much set. The funniest part about this period of my life was that I actually wasn’t really a particularly rebellious child: I maintained pretty damn good grades, save for a few semesters, I still went to church, and I was generally a good kid. I didn’t even smoke pot! Yet even the act of possessing these albums was heinously offensive. I got grounded for half a year for possessing Blink182’s “Take off your Pants and Jacket” and Limp Bizkit’s “Three Dollar Bill, Y’all$.” This constant conflict over music with my parents dominates my memories at the time. I literally printed out all the lyrics and wrote two pages explaining why I should be allowed to own a copy of the Offspring’s “Americana” even though they said ‘shit’ a few times. Punk, even third generation pop-punk, really spoke to my desire for autonomy and my attempts to be my own person.</p>
<h2>Metal</h2>
<p>Of course, slippery slopes are everywhere, and so I started searching for harder and harder music. Always the <a href="http://words.steveklabnik.com/how-do-you-find-the-time">min/maxer</a>, I looked for loopholes in my parent’s rules. The biggest hole: anything from the Christian Family Bookstore was fair game. So I scoured it for anything awesome. That’s when I found Zao:</p>
<p>Screw punk, metal was way more awesome. I grew my hair out long and learned to headbang.</p>
<p>I don’t think I need to explain to you why metal is ridiculous. I mean, come on:</p>
<p><img src="/img/2012-10-12/dethklokbandfeb2024_638.webp" alt="dethklok" /></p>
<h2>Hardcore</h2>
<p>Of course, there has to be a way to reconcile all this, right? Take two steps forward, one step back? Well, that’s hardcore.</p>
<p>My fascination with hardcore in late high school and early college was two-fold, like ‘hardcore’ itself: there’s ‘hardcore’ and then there’s ‘hardcore punk.’ Hardcore punk is a specific sub-genre of punk rock from the 80s, and hardcore is its continuation in the modern day, after that whole emo thing happened.</p>
<p>To recap: hardcore punk:</p>
<p><img src="/img/2012-10-12/a18-MinorThreat.jpg" alt="minor threat poster" /></p>
<p>hardcore:</p>
<p><img src="/img/2012-10-12/tumblr_lpqekrWMCt1qh3e7yo1_400.gif" alt="slam dancing" /></p>
<p>Uuhhh yeah. Hardcore punk also brought me to straightedge, which leads to silliness like this:</p>
<p><img src="/img/2012-10-12/KmCWp.jpg" alt="my straightedge tattoo" /></p>
<p>To this day, I still don’t regret this tattoo, and anyone that’s seen me at a Ruby conference knows that I’m not straightedge anymore. But this music and philosophy defined who I was for about 8 years, which was, at the end of it, about a third of my life. It changed my college experience. I’m (basically) permanently marked by it.</p>
<p>But no, seriously:</p>
<p><img src="/img/2012-10-12/tumblr_lpqekrWMCt1qh3e7yo1_400.gif" alt="slam dancing" /></p>
<h2>Party Rock</h2>
<p>Anyway, so nowadays, this is what I listen to:</p>
<p>&lt;iframe width="560" height="315" src="https://www.youtube.com/embed/edP0L6LQzZE?si=uTrbb2_tUchSk1FC" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen&gt;&lt;/iframe&gt;</p>
<p>&lt;iframe width="560" height="315" src="https://www.youtube.com/embed/k0BWlvnBmIE?si=m4OITv0l29x0zot5" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen&gt;&lt;/iframe&gt;</p>
<p>Here’s the thing: once I finally admitted to myself that I enjoy terrible pop music and stopped trying to apologize for my taste, I realized I was way more productive. Upbeat music with a vaguely positive vibe means I get shit done. Way better than the heavy, dreary, angry stuff I listened to in the past. I listen to hours of music a day, and I’d much rather have it be happy…</p>
<p>But that’s when I realized I’ve basically always had bad taste in music.</p>
<p>The thing is, that’s totally okay! Music has always provided an interesting backdrop for the rest of my life, and it’s fit whatever moment I was living. Writing this post was a blast, because I’d forgotten about half of this stuff, and re-living it is a good time. I can remember debating my friends about the Casualties being street punk or hardcore, if some band had really sold out, or if straightedge still matters now that Ian MacKaye smokes pot.</p>
<p>I wonder what I’ll be listening to in five years.</p>
</content:encoded></item><item><title>Run Rails with custom patches</title><link>https://steveklabnik.com/writing/run-rails-with-custom-patches/</link><guid isPermaLink="true">https://steveklabnik.com/writing/run-rails-with-custom-patches/</guid><pubDate>Thu, 04 Oct 2012 00:00:00 GMT</pubDate><content:encoded><p>I often see comments <a href="https://github.com/rails/rails/pull/7397#issuecomment-9132009">like this</a> in the Rails bugtracker. Generally, someone is running an older version of Rails, and some bug they face has been fixed on edge. But they may be running a version that’s too old to recieve fixes, or need a fix that has yet to be included in an actual release. What to do?</p>
<p>Luckily, <a href="http://gembundler.com/">Bundler</a> exists. It makes it super easy to run your own Rails. Check it out:</p>
<h2>Step 1: Fork you!</h2>
<p>Go to GitHub, hit up <a href="http://github.com/rails/rails">rails/rails</a>, and click that fork button. My own personal fork of Rails is <a href="https://github.com/steveklabnik/rails">here</a>, for example, so in the rest of these examples, I’ll be using my own username. If you’re me, you can use it too, but seriously, stop being me. If you’re not me, substitute your own username.</p>
<h2>Step 2: apply patch</h2>
<p>Let’s get our copy going locally:</p>
<pre><code>$ git clone [email protected]:steveklabnik/rails.git
$ cd rails
</code></pre>
<p>We’ll apply the patch from the example comment above. In this case, the person wants to have this patch work on Rails 3.1.</p>
<p>You have two options here: Rails keeps a branch open for every minor version, and also tags every release. You can pick between the stable branch and a particular release. The stable branch should work just fine, but maybe you’re on a specific Rails release for a reason. For example, the latest version of Rails 3.1 is 3.1.8, but this person says they’re on 3.1.1. If they use the 3-1-stable branch, they’ll also get all the changes from 3.1.1-3.1.8, as well as unreleased changes that may exist on that branch. That’s possibly too much for you, so we’ll just work off of 3.1.1. This is probably a bad idea, since <a href="http://weblog.rubyonrails.org/2012/6/12/ann-rails-3-1-6-has-been-released/">3.1.6 included several security fixes</a>, but you’re an adult, do dumb things if you want to.</p>
<pre><code>$ git checkout v3.1.1
</code></pre>
<p>You’ll see a big message about a detached HEAD. Don’t worry about it.</p>
<p>If you want the stable branch for extra fixes,</p>
<pre><code>$ git checkout 3-1-stable
</code></pre>
<p>Now, it’s a good idea to make our own branches, so that we don’t get confused with upstream. So let’s make a new branch:</p>
<pre><code>$ git checkout -b my_patched_rails
</code></pre>
<p>Awesome. Now, we can grab the commit we wanted. We can do this with <code>cherry-pick</code>:</p>
<pre><code>$ git cherry-pick 8fc8763fde2cc685ed63fcf640cfae556252809b
</code></pre>
<p>I found this SHA1 by checking out <a href="https://github.com/rails/rails/pull/7397/commits">this page</a>, which is linked at the top of the pull request.</p>
<p>If there are multiple commits you need, you can do one of two things: either <code>cherry-pick</code> the merge commit, in which case you’ll probably need to pass the <code>-m1</code> option, or simply <code>cherry-pick</code> them all in order.</p>
<p>You may get conflicts. Yay backporting! If <code>git</code> whines at you, do what you normally do. Resolve the merge, then <code>git commit</code>.</p>
<p>Finally, push it back up to your GitHub:</p>
<pre><code>$ git push origin my_patched_rails
</code></pre>
<p>Congrats! You’ve got your own custom Rails. Now it’s time to use it.</p>
<h2>Step 3: update Gemfile</h2>
<p>Go to your Rails app, and edit this line in your Gemfile:</p>
<pre><code>gem 'rails', "3.1.1"
</code></pre>
<p>change it to this:</p>
<pre><code>gem 'rails', :git =&gt; "https://github.com/steveklabnik/rails", :branch =&gt; "my_patched_rails"
</code></pre>
<p>It’s that easy! Now bundle:</p>
<pre><code>$ bundle update
</code></pre>
<p>You should see it mention something about checking out a certain copy of Rails.</p>
<h2>Step 4: Profit!</h2>
<p>That’s it! Congrats, you’re using Rails with your patch.</p>
<h2>But I’m on 2.3, I’m not running Bundler!</h2>
<p><a href="http://gembundler.com/v0.9/rails23.html">Do this</a>.</p>
</content:encoded></item><item><title>How do you find the time?</title><link>https://steveklabnik.com/writing/how-do-you-find-the-time/</link><guid isPermaLink="true">https://steveklabnik.com/writing/how-do-you-find-the-time/</guid><pubDate>Thu, 27 Sep 2012 00:00:00 GMT</pubDate><content:encoded><p>I’m often asked the question, “How do you find the time?” Mostly, it’s around open source, sometimes it’s about books, and occasionally, on other things. I’ve generally deflected the question with something like “I don’t know, man, I just <em>do</em> it.” But that answer isn’t very helpful. And I’ve been wondering if I can do even better. So I’ve done some reflection this week, and here’s how I find the time:</p>
<h2>Min/max</h2>
<p><img src="http://2.bp.blogspot.com/_6VPzVlB8N28/SwC5AWnireI/AAAAAAAABgA/O883tYn78As/s1600/DDplayershandbook2000.jpg" alt="http://2.bp.blogspot.com/_6VPzVlB8N28/SwC5AWnireI/AAAAAAAABgA/O883tYn78As/s1600/DDplayershandbook2000.jpg" /></p>
<p>player’s handbook</p>
<p>I have this personality type that is an asset to any competitive gamer: Dungeons and Dragons nerds call it “min/maxing”. The basic idea is this: you try to minimize the things that are bad, and maximize those that are good. This sounds super obvious, but there are many times when maximizing ‘winning’ isn’t what you’re going for. One great example is ‘pretending to be a fantasy adventurer.’ If you’re not shooting for strictly winning, then min/maxing doesn’t make any sense. But if you’re looking to compete, it’s incredibly important.</p>
<p>Speaking of competition, you should really read this book:</p>
<p><img src="http://ecx.images-amazon.com/images/I/41SZq3uDRVL._SS500_.jpg" alt="http://ecx.images-amazon.com/images/I/41SZq3uDRVL.SS500.jpg" /></p>
<p>playing to win</p>
<p><a href="http://www.sirlin.net/ptw/">“Playing to Win”</a> is an excellent book by one of the best Street Fighter players in the world. It is a must-read for anyone who wishes to compete in any way.</p>
<p>In it, he develops the theory of “Players” and “Scrubs”:</p>
<blockquote>
<p>A scrub is a player who is handicapped by self-imposed rules that the game knows nothing about.</p>
</blockquote>
<p>A great tell of someone being a scrub is when they mention ‘fairness.’ A game is an impartial, closed system. There are rules to the system that constrains the players. Simply, the more rules, the harder it is for you to achieve your win condition. If you invent extra rules, you handicap yourself. Games are not ‘fair.’ Players don’t handicap themselves in this way:</p>
<blockquote>
<p>A player should use any tournament legal move available to him that maximizes his chances of winning the game.</p>
</blockquote>
<p>If a move is legal, then it’s fair game. Good games are written in such a way that bad moves don’t exist. Use all the tools at your disposal, and use them ruthlessly.</p>
<p>Here’s the thing about the scrub mindset: it’s easy to do accidentally. Extra rules can be self-imposed, but they can also happen due to a poor analysis of what the rules actually are. For example, the biggest time I scrubbed out in my entire life was a choice that I made almost a decade ago: going to college. Every adult in my life told me that going to college was a good idea. They all said it was needed to get a job as a programmer. I didn’t bother to challenge them, and went along with it. Now I have a useless piece of paper and $70,000 in debt. College wasn’t all bad, but it certainly wasn’t worth the debt. I cherish the friendships I made, but not much else. This mistake was made because of my apathy towards the systems governing my life, and it’s one I try not to repeat.</p>
<p>Anyway, these two principles are the backdrop for everything else: properly analyze your situation, maximize the outcomes you want, and minimize the ones you don’t.</p>
<h2>Work</h2>
<p>“So what do you want out of life, Steve?”</p>
<p>“I want to teach, I want to work on open source, and I want to travel.”</p>
<p>“I think we can work something out.”</p>
<p>If you work a 9-5 job, you spend 40 hours a week at work. Let’s add a one hour commute in there too, which puts you up to 50 hours. And as Ruby will tell you…</p>
<pre><code>1.9.3-p194 :001 &gt; 24 * 7
=&gt; 168
1.9.3-p194 :002 &gt; 50 / 168.0
=&gt; 0.297619047619048
</code></pre>
<p>30% of your life. Let’s analyze this like a pro gamer, shall we? Here’s the situation we’re trying to maximize: “I want to live a full life.” In order to maximize this, we need to discover the constraints. The biggest one, of course, is your time on this Earth. For now, at least, we all will die someday. The primary resource that we all have to work with is time. You should guard your time more closely than any other thing you have. You can’t ever get it back. Every single thing you do consumes this most precious of resources.</p>
<p>When you enter into an employment agreement, you are literally selling your most precious thing in order to acquire other resources. Don’t forget that.</p>
<p>So, 9-5 job. 30% of your life. Is it worth it? Maybe! I can’t tell you. But in order to maximize your outcomes, you need proper analysis. Employment is a tricky thing, in that regard. Capitalism forces us to interact with the market in order to gain our subsistence, so we have to enter into this transaction somehow. What to do?</p>
<p>Quick, startup crowd: what’s the first rule of pricing? If you say “do it value-based, not cost-based,” then you win a <a href="http://www.kalzumeus.com/blog/">patio11</a> sticker. You’ll never get anywhere selling yourself on an hourly basis, you have to sell yourself on value. That’s another topic for another time. Point is, don’t just assume the rules: truly analyze your situation.</p>
<p>Anyway, that’s neither here nor there. On a day-to-day basis, the only reason that I can do what I do is because <a href="https://twitter.com/j3">Jeff</a> is a stand-up guy, and working with <a href="http://jumpstartlab.com/">Jumpstart Lab</a> is awesome. We do the best Ruby and Rails classes in the world, and I mean that. I don’t want to get into the details of my employment, but basically, teaching classes, contributing to open source, and speaking at conferences are all part of my job. You can see how that directly maps to my statement above.</p>
<p>So, if you appreciate</p>
<ul>
<li><a href="http://hackety.com/">My work on Hackety Hack</a></li>
<li><a href="http://contributors.rubyonrails.org/contributors/steve-klabnik/commits">My contributions to Rails</a></li>
<li><a href="https://github.com/defunkt/resque">My contributions to Resque</a></li>
<li><a href="http://vimeo.com/search?q=klabnik">My public speaking</a></li>
<li><a href="https://github.com/steveklabnik?tab=activity">All the other Open Source work that I do</a></li>
<li><a href="http://confreaks.com/videos/1089-madisonruby2012-anti-opression-101">My advocate work for feminism and social justice</a></li>
</ul>
<p>then you should consider hiring us over at Jumpstart. If I had to do normal consulting to pay my bills, I would have much less time for all of this stuff.</p>
<h3>But I don’t work at Jumpstart!</h3>
<p>So, you’re not me. You work a 9-5 job. So you’re screwed, right?</p>
<p>No way! Analyze your situation. Figure out how to maximize it. Before I did software development, I <a href="http://www.vocellipizza.com/consumer/index.php">did this</a>. About 30 hours a week while at high school, and 40 hours a week through college. I <em>still</em> managed to put in open source time. How?</p>
<p>The thing is, working on open source is something I truly love to do. For me, it’s not ‘work.’ That does not mean that it’s always roses. Sometimes it’s hard. Sometimes, it really, really sucks. Hard. However, I truly love it, so I stick it out through the hard times, and it ends up being good again. Once, I suffered a really bad break-up. My mom told me this maxim that her grandmother always told her:</p>
<blockquote>
<p>What bone you were meant to have, no dog will drag away.</p>
</blockquote>
<p>Grandmas. They’re so opaque! The point is, contributions to Open Source is something that’s a part of me, no matter what. So I’ll always find a way to do it. I don’t give up on it easily. When I first took over Shoes, it took me <strong>six months</strong> to figure out how to compile it. Six months. No compilation. That sucked. But I stuck it out.</p>
<blockquote>
<p>Eighty percent of success is showing up.Woody Allen</p>
</blockquote>
<h2>Relationships</h2>
<p>Having a full life means interacting with other people. Don’t let some productivity fetish negatively affect relationships. I mean that in a broad sense, both romantic and bromantic. ;) Make time for stuff that’s not ‘ZOMG MUST GET WORK DONE’ in your schedule too. I make sure to talk to significant others at least once or twice a day, for example. Assuming they want me to. Tonight, I’m gonna go for a bike ride on the beach. I still play StarCraft from time to time. Every Saturday, my friends and I used to get together in the computer lab and hack.</p>
<p>If you spend all of your time working, and none of your life living, you’ve failed at life. Open source is fun, but go outside, dammit. Or read a book. Or something. But if you only program computers, you’ve failed at being a human.</p>
<h2>Music</h2>
<p>Listen to awesome music while you work. I first discovered this while mowing my parent’s’ lawn: when I listened to Metallica, that grass got cut way quicker. So, while I work, I listen to shitty pop music. <a href="http://www.pandora.com/?sc=sh588283719292597139&amp;shareImp=true">Here’s my Pandora station</a>. I find that upbeat, happy music makes me upbeat and happy. I’m over being embarrassed about my love for Ke$ha. I just bob my head and write more code.</p>
<h2>Parallelize</h2>
<p>I actually cannot be productive without doing six different things at once. This is related to <a href="http://www.structuredprocrastination.com/">“Structured Procrastination”</a>. I do a little bit of work on a bunch of different things, and cycle between them quickly. One day, it’s Rails, another, it’s Resque, another is training, then back to Rails, then Hackety…</p>
<h2>lift.do</h2>
<p>I’ve been trying out <a href="http://lift.do/">Lift</a> lately, and I really like it. You can build habits with some light social interaction. You might find it useful.</p>
<h2>Cut distractions.</h2>
<p>That’s a bunch of things I do to maximize good outcomes, but what about minimizing bad ones? In the interest of minimalization, I won’t dwell on them too much, but here are some things that I’ve cut down or cut out in some way:</p>
<ul>
<li>TV</li>
<li>Reddit/HN</li>
<li>Cell phones</li>
<li>Video Games</li>
</ul>
<p>I still indulge in some of these from time to time, but I’ve cut them out of the ‘every day’ loop. My life is better off for it. Figure out what you don’t actually need, and stop it.</p>
<p>;)</p>
</content:encoded></item><item><title>Seriously: numbers: use them!</title><link>https://steveklabnik.com/writing/seriously--numbers--use-them-/</link><guid isPermaLink="true">https://steveklabnik.com/writing/seriously--numbers--use-them-/</guid><pubDate>Thu, 27 Sep 2012 00:00:00 GMT</pubDate><content:encoded><p><a href="http://vimeo.com/50340416">Turbolinks test</a> from <a href="http://vimeo.com/steveklabnik">Steve Klabnik</a> on <a href="http://vimeo.com/">Vimeo</a>.</p>
<p>If you don’t feel like watching…</p>
<p><a href="https://github.com/steveklabnik/turbolinks_test">https://github.com/steveklabnik/turbolinks_test</a></p>
<p>Results with 1000 pages:</p>
<p>Blank Rails app:</p>
<pre><code>$ rspec
user system total real
no turbolinks 11.170000 0.980000 12.460000 (138.656728)
yes turbolinks 10.800000 0.870000 11.670000 ( 80.436286)
</code></pre>
<p>With Basecamp Next’s CSS file:</p>
<pre><code>$ rspec
user system total real
no turbolinks 14.470000 1.540000 16.320000 (235.404727)
yes turbolinks 10.730000 0.870000 11.600000 ( 82.176967)
</code></pre>
<p>With Basecamp Next’s JS file:</p>
<pre><code>$ rspec
user system total real
no turbolinks 15.300000 1.700000 17.270000 (433.880904)
yes turbolinks 10.540000 0.890000 11.430000 (170.545663)
</code></pre>
<p>With Both:</p>
<pre><code>$ rspec
user system total real
no turbolinks 21.990000 2.890000 25.150000 (581.822206)
yes turbolinks 10.970000 0.910000 11.880000 (196.481247)
</code></pre>
<h2>TL;DR</h2>
<p>Turbolinks seems to speed up apps.</p>
</content:encoded></item><item><title>Resque: let's do this</title><link>https://steveklabnik.com/writing/resque--let-s-do-this/</link><guid isPermaLink="true">https://steveklabnik.com/writing/resque--let-s-do-this/</guid><pubDate>Sat, 22 Sep 2012 00:00:00 GMT</pubDate><content:encoded><p>If you’ve ever done background job stuff with Rails, you’ve probably used <a href="https://github.com/defunkt/resque">Resque</a>. Resque is “Yet Another GitHub Project They’ve Open Sourced and Then Abandoned.” But Resque is super awesome: it builds a job queue on top of Redis, which is a totally awesome piece of software.</p>
<p>Anyway, Resque has been psuedo-abandonware for a while now. In January, <a href="http://hone.heroku.com/">Terence Lee</a> from Heroku got in charge of the project, but it’s a big job. Too big of a job to properly manage alone. It’s hard to sift through <a href="https://github.com/defunkt/resque/issues?state=open">60 pulls and 120 open issues</a>, some of which have been open for a few <em>years</em>. And manage the 1.x line while working on new stuff for 2.0. And debug issues that basically boil down to “When I upgrade Resque and this random gem, everything breaks, but when I downgrade that other gem, it works again. But Resque throws the error.”</p>
<p>Even outside of heroics, the bus factor on a project that’s as important as Resque should be higher than one.</p>
<p>So Terrence gave a presentation at Frozen Rails, and in it, he outlined what needs to be done for Resque 2.0, and asked for some help getting it out the door. So myself and a few other people are gonna pitch in and help out, and we’d love to have you.</p>
<p>Here’s the deal: I will personally be looking at (and already have read) every single open issue on GitHub. Feel ignored about something involving Resque? You can come to me. I can’t guarantee that I can get it fixed, but at least I’ll help you figure out wtf is going on, and get us a bug report that we can use to build a fix.</p>
<p>Think of it as decoupling the front end from the back end: Terrence can focus on writing new code, and I can focus on paying attention to your problems, helping you solve them, and getting fixes applied to the 1.x branch.</p>
<p>If you’re interested in helping out, we’d love to have you. Helping triage issues would be awesome, fixing bugs would be awesome, writing code would be awesome, updating docs would be awesome.</p>
<p>Let’s do this.</p>
</content:encoded></item><item><title>Software and community</title><link>https://steveklabnik.com/writing/software-and-community/</link><guid isPermaLink="true">https://steveklabnik.com/writing/software-and-community/</guid><pubDate>Tue, 18 Sep 2012 00:00:00 GMT</pubDate><content:encoded><p>“Are you sure you’re not vegan? You look like a vegan.”</p>
<p>Chad Fowler made me laugh pretty hard that day. We were in Austin, at a Ruby conference, and getting some very non-vegan food at the excellent Iron Works BBQ. I had only talked to Chad a few times before, and while chowing down on some excellent ribs, we talked about something that we’d discussed in a previous conversation: “the Ruby community.”</p>
<p>I gave a talk at Lone Star Ruby last year about my involvement with Hackety Hack and Shoes, and at the end, discussed some social issues I thought Ruby has:</p>
<p>The fun starts at 27:49. But ultimately, this whole section relies on the idea that in Ruby, we have a cohesive ‘community’ of people. After I gave my talk, Chad mentioned to me that he didn’t like the idea of “the Ruby community.” At the time, I was so starstruck that I said something like “sure, okay, I’ll think about that,” not even realizing the irony. So, while trying valiantly to eat barbecue in a not completely messy way, I asked Chad more about his thoughts on this.</p>
<p>If you were wondering, if it’s good barbecue, it’s impossible to eat it neatly.</p>
<p>I don’t remember his exact words, but Chad’s point was that if we talk about Ruby like a big cohesive group, we forget about all the people who use it who don’t post on blogs, or Twitter, or contribute to GitHub. Those people are people too, but they’re effectively outside ‘the community,’ either by choice or by ignorance.</p>
<h2>Kill Your Idols</h2>
<p>It seems to me that much of software is driven by fashion, celebrity, and superstition. This shouldn’t surprise anyone who’s not a software developer, as programmers are just people, and much of the human world is driven by fashion, celebrity, and superstition. Programmers, though, regard themselves as bastions of rationality.</p>
<blockquote>
<p>I don’t argue about sports, it’s stupid. I prefer to argue about important things, like whether or not to use semicolons in JavaScript.</p>
</blockquote>
<p>— Matthew Nielsen (@xunker) September 15, 2012</p>
<blockquote></blockquote>
<p>Right? This whole idea makes us, or at least me, very uncomfortable. I’m a scientist, dammit! I measure twice, cut once. I use the right tool for the job. I swear, MongoDB, CoffeeScript, and Ember.js fits my use case perfectly!</p>
<p>Ideally, we’d make technical decisions on technical means only. However, we’re just humans. I’m not sure it’s possible to divorce our decision making from our squishy human tendencies, and after typing that sentence, I’m no longer sure that it’s a good idea, either.</p>
<h2>Leaders vs. Bosses</h2>
<blockquote>
<p>Does it follow that I reject all authority? Far from me such a thought. In the matter of boots, I refer to the authority of the bootmaker; concerning houses, canals, or railroads, I consult that of the architect or the engineer. For such or such special knowledge I apply to such or such a savant. But I allow neither the bootmaker nor the architect nor the savant to impose his authority upon me. I listen to them freely and with all the respect merited by their intelligence, their character, their knowledge, reserving always my incontestable right of criticism and censure. I do not content myself with consulting a single authority in any special branch; I consult several; I compare their opinions, and choose that which seems to me the soundest. But I recognise no infallible authority, even in special questions; consequently, whatever respect I may have for the honesty and the sincerity of such or such an individual, I have no absolute faith in any person. Such a faith would be fatal to my reason, to my liberty, and even to the success of my undertakings; it would immediately transform me into a stupid slave, an instrument of the will and interests of others.Mikhail Bakunin, “What is Authority,” 1882</p>
</blockquote>
<p>There is a difference between a leader and a boss. There’s also a difference between a leader and a celebrity. What I’m concerned with is that we occasionally turn leaders into bosses, and we turn celebrities into leaders.</p>
<p>One other Ruby person that straddles this line is Aaron “Tenderlove” Patterson. He is, for good reason, almost universally beloved by Rubyists. He’s known to all for two different reasons: as a leader, and as a celebrity. Aaron is the only person on both the core Ruby as well as core Rails teams. He is one of the most technically brilliant people I know.</p>
<p>He also posts amazing pictures of his cat on Tumblr. (The caption is “My new Yoga outfit.”)</p>
<p><img src="http://25.media.tumblr.com/tumblr_m3v8keZVhR1rt6kj7o1_1280.png" alt="http://25.media.tumblr.com/tumblr_m3v8keZVhR1rt6kj7o1_1280.png" /></p>
<p>gorbypuff</p>
<p>Another amazingly technical thing Aaron has done is <a href="http://confreaks.com/videos/427-rubyconf2010-zomg-why-is-this-code-so-slow">re-writing the AST -&gt; SQL generation stuff in ActiveRecord</a>, which sped up ActiveRecord by 5x. He deserves every single last bit of his popularity, on both sides. I happen to find the split reasons interesting.</p>
<h2>“The community” as useful abstraction</h2>
<p>While I think ‘the community’ may help drive a celebrity culture over a culture of leadership, there’s no denying that we do, in fact, have a community. If I had a nickel for every time someone in Ruby did something nice for me just because, I’d have a lot of nickels. I’m writing this post from the couch of <a href="https://twitter.com/kiiiir">a Rubyist</a> who happened to be on the same plane as me to Helsinki, and rather than stay at the airport for 10 hours overnight (checkin isn’t for another few hours yet), I crashed here. On my last trip to Germany, Konstantin and Josh from Travis not only gave me their couches to sleep on, but bought me food when I fucked up my debit card situation. One Saturday, Luis Lavena spent 8 hours on a Saturday randomly pairing with me on some Ruby on Windows stuff, and left with an “I’m sorry, but my fiance is bugging me, I told her we’d get dinner an hour ago, I need to leave.” That’s crazy, I’m some dude from another hemisphere, hang out with your fiance! Aaron helped me debug an issue in Rails the other day, even though I kept asking him stupid questions. I have tons of stories like this, and so do many other people who participate in open source.</p>
<p>In “Debt: The first 5,000 Years,” Graeber defines “everyday communism” as "any community who follows the maxim ‘from each according to their abilities, to each according to their needs." Of course, communism and community share a root for a reason, and all of these examples above are great descriptions of ’everyday communism.’ Not every exchange must go through some sort of monetary exchange. It’s often simpler to just do each other favors, and know that it will all work out in the end. That’s the basis of a gift economy.</p>
<p>In this way, I think community is a useful abstraction; we just need to make sure that we don’t get <a href="http://rubydramas.com/">too caught up in ourselves</a>. Focus on the friendships, helping each other out, and writing great software. Try not to get too trendy. Pay attention to what matters.</p>
<p>In the end, I wouldn’t give up the <strong>real</strong> friendships I’ve made via the Ruby community for the world.</p>
</content:encoded></item><item><title>Random Ruby Tricks: Class.new</title><link>https://steveklabnik.com/writing/random-ruby-tricks--class-new/</link><guid isPermaLink="true">https://steveklabnik.com/writing/random-ruby-tricks--class-new/</guid><pubDate>Sun, 09 Sep 2012 00:00:00 GMT</pubDate><content:encoded><p>If you didn’t know, classes are first-class objects in Ruby:</p>
<pre><code>1.9.3p194 :001 &gt; String.class
=&gt; Class
1.9.3p194 :002 &gt; Class.class
=&gt; Class
</code></pre>
<p>How is this useful, though?</p>
<h2>Inheritance</h2>
<p>You may create very simple classes at times. This often happens when subclassing an error of some sort:</p>
<pre><code>class MyException &lt; StandardError
end
raise MyException
</code></pre>
<p>That whole thing is awkward to me, though. Why bother with the end if you don’t need it? Turns out, it’s easy to make your own <code>Class</code> object: just use <code>Class.new</code>:</p>
<pre><code>1.9.3p194 :001 &gt; Class.new
=&gt; #&lt;Class:0x007fee209f42f8&gt;
</code></pre>
<p>But that doesn’t help us a ton. Turns out <code>Class.new</code> takes an argument: the class this new class inherits from:</p>
<pre><code>1.9.3p194 :002 &gt; Class.new(StandardError)
=&gt; #&lt;Class:0x007fee209ee2e0&gt;
</code></pre>
<p>We can then save this in our own constant:</p>
<pre><code>1.9.3p194 :003 &gt; MyException = Class.new(StandardError)
=&gt; MyException
1.9.3p194 :004 &gt; MyException.new
=&gt; #&lt;MyException: MyException&gt;
1.9.3p194 :005 &gt; MyException.ancestors
=&gt; [MyException, StandardError, Exception, Object, Kernel, BasicObject]
</code></pre>
<p>I think I like this notation a bit better than <code>class...end</code>. But many people may not be familiar with it.</p>
<h2>Passing a block</h2>
<p>Of course, you can pass a block to <code>Class.new</code> and it’ll work like you’d expect:</p>
<pre><code>1.9.3p194 :001 &gt; Foo = Class.new do
1.9.3p194 :002 &gt; def bar
1.9.3p194 :003?&gt; puts "bar"
1.9.3p194 :004?&gt; end
1.9.3p194 :005?&gt; end
=&gt; Foo
1.9.3p194 :006 &gt; Foo.new.bar
bar
=&gt; nil
</code></pre>
<p>The block gets <code>class_eval</code>’d. I haven’t found a good use for this one, exactly, but it’s good to know about!</p>
<p>EDIT: <a href="https://twitter.com/pete_higgins/status/244903490588119040">@pete_higgins has chimed in on Twitter.</a> I knew I’d done this before!</p>
<h2><code>new</code> vs <code>initialize</code></h2>
<p>Ever wonder why you call <code>Foo.new</code> but define <code>Foo.initialize</code>? It’s pretty simple. Here:</p>
<pre><code>VALUE rb_class_new_instance(int argc, VALUE *argv, VALUE klass)
{
VALUE obj;
obj = rb_obj_alloc(klass);
rb_obj_call_init(obj, argc, argv);
return obj;
}
</code></pre>
<p>Obviously! This is the source for <code>Foo.new</code>. You might not read C, but it’s pretty simple: it first allocates the space for the object using <code>rb_obj_alloc</code>, and then calls <code>initialize</code> using <code>rb_obj_call_init</code>.</p>
</content:encoded></item><item><title>Protocol and language</title><link>https://steveklabnik.com/writing/protocol-and-language/</link><guid isPermaLink="true">https://steveklabnik.com/writing/protocol-and-language/</guid><pubDate>Mon, 03 Sep 2012 00:00:00 GMT</pubDate><content:encoded><p>This is the second part of my series on protocol. The <a href="http://words.steveklabnik.com/protological-control-an-introduction">first part</a> contained a lot of background information, but now we’re ready to get into what Protocol actually <em>is</em>.</p>
<hr />
<p>I live in a pretty unique place, <a href="http://farmhouse.la/">the Farmhouse</a>. It’s a psuedo-public space in which we live, others work, and everybody has a good time. We do have <a href="http://farmhouse.la/rules">some rules</a>, though:</p>
<p><img src="http://farmhouse.s3.amazonaws.com/images/home/farmhouse-rules-chalkboard-wall-by-heather-peterson.jpg" alt="http://farmhouse.s3.amazonaws.com/images/home/farmhouse-rules-chalkboard-wall-by-heather-peterson.jpg" /></p>
<p>They’re written up on the wall in chalk. Since we have lots of visitors to the house, and these are hard and fast rules, it’s important that we communicate them in a clear way. These rules form the protocol of our house. Phrased another way, they’re the rules that govern the way that we interact with each other and the house itself while at the house.</p>
<p>Ultimately, this is what protocol is: a language. To Galloway, protocol is a very specific kind of language:</p>
<blockquote>
<p>Protocol is a language that regulates flow, directs netspace, codes relationships, and connects life-forms.</p>
</blockquote>
<p>Some of these properties are contradictory. Stated more accurately, there exists a tension between opposing forces within protocol itself. Let’s expand on these meanings.</p>
<h2>‘regulates flow’</h2>
<p>If you’ve ever read <em>1984</em>, you’re familiar with linguistic relativity, more commonly known as the Sapir-Whorf hypothesis.</p>
<blockquote>
<p>How could you have a slogan like “freedom is slavery” when the concept of freedom has been abolished?</p>
</blockquote>
<p>What’s truly interesting is that Orwell himself actually advocated for simplifying language in <a href="http://www.resort.com/~prime8/Orwell/patee.html">Politics and the English Language</a>, but specifically because he felt more complicated language was <em>obfuscatory</em>, not illuminating.</p>
<p>In any case, language enumerates the possible, and while <a href="http://en.wikipedia.org/wiki/Longest_words#German">some languages can have really long words</a>, ultimately, any language will have some things that are not expressible. <a href="http://byfat.xxx/rien-ne-tient-en-place">This often happens with translations</a>.</p>
<p>If you are the proud recipient of a computer science degree, you may have run into the <a href="http://en.wikipedia.org/wiki/Chomsky_hierarchy">Chomsky hierarchy</a> in your compilers class. If you weren’t required to take a compilers class, please call up your alma mater and demand a refund. Here is the hierarchy:</p>
<ol>
<li>unrestricted grammars</li>
<li>context-sensitive grammars</li>
<li>context-free grammars</li>
<li>regular grammars</li>
</ol>
<p>As you go up the hierarchy, each language type is enclosed by the type above it. All context-sensitive grammars are unrestricted grammars, and all context-free grammars are context-sensitive.</p>
<p>Regular expressions are one area where a language was forced to move up the hierarchy in order to express more things. Strictly speaking, a regular expression cannot match nested parenthesis. If you are interested in the math, the <a href="http://en.wikipedia.org/wiki/Pumping_lemma_for_regular_languages">Pumping Lemma</a> demonstrates why. However, this kind of thing is useful, and so extensions were developed that move them up the chain. <a href="http://perldoc.perl.org/perlfaq6.html#Can-I-use-Perl-regular-expressions-to-match-balanced-text%3f">Now you can write this</a>:</p>
<blockquote>
<p>qr/(&lt;(?:[^&lt;&gt;]++|(?1))*&gt;)/</p>
</blockquote>
<p>Maybe Orwell was on to something.</p>
<p>Others have speculated that linguistic relativity would also apply to software. Two notable examples are Kenneth E. Iverson, who created APL, and Paul Graham, Lisp enthusiast and investor extraordinaire. Iverson wrote a paper titled <a href="http://www.jsoftware.com/papers/tot.htm">“Notation as a tool of thought”</a>, which begins:</p>
<blockquote>
<p>The importance of nomenclature, notation, and language as tools of thought has long been recognized.</p>
</blockquote>
<p>Graham authored an essay called <a href="http://www.paulgraham.com/avg.html">“Beating the Averages”</a> in which describes the ‘Blub paradox’. The paradox relies on linguistic relativity:</p>
<blockquote>
<p>You can’t trust the opinions of the others, because of the Blub paradox: they’re satisfied with whatever language they happen to use, because it dictates the way they think about programs.</p>
</blockquote>
<p>I don’t mean to rain on the relativity parade, but there’s one small detail that we haven’t considered: Turing completeness. At the end of the day, most programming languages are Turing complete. This means that COBOL can do everything Lisp can, and PHP can do everything that Ruby can.</p>
<p>So are Graham and Iverson wrong?</p>
<p>Turns out there are two variations of linguistic relativity: <em>strong</em> relativity states that language <strong>determines</strong> thought, and <em>weak</em> relativity that states language <strong>influences</strong> thought. This was coined later, as it would seem that current research has found little evidence to support strong relativity, but some to support the notion of weak relativity. Graham notes this, and confines the paradox to the realm of weak:</p>
<blockquote>
<p>All languages are equally powerful in the sense of being Turing equivalent, but that’s not the sense of the word programmers care about. (No one wants to program a Turing machine.)</p>
</blockquote>
<p>To put it another way, language features create friction that encourages or discourages certain modes of thinking.</p>
<p>Python and Ruby are two very similar languages, yet they engender very different styles. One area in which this division is sharp is first-class functions. Both languages have them, but Python’s are simply expressions rather than fully-featured . <a href="http://www.artima.com/weblogs/viewpost.jsp?thread=147358">Guido van Rossum likes them this way</a>:</p>
<blockquote>
<p>But such solutions often lack “Pythonicity” – that elusive trait of a good Python feature. It’s impossible to express Pythonicity as a hard constraint. Even the Zen of Python doesn’t translate into a simple test of Pythonicity.</p>
</blockquote>
<p>Guido specifically wants to limit the scope of language in order to encourage a particular style of thinking about problems: a Pythonic one.</p>
<p>Galloway is very explicit about this regulation.</p>
<blockquote>
<p>Protocol is synonymous with possibility. From the perspective of protocol, if you can do it, it can’t be bad, because if it were bad, then it would have been outlawed years ago by protocol.</p>
</blockquote>
<h2>‘directs netspace’</h2>
<p>After that massive first part, I’m basically going to punt on the second. I don’t find it very interesting or worth elaborating on: languages and protocols direct the direction of the Internet. I don’t think anyone disputes this.</p>
<h2>‘codes relationships’</h2>
<p>Communication has an intimate connection to relationships. The act of communicating is what creates the relationship in the first place. And language is the encoding of that particular relationship.</p>
<p>In “A user’s guide to Capitalism and Schizophrenia: Deviations from Deleuze and Guattari,” Brian Massumi discusses an example of marriage vows:</p>
<blockquote>
<p>A particular man and a particular woman say “I do.” Their words undoubtedly have personal meaning for them in their heart of hearts. But their personal intention is not in itself responsible for the magical transformation that has touched their lives. What has brought them to say these words and what makes those words effectively transformative is too big to fit into a single mid. It is a complex interplay of laws, customs, social pressure, and tax law.</p>
</blockquote>
<p>Marriage is a social protocol that encodes a particular relationship. Once that relationship has been encoded, it’s made legible to others who weren’t privy to the details of that relationship. A wedding ring is only one form of relationship encoding; a couple holding hands is a similar example, whose bond is weaker both in symbol as well as in reality.</p>
<p>In the software world, HTTP encodes relationships as well. The client/server relationship is integral to the protocol’s design, as Roy Fielding <a href="http://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch_styles.htm#sec_3_4_1">describes in section 3 of his dissertation</a>:</p>
<blockquote>
<p>A client is a triggering process; a server is a reactive process. Clients make requests that trigger reactions from servers. Thus, a client initiates activity at times of its choosing; it often then delays until its request has been serviced. On the other hand, a server waits for requests to be made and then reacts to them. A server is usually a non-terminating process and often provides service to more than one client.</p>
</blockquote>
<p>He further goes on to describe why this relationship is important:</p>
<blockquote>
<p>Separation of concerns is the principle behind the client-server constraints. A proper separation of functionality should simplify the server component in order to improve scalability. This simplification usually takes the form of moving all of the user interface functionality into the client component. The separation also allows the two types of components to evolve independently, provided that the interface doesn’t change.</p>
</blockquote>
<p>In other words, the protocol encodes the relationship so that others know what a client should do, and what a server should do.</p>
<p>If you remember our good friend Saussure, you’ll remember that he made this encoding explicit, and in fact central to language itself. While Saussure’s work isn’t directly useful to linguists in the present, much of his work is foundational to the ways that we approach language today.</p>
<p>One of the most fundamental concepts in structural linguistics is the ‘sign,’ and its component parts, the ‘signifier’ and the ‘signified.’ The signified is a particular concept, and the signifier is the way the signified is expressed. A sign is a combination of a signifier and a signified. Seems pretty simple, but Saussure also posited that a sign can only gain <a href="http://en.wikipedia.org/wiki/Meaning_(semiotics)">meaning</a> and <a href="http://en.wikipedia.org/wiki/Meaning_(semiotics)">value</a> within their relationship to other signs, which means that you cannot consider an idea outside of the way in which the idea is expressed.</p>
<p>The study of signs is called <a href="http://en.wikipedia.org/wiki/Semiotics">semiotics</a>. In Semiotics, a particular set of conventions used to convey meaning are called <a href="http://en.wikipedia.org/wiki/Code_(semiotics)">codes</a>. Hence ‘encoding’ can also refer to this kind of code, as well.</p>
<p>Hence, if we combine Fielding, Galloway, and Saussure, the word ‘client’ is a signifier that holds a relationship to the signified concept ‘a triggering process’ (among other things), that forms the sign ‘client.’ This is encoded by the protocol HTTP.</p>
<h2>‘connects life-forms’</h2>
<p>From the Massumi marriage example:</p>
<blockquote>
<p>Say “I do,” and your life will never be the same. Your legal, social, and familial status instantly changes, along with your entire sexual, psychological, and financial economy. You have been pronounced man and wife. You may file a joint tax return.</p>
</blockquote>
<p>This connection is related to the previous coded relationship. In this case, ‘connects’ is an active verb rather than a passive noun. It not only <em>creates</em> a connection, but it ‘connects’. It not only relates two objects, it <em>is</em> the relation.</p>
<p>We spent most of the last section discussing connections, though, so let’s get to the interesting part: life. Galloway is very explicit about the role protocol plays with life:</p>
<blockquote>
<p>…protocol is an affective, aesthetic force that has control over “life itself.”</p>
</blockquote>
<p>In <a href="http://www.amazon.com/The-Exploit-Networks-Electronic-Mediations/dp/0816650446">the Exploit</a>, Galloway discusses the relationship between life and protocol as well:</p>
<blockquote>
<p>While the first type of network (Internet protocols) is silicon based and may use biological concepts (intelligent agents, artificial life, genetic algorithms), the second (DNA algorithms) is fully biological and yet recodes itself in computational terms (biology as computation, as opposed to evolution)</p>
</blockquote>
<p>The relationship between life and software is an interesting one.</p>
<p><img src="http://upload.wikimedia.org/wikipedia/commons/e/e5/Gospers_glider_gun.gif" alt="http://upload.wikimedia.org/wikipedia/commons/e/e5/Gospers_glider_gun.gif" /></p>
<p>In Conway’s game of life, one builds cellular automata and lets them go. There are a few simple rules that govern their reproduction, death, and life. This protocol governs all that happens within the game. Someone playing the game is an impotent god, as there is no room in the protocol for communicating with creations. Likewise, creations have no way of perceiving an ‘outside’ of the game.</p>
<p>Galloway references this quote by Frederick Cohen from <a href="http://www.amazon.com/Course-Computer-Viruses-Professional-Computing/dp/0471007684">A Short Course on Computer Viruses</a>:</p>
<blockquote>
<p>I personally believe that reproducing programs are living beings in the information environment.</p>
</blockquote>
<p>Viruses, Conway cells, <a href="http://thesims.com/en_us/home">Sims</a>, and all other kinds of ‘artificial life’ exist inside of computers. They’re native to protocol. But what about actual, real life?</p>
<p>Galloway reaches to Norbert Wiener, a mathematician who was the originator of cybernetics, for this:</p>
<blockquote>
<p>if one views the world in terms of information, then there is little instrumental difference between man and machine since both are able to affect dynamic systems via feedback loops. In this way the cybernetic system of man and machine are born. Its virtues are balance, self-regulation, circularity, and control. In a word, protocol.</p>
</blockquote>
<p>Deleuze also forms a connection between life and information in <a href="http://www.n5m.org/n5m2/media/texts/deleuze.htm">“Postscript on the Societies of Control”</a>:</p>
<blockquote>
<p>The numerical language of control is made of codes that mark access to information, or reject it. We no longer find ourselves dealing with the mass/individual pair. Individuals have become “dividuals,” and masses, samples, data, markets, or “banks.”</p>
</blockquote>
<p>A <em>dividual</em> is a play on the process of <a href="http://en.wikipedia.org/wiki/Individuation">individuation</a>. Galloway elaborates:</p>
<blockquote>
<p>Deleuze’s neologism comes from the word ‘individuate.’ Dividuation would thus be the opposite: the dissolving of the individual entity into distributed networks of information.</p>
</blockquote>
<p>Let’s use a personal example to demonstrate life submitting itself to dividuation in order to interface with protocol: me. <a href="http://steveklabnik.com/">This is my personal website</a>. In it, my public face on the Internet, I’ve split myself up into five broad categories, and link out to all of the various sub-components. Examining the last category, you can see how if you just follow me on twitter <a href="https://twitter.com/steveklabnik">;)</a>, you will not get the whole picture. You could also <a href="https://github.com/steveklabnik">check out my code on GitHub</a>, but that wouldn’t be me, either. Even the sum of these two components would not tell you everything about me. Yet I can’t possibly put <em>me</em> on the network. I must break ‘me’ into a form that conforms with protocol.</p>
<p>Finally, there’s an important relationship with Foucault’s concept of ‘biopolitics.’</p>
<blockquote>
<p>Foucault defines biopolitics as “the endeavor, begun in the eighteenth century, to rationalize the problems presented to governmental practice by the phenomena characteristic of a group of living human beings constituted as a population: health, sanitation, birthrate, longevity, race.”</p>
</blockquote>
<p>Obviously, dividuating a population into informational components would be relevant to protocol. And, in fact, life provides a path to resist protocol. Deleuze in his book <em>Foucault</em> (because that’s not confusing…):</p>
<blockquote>
<p>When power becomes bio-power resistance becomes the power of life, a vital power that cannot be confined within species, environment or the paths of a particular diagram.</p>
</blockquote>
<p>Galloway expands:</p>
<blockquote>
<p>Is life resistance a way of engaging with distributed forms of protological management? Part III of this book, “Protocol Futures,” answers yes. While the new networked technologies have forced an ever more reticent public to adapt to the control structures of global capital, there has emerged a new set of social practices that inflects or otherwise diverts these protological flows toward the goal of a utopian form of unalienated social life.</p>
</blockquote>
<h2>To conclude</h2>
<p>Protocol has an intimate and complex relationship with language. In many ways, it is a particular language, with all of the expressive power and limiting abilities that that brings.</p>
<p>The implications of this language, though, will have to wait for another time.</p>
</content:encoded></item><item><title>Random Ruby Tricks: Struct.new</title><link>https://steveklabnik.com/writing/random-ruby-tricks--struct-new/</link><guid isPermaLink="true">https://steveklabnik.com/writing/random-ruby-tricks--struct-new/</guid><pubDate>Sat, 01 Sep 2012 00:00:00 GMT</pubDate><content:encoded><p>One of my favorite classes in Ruby is <code>Struct</code>, but I feel like many Rubyists don’t know when to take advantage of it. The standard library has a lot of junk in it, but <code>Struct</code> and <code>OStruct</code> are super awesome.</p>
<h2>Struct</h2>
<p>If you haven’t used <code>Struct</code> before, here’s <a href="http://www.ruby-doc.org/core-1.9.3/Struct.html">the documentation of Struct from the Ruby standard library</a>.</p>
<p>Structs are used to create super simple classes with some instance variables and a simple constructor. Check it:</p>
<pre><code>Struct.new("Point", :x, :y) #=&gt; Struct::Point
origin = Struct::Point.new(0,0) #=&gt; #&lt;struct Struct::Point x=0, y=0&gt;
</code></pre>
<p>Nobody uses it this way, though. Here’s the way I first saw it used:</p>
<pre><code>class Point &lt; Struct.new(:x, :y)
end
origin = Point.new(0,0)
</code></pre>
<p>Wait, what? Inherit…from an instance of something? Yep!</p>
<pre><code>1.9.3p194 :001 &gt; Struct.new(:x,:y)
=&gt; #&lt;Class:0x007f8fc38da2e8&gt;
</code></pre>
<p><code>Struct.new</code> gives us a <code>Class</code>. We can inherit from this just like any other <code>Class</code>. Neat!</p>
<p>However, if you’re gonna make an empty class like this, I prefer this way:</p>
<pre><code>Point = Struct.new(:x, :y)
origin = Point(0,0)
</code></pre>
<p>Yep. Classes are just constants, so we assign a constant to that particular <code>Class</code>.</p>
<h2>OStruct</h2>
<p><code>[OStruct](http://ruby-doc.org/stdlib-1.9.3/libdoc/ostruct/rdoc/OpenStruct.html)</code>s are like <code>Struct</code> on steroids. Check it:</p>
<pre><code>require 'ostruct'
origin = OpenStruct.new
origin.x = 0
origin.y = 0
origin = OpenStruct.new(:x =&gt; 0, :y =&gt; 0)
</code></pre>
<p><code>OStruct</code>s are particularly good for configuration objects. Since any method works to set data in an <code>OStruct</code>, you don’t have to worry about enumerating every single option that you need:</p>
<pre><code>require 'ostruct'
def set_options
opts = OpenStruct.new
yield opts
opts
end
options = set_options do |o|
o.set_foo = true
o.load_path = "whatever:something"
end
options #=&gt; #&lt;OpenStruct set_foo=true, load_path="whatever:something"&gt;
</code></pre>
<p>Neat, eh?</p>
<h2>Structs for domain concepts</h2>
<p>You can use <code>Struct</code>s to help reify domain concepts into simple little classes. For example, say we have this code, which uses a date:</p>
<pre><code>class Person
attr_accessor :name, :day, :month, :year
def initialize(opts = {})
@name = opts[:name]
@day = opts[:day]
@month = opts[:month]
@year = opts[:year]
end
def birthday
"#@day/#@month/#@year"
end
end
</code></pre>
<p>and we have this spec</p>
<pre><code>$:.unshift("lib")
require 'person'
describe Person do
it "compares birthdays" do
joe = Person.new(:name =&gt; "Joe", :day =&gt; 5, :month =&gt; 6, :year =&gt; 1986)
jon = Person.new(:name =&gt; "Jon", :day =&gt; 7, :month =&gt; 6, :year =&gt; 1986)
joe.birthday.should == jon.birthday
end
end
</code></pre>
<p>It fails, of course. Like this:</p>
<pre><code>$ rspec
F
Failures:
1) Person compares birthdays
Failure/Error: joe.birthday.should == jon.birthday
expected: "7/6/1986"
got: "5/6/1986" (using ==)
# ./spec/person_spec.rb:9:in `block (2 levels) in &lt;top (required)&gt;'
Finished in 0.00053 seconds
1 example, 1 failure
Failed examples:
rspec ./spec/person_spec.rb:5 # Person compares birthdays
</code></pre>
<p>Now. We have these two birthdays. In this case, we know about why the test was failing, but imagine this failure in a real codebase. Are these month/day/year or day/month/year? You can’t tell, it could be either. If we switched our code to this:</p>
<pre><code>class Person
attr_accessor :name, :birthday
Birthday = Struct.new(:day, :month, :year)
def initialize(opts = {})
@name = opts[:name]
@birthday = Birthday.new(opts[:day], opts[:month], opts[:year])
end
end
</code></pre>
<p>We get this failure instead:</p>
<pre><code>$ rspec
F
Failures:
1) Person compares birthdays
Failure/Error: joe.birthday.should == jon.birthday
expected: #&lt;struct Person::Birthday day=7, month=6, year=1986&gt;
got: #&lt;struct Person::Birthday day=5, month=6, year=1986&gt; (using ==)
Diff:
@@ -1,2 +1,2 @@
-#&lt;struct Person::Birthday day=7, month=6, year=1986&gt;
+#&lt;struct Person::Birthday day=5, month=6, year=1986&gt;
# ./spec/person_spec.rb:9:in `block (2 levels) in &lt;top (required)&gt;'
Finished in 0.00092 seconds
1 example, 1 failure
Failed examples:
rspec ./spec/person_spec.rb:5 # Person compares birthdays
</code></pre>
<p>We have a way, way more clear failure. We can clearly see that its our days that are off.</p>
<p>Of course, there are other good reasons to package related instance variables into <code>Struct</code>s, too: it makes more conceptual sense. This code represents our intent better: a Person has a Birthday, they don’t have three unrelated numbers stored inside them somehow. If we need to add something to our concept of birthdays, we now have a place to put it.</p>
</content:encoded></item><item><title>New blog: Words</title><link>https://steveklabnik.com/writing/new-blog--words/</link><guid isPermaLink="true">https://steveklabnik.com/writing/new-blog--words/</guid><pubDate>Fri, 31 Aug 2012 00:00:00 GMT</pubDate><content:encoded><p>I’ve started a new blog: <a href="http://words.steveklabnik.com/">Words</a>.</p>
<p>You may have noticed that it’s a part of <a href="https://svbtle.com/">the Svbtle blogging network</a>. I’m super excited about this, but you may be confused. Let me explain.</p>
<h2>My relationship with writing</h2>
<p>I’ve been involved in Internet communities for most of my life. One of the nice things about that is that it’s forced me to express myself in writing for a very, very long time.</p>
<p>I don’t remember what site this is, but there was one that generated a PDF of your top 100 comments on Hacker News. The idea was “You’ve already written a book!” You can see this kind of thinking in many “Blog to book” books that are out there, but I found this really powerful. My top 100 comments span 19 pages; I was (at one point) number 60 on the top of Hacker News by karma. And I really was only active on HN for about a year, February 2010 to 2011. I spent much more time on /., Reddit, and a half dozen forums. I bet if you compiled a list of everything I ever wrote in a <code>&lt;textarea name="comment" id="comment"&gt;</code>, I would have written at least 3 novels by now. They’d be shit, of course, but the point is that if you’re active at all on the read/write web, then you’ve probably written a ton.</p>
<p>As a side point, I really enjoyed the anonymity that posting on the web gave me. I could be 13, yet arguing with people who were 30! And they’d actually take me seriously, since they didn’t know I wasn’t an adult! Of course, the only way they wouldn’t know is if I posted proper English… grammar nazis helped me out, in a way.</p>
<p>Anyway.</p>
<h2>Practice makes perfect</h2>
<p>Of course, this means that at first, I <em>wasn’t</em> practiced at writing, and that means that <a href="http://games.slashdot.org/comments.pl?sid=87657&amp;cid=7604346">it was any good…</a> That’s the only way that you get better, though: write and write and write and write. And get feedback on that writing. Originally, it was just people commenting back. Then upvotes. I’m pretty sure that I’ve averaged over two or three comments a day on the Internet over the past decade or so. If you include tweets, I bet the median is about the same, but the mean would go up to 6 or 8.</p>
<p>Regardless, I tend to be very off-the-cuff when writing. When I’m done writing this blog post, I will type <code>:wq&lt;enter&gt;git commit -am "add 'words'"&lt;enter&gt;git push origin,heroku&lt;enter&gt;&lt;alt-tab&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;&lt;f5&gt;</code> . Then tweet. I end up doing very little proofreading of the things I write, and that means that sometimes, my writing suffers. I’ve settled into a local maximum: I’m Good Enough at writing that people like what I write, and I can quickly type something out and get it up there. But I don’t think I’m getting any <strong>better</strong> at writing. And I’d like to.</p>
<h2>Gamification and Exhibitionism</h2>
<p>I’ve also come to realize something about myself over the years: I respond really well to gamification, and am somewhat of an exhibitionist. Or at the very least, an extrovert. These two things in combination are a powerful mixture.</p>
<p>When I started playing StarCraft II, I knew I wanted to get better. So I set up <a href="http://www.twitch.tv/emmagoldmansc">an account on Twitch</a>. I sometimes stream games there. I’m in Silver league, which means I’m not that good. But knowing that someone may be watching makes me want to be better at it. I sometimes <a href="https://vimeo.com/37978370">release videos of me live coding</a>. Almost everything <a href="https://github.com/steveklabnik">I code is open source</a>. Knowing that others might be watching makes me want to be at my best.</p>
<p>Gamification can help with this, too. Monotonically increasing numbers are terribly addicting, and trigger something deep down in my brain. I’ve been known to refresh my retweet count or my upvote count every so often just to watch that number get higher. Combining this with public-ness means that I can compare myself to everyone else with a number too, and I can get to the top of the leaderboard! I’m terribly competitive at gaming, and quantification is the first step towards gamification and competition. I’ve been watching the Quantified Self movement from afar specifically because I’m afraid that I’ll do nothing but ‘self-improve’ and then not use that improvement for anything worthwhile.</p>
<h2>Svbtle</h2>
<p>If you haven’t heard, <a href="http://dcurt.is/codename-svbtle">Svbtle</a> is Dustin Curtis’ blogging network. I remember reading that post when Dustin started it, and this was the part that stuck out to me:</p>
<blockquote>
<p>The goal is simple: when you see the Svbtle design, you should know that the content is guaranteed to be great.</p>
</blockquote>
<p>I’m always looking for good things to read on the Internet, so I kept this idea in the back of my head. When I saw a Svbtle blog, I raised my expectations. I can really enjoy bad things as long as I go into them expecting that they’re bad; I’m not the type of person that only enjoys high-brow stuff. And generally, blogs have low-brow expectations for me.</p>
<p>Anyway, after a couple of months, I noticed that I really did think that most posts on Svbtle were pretty damn good. At the very least, they weren’t bad in the way that many blog posts are: super flamebait, shit-talking, or half-baked. I also kept looking at my blog, and realized that many of my posts are sort of trivial, or not really up to my current standards, or have those kinds of flaws. And finally, I <em>hate</em> CSS. Like, I’m no good at it. At all. And I’m sick of messing around with my own style. But most blogging software sucks…</p>
<p>So I applied.</p>
<p>It’s sort of funny to talk about hating startups and centralized internet companies, then go from my own Jekyll blog to a hosted solution. There are a few huge wins for me with Svbtle, though:</p>
<p>Ultimately, it’s all posted on my own domain via the magic of CNAME. This means that in the future, I can re-host everything myself if I want to. I’ve already done this previously when I moved from Posterous to my own blog, and even have <a href="https://github.com/steveklabnik/blog/blob/master/redirects.rb#L32">redirects.rb</a> already set up. Secondly, Svbtle will send me an archive of all my posts if I ask, which means that I’m never going to lose the data. I will probably keep my own backups locally as well. But in the event that Svbtle goes south, I’m not locked in.</p>
<p>Getting kicked off of Svbtle for having bad content would make me really upset. If it’s a blog network that’s high quality, and my writing isn’t up to snuff, that would make me sad. Time to level up! Now that I’m there, I want to stay there. This gives me lots of motivation to write high quality things, and to improve. And Svbtle provides tools to help me become a better writer, like vetting and copy editing. That’s a pretty huge attraction to me.</p>
<p>Finally, there’s light gamification with the whole Kudos thing. I know that I’ll be trying to get as many as possible with each post, too, and that it can give me reasonable feedback in some fashion. Not as good as well-written responses, but it’s easier to give someone internet points than it is to write a full rebuttal.</p>
<h2>Two blogs</h2>
<p>I’m still going to keep this blog going; it’s gonna shift to something that’s more about code and less about politics. Words will be more politics and theory with a de-emphasis on code. This blog will be more like a Tumblr in that I’ll post more often, do less work on each post, and make the good but quick. Words will become a place where I’ll take my time, post less frequently, but really work on the polish.</p>
<p>I hope you’ll enjoy continuing to read the things I write about, here or there.</p>
</content:encoded></item><item><title>Protological Control: an introduction</title><link>https://steveklabnik.com/writing/protological-control-an-introduction/</link><guid isPermaLink="true">https://steveklabnik.com/writing/protological-control-an-introduction/</guid><pubDate>Mon, 13 Aug 2012 00:00:00 GMT</pubDate><content:encoded><blockquote>
<p>“I finished by pointing out that, like scientists, people in the web development community had to be ethically and morally aware of what they were doing. I thought this might be construed as a bit out of line by the geek side, but the people present were the ones now creating the Web, and therefore were the only ones who could be sure that what the systems produced would be appropriate to a reasonable and fair society.”Tim Berners-Lee, “Weaving the Web,” p86.</p>
</blockquote>
<p>Humans are social creatures. Regardless of your position on how human society should be, we all acknowledge that it does exist. While there may be some who live a lonely life in the wilderness, they still live within the larger social organization of a state. And even though those few people do exist, they are a tiny, tiny portion of the overall population. For the vast majority of humanity, society is a reality.</p>
<p>What do I mean when I say ‘society?’ For this, I turn to Marx:</p>
<blockquote>
<p>“Society does not consist of individuals, but expresses the sum of interrelations, the relations within which these individuals stand.”Karl Marx, Grundrisse der Kritik der Politischen Ökonomie</p>
</blockquote>
<p>Regardless of your opinions of Marx, I also don’t think this is a particularly controversial opinion: society is made up of our relationships with each other. A single person does not a society make, but our connections to each other. These connections are the foundational aspect of society; they are atomic.</p>
<p>Society is a fractal, rhizomatic object: there is not only the broader ‘society,’ but many smaller, overlapping, societies. These societies are a set of these relations within a given context. These contexts provide us with guidelines of what sorts of qualities these relations possess. For example, when amongst friends at home, a person will act differently than when around those same set of friends in a public place. Often, “social awkwardness” is a transgression of one of these boundaries; someone assumes that they exist in a different social space than those they’re interacting with.</p>
<blockquote>
<p>In an extreme view, the world can be seen as only connections, nothing else. We think of a dictionary as the repository of meaning, but it defines words only in terms of other words. I liked the idea that a piece of information is really defined only by what it’s related to, and how it’s related. There really is little else to meaning. The structure is everything. There are billions of neurons in our brains, but what are neurons? Just cells. The brain has no knowledge until connections are made between neurons. All that we know, all that we are, comes from the way our neurons are connected.Tim Berners-Lee, “Weaving the Web,” p12</p>
</blockquote>
<p>As it turns out, Tim isn’t the only person to find this line of thinking interesting. Ferdinand de Saussure was a linguist in the early 1900s who developed the idea of “structural linguistics,” and in the 1950’s and 1960’s, a French anthropologist named Claude Lévi-Strauss took his concepts and applied them to anthropology, birthing Structuralism. Others followed his lead and used this mode to develop an analysis of psychology, sociology, and more.</p>
<p>Societies exist naturally, but can also be man-made. Any opening of a new avenue for interpersonal relationships creates a new society amongst the multitude. The web is an example of this, and websites are a second, inner socius. The major difference between artificial and natural societies is not one of effect, but of cause. The end result is the same, but the initial conditions for the formation of the society determine the acceptable rules for the given relations that exist within it. Therefore, potential creators of said social enclosures should understand the power that they wield, and use said power to create the exact form of society they wish it make, with deliberate strokes.</p>
<h2>Society has a very clean mirror</h2>
<p>When people create, they end up creating something that reflects themselves and the societies that they live in. Linguistic relativity in even its weak form implies that language shapes who we are, and it is such with our creations. Many <em>ex</em> <em>nihilo</em> creation myths touch upon this property, like Adam’s receipt of God’s breath, or Atum from ancient Egypt, who not only created himself (?!?) but then other gods (first from his own shadow) and then man from his tears.</p>
<p>Here’s a fun, terrible example of this happening:</p>
<p>Amusingly, even though the tone is wrong, this YouTube user nails it:</p>
<p>(image lost to time)</p>
<p>Exactly. They didn’t test it on darker skinned folks. Of course, it may not be malicious, but this certainly meant that no dark-skinned people were involved in the production of this device, the whole way up to shipping it. If they were, it would have been caught early on before it was even beginning manufacture.</p>
<p>Because we grow up in a society, and we create new societies through technology, it stands to reason that society and our own biases influence the societies that we create. Even beyond that, if we create societies, we should be able to use the same tools that we use to analyze naturally occurring ones on artificial ones.</p>
<p>So how do we analyze societies?</p>
<h2>It’s all about power and control</h2>
<p>Remember Structuralism? Well, it has some issues, and not just the opposition that Saussure faced from Chomsky. Along comes a bunch of French philosophers, and they have beef. I won’t get into it, except to mention the name of one: Michel Foucault. He wrote this book titled “Surveiller et punir: Naissance de la Prison,” which has some awkward title translation issues, and so ends up as “Discipline and Punish: The Birth of the Prison” in English. Long time readers of my blog will remember <a href="http://blog.steveklabnik.com/posts/2011-01-22-a-few-notes-on-foucault">my notes</a> on this book, but the basics are as such:</p>
<h3>Foucault</h3>
<p>Feudal society was predicated on divine right. The body of a sovereign is literally the body of God, and since God also created everything, the sovereign is everything. Therefore, if you commit a crime, you commit it against the body of the sovereign, and therefore, he must exact punishment against your body in kind. Hence torture.</p>
<p>Eventually, though, torture became socially inconvenient, and people started challenging the very idea of divine right, and wanted democracy in some form. Therefore, the power to punish would need to take on a new form to survive in this new society. That mechanism is called “discipline.” Discipline (specifically of the body) is how control manifests itself in society. One of the great advances of Taylorism, for example, was studying how the bodies of assembly line workers operated and streamlining their motions.</p>
<p>Foucault also illustrates that control and power in modern society mirror Jeremy Bentham’s concept of the ‘Panopticon,’ which is a prison in a circular shape by which the rooms can all observe each other. The observation would help to keep them all in line, since the all-seeing tower in the middle would be able to observe everyone at all times.</p>
<p><img src="/img/2012-08-13/panopticon.png" alt="panopticon" /></p>
<p>I found this concept amusing, <a href="http://en.wikipedia.org/wiki/Litchfield_Towers">here</a> are the dorms I stayed in during my time at the University of Pittsburgh.</p>
<h3>Those in charge get to make the rules?</h3>
<p>Anyway, all of that implies this: the people who are ‘in charge’ are so because they get to define the rules. Most people would say this the other way around, but that’s wrong: You don’t get to make the rules because you’re in charge, you’re in charge because you get to make the rules. Regardless, what does it mean to make the rules?</p>
<p>In a simple sense, discipline is pretty easy to grasp: there’s a king (or boss). He makes the rules. Done. And it’s the same with our ‘creation of societies’ from above. When you make a society, you make it, so you get to make the rules! This ‘rule-making’ property is everywhere in software: operating systems are absolutely about creating a society of software programs on your computer. APIs are all about the connections between various systems, and the web, with its hyperlinks, even more explicitly so.</p>
<p>That’s why this stuff matters to programmers. We create societies all the time, and need to be careful of the kinds of societies that we create. We also participate in many, and need to be sure that we participate in the kinds of societies that we agree with.</p>
<p>One problem: we’ve already moved beyond discipline society, twice.</p>
<h2>Deleuze and Guattari</h2>
<p>Oh these two! What a pair! If you picked up a random copy of either volume of “Capitalism and Schizophrenia” without knowing what it was, you would be oh so confused. I mean, I vaguely knew what I was getting into, and I was still confused.</p>
<p>Deleuze specifically wrote an essay called “Postscript on the Societies of Control”. You can find it online <a href="http://www.n5m.org/n5m2/media/texts/deleuze.htm">here</a>.</p>
<blockquote>
<p>These are the societies of control, which are in the process of replacing disciplinary societies. “Control” is the name Burroughs proposes as a term for the new monster, one that Foucault recognizes as our immediate future.</p>
</blockquote>
<p>He also discusses it in <a href="http://www.amazon.com/Negotiations-1972-1990-Gilles-Deleuze/dp/0231075812">Negotiations</a>.</p>
<p>Control societies are different from disciplinary societies because discipline is all about confinement. Control is a slippery beast compared to discipline:</p>
<blockquote>
<p>Enclosures are molds, distinct castings, but controls are a modulation, like a self-deforming cast that will continuously change from one moment to the other, or like a sieve whose mesh will transmute from point to point.</p>
</blockquote>
<p>Discipline is direct, while control is indirect. I won’t get into control too much more here, because we’re past that too; control’s time on this earth was fleeting. Now we live in the era of protocol.</p>
<h2>Galloway</h2>
<p>Alexander Galloway is an Associate Professor of Media, Culture, and Communication at NYU, and every programmer should read him. Specifically, his book <a href="http://www.amazon.com/Protocol-Control-Exists-Decentralization-Leonardo/dp/0262072475">Protocol: How control exists after decentralization</a>. Galloway takes Deleuze’s work and applies it to our modern computer systems, and terms that ‘protocol.’</p>
<blockquote>
<p>Protocol is a system of distributed management.Protocol facilitates peer-to-peer relationships between autonomous entities.Protocol is anti-hierarchy and anti-authority.Protocol engenders localized decision making, not centralized.Protocol is robust, flexible, and universal.Protocol can accommodate massive contingency.Protocol is the outcome (not the antecedent) of distributed power.Galloway, “Protocol”, p82</p>
</blockquote>
<p>Sounds awesome, right? Sounds like the web.</p>
<blockquote>
<p>I saw one [advocate of informational organization systems] after the next shot down because the developers were forcing them to reorganize their work to fit the system. I would have to create a system with common rules that would be acceptable to everyone. This meant as close as possible to no rules at all.This notion seemed impossible until I realized that the diversity of different computer systems could be a rich resource– something to be represented, not a problem to be eradicated. The model I chose for my minimalist system was hypertext.Tim Berners-Lee, “Weaving the Web,” p15.</p>
</blockquote>
<p>Tim recognized that centralization was the root of exploitation:</p>
<blockquote>
<p>It also shows how a technical decision to make a single point of reliance can be exploited politically for power and commercially for profit, breaking the technology’s independence from those things, and weakening the web as a universal space.Tim Berners-Lee, “Weaving the Web,” p129.</p>
</blockquote>
<p>Tim saw that control was an issue:</p>
<blockquote>
<p>Whether inspired by free-market desires or humanistic ideals, we all felt that control was the wrong perspective. … Technically, if there was any centralized point of control, it would rapidly become a bottleneck that restricted the web’s growth, and the web would never scale up. Its being ‘out of control’ was very important.Tim Berners-Lee, “Weaving the Web,” p99</p>
</blockquote>
<p>Galloway, however, contends something else, and this is the crux of it:</p>
<blockquote>
<p>Thus it is an oversight for theorists like Lawrence Lessig (despite his strengths) to suggest that the origin of Internet communication was one of total freedom and lack of control. Instead, it is clear to me that the exact opposite of freedom – that is, control – has been the outcome of the last forty years of developments in networked communications. The founding principle of the Net is control, not freedom. Control has existed from the beginning.Perhaps it is a different type of control than we are used to seeing. It is control borne from high degrees of technical organization (protocol) not this or that limitation on individual freedom or decision making (fascism).To put it another way, in order for protocol to enable radically distributed communications between autonomous entities, it must employ a strategy of universalization, and of homogeneity. It must be anti-diversity. It must promote standardization in order to enable openness. It must organize peer groups into bureaucracies like the IETF in order to create free technologies.In short, control in distributed networks is not monolithic. It proceeds in multiple, parallel, contradictory, and often unpredictable ways. It is a complex of interrelated currents and counter-currents.Galloway, “Protocol”, p141-143.</p>
</blockquote>
<p>This is the kind of control that startups exert on the rest of the world. This is the kind of control that’s hard to see coming, because it’s inexact. This is the kind of control that lets you <em>feel</em> like you have freedom, even when you don’t. This is the kind of control that <em>likes</em> open standards, and uses them against you.</p>
<p>Discipline is Microsoft. Protocol is Google. Protocol is Facebook. Protocol is Twitter.</p>
<p>It’s in this tradition that I critique App.net and GitHub.</p>
</content:encoded></item><item><title>DayZ</title><link>https://steveklabnik.com/writing/dayz/</link><guid isPermaLink="true">https://steveklabnik.com/writing/dayz/</guid><pubDate>Wed, 25 Jul 2012 00:00:00 GMT</pubDate><content:encoded><p>I’ve recently come across one of the most addicting, interesting experiences I’ve ever had online: <a href="http://dayzmod.com/">DayZ</a>. I hesitate to call it a ‘video game,’ because while it is a mod for a first person shooter game, it basically ignores all common wisdom about what makes for a good game. I think that the broader implications are also interesting across other domains: ‘common wisdom’ isn’t always correct. Sometimes communities reach local maximums, and something drastic needs to be done to break out of the rut. So what is DayZ, anyway?</p>
<p>Essentially, DayZ is a simulation of a zombie apocalypse.</p>
<p>You wake up on a beach with essentially nothing in your pockets. This means that you have no way to defend yourself, and no supplies. Check out the bottom right of that screenshot: See that canteen and knife and fork? Those are water and food indicators, and if you don’t eat or drink often enough, you’ll faint from fatigue, and even lose blood (which is a proxy for health, and is the middle indicator). So you have to sneak around until you can find a hatchet, food, and water. See those two things on the upper right? That’s how much audio and visual ‘noise’ you give off. Right now, this player is basically invisible, so they’re safe.</p>
<p>The game takes realism very seriously. There’s the food and water bit, but this also isn’t some Left 4 Dead run and gun bullshit: if you take damage, you’ll start to bleed, and will need to bandage yourself or bleed out. If you get shot, you’re basically dead. You <em>might</em> be able to take two bullets from small arms. I already mentioned the food and water. You can break bones, which seriously limits mobility, and will probably cause the shakes. Better take some painkillers! There are a few vehicles, but they’re all in disrepair, so find some parts, fix them up, and then stop and put some gas in them! Voice chat only works for a 40 yard radius around where you stand, so you can’t talk to people that are far away. You don’t have a map at first, so you won’t know where the hell you are until you find one. Of course, you can read the street signs, so obviously, you’ll know when you’re in черногорск. ;). (That’s ‘Cherno’ as it’s affectionately called by players, because ‘Chernogorsk’ is too long to say.) Oh, and when you die, you die. You lose all your stuff. This also adds to the crazy tenseness of the game. There’s actually 24 hours on each server, so if you play on one with your GMT offset, it’ll be light when it’s light and dark when it’s dark outside.</p>
<p>The biggest thing, though, is that this is a true sandbox: there are no ‘levels’ or ‘experience points’ or ‘missions’ or ‘objectives.’ You just play. Average survival time is about 45 minutes, and as you live longer, the tension goes way, way up. Because there are no missions, you develop a real attachment to the ‘emergent stories’ that come out of the game, which is the big, big draw for me. Here’s an example:</p>
<p>I spawned just outside of Электрозаводск (‘Electro’), and one of my roommates had been playing for about twenty minutes, and was in Пута (“Puta”), just north. He had found a barn with two hatchets in it, and obviously, as a new player, that’s a big deal. Электро and Пута are about <a href="http://dayzdb.com/map#4.099.124">two kilometers away from each other</a>, so he decided to wait for me. I ran up over that hill, and as I got closer, I didn’t pay close enough attention, and a zombie started chasing me. I decided to sprint to the barn, so I’d at least be able to defend myself, and Kyle decided to brace himself: my running would obviously draw more zombies, and the barn had three doors: we couldn’t cover them all. If we weren’t careful, this could be the end.</p>
<p>As I sprint through the door, Kyle steps into it, and swings his hatchet. “Holy shit dude, more coming, hurry up!” I quickly located the other one, as Kyle took out the second undead trying to eat us. I stepped up too, and covered the other door. Thirty seconds of frantic hatchet-ing later, we were safe. But Kyle was starting to get thirsty, and Электро is really dangerous. We searched the rest of the settlement, but no luck. No water to be found. We decided that rather than risk the big costal town, it’d be best to <a href="http://dayzdb.com/map#4.085.109">make the two klick journey north to Могилевка</a>. While trying to locate the road, Kyle attracted the attention of another zombie, and after dispatching it, found an empty watter bottle! Luckily, Пута has a public well, so he could fill it and drink up. Problem solved.</p>
<p>Now, if this scenario had been a scripted part of some zombie RPG, I wouldn’t have cared about it at all. I’ve now told this story three times in person, and it doesn’t get old. This tense five minutes emerged naturally out of putting players in the world and letting them go. As a result, since they’re <em>your</em> stories, you care about them a lot more. People end up sharing these stories, too: <a href="http://www.youtube.com/watch?v=0oMdOPF8a8U">Check out this Jeep theft on YouTube</a>. He spends the first 3 minutes or so telling the story.</p>
<h2>Ignoring video game tropes</h2>
<p>So what really makes DayZ special? There are lots of sandbox games, and there are lots of zombie games. Well, for me, it’s that DayZ ignores much about what makes for conventional games.</p>
<p>See, I’ve been playing video games for most of my life. I started with an NES long ago, and spent many, many, many hours with various games across genres. I play fewer games these days, and actually took a pretty big break from them over the last two years or so. I would go to GameStop, look at what was offered, shrug, and walk out. It’s not that I’m growing up <a href="http://www.theesa.com/facts/pdfs/ESA_EF_2011.pdf">(the average gamer is 37)</a>, it’s that I’ve played all these games before. I spent over a hundred hours with Diablo II, so why bother with Diablo III?</p>
<p>When there’s a game that breaks out of these standard genres and franchises, I get excited. I really got into music games until Activision drove Guitar Hero into the ground, Minecraft was incredibly addicting, and now DayZ with its emergent gameplay, focus on realism, and tense atmosphere.</p>
<h3>Games need to be accessible</h3>
<p>DayZ does not have mass appeal. Its uncompromising approach to realism means that it will alienate many people. You will die often, and it will feel random much of the time. You will get killed by other players. You’ll break a bone miles away from the nearest hospital with no morphine, and have to crawl for an hour or die.</p>
<h3>Games should be polished</h3>
<p>Frankly, <a href="http://www.dayzwiki.com/wiki/index.php?title=Known_Bugs">DayZ is buggy as shit</a>. And not trivial ones, either:</p>
<blockquote>
<p>Dropping Items can cause them to disappear</p>
</blockquote>
<p>Honestly, in many ways, it makes the game even more tense. Bad things can happen to you at any time. The world is scary.</p>
<p>The interface sucks. This is more of an ARMAII issue, but one of my roommates was trying to get something out of another roommate’s backpack, and since you both click to manipulate the UI and click to shoot, accidentally shot him in the stomach. This totally could happen in a zombie apocalypse: fumble around with the safety off… the interface can be confusing, seemingly arbitrary, and complicated. Still totally worth it.</p>
<p>In many ways, this is the <a href="http://www.jwz.org/doc/worse-is-better.html">worse is better</a> principle.</p>
<h3>More is better</h3>
<p>Many games are ‘just like this other game, but with more stuff.’ DayZ has a really pleasant minimalism in many ways: the HUD is basically nonexistent, the core gameplay is pretty simple, and while there are a lot of details, most of them are not too complicated: break a leg, use some morphine. Hungry? Eat some food.</p>
<p>But these simple components come together in a way that gives the game a large amount of depth. But that depth is not from some big crazy feature that took weeks to build.</p>
<p>In fact, the author of the mod has <em>removed</em> entire features. There was a ‘humanity’ system at one time, and ‘bandits,’ which were players that killed other players and had their skin changed. That’s been taken out. I didn’t play the game with that particular feature in, but the author isn’t afraid to cut stuff out if it isn’t a good idea, even if he’s spent lots of time building the feature.</p>
<h3>A good story matters</h3>
<p>Most simulation games have some sort of goal. Even many ‘sandbox’ games, like Grand Theft Auto, have some sort of points, goals, or missions, and you’re free to ignore them, but they’re there. Minecraft is a great example of a true sandbox game, and I’m not suggesting that DayZ has a monopoly here. But it’s a genre that doesn’t have that many games in it, so another entry is very welcome; the genre isn’t as tired as many others.</p>
<p>Story is often used as a reason for progression; as you go through the story, you get bigger, better, and generally more powerful. DayZ manages to have progression without a story; there’s sort of five broad stages to the game: pre-hatchet, hatchet, gun, tent, PvP. This is hard to do! A story provides nice justification. But DayZ doesn’t need it.</p>
<p>Also, as I mentioned earlier, you sort of invent stories as you play. They emerge. And therefore, you care about <em>your</em> DayZ story more than the one that would have been concocted if this game was not a sandbox.</p>
<h3>A player should be quite powerful</h3>
<p>Often, the player is a total badass. Video games are often dripping in machismo, and so they are <strong>THE ULTIMATE DESTROYER OF WORLDS</strong>. Maybe, after killing hundreds of enemy soldiers, getting worn down over time, one gets a lucky shot and you go down.</p>
<p>DayZ is different. You are weak. The environment is dangerous. Even though you can kill zombies, often, the noise attracts others, and they get dangerous in packs. And you can always run out of ammo…</p>
<p>Also, other players are just as desperate and dangerous as you are. Sometimes, you’ll be friendly, but more often, you’re gonna end up getting shot. After all, if you’re not friendly, <em>I</em> could get shot, so maybe I’d better take you out first…</p>
<h2>DayZ is awesome</h2>
<p>Seriously, you should give it a try. ARMAII goes on sale on Steam often, I was able to pick it up for $17 the other day. If you’re bored with the mainstream of gaming, it just might be enough of a shake-up to be interesting to you.</p>
</content:encoded></item><item><title>Sparrow</title><link>https://steveklabnik.com/writing/sparrow/</link><guid isPermaLink="true">https://steveklabnik.com/writing/sparrow/</guid><pubDate>Sat, 21 Jul 2012 00:00:00 GMT</pubDate><content:encoded><p>I purchased Sparrow back when it was $9, and didn’t find it particularly amazing. I did like it, but ended up using the web interface more. I certainly got my $9 of value out of it, though, even though I used it for a short time.</p>
<p>However.</p>
<p>There’s been a lot of complaining about <a href="http://sprw.me/">Sparrow’s acquisition</a>, and a lot of complaining about complaining. Most of the complaining about the complaining argues that those who complain should feel like me: You bought it, you probably got a ton of value out of it. You can’t (and shouldn’t) complain.</p>
<p>But.</p>
<p>Theoretically, a market works through feedback mechanisms. That’s the whole ‘free market’ argument, right? People that participate in the market come together, the magic happens, and everything gets allocated correctly. Vote with your dollars. Shop somewhere else, and poof! A store that’s too expensive won’t be able to keep its doors open. A triumph for ‘free commerce.’</p>
<p>So why does everyone get all huffy when certain people provide the market with feedback about the way that business has been conducted?</p>
</content:encoded></item><item><title>Why I don't like factory_girl</title><link>https://steveklabnik.com/writing/why-i-don-t-like-factory_girl/</link><guid isPermaLink="true">https://steveklabnik.com/writing/why-i-don-t-like-factory_girl/</guid><pubDate>Sat, 14 Jul 2012 00:00:00 GMT</pubDate><content:encoded><p>Once upon a time, I was building my First Serious Rails App. I was drawn to Rails in the first place because of automated testing and ActiveRecord; I felt the pain of not using an ORM and spending about a week on every deploy making sure that things were still okay in production. So of course, I tried to write a pretty reasonable suite of tests for the app.</p>
<p>To gloss over some details to protect the innocent, this app was a marketplace: some people owned Things, and some people wanted to place Orders. Only certain Things could fulfill an Order, so of course, there was also a ThingType table that handled different types of Things. Of course, some Types came in multiple Sizes, so there also needed to be a Size Table and a ThingTypeSize table so that a User could own a Thing of a certain Type and a certain Size.</p>
<p>Stating that creating my objects for tests was difficult would be an understatement.</p>
<p>Then I read a blog post about FactoryGirl. Holy crap! This would basically save me. With one simple <code>Factory(:thing)</code> I could get it to automatically build a valid list of all that other crap that I needed!</p>
<p>So of course, I had to write my spec for a thing:</p>
<pre><code>describe Order do
it "generates quotes only from Things that are of the right size" do
order = Factory(:order)
thing = Factory(:thing, :size =&gt; order.size)
thing = Factory(:thing)
order.quote!
order.quote.thing.should == thing
end
end
</code></pre>
<p>This test worked. It also generated around 15 objects, saved them in the database, and queried them back out. I don’t have the code running anymore, but it was like 30-40 queries, and took a second or two to run.</p>
<p>That was one test. I was trying to test a lot, even though I wasn’t good at test first yet, so my suite got to be pretty big. Also, sometimes my factories weren’t the best, so I’d spend a day wondering why certain things would start failing. Turns out I’d defined them slightly wrong, validations started to fail, etc.</p>
<h2>How did we get here?</h2>
<p>This story is one of Ruby groupthink gone awry, basically. Of course, we know that fixtures get complicated. They get complicated because we have these crazy ActiveRecord models, don’t use plain Ruby classes when appropriate, and validations make us make extra objects just to get tests to pass. Then fixtures get out of date. So let’s introduce a pattern!</p>
<p>Of course, since we know that Factories are really useful when things get complicated, let’s make sure to use them from the start, so we don’t have to worry about them later. Everyone started doing this. Here’s how new Rails apps get started:</p>
<pre><code>steve at thoth in ~/tmp
$ rails new my_app
create
create README.rdoc
&lt;snip&gt;
create vendor/plugins/.gitkeep
run bundle install
Fetching gem metadata from https://rubygems.org/.........
&lt;snip&gt;
Using uglifier (1.2.6)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
steve at thoth in ~/tmp
$ cd my_app
steve at thoth in ~/tmp/my_app
$ cat &gt;&gt; Gemfile
gem "rspec-rails"
gem "factory_girl_rails"
^D
steve at thoth in ~/tmp/my_app
$ bundle
Fetching gem metadata from https://rubygems.org/........
&lt;snip&gt;
steve at thoth in ~/tmp/my_app
steve at thoth in ~/tmp/my_app
$ rails g rspec:install
create .rspec
exist spec
create spec/spec_helper.rb
$ rails g resource foo
invoke active_record
create db/migrate/20120714180554_create_foos.rb
create app/models/foo.rb
invoke rspec
create spec/models/foo_spec.rb
invoke factory_girl
create spec/factories/foos.rb
invoke controller
create app/controllers/foos_controller.rb
invoke erb
create app/views/foos
invoke rspec
create spec/controllers/foos_controller_spec.rb
invoke helper
create app/helpers/foos_helper.rb
invoke rspec
create spec/helpers/foos_helper_spec.rb
invoke assets
invoke coffee
create app/assets/javascripts/foos.js.coffee
invoke scss
create app/assets/stylesheets/foos.css.scss
invoke resource_route
route resources :foos
steve at thoth in ~/tmp/my_app
$ rake db:migrate
== CreateFoos: migrating =====================================================
-- create_table(:foos)
-&gt; 0.0065s
== CreateFoos: migrated (0.0066s) ============================================
$ cat &gt; spec/models/foo_spec.rb
require 'spec_helper'
describe Foo do
it "does something" do
foo = Factory(:foo)
foo.something!
end
end
^D
$ bundle exec rake spec
/Users/steve/.rvm/rubies/ruby-1.9.3-p194/bin/ruby -S rspec ./spec/controllers/foos_controller_spec.rb ./spec/helpers/foos_helper_spec.rb ./spec/models/foo_spec.rb
*DEPRECATION WARNING: Factory(:name) is deprecated; use FactoryGirl.create(:name) instead. (called from block (2 levels) in &lt;top (required)&gt; at /Users/steve/tmp/my_app/spec/models/foo_spec.rb:5)
F
Pending:
FoosHelper add some examples to (or delete) /Users/steve/tmp/my_app/spec/helpers/foos_helper_spec.rb
# No reason given
# ./spec/helpers/foos_helper_spec.rb:14
Failures:
1) Foo does something
Failure/Error: foo.something!
NoMethodError:
undefined method `something!' for #&lt;Foo:0x007f82c70c07a0&gt;
# ./spec/models/foo_spec.rb:6:in `block (2 levels) in &lt;top (required)&gt;'
Finished in 0.01879 seconds
2 examples, 1 failure, 1 pending
Failed examples:
rspec ./spec/models/foo_spec.rb:4 # Foo does something
Randomized with seed 27300
rake aborted!
/Users/steve/.rvm/rubies/ruby-1.9.3-p194/bin/ruby -S rspec ./spec/controllers/foos_controller_spec.rb ./spec/helpers/foos_helper_spec.rb ./spec/models/foo_spec.rb failed
Tasks: TOP =&gt; spec
(See full trace by running task with --trace)
</code></pre>
<p>Woo! Failing test! Super easy. But what about that test time?</p>
<pre><code>Finished in 0.01879 seconds
</code></pre>
<p>Now, test time isn’t everything, but a hundredth of a second. Once we hit a hundred tests, we’ll be taking almost two full seconds to run our tests.</p>
<p>What if we just <code>Foo.new.something!</code>?</p>
<pre><code>Finished in 0.00862 seconds
</code></pre>
<p>A whole hundredth of a second faster. A hundred tests now take one second rather than two.</p>
<p>Of course, once you add more complicated stuff to your factories, your test time goes up. Add a validation that requires an associated model? Now that test runs twice as slow. You didn’t change the test at all! But it got more expensive.</p>
<p>Now, a few years in, we have these massive, gross, huge, long-running test suites.</p>
<h2>What to do?</h2>
<p>Now, I don’t think that test times are the end-all-be-all of everything. I really enjoyed <a href="http://gmoeck.github.com/2012/07/09/dont-make-your-code-more-testable.html">this post</a> that floated through the web the other day. I think that the ‘fast tests movement’ or whatever (which I am/was a part of) was a branding mistake. The real point wasn’t about fast tests. Fast tests are really nice! But that’s not a strong enough argument alone.</p>
<p>The point is that we forgot what testing is supposed to help in the first place.</p>
<h2>Back to basics: TDD</h2>
<p>A big feature of tests is to give you feedback on your code. Tests and code have a symbiotic relationship. Your tests inform your code. If a test is complicated, your code is complicated. Ultimately, because tests are a client of your code, you can see how easy or hard your code’s interfaces are.</p>
<p>So, we have a pain in our interface: our objects need several other ones to exist properly. How do we fix that pain?</p>
<p>The answer that the Rails community has taken for the last few years is ‘sweep it under the rug with factories!’ And that’s why we’re in the state we’re in. One of the reasons that this happened was that FactoryGirl is a pretty damn good implementation of Factories. I do use FactoryGirl (or sometimes Fabrication) in my request specs; once you’re spinning up the whole stack, factories can be really useful. But they’re not useful for actual unit tests. Which I guess is another way to state what I’m saying: we have abandoned unit tests, and now we’re paying the price.</p>
<p>So that’s what it really boils down to: the convenience of factories has set Rails testing strategies and software design back two years.</p>
<p>You live and learn.</p>
</content:encoded></item><item><title>How can I contribute to Ruby on Rails?</title><link>https://steveklabnik.com/writing/how-can-i-contribute-to-ruby-on-rails-/</link><guid isPermaLink="true">https://steveklabnik.com/writing/how-can-i-contribute-to-ruby-on-rails-/</guid><pubDate>Thu, 05 Jul 2012 00:00:00 GMT</pubDate><content:encoded><p>After RailsConf this year, I joined the Rails Issue Team. This means that I help triage issues that people have filed, try to reproduce errors, and point core team members at ones that are most important. Since doing that, a few people have asked me how to get started, so I decided to draw up my thoughts here.</p>
<p>Note that there is also an <a href="http://edgeguides.rubyonrails.org/contributing_to_ruby_on_rails.html">official Rails Guide on contribution</a> too.</p>
<h2>Who helps with Rails?</h2>
<p>First up is the Rails Core team: <a href="http://rubyonrails.org/core/">http://rubyonrails.org/core/</a> Core team members essentially have the authority to merge code into the main Rails codebase. Someone from core reviews every patch that goes into Rails.</p>
<p>Issues team: This group of people has the authority to tag, close, and edit tickets, but can’t actually merge code.</p>
<p>Documentation team: A few awesome people contribute by writing documentation rather than just code. Docs are super important, so all these people are also awesome.</p>
<p>Everyone else!: Rails is open source after all, and 1224 people have contributed at least one patch.</p>
<p>You can find everyone’s commits and count here: <a href="http://contributors.rubyonrails.org/">http://contributors.rubyonrails.org/</a></p>
<h3>About payment</h3>
<p>Nobody is paid to just work on Rails itself. Everyone is a volunteer. Rails runs on a ‘scratch your own itch’ policy. This has two big effects:</p>
<ol>
<li>Pull requests can sit sometimes. Pull requests must be merged by a Core member, so after your patch is submitted, you’re basically waiting on someone from a group of about 12 to decide ‘hey, I should merge some pull requests today’ and get on it. There are currently 168 pull requests open on Rails, the oldest of which is three months since its last comment.</li>
<li>Feature requests may simply fall on deaf ears. If you have a suggestion for Rails, and you don’t want to implement it, you need to convince someone to write the code for you. But it’s not as if the core team has a list of requested features that it goes over when working. Of course, they have a list of the features that they want to add, but it’s not like it’s comprehensive from every feature requested. That said, it’s possible that your idea may spark the attention of someone, so sharing a feature request is valuable. But don’t feel entitled here.</li>
</ol>
<h2>I found a bug, what should I do?</h2>
<p>File a ticket on the <a href="https://github.com/rails/rails/issues">Rails issue tracker</a> and give us as much information as possible.</p>
<p>Note that only 3.2 is getting bug fixes. 3.1 and earlier are security fix only.</p>
<p>Good: Clear bug report, listed versions of Rails and/or Ruby and backtrace if applicable</p>
<p>Better: Steps to reproduce, maybe with a sample app.</p>
<p>Best: All of the above with a patch that fixes the problem.</p>
<h2>I’ve found a security vulnerability, what should I do?</h2>
<p>Don’t submit something to the tracker. Instead, check out <a href="http://rubyonrails.org/security">this page</a> which outlines the security policy. Basically, send an email to <a href="mailto:[email protected]">[email protected]</a>.</p>
<h2>I have a feature request, what should I do?</h2>
<p>Please don’t file an Issue; it’s a bad medium for discussing these kinds of things. It also makes it harder to keep track of what’s a bug and what’s a wish for future changes. Please post to the <a href="https://groups.google.com/forum/?fromgroups#!forum/rubyonrails-core">rails-core mailing list</a> instead. Your idea is much more likely to be seen and discussed there.</p>
<p>You may get no response from the list. People tend to say nothing if they’re not interested. It happens.</p>
<h2>I want feedback on an idea before I work up a patch</h2>
<p>Same as a feature request, please ping the <a href="https://groups.google.com/forum/?fromgroups#!forum/rubyonrails-core">rails-core mailing list</a>.</p>
<h2>I’ve written a patch, now what?</h2>
<p>Submit a pull request with your patch. Someone from core will review it, and may ask you to modify what you’ve done. Eventually it will either get merged into Rails or rejected.</p>
<h2>I want to write docs, now what?</h2>
<p>Don’t sumbit them as issues to the main Rails repo, instead, check out <a href="http://weblog.rubyonrails.org/2012/3/7/what-is-docrails/">docrails</a>.</p>
<h2>I want to help, but don’t know what to work on!</h2>
<p>We try to tag each issue with the part of Rails it affects. If you don’t know where to get started, pick your favorite part of Rails and sort the issues by that part. For example <a href="https://github.com/rails/rails/issues?labels=asset+pipeline&amp;page=1&amp;sort=updated&amp;state=open">here are all the issues relating to the asset pipeline</a>.</p>
<p>Try to reproduce the bug locally, write a test that fails, write a patch, boom!</p>
<h2>How do I set up my local machine to work on Rails</h2>
<p>Check the Rails Guide <a href="http://edgeguides.rubyonrails.org/contributing_to_ruby_on_rails.html#running-the-test-suite">here</a>.</p>
</content:encoded></item><item><title>Introducing issue2pr</title><link>https://steveklabnik.com/writing/introducing-issue2pr/</link><guid isPermaLink="true">https://steveklabnik.com/writing/introducing-issue2pr/</guid><pubDate>Fri, 29 Jun 2012 00:00:00 GMT</pubDate><content:encoded><p>Ever had an issue on GitHub, and then you fixed the bug? Opened a brand new pull request, which becomes a new issue, and then you have to remember to link the two via a comment, or manually go back and close the issue as well?</p>
<p>Turns out that GitHub has the power to turn Issues into Pull Requests in the API, but not in the web UI. So I wrote a tiny web app that lets you use the API call yourself: <a href="http://issue2pr.herokuapp.com/">http://issue2pr.herokuapp.com/</a></p>
<p>Of course, the code is up on GitHub, here: <a href="https://github.com/steveklabnik/issue2pr">https://github.com/steveklabnik/issue2pr</a></p>
<p>Issues (and pull requests) welcome!</p>
</content:encoded></item><item><title>Show solidarity to all who resist</title><link>https://steveklabnik.com/writing/show-solidarity-to-all-who-resist/</link><guid isPermaLink="true">https://steveklabnik.com/writing/show-solidarity-to-all-who-resist/</guid><pubDate>Thu, 31 May 2012 00:00:00 GMT</pubDate><content:encoded><p>I found this on a flyer in my neighborhood.</p>
<hr />
<p>On may 19th, three people associated with the Occupy movement were arrested in Chicago, just before the anti-NATO protest. Two infiltrators had been working with them for months, trying to persuade them to escalate their tactics – just as in the recent Cleveland bomb scare. Whether the infiltrators would have succeeded in pushing them further than they wanted to go, we’ll never know. Desperate to make an arrest that would intimidate the NATO protesters, divide the Occupy movement from its most radical members, and create an impressive spectacle, the police attacked. Their most substantial evidence is beer-making supplies at the house the NATO 3 were arrested in, which they ridiculously claim are bomb materials.</p>
<p>While we are confident that the NATO 3 will beat these ludicrous charges, they are being held in solitary confinement on 1.5 million dollars bond each. It often comforts prisoners to know that people on the outside love them and are thinking about them, so a solidarity demonstration was called for May 24th, in St Louis, MO. It was noissy, vibrant, nighttime march, enjoyed by the people it passed. “Solidarity with all who resist!” was painted on a business window, and, of course, a businessman got angry, and, of course, there was an altercation; the cops showed up, hit people, were allegedly hit in return, and so on. By the end of the night, ten people were in jail – one charged with five felony counts of assault, property damage and riot, as well as two misdemeanors. Others are charged with felony resisting, interference, and misdemeanor riot. Upon their release, we learned that while being interrogated one prisoner had a knife held to their throat, that they were choked and lifted off the ground and punched in the chest for non-compliance. One comrade, who is facing the most serious charges, was brought to the interrogation room where a detective repeatedly struck him in the face, breaking his nose while he was shackled to the floor. They continued to make threats of violence if he continued to “not comply.”</p>
<p>Fuck.</p>
<p>This.</p>
<p>Shit.</p>
<p>As a wise woman once said, “If you hit me and I hit back, we still ain’t even.” There is no way in society as it exists today to have a fair fight with the police; they have vast, systemic power, and will always be better equipped and better financed than we are. Nor can we let them hurt anyone, particularly those close to us, with complete impunity, without a response from us. We must fight, but we must fight asymmetrically. If we must respond, we are forced into reaction instead of attack, let it be waves of reaction – if strange and inadequate, funny and off putting – that build off each other to create an ocean of solidarity. Solidarity actions in response to the repression of solidarity actions with people we’ve never met accused of crimes that never occurred – this is utterly bizarre, but this is our life.</p>
<p>You should care about this. You should care that the government is making tactical strikes against its enemies. Maybe you don’t think of yourself as an enemy of the government or capitalism; maybe you would never consider it publicly, in a quiet, accepted way or otherwise. Still, think about your life: do you feel safer, happier, more fulfilled because of your job, the police, the taxes you pay? Even if you feel those things on a surface level, do you feel like something is missing, that you could be living in a way that is more glorious and beautiful? Do you see people around you who suffer, or read books about terrible things in America’s past, or hear of people in far-off lands whose lives seem impossibly hard, and wish that you could do something about it – something more meaningful than giving to a charity or voting in the election? We suggest a first step:</p>
<p>Show solidarity to all who resist.</p>
</content:encoded></item><item><title>Finale</title><link>https://steveklabnik.com/writing/finale/</link><guid isPermaLink="true">https://steveklabnik.com/writing/finale/</guid><pubDate>Wed, 30 May 2012 00:00:00 GMT</pubDate><content:encoded><p>This morning, my mother woke up and found that my father had passed away in the night.</p>
<p>Thanks for all the love and support you all have shown me over the last two months. It’s been hard, but you all have made it easier.</p>
<p>Don’t be worried if you don’t see me around a lot for the next few days. Just trying to take some time off from the hustle to help my family heal. And to just play video games and be a bit mindless.</p>
<p>No, seriously. Thank you.</p>
</content:encoded></item><item><title>Shoes 4 work is starting: get involved!</title><link>https://steveklabnik.com/writing/shoes-4-work-is-starting--get-involved/</link><guid isPermaLink="true">https://steveklabnik.com/writing/shoes-4-work-is-starting--get-involved/</guid><pubDate>Fri, 25 May 2012 00:00:00 GMT</pubDate><content:encoded><p>As you know, I’ve been doing a lot of work over the last few years to keep _why’s <a href="http://shoesrb.com/">Shoes</a> project going. A few other intrepid individuals have been doing a lot as well. We’re starting to work on Shoes 4, and we’d love your help.</p>
<h2>A little about Shoes</h2>
<p>If you didn’t know, Shoes is a GUI toolkit for Ruby. It uses blocks heavily:</p>
<pre><code>Shoes.app do
para "Push the button"
button "Me! Me!" do
alert "Good job."
end
end
</code></pre>
<p>Super simple. You get native widgets on all platforms.</p>
<h2>Shoes 4</h2>
<p>Shoes 4 is a total re-write of Shoes. It’s being done in JRuby. There are a few related projects:</p>
<ul>
<li>ShoesSpec: An executable specification for Shoes implementations, so that others can co-exist. A few other Shoes projects exist already, and a unified interface will be great.</li>
<li>Shoes-mocks: Sort of the opposite, a dummy set of Shoes that you can use to test applications written in Shoes.</li>
<li>The website: A new website is being made as well. Help here would be just as awesome as work on Shoes itself.</li>
</ul>
<h2>What we need</h2>
<p>Of course, pull requests are always welcome. But if you use Shoes or want to, please get involved in <a href="http://github.com/shoes/shoes4/issues">the discussions</a> about what Shoes 4 will look like on the bug tracker. Feel free to open an Issue with any topic for discussion. Thoughts from users, current and potential, are always awesome!</p>
<p>Let’s all have fun developing Shoes 4!</p>
</content:encoded></item><item><title>Mixins: a refactoring anti-pattern</title><link>https://steveklabnik.com/writing/mixins--a-refactoring-anti-pattern/</link><guid isPermaLink="true">https://steveklabnik.com/writing/mixins--a-refactoring-anti-pattern/</guid><pubDate>Mon, 07 May 2012 00:00:00 GMT</pubDate><content:encoded><p>I spend an unusually large amount of time thinking about interactions between what I call ‘past me’ and ‘future me.’ It seems that my life changes significantly every few years, and I like to ground myself by imagining how odd it would be if ‘current me’ could tell ‘past me’ things like ‘<a href="http://www.oscon.com/oscon2012/public/schedule/detail/24042">Someday, you’ll be speaking at OSCON</a>.’</p>
<p>It’s not always general life stuff, though, it’s often a trick or technique with code. How many times have you learned something new, and then had that terrible urge to go back and re-write all of that terrible, terrible code you’ve written in the past? This happens to me constantly.</p>
<p>So here, let me show you something I realized that I used to do wrong all the time.</p>
<p>Let’s do it via a quiz. Of course, because I’m framing it this way, you’ll get the answer, but seriously, just play along.</p>
<h2>Two factorizations</h2>
<p>Which code is better factored?</p>
<pre><code># app/models/post.rb
class Post
include Behaviors::PostBehavior
end
# app/models/behaviors/post_behavior.rb
module Behaviors
module PostBehavior
attr_accessor :blog, :title, :body
def initialize(attrs={})
attrs.each do |k,v| send("#{k}=",v) end
end
def publish
blog.add_entry(self)
end
# ... twenty more methods go here
end
end
</code></pre>
<p>or</p>
<pre><code>class Post &lt; ActiveRecord::Base
def publish
blog.add_entry(self)
end
# ... twenty more methods go here
end
</code></pre>
<p>One line of reasoning asserts that the first example is better. Here’s why: all of the behavior is in one place, and the persistence is in another. A clear win.</p>
<p>Another line of reasoning asserts that the second is better. Here’s why: it’s significantly simpler. A clear win.</p>
<p>So which is right?</p>
<h2>They’re both wrong</h2>
<p>Both of these justifications are wrong. As far as I’m concerned, these two classes are equally complex, but because there is a lot more ceremony in version one, version two is preferable. Assertion one is wrong because both are still in the same place, they’re just in two different files. Assertion two is wrong because it’s not simpler than version one, it’s equivalently complex.</p>
<h2>Measures of complexity</h2>
<p>Whenever we refactor, we have to consider what we’re using to evaluate that our refactoring has been successful. For me, the default is complexity. That is, any refactoring I’m doing is trying to reduce complexity. So how do I measure complexity?</p>
<p>One good way that I think about complexity on an individual object level comes from security practices: the ‘attack surface.’ We call this ‘encapsulation’ in object oriented software design. Consider these two objects:</p>
<pre><code>class Foo
def bar
end
def baz
end
end
class Bar
def quxx
end
end
</code></pre>
<p>Bar has one less way into the object than Foo does. Hence the idea of a ‘surface’ that an attacker would have access to. A bigger surface needs more defense, more attention, and is harder to lock down.</p>
<p>Note that I said ‘one less.’ If you said “Foo has a surface of two, and Bar has a surface of one,” you’d be wrong. Luckily, Ruby provides us with a simple way to calculate attack surface. Just use this monkey patch: (I’m not suggesting that you actually use this monkey patch.)</p>
<pre><code>class BasicObject
def attack_surface
methods.count
end
end
</code></pre>
<p>This gives us</p>
<pre><code>1.9.2p318 &gt; Foo.new.attack_surface
=&gt; 59
1.9.2p318 &gt; Bar.new.attack_surface
=&gt; 58
</code></pre>
<p>Oooooh yeah. <code>Object.new.methods.count</code> is 57… we forgot about those. This is why mixins do not really reduce the complexity of your objects. Both versions of Post above have the same number of methods. How many, you ask?</p>
<pre><code>&gt; Post.new.methods.count
=&gt; 337
&gt; Post.new.methods.count - Object.new.methods.count
=&gt; 236
</code></pre>
<p>In this case, my <code>Post</code> is an object that inherits from <code>ActiveRecord::Base</code> and defines nothing else. ActiveRecord adds in 236 methods to our object. That is a pretty huge increase.</p>
<h2>Reducing complexity through encapsulation</h2>
<p>Several people have been dancing around this issue for a while by saying something like ‘Consider ActiveRecord a private interface, and never use it from outside of your class.’ They feel this exact pain, but enforce it through social means rather than reducing the complexity of objects. Now, there are good reasons to do that, so I don’t think it’s categorically wrong, but it certainly is a compromise.</p>
<p>An implementation of DataMapper would be able to solve this problem,</p>
<pre><code>class Post
def initialize(title, body)
@title = title
@body = body
end
end
class PostMapper
def save(post)
@database.insert(:posts, [post.title, post.body])
end
end
PostMapper.new.save(Post.new("Hello DataMapper", "DM rocks"))
</code></pre>
<p>or the repository pattern:</p>
<pre><code>class PostRepository
def first
title, body = @database.select(:posts, "LIMIT 1")
Post.new(title, body)
end
end
</code></pre>
<p>All incredibly theoretical and pretend. But note that the <code>Post</code> in both instances has no methods defined on it that deal with persistence whatsoever. We’ve used encapsulation to hide all of the details of the database in the database, the details of the post logic in the <code>Post</code>, and we use the Mapper or Repository to mediate between the two.</p>
<h2>Using Mixins the right way</h2>
<p>Mixins are awesome, though, and can be used to reduce duplication. They’re for cross-cutting concerns:</p>
<pre><code>class Post
def summary
"#{author}: #{body[0,200]}"
end
end
class Comment
def summary
"#{author}: #{body[0,200]}"
end
end
</code></pre>
<p>moves to</p>
<pre><code>class Post
include Summarizer
end
class Comment
include Summarizer
end
module Summarizer
def summary
"#{author}: #{body[0,200]}"
end
end
</code></pre>
<p>This example isn’t fantastic, but whatever. We’ve eliminated some duplication, even if we didn’t reduce our method surface. This is a win along a different axis, we just shouldn’t fool ourselves that we’ve made our classes ‘simpler.’</p>
<h2>This isn’t the only metric</h2>
<p>Now, I don’t mean to imply that method count is the be-all, end-all metric of complexity. Real software development is too complicated to wrap up in one tiny number. However, I do think there’s a strong correlation between small, focused classes and easy to understand software systems. Aggregates can be used to make it easier to understand the use of a cluster of smaller objects. But that’s another post.</p>
</content:encoded></item><item><title>On the eve of #m1gs</title><link>https://steveklabnik.com/writing/on-the-eve-of--m1gs/</link><guid isPermaLink="true">https://steveklabnik.com/writing/on-the-eve-of--m1gs/</guid><pubDate>Mon, 30 Apr 2012 00:00:00 GMT</pubDate><content:encoded><p>Walking down the street, a woman cries for help. She’s blind, and the curb is large, and she’s unsure of herself. Knowing it’s not safe, she seeks the help of a stranger to guide her on her way. Three or four people pass by, but then someone stops, gives her their arm, and guides her safely to her destination.</p>
<p><strong>HOOOONK!!!</strong> A large truck is blocked from going down the street. A Lexus with out-of-state-plates is in the way, and the driver is standing outside of the car, looking agitated. A passer-by stops to ask if they can help, and the man frantically tries to explain. His English is poor, but not that poor, but due to the stress of the situation, he’s losing his words. Luckily, the stranger speaks a little of his native tongue. The astute observer would notice that this tiny bit of familiarity causes a massive change in demeanor in this chaotic moment. 311 is called, a translator is found, and the man will be able to move his car soon.</p>
<p>These moments, these tiny acts of selflessness and mutual aid, are what revolutions are truly made of.</p>
</content:encoded></item><item><title>Cards for dad - follow-up</title><link>https://steveklabnik.com/writing/cards-for-dad--follow-up/</link><guid isPermaLink="true">https://steveklabnik.com/writing/cards-for-dad--follow-up/</guid><pubDate>Sun, 22 Apr 2012 00:00:00 GMT</pubDate><content:encoded><p>Hey everyone! It’s been a few weeks, and I wanted to update you guys on the card situation.</p>
<p>First of all, thank you so much. Every single one of you who sent cards, thank you! It was truly amazing to see so much support. I was even more surprised that I basically put my address on the internet and nothing particularly bad happened. Zero trolls. Restored some of my faith in humanity.</p>
<p>We ended up with about 500 cards. My dad is well enough to stay at home, but the chemo is pretty rough, so he has good days and bad days. I go home to visit about once a week, but we don’t really know how well and if it’s really working yet, so no actual understanding of how long he has. Regardless, it all made a big impact, so thank you again.</p>
<p>Here’s some pictures:</p>
<p><img src="http://i.imgur.com/mcQSG.jpg" alt="http://i.imgur.com/mcQSG.jpg" /></p>
<p><img src="http://i.imgur.com/GmmhJ.jpg" alt="http://i.imgur.com/GmmhJ.jpg" /></p>
<p><img src="http://i.imgur.com/RV4k8.jpg" alt="http://i.imgur.com/RV4k8.jpg" /></p>
<p><img src="http://i.imgur.com/QVTJK.jpg" alt="http://i.imgur.com/QVTJK.jpg" /></p>
<p><img src="http://i.imgur.com/YxXqj.jpg" alt="http://i.imgur.com/YxXqj.jpg" /></p>
<p>Oh, and I didn’t read very many of them personally, since they were for my dad, not for me. My parents read and cherished all of them, though. They said that you all said too many nice things about me, so thank you for all of that as well.</p>
</content:encoded></item><item><title>Building StoreEngine</title><link>https://steveklabnik.com/writing/building-storeengine/</link><guid isPermaLink="true">https://steveklabnik.com/writing/building-storeengine/</guid><pubDate>Thu, 05 Apr 2012 00:00:00 GMT</pubDate><content:encoded><p>So I said I was gonna build antisocialne.ws. But I’m changing gears instead: I’d rather do this project, which has a better spec and needs a reference implementation.</p>
<p><a href="http://vimeo.com/39850969">Building StoreEngine</a> from <a href="http://vimeo.com/steveklabnik">Steve Klabnik</a> on <a href="http://vimeo.com/">Vimeo</a>.</p>
</content:encoded></item><item><title>About antisocialne.ws</title><link>https://steveklabnik.com/writing/about-antisocialne-ws/</link><guid isPermaLink="true">https://steveklabnik.com/writing/about-antisocialne-ws/</guid><pubDate>Tue, 03 Apr 2012 00:00:00 GMT</pubDate><content:encoded><p><a href="http://vimeo.com/39718634">About Antisocialne.ws</a> from <a href="http://vimeo.com/steveklabnik">Steve Klabnik</a> on <a href="http://vimeo.com/">Vimeo</a>.</p>
</content:encoded></item><item><title>Anti-capitalism 101</title><link>https://steveklabnik.com/writing/anti-capitalism-101/</link><guid isPermaLink="true">https://steveklabnik.com/writing/anti-capitalism-101/</guid><pubDate>Sat, 31 Mar 2012 00:00:00 GMT</pubDate><content:encoded><p>People are often surprised to find out that I identify as an anti-capitalist. At least in America, many people see capitalism as inevitable, or even as the way things always were. Unfortunately, there’s so much rhetoric and confusion around this topic that it always takes a long time to break it down. I’d been meaning to write something about this topic, but then I found this really great work called <a href="http://libcom.org/files/Capitalism_ex-dummies_web.pdf">Capitalism, Class, and Class Struggle for (ex) Dummies</a>. I’ve adapted it with my own thoughts and edits below. It in turn is adapted from <a href="http://libcom.org/library/capitalism-introduction">this essay</a> and <a href="http://libcom.org/library/class-class-struggle-introduction">this essay</a> from LibCom.org, so if you’d like to see the history, you can read those versions as well.</p>
<p>I plan on refining this as things go along, as it’s still a bit jargony in places. Any suggestions are very welcome!</p>
<hr />
<h2>Capitalism: an introduction</h2>
<p>At its root, capitalism is an economic system based on three things: wage labour (working for a wage), private ownership of the means of production (things like factories, machinery, farms, and offices), and production for exchange and profit.</p>
<p>While some people own means of production, or capital, most of us don’t and so to survive we need to sell our ability to work in return for a wage, or else scrape by on benefits. This first group of people is the capitalist class or “bourgeoisie” in Marxist jargon, and the second group is the working class or “proletariat”. This jargon is only included so that you can make connections to other literature, we’ll stick away from such things later in this text.</p>
<p>Capitalism is based on a simple process—money is invested to generate more money. When money functions like this, it functions as capital. For instance, when a company uses its profits to hire more staff or open new premises, and so make more profit, the money here is functioning as capital. As capital increases (or the economy expands), this is called ‘capital accumulation’, and it’s the driving force of the economy.</p>
<p>Those accumulating capital do so better when they can shift costs onto others. If companies can cut costs by not protecting the environment, or by paying sweatshop wages, they will. So catastrophic climate change and widespread poverty are signs of the normal functioning of the system. Furthermore, for money to make more money, more and more things have to be exchangeable for money. Thus the tendency is for everything from everyday items to DNA sequences to carbon dioxide emissions—and, crucially, our ability to work—to become commodified.</p>
<p>And it is this last point—the commodification of our creative and productive capacities, our ability to work—which holds the secret to capital accumulation. Money does not turn into more money by magic, but by the work we do every day.</p>
<p>In a world where everything is for sale, we all need something to sell in order to buy the things we need. Those of us with nothing to sell except our ability to work have to sell this ability to those who own the factories, offices, etc. And of course, the things we produce at work aren’t ours, they belong to our bosses.</p>
<p>Furthermore, because of long hours, productivity improvements etc, we produce much more than necessary to keep us going as workers. The wages we get roughly match the cost of the products necessary to keep us alive and able to work each day (which is why, at the end of each month, our bank balance rarely looks that different to the month before). The difference between the wages we are paid and the value we create is how capital is accumulated, or profit is made.</p>
<p>This difference between the wages we are paid and the value we create is called “surplus value”. The extraction of surplus value by employers is the reason we view capitalism as a system based on exploitation—the exploitation of the working class. This process is essentially the same for all wage labour, not just that in private companies. Public sector workers also face constant attacks on their wages and conditions in order to reduce costs and maximise profits across the economy as a whole.</p>
<h3>Unwaged labour</h3>
<p>The accumulation of capital also relies on unwaged work, such as housework or domestic labour. This includes the reproduction of labour power in the form of producing and raising children—the next generation of workers—and servicing the current workforce—physically, emotionally, and sexually. This unpaid labour is predominantely carried out by women.</p>
<p>Servicing men and children at home serves capital: by making housework and reproduction a women’s ‘natural and feminine’ process rather than work, capitalism benefits in the form of free labour. When capital pays husbands they get two workers, not one. Denying domestic labour a wage makes this work invisible, and divides the working class into waged and unwaged at the expense of both.</p>
<h3>Competition</h3>
<p>In order to accumulate capital, our boss must compete in the market with bosses ofother companies. They cannot afford to ignore market forces, or they will lose ground to their rivals, lose money, go bust, get taken over, and ultimately cease to be our boss. Therefore even the bosses aren’t really in control of capitalism, capital itself is. It’s because of this that we can talk about capital as if it has agency or interests of its own, and so often talking about ‘capital’ is more precise than talking about bosses.</p>
<p>Both bosses and workers, therefore, are alienated by this process, but in different ways. While from the workers’ perspective, our alienation is experienced through being controlled by our boss, the boss experiences it through impersonal market forces and competition with other bosses.</p>
<p>Because of this, bosses and politicians are powerless in the face of ‘market forces,’ each needing to act in a way conducive to continued accumulation (and in any case they do quite well out of it!). They cannot act in our interests, since any concessions they grant us will help their competitors on a national or international level.</p>
<p>So, for example, if a manufacturer develops new technology for making cars which doubles productivity it can lay off half its workers, increase its profits and reduce the price of its cars in order to undercut its competition. If another company wants to be nice to its employees and not sack people, eventually it will be driven out of business or taken over by its more ruthless competitor—so it will also have to bring in the new machinery and make the layoffs to stay competitive.</p>
<p>Of course, if businesses were given a completely free hand to do as they please, monopolies would soon develop and stifle competition which would lead to the system grinding to a halt. The state intervenes, therefore to act on behalf of the long-term interests of capital as a whole. We observed this happen in America back in the Robber Baron days.</p>
<h3>The State</h3>
<p>The primary function of the state in capitalist society is to maintain the capitalist system and aid the accumulation of capital. As such, the state uses repressive laws and violence against the working class when we try to further our interests against capital. For example, bringing in anti-strike laws, or sending in police or soldiers to break up strikes and demonstrations.</p>
<p>The “ideal” type of state under capitalism at the present time is liberal democratic, however in order to continue capital accumulation at times different political systems are used by capital to do this. State capitalism in the USSR, and fascism in Italy and Germany are two such models, which were necessary for the authorities at the time in order to co-opt and crush powerful working-class movements which threatened the very continuation of capitalism.</p>
<p>When the excesses of bosses cause workers to fight back, alongside repression the state occasionally intervenes to make sure business as usual resumes without disruption. For this reason national and international laws protecting workers’ rights and the environment exist. Generally the strength and enforcement of these laws ebbs and flows in relation to the balance of power between employers and employees in any given time and place. For example, in France where workers are more well-organised and militant, there is a maximum working week of 35 hours. In the UK, where workers are less militant the maximum is 48 hours, and in the US where workers are even less likely to strike there is no maximum at all.</p>
<h3>History</h3>
<p>Capitalism is presented as a ‘natural’ system, formed a bit like mountains or land masses by forces beyond human control, that it is an economic system ultimately resulting from human nature. However it was not established by ‘natural forces’ but by intense and massive violence across the globe. First in the ‘advanced’ countries, enclosures drove self-sufficient peasants from communal land into the cities to work in factories. Any resistance was crushed. People who resisted the imposition of wage labour were subjected to vagabond laws and imprisonment, torture, deportation or execution. In England under the reign of Henry VIII alone 72,000 people were executed for vagabondage.</p>
<p>Later capitalism was spread by invasion and conquest by Western imperialist powers around the globe. Whole civilisations were brutally destroyed with communities driven from their land into waged work. The only countries that avoided conquest were those—like Japan—which adopted capitalism on their own in order to compete with the other imperial powers. Everywhere capitalism developed, peasants and early workers resisted, but were eventually overcome by mass terror and violence.</p>
<p>Capitalism did not arise by a set of natural laws which stem from human nature: it was spread by the organised violence of the elite. The concept of private property of land and means of production might seem now like the natural state of things, however we should remember it is a man-made concept enforced by conquest. Similarly, the existence of a class of people with nothing to sell but their labour power is not something which has always been the case—common land shared by all was seized by force, and the dispossessed forced to work for a wage under the threat of starvation or even execution. As capital expanded, it created a global working class consisting of the majority of the world’s population whom it exploits but also depends on.</p>
<h3>The Future</h3>
<p>Capitalism has only existed as the dominant economic system on the planet for a little over 200 years. Compared to the half a million years of human existence it is a momentary blip, and therefore it would be naive to assume that it will last for ever. It is entirely reliant on us, the working class, and our labour which it must exploit, and so it will only survive as long as we let it.</p>
<h2>Class &amp; Class Struggle: an introduction</h2>
<p>The first thing to say is that there are various ways of referring to class. Often, when people talk about class, they talk in terms of cultural/sociological labels. For example, middleclass people like foreign films, working class people like football, upper-class people like top hats and so on.</p>
<p>Another way to talk about class, however, is based on classes’ economic positions. We talk about class like this because we see it as essential for understanding how capitalist society works, and consequently how we can change it.</p>
<p>It is important to stress that our definition of class is not for classifying individuals or putting them in boxes, but in order to understand the forces which shape our world, why our bosses and politicians act the way they do, and how we can act to improve our conditions.</p>
<h3>Class and Capitalism</h3>
<p>The economic system which dominates the world at present is called capitalism. As mentioned earlier, capitalism is essentially a system based on the self-expansion of capital—commodities and money making more commodities and more money.</p>
<p>This doesn’t happen by magic, but by human labour. For the work we do, we’re paid for only a fraction of what we produce. The difference between the value we produce and the amount we’re paid in wages is the “surplus value” we’ve produced. This is kept by our boss as profit and either reinvested to make more money or used to buy swimming pools or fur coats or whatever.</p>
<p>In order for this to take place, a class of people must be created who don’t own anything they can use to make money i.e. offices, factories, farmland or other means of production. This class must then sell their ability to work in order to purchase essential goods and services in order to survive. This class is the working class.</p>
<p>So at one end of the spectrum is this class, with nothing to sell but their ability to work. At the other, those who do own capital to hire workers to expand their capital. Individuals in society will fall at some point between these two poles, but what is important from a political point of view is not the positions of individuals but the social relationship between classes.</p>
<h3>The Working Class</h3>
<p>The working class then, or ‘proletariat’ as it is sometimes called, the class who is forced to work for wages, or claim benefits if we cannot find work or are too sick or elderly to work, to survive. We sell our time and energy to a boss for their benefit.</p>
<p>Our work is the basis of this society. And it is the fact that this society relies on the work we do, while at the same time always squeezing us to maximise profit, that makes it vulnerable.</p>
<h3>Class Struggle</h3>
<p>When we are at work, our time and activity is not our own. We have to obey the alarm clock, the time card, the managers, the deadlines and the targets.</p>
<p>Work takes up the majority of our lives. We may see our managers more than we see our friends and partners. Even if we enjoy parts of our job we experience it as something alien to us, over which we have very little control. This is true whether we’re talking about the nuts and bolts of the actual work itself or the amount of hours, breaks, time off etc.</p>
<p>Work being forced on us like this compels us to resist.</p>
<p>Employers and bosses want to get the maximum amount of work from us, from the longest hours, for the least pay. We, on the other hand, want to be able to enjoy our lives: we don’t want to be over-worked, and we want shorter hours and more pay.</p>
<p>This antagonism is central to capitalism. Between these two sides is a push and pull: employers cut pay, increase hours, speed up the pace of work. But we attempt to resist: either covertly and individually by taking it easy, grabbing moments to take a break and chat to colleagues, calling in sick, leaving early. Or we can resist overtly and collectively with strikes, slow-downs, occupations etc.</p>
<p>This is class struggle. The conflict between those of us who have to work for a wage and our employers and governments, who are the capitalist class, or ‘bourgeoisie’. By resisting the imposition of work, we say that our lives are more important than our boss’s profits. This attacks the very nature of capitalism, where profit is the most important reason for doing anything, and points to the possibility of a world without classes and privately owned means of production. We are the working class resisting our own existence. We are the working class struggling against work and class.</p>
<h3>Beyond the Workplace</h3>
<p>Class struggle does not only take place in the workplace. Class conflict reveals itself in many aspects of life. For example, affordable housing is something that concerns all working class people. However, affordable for us means unprofitable for them. In a capitalist economy, it often makes more sense to build luxury apartment blocks, even while tens of thousands are homeless, than to build housing which we can afford to live in. So struggles to defend social housing, or occupying empty properties to live in are part of the class struggle.</p>
<p>Similarly, healthcare provision can be a site of class conflict. Governments or companies attempt to reduce spending on healthcare by cutting budgets and introducing charges for services to shift the burden of costs onto the working class, whereas we want the best healthcare possible for as little cost as possible.</p>
<h3>The ‘Middle Class’</h3>
<p>While the economic interests of capitalists are directly opposed to those of workers, a minority of the working class will be better off than others, or have some level of power over others. When talking about history and social change it can be useful to refer to this part of the proletariat as a “middle class”, despite the fact that it is not a distinct economic class, in order to understand the behaviour of different groups.</p>
<p>Class struggle can sometimes be derailed by allowing the creation or expansion of the middle class—Margaret Thatcher encouraged home ownership by cheaply selling off social housing in the UK during the big struggles of the 1980s, knowing that workers are less likely to strike if they have a mortgage, and allowing some workers to become better off on individual levels, rather than as a collective. And in South Africa the creation of a black middle class helped derail workers’ struggles when apartheid was overturned, by allowing limited social mobility and giving some black workers a stake in the system.</p>
<p>Bosses try to find all sorts of ways to materially and psychologically divide the working class, including by salary differentials, professional status, race and by gender. It should be pointed out again that we use these class definitions in order to understand social forces at work, and not to label individuals or determine how individuals will act in given situations.</p>
<h3>Conclusion</h3>
<p>Talking about class in a political sense is not about which accent you have but the basic conflict which defines capitalism—those of us who must work for a living vs. those who profit from the work that we do. By fighting for our own interests and needs against the dictates of capital and the market we lay the basis for a new type of society—a society without money, class, or capitalism, organised for the direct fulfilment of our needs: a Libertarian Communist society.</p>
</content:encoded></item><item><title>Birthday cards for Dad</title><link>https://steveklabnik.com/writing/birthday-cards-for-dad/</link><guid isPermaLink="true">https://steveklabnik.com/writing/birthday-cards-for-dad/</guid><pubDate>Mon, 26 Mar 2012 00:00:00 GMT</pubDate><content:encoded><p>Hey everyone. If you follow me on Twitter, you’ll know that I’ve been a bit absent lately. Basically, my dad has <a href="http://en.wikipedia.org/wiki/Angiosarcoma">angiosarcoma</a>. It’s a really rare cancer of the blood vessels. A city the size of Pittsburgh sees two cases every ten years, roughly. It’s certainly terminal, while he had some chemotherapy a few days ago, we won’t know if it’s helped at all for another week or so, and if it works, it’ll give him another few weeks to months at best. It came on really fast: he felt some shortness of breath the first week of February, went into the hospital in week three, by week four he almost died from a blood vessel bursting due to weakness from the disease.</p>
<p>Anyway, you all have sent me tons of well-wishes and such over the last week, and it’s truly appreciated. It’s nice to know that I have support, as I try to support my family. I’m the oldest son, so that carries a lot of responsibilities, and you guys have helped me a lot.</p>
<p>However, if I can ask you for one more thing…</p>
<p>My dad turns 54 on Sunday. My aunt told my mom that for his birthday, she’s going to email everyone she knows, ask them to send him a birthday card, and she’s hoping to get 100. I think that’s a pretty awesome idea, but I also happen to know that while I’ve mentioned to my dad that ‘people on the internet I know send well-wishes,’ I think it’d be super awesome to blow my aunt out of the water and give my dad a zillion cards for his last birthday.</p>
<p>So yeah, if you have a few minutes this week, and you’re thinking of my father, send a birthday card to my house. Here’s the address:</p>
<p><em>note</em>: Five years later, I’ve removed the address. I haven’t lived there in years, please don’t see this now and send mail to some random person kthx.</p>
<p>Oh yeah: My dad is also Steve. I’m not a junior because he has no middle name: his mother said they were too poor to give him one.</p>
<p>Additionally, I mean April 1. That’s his birthday. However, if you live far away and it’ll be a bit late, that’s perfectly fine, still send one. I think that ‘oh, this card came from so far away that it took some time’ is pretty awesome in its own way.</p>
<p>I really think this would be an awesome last gift to my dad. One of the worst things about being in the hospital is that you feel cut off from everything outside of that room, and while I’ve spent almost every minute of the last 8 days in there with him, keeping your spirits up helps the healing process quite a bit.</p>
<p>Thanks. &lt;3 &lt;3 &lt;3</p>
</content:encoded></item><item><title>Transmuting Philosophy into Machinery</title><link>https://steveklabnik.com/writing/transmuting-philosophy-into-machinery/</link><guid isPermaLink="true">https://steveklabnik.com/writing/transmuting-philosophy-into-machinery/</guid><pubDate>Thu, 08 Mar 2012 00:00:00 GMT</pubDate><content:encoded><p>I’m so very close to releasing the beta of <a href="http://designinghypermediaapis.com/">Get Some REST</a>. However, I’m about to get on a plane to Poland, and that’s a bad time to launch things. ;) <a href="http://wrocloverb.com/">wroc_love.rb</a>, here I come!</p>
<p>Anyway, I figured I’d give you a preview of some of the content that I’ve been working on. This particular article is short, but it’s also a public response to something, and so, to whet your appetite, I figured I’d give you a little taste of what’s to come. It’s out of context, so I’m a little light on explaining why you’d do each step of the process; there is much more on that elsewhere in the book! Rather than explaing all the <em>why</em>, I just actually do it. This is one of the last articles, and it’s a fairly simple example once you’ve grokked all the fundamentals, so it’s a bit small, lengthwise.</p>
<p>If all goes well, I’ll be releasing at wroc_love.rb. So keep your eyes peeled.</p>
<p>Without further ado, the article.</p>
<hr />
<blockquote>
<p>Kessler’s idea was, that besides the law of mutual struggle there is in nature the law of mutual aid, which, for the success of the struggle for life, and especially for the progressive evolution of the species, is far more important than the law of mutual contest. This suggestion - which was, in reality, nothing but a further development of the ideas expressed by Darwin himself in The Descent of Man, seemed to me so correct and of so great an importance, that since I became aquainted with it I began to collect materials for further developing the idea, which Kessler had only cursorily sketched in his lecture, but had not lived to develop. He died in 1881.Peter Kropotkin, “Mutual Aid: A Factor of Evolution”, p21</p>
</blockquote>
<p>Rob Conery is a pretty cool guy. While I always enjoy reading <a href="http://blog.wekeroad.com/">his blog</a>, he’s been at battle with the Hypermedia crowd recently. It’s always good natured, though, and he means well.</p>
<p>He recently posed <a href="http://wekeroad.com/2012/03/03/moving-the-philosophy-into-machinery/">an interesting question to his blog</a>:</p>
<blockquote>
<p>I would like to invite the good people who have engaged with me over the last few days to jump in and write me up an API – and by way of explanation – show how their ideas can be translated into reality.</p>
</blockquote>
<p>Great! After all, social exchange is one of the building blocks of society:</p>
<blockquote>
<p>In so far as the process of exchange transfers commodities from hands in which they are non-use-values to hands in which they are use-values, it is a process of social metabolism.Karl Marx, “Capital, Volume 1”, p198</p>
</blockquote>
<p>Let’s apply what we’ve learned about the basics of designing hypermedia APIs. Here are his requirements:</p>
<h2>Use Cases</h2>
<p>This is step one: simple authentication and then consumption of basic data. The client will be HTML, JS, and Mobile.</p>
<h3>Logging In</h3>
<p>Customer comes to the app and logs in with email and password. A token is returned by the server upon successful authentication and a message is also received (like “thanks for logging in”).</p>
<h3>Productions</h3>
<p>Joe User is logged in and wants to see what he can watch. He chooses to browse all productions and can see in[sic] the app which ones he is allowed to watch and which ones he isn’t. He then chooses to narrow his selection by category: Microsoft, Ruby, Javascript, Mobile. Once a production is selected, a list of Episodes is displayed with summary information. Joe wants to view Episode 2 of Real World ASP.NET MVC3 – so he selects it. The video starts.</p>
<h3>Episodes.</h3>
<p>Kelly User watches our stuff on her way to work every day, and when she gets on the train [sic] will check and see if we’ve pushed any new episodes recently. A list of 5 episodes comes up – she chooses one, and watches it on her commute.</p>
<h2>The design process</h2>
<h3>Step 1: Evaluate Process</h3>
<p>Fortunately, this has been done for us, in the Use Cases above. Sweet!</p>
<h3>Step 2: Create state machine</h3>
<p>Taking all of this into account, I drew out this state machine:</p>
<p><img src="/img/2012-03-08/state-machine.png" alt="state machine" /></p>
<p>Basically, you start at a root. Two options: see the newest list of productions, or see them all. You can filter all of them by a category. Eventually, you end up picking one. This workflow should be enough to support all of our use cases.</p>
<h3>Step 3: Evaluate Media Type</h3>
<p>Okay, so, he mentions this in use cases:</p>
<blockquote>
<p>The client will be HTML, JS, and Mobile.</p>
</blockquote>
<p>I’m not 100% sure what he means here: I think it’s that we’ll be building a site on this API (html), it’ll have heavy JS usage (js), and probably a mobile version or possibly a native client (mobile).</p>
<p>Given this information, I’m going to choose JSON as a base format. Besides, our developers tend to like it. ;)</p>
<p>After that choice is made, we also need these things:</p>
<ul>
<li>Filtering things means a templated query of some kind, so we’ll need some kind of templating syntax.</li>
<li>We need lists of things as well as singular things. I like to simply this by representing singular things as a list of one item. So, lists and individual items.</li>
<li>We also have a few attributes we need to infer from these loose requirements. No biggie. :)</li>
</ul>
<h3>Step 4: Create Media Types</h3>
<p>Based on this, I’ve made up <a href="/tekpub-productions.html">the application/vnd.tekpub.productions+json media type</a>. Key features, based on our evaluation:</p>
<ul>
<li>Each transition in our state machine has a relation attribute</li>
<li>Each transition that needs to be parameterized has some sort of template syntax</li>
<li>Each attribute that we need someone to know about has a definition</li>
<li>Everything is always a list. It may contain just one item. Our client’s interface can detect this special case and display something different if it wants.</li>
</ul>
<h3>Step 5: Implementation!</h3>
<p>That’s for Rob! ahaha!</p>
<p>However, you might want a sample implementation. I’m willing to make one if there’s confusion about it, but I figured I’d put the article out and see if that’s interesting to people before I went through the effort.</p>
<h2>What about auth?</h2>
<p>Oh, I didn’t handle the auth case. That’s because auth happens at the HTTP level, not at the application level. HTTP BASIC + SSL or Digest should be just fine.</p>
<h2>But, but, but… I didn’t get any verbs! Or URLS!</h2>
<p>I know. Fielding:</p>
<blockquote>
<p>A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state, or in defining extended relation names and/or hypertext-enabled mark-up for existing standard media types. Any effort spent describing what methods to use on what URIs of interest should be entirely defined within the scope of the processing rules for a media type (and, in most cases, already defined by existing media types). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]</p>
</blockquote>
<p>As well as</p>
<blockquote>
<p>A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations. [Failure here implies that clients are assuming a resource structure due to out-of band information, such as a domain-specific standard, which is the data-oriented equivalent to RPC’s functional coupling].</p>
</blockquote>
<p>And</p>
<blockquote>
<p>A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]</p>
</blockquote>
<p>Soooooooooo yeah.</p>
<h2>Further Exploration</h2>
<h3>Sources Cited</h3>
<ul>
<li><a href="http://www.amazon.com/Mutual-Aid-Evolution-Peter-Kropotkin/dp/0875580246">“Mutual Aid: A Factor of Evolution”, Kropotkin</a></li>
<li><a href="http://www.amazon.com/Capital-Critique-Political-Economy-Classics/dp/0140445684/ref=sr_1_1?s=books&amp;ie=UTF8&amp;qid=1331214700&amp;sr=1-1">“Capital, Volume I”, Karl Marx</a></li>
<li><a href="http://wekeroad.com/2012/02/28/someone-save-us-from-rest/">“Someone save us from REST”</a></li>
<li><a href="http://wekeroad.com/2012/03/03/moving-the-philosophy-into-machinery/">“Moving the Philosophy into Machinery”</a></li>
<li><a href="http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven">“REST APIs Must be Hypertext Driven”</a></li>
<li><a href="/nodes/the-design-process-an-overview">The Design Process: An Overview</a></li>
</ul>
<h3>Next Articles</h3>
<p>For more on why media types are awesome, check <a>Programming the Media Type</a>.</p>
<h3>Terms Used</h3>
<p>None really at the moment! This article stands alone.</p>
</content:encoded></item><item><title>Hypermedia API reading list</title><link>https://steveklabnik.com/writing/hypermedia-api-reading-list/</link><guid isPermaLink="true">https://steveklabnik.com/writing/hypermedia-api-reading-list/</guid><pubDate>Mon, 27 Feb 2012 00:00:00 GMT</pubDate><content:encoded><p>Originally, this post was titled “A RESTful Reading List,” but please note that <a href="/posts/2012-02-23-rest-is-over">REST is over. Hypermedia API is the new nomenclature.</a></p>
<p>I’ve been doing an intense amount of research on Hypermedia APIs over the last few months, and while I didn’t save every resource I found, I’ve made a list here of the most important.</p>
<p>I’ll be updating this post as I get new resources, so check back!</p>
<h2>The book list</h2>
<p>If you want to go from ‘nothing to everything,’ you can do it by reading just a few books, actually. I’m going to make all of these links affiliate. I purchase a <em>lot</em> of books myself, maybe suggesting things to you can help defray the cost of my massive backlog. All are easily searchable on Amazon if you’re not comfortable with that.</p>
<p>Start off with <a href="http://www.amazon.com/gp/product/0596529260/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0596529260">Restful Web Services</a> by Leonard Richardson and Sam Ruby. This book is fantastic from getting you from zero knowledge to “I know how Rails does REST by default,” which is how most people do REST. But as you know, that’s flawed. However, understanding this stuff is crucial, not only for when you interact with REST services, but also for understanding how Hypermedia APIs work differently. This baseline of knowledge is really important.It also comes in <a href="http://www.amazon.com/gp/product/0596801688/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0596801688">cookbook form</a>. You really only need one or the other; pick whichever format you like.</p>
<p>Next up, read <a href="http://www.amazon.com/gp/product/0596805829/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0596805829">REST in Practice: Hypermedia and Systems Architecture</a>. This book serves as a great <em>bridge</em> to understanding Hypermedia APIs from the RESTful world. Chapters one through four read like Richardson &amp; Ruby; yet they start slipping in the better hypermedia terminology. Chapter five really starts to dig into how Hypermedia APIs work, and is a great introduction. Chapter six covers scaling, chapter seven is an introduction to using ATOM for more than an RSS replacement, nine is about security, and eleven is a great comparison of how Hypermedia and WS-* APIs differ. All in all, a great intermediate book.</p>
<p>To really start to truly think in Hypermedia, though, you <em>must</em> read <a href="http://www.amazon.com/gp/product/1449306578/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=1449306578">Building Hypermedia APIs with HTML5 and Node</a>. Don’t let the title fool you, as Mike says in the introduction:</p>
<blockquote>
<p>[HTML5, Node.js, and CouchDB] are used as tools illustrating points about hypermedia design and implementation.</p>
</blockquote>
<p>This is not a Node.js book. I find Node slightly distasteful, but all the examples were easy to follow, even without more than a cursory working knowledge.</p>
<p>Anyway, the book: Mike says something else that’s really important in the intro:</p>
<blockquote>
<p>While the subject of the REST architectural style comes up occasionally, this book does not explore the topic at all. It is true that REST identifies hypermedia as an important aspect of the style, but this is not the case for the inverse. Increasing attention to hypermedia designs can improve the quality and functionality of many styles of distributed network architecture including REST.</p>
</blockquote>
<p>And, in the afterward:</p>
<blockquote>
<p>However, the aim of this book was not to create a definitive work on designing hypermedia APIs. Instead, it was to identify helpful concepts, suggest useful methodologies, and provide pertinent examples that encourage architects, designers, and developers to see the value and utility of hypermedia in their own implementations.</p>
</blockquote>
<p>I think these two statements, taken together, describe the book perfectly. The title is “Building Hypermedia APIs,” not “Designing.” So why recommend it on an API design list? Because understanding media types, and specifically hypermedia-enabled media types, is the key to understanding Hypermedia APIs. Hence the name.</p>
<p>Mike is a great guy who’s personally helped me learn much of the things that I know about REST, and I’m very thankful to him for that. I can’t recommend this book highly enough.</p>
<p>However, that may leave you wondering: Where’s the definitive work on how to actually build and design a Hypermedia API? Did I mention, totally unrelated of course, that <a href="http://designinghypermediaapis.com/">I’m writing a book</a>? ;)</p>
<p>Yes, it still has REST in the title. Think about that for a while, I’m sure you can see towards my plans. I’m planning on a beta release as soon as I’m recovered from some surgery this week, but I’m not sure how long that will take, exactly. So keep your eyes peeled.</p>
<h3>Books I don’t recommend</h3>
<p><a href="http://www.amazon.com/gp/product/1449310508/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=1449310508">REST API Design Rulebook</a>, while I haven’t actually read it, seems quite terrible. Let me copy an <a href="http://www.amazon.com/review/R2F4STDF7XS7U3/ref=cm_cr_dp_perm?ie=UTF8&amp;ASIN=1449310508&amp;nodeID=283155&amp;tag=&amp;linkCode=">Amazon review</a>:</p>
<blockquote>
<p>The first chapters give a good feel for the vocabulary, and some good techniques for implementing REST. A lot of the ‘rules’, especially those related to basic CRUD operations, are clean and simple with useful examples.Unfortunately, the later chapters get more and more focused on specifying something called ‘WRML’, which is a concept/language newly introduced in this book as far as I can tell.Personally I would recommend ignoring the sections dealing with WRML (or keep them in mind as a detailed example of one possible way of handling some of the REST issues).As to WRML itself: yuck. It appears to be an attempt to drag in some of the unnecessary complexity of SOAP with little added benefit. Not recommended.</p>
</blockquote>
<p>Looking up information about WRML, I can agree, 100%. Ugh. So nasty. So this gets a big fat downvote from me.</p>
<h3>Books I want to read</h3>
<p>There aren’t any in this category. Should there be? You tell me!</p>
<h2>Web resources</h2>
<p>There are so many, this will just be a partial list for now.</p>
<p>Of course, <a href="http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm">Fielding’s dissertation</a> is essential.</p>
<p>Roy has also written <a href="http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven">REST APIs Must be Hypertext Driven</a>. This post is central for understanding why “Hypermedia API” is a much better name than REST, and why hypermedia in general is so essential.</p>
<p>I’ve written <a href="http://timelessrepo.com/haters-gonna-hateoas">this post about HATEOAS</a>. It’s a pretty basic, simple introduction to the topic.</p>
<p>I gave a talk called <a href="http://vimeo.com/30764565">Everything you know about REST is wrong</a>.</p>
<p><a href="http://vimeo.com/20781278">This talk by Jon Moore</a> was instrumental in giving me a mental breakthrough about HATEAOS. He was kind enough to <a href="https://gist.github.com/1445773">share some code with me</a> that he used in the presentation as well. This is also an earlier example of the usage of “Hypermedia APIs.”</p>
<p>A classic: <a href="http://tomayko.com/writings/rest-to-my-wife">How I explained REST to my wife</a>. A great story, simple and easy to explain.</p>
<p>Another classic is <a href="http://www.infoq.com/articles/webber-rest-workflow">How to GET a cup of coffee</a>. It does a great job of explaining how to model your business processes as state machines, and then convert them to HTTP.</p>
<p>In which Mike Mayo has a realization that HATEOAS is not simply academic: <a href="http://mikemayo.org/2012/how-i-learned-to-stop-worrying-and-love-rest">http://mikemayo.org/2012/how-i-learned-to-stop-worrying-and-love-rest</a></p>
<p>A recent resource that’s popped up is <a href="http://publish.luisrei.com/rest.html">Designing a RESTful Web API</a>. It’s a nice, basic overview of lots of things.</p>
<h2>Related resources</h2>
<p><a href="http://www.amazon.com/gp/product/1449308929/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=1449308929">APIs: A Strategy Guide</a> seems really interesting. This isn’t about REST or Hypermedia APIs specifically, but more making a case for why you’d want an API in the first place. Which is a related topic for all of us API enthusiasts, for sure. I haven’t read it yet, but it’s on the related list.</p>
<p><a href="http://www.amazon.com/gp/product/0262572338/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0262572338">Protocol: How Control Exists after Decentralization</a> is one of my favorite books ever. This book manages to be both a hard computer science book as well as referencing a broad range of philosophy, history, and other fields as well.</p>
<p>If sentences like</p>
<blockquote>
<p>A perfect exmaple of a distributed network is the rhizome described in Deleuze and Guattari’s A Thousand Plateaus. Reacting specifically to what they see as the totalitarianism inherent in centralized and even decentralized networks, Deleuze and Guattari instead describe the rhizome, a horizontal meshwork derived from botany. The rhizome links many autonomous nodes together in a manner that is neither linear nor hierarchical. Rhizomes are heterogeneous and connective, that is to say, “Any point of a rhizome can be connected to anything other.”</p>
</blockquote>
<p>immediately followed by a hardcore, low-level diagram of the four levels of networking: application layer, transport layer, internet layer, and link layer. With full (and accurate) descriptions of TCP, IP, DNS, the SYN-ACK/SYN-ACK handshake, and HTTP following gets you all hot and bothered, you <em>need</em> to read this book. Hell, if you don’t like literature stuff, the politics in this book are amazing. This book draws the connections between <em>why</em> the APIs we’re building matter, and for that matter, the systems that we programmers create. Seriously, I can’t recommend this book enough.</p>
<p><a href="http://www.amazon.com/gp/product/0801882575/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0801882575">Hypertext 3.0: Critical Theory and New Media in an Era of Globalization</a> is related, and absolutely interesting. Here’s the eight main sections:</p>
<ol>
<li>Hypertext: An introduction</li>
<li>Hypertext and Critical Theory</li>
<li>Reconfiguring the Text</li>
<li>Reconfiguring the Author</li>
<li>Reconfiguring Writing</li>
<li>Reconfiguring Narrative</li>
<li>Reconfiguring Literary Education</li>
<li>The Politics of Hypertext: Who Controls the Text?</li>
</ol>
<p>While not <em>directly</em> useful for those designing APIs, those of us who like to draw broad connections between disciplines will find that this book has lots of interesting parallels. Especially around the politics angle, as well as reconfiguring narrative.</p>
</content:encoded></item><item><title>REST is over</title><link>https://steveklabnik.com/writing/rest-is-over/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rest-is-over/</guid><pubDate>Thu, 23 Feb 2012 00:00:00 GMT</pubDate><content:encoded><h1>REST is</h1>
<p><img src="/img/2012-02-23/restisover.png" alt="Rest is OVER" /></p>
<hr />
<p>Yep. Sorry to have to inform you. REST is totally over. The cool kids are moving on. We’re building “Hypermedia APIs” now. Such is life.</p>
<h2>A lesson from the anti-globalization movement</h2>
<p>Way back in the day, <a href="http://en.wikipedia.org/wiki/COINTELPRO">COINTELPRO</a> was at the forefront of America’s fight against “subersive” organizations and individuals. One goal of COINTELPRO was to create tension and division amongst radical groups, in order to disrupt their operations. Techniques such as Concern Trolling are really effective at this kind of thing.</p>
<p>In 2008, for the Republican National Convention in St. Paul, a document was passed around:</p>
<blockquote>
<p>At an anti-RNC conference held over the weekend of February 9th and 10th, a broad spectrum of groups revealed what are being called the “St. Paul Principles” of unity for resisting the 2008 Republican National Convention (RNC).</p>
</blockquote>
<p>This is a departure from the sectarian squabbles that have plagued past years’ anti-convention organizing. Pitting groups of differing political beliefs against each other has been a frequent tactic of state repression since the days of COINTELPRO.</p>
<p>By drafting the principles together, the co-signing organizations are taking historic steps to actively extinguish divisiveness from their respective groups. The principles will ensure respect for the soon-to-be-permitted march on September 1 by people planning non-permitted activities, and in turn, participants in the September 1 march will adhere to the principles and do nothing to sow division among the many activists coming to the Twin Cities to protest the RNC.</p>
<p>The principles are:</p>
<ol>
<li>Our solidarity will be based on respect for a diversity of tactics and the plans of other groups.</li>
<li>The actions and tactics used will be organized to maintain a separation of time or space.</li>
<li>Any debates or criticisms will stay internal to the movement, avoiding any public or media denunciations of fellow activists and events.</li>
<li>We oppose any state repression of dissent, including surveillance, infiltration, disruption and violence. We agree not to assist law enforcement actions against activists and others.</li>
</ol>
<blockquote></blockquote>
<p>Please draw your attention to the third principle. The reasons behind this rule are interesting:</p>
<ol>
<li>‘Solidarity’ kinda goes out the window when you’re busy arguing about each other’s drama.</li>
<li>Every second you have in the media is precious. Why waste it talking about each other when you could be talking about your issue?</li>
<li>Media will jump on any kind of debate as a weakness. The only way to make these sort of self-reflective discussions productive is to keep them internal.</li>
</ol>
<p>So, keeping that all in mind, why are we arguing about what REST means all the time? Yes, it’s annoying that the common usage of REST is not actually REST. Yes, it’s really hard when someone is wrong on the Internet. Yes, words do matter. However, it’s a question of what’s most productive with our time. Every moment we waste arguing over what REST means could have been spent discussing how to properly build APIs instead. But why bother being productive when we can be critical?</p>
<h2>‘Hypermedia API’ is more clear</h2>
<p>The real problem is that REST is just bad branding. This isn’t Roy’s fault, he wasn’t thinking about such things when trying to write his thesis. But really, from an outside perspective, a ‘real RESTful API’ does two things:</p>
<ol>
<li>Uses HTTP correctly.</li>
<li>Serves hypermedia responses.</li>
</ol>
<p>HTTP does most of the heavy lifting in terms of bringing us into REST compliance. So why are we talking about how to transfer representations of state?</p>
<p>The phrase “Hypermedia API” is much more direct: it’s an API where hypermedia is at the center. Those RESTish APIs could never be called ‘Hypermedia APIs,’ as it’s quite obvious that they don’t use hypermedia. It is not quite clear that they don’t transfer state and representations, though. ;)</p>
<h2>We just really shouldn’t fight</h2>
<p>Ultimately, it really comes back to the first point, though. Arguing about this is just wasting everyone’s time. It’s time to take a deep breath, step back, and just let REST go. Language changes. It happens. It might be a little bit sad, but life will move on. Let’s build fantastic APIs instead. &lt;3 &lt;3 &lt;3</p>
<p>Oh, and I didn’t actually kick this off, credit for that goes to Mike Amundsen and O’Reilly with <a href="http://www.amazon.com/Building-Hypermedia-APIs-HTML5-Node/dp/1449306578/ref=sr_1_1?ie=UTF8&amp;qid=1330039178&amp;sr=8-1">Building Hypermedia APIs with HTML5 and Node</a>. Once something has an O’Reilly book about it, it’s legit. ;) Additionally, the term has been bandied about in the past, in various presentations and talks, but I feel that now’s the time to really step forward and start calling a spade a spade.</p>
</content:encoded></item><item><title>An API ontology</title><link>https://steveklabnik.com/writing/an-api-ontology/</link><guid isPermaLink="true">https://steveklabnik.com/writing/an-api-ontology/</guid><pubDate>Mon, 13 Feb 2012 00:00:00 GMT</pubDate><content:encoded><p><strong>NOTE</strong>: The alpha of my book on APIs is out! Check it out at <a href="http://designinghypermediaapis.com/">http://designinghypermediaapis.com</a>.</p>
<p>As I’ve done research on APIs for <a href="http://designinghypermediaapis.com/">Designing Hypermedia APIs</a>, I’ve become increasingly interested in different styles of API. I currently see most real-world deployed APIs fit into a few different categories. All have their pros and cons, and it’s important to see how they relate to one other.</p>
<p>You may find this amusing if you’ve <a href="http://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch_styles.htm">read some of the literature</a> on the topic, but I’ve created this list in a top-down way: APIs as black boxes, rather than coming up with different aspects of an API and categorizing them based on that. I also decided to look at actually deployed APIs, rather than theoretical software architectures.</p>
<p>If you have an API that doesn’t fit into one of these categories, I’d love to hear about it. I’d also like to further expand these descriptions, if you have suggestions in that regard, please drop me a line, too.</p>
<h2>HTTP GET/POST</h2>
<h3>Synopsis:</h3>
<p>Provide simple data through a simple GET/POST request.</p>
<h3>Examples:</h3>
<ul>
<li><a href="http://placekitten.com/">http://placekitten.com/</a></li>
<li><a href="http://code.google.com/apis/maps/documentation/staticmaps/">http://code.google.com/apis/maps/documentation/staticmaps/</a></li>
<li><a href="http://loripsum.net/api">http://loripsum.net/api</a></li>
</ul>
<h3>Description:</h3>
<p>Simple data is made available via an HTTP GET or POST request. The vast majority of these services seem to return images, but data is possible as well.</p>
<p>These API are technically a sub-type of *-RPC, but I feel that their lack of business process makes them feel different. It’s basically just one specific remote procedure, available over HTTP.</p>
<h2>*-RPC</h2>
<h3>Synopsis:</h3>
<p>Remote procedure call; call a function over the web.</p>
<h3>Examples:</h3>
<ul>
<li><a href="http://codex.wordpress.org/XML-RPC_Support">http://codex.wordpress.org/XML-RPC_Support</a></li>
<li><a href="http://www.flickr.com/services/api/request.rest.html">http://www.flickr.com/services/api/request.rest.html</a></li>
<li><a href="http://services.sunlightlabs.com/docs/Sunlight_Congress_API/">http://services.sunlightlabs.com/docs/Sunlight_Congress_API/</a></li>
</ul>
<h3>Description:</h3>
<p>Similiar to how structured programming is built around functions, so is RPC. Rather than call functions from your own programs, RPC is a way to call functions over the Internet.</p>
<p>All calls are made through some sort of API endpoint, and usually sent over HTTP POST.</p>
<p>Major flavors include XML-RPC and JSON-RPC, depending on what format data is returned in.</p>
<p>Note that while Flickr’s API says REST, it is very clearly RPC. Yay terminology!</p>
<h2>WS-* (or SOAP)</h2>
<h3>Synopsis:</h3>
<p>Serialize and send objects over the wire.</p>
<h3>Examples:</h3>
<ul>
<li><a href="http://www.flickr.com/services/api/request.soap.html">http://www.flickr.com/services/api/request.soap.html</a></li>
<li><a href="https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&amp;content_ID=developer/e_howto_api_soap_PayPalSOAPAPIArchitecture">https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&amp;content_ID=developer/e_howto_api_soap_PayPalSOAPAPIArchitecture</a></li>
<li><a href="http://www.salesforce.com/us/developer/docs/api/Content/sforce_api_quickstart_intro.htm">http://www.salesforce.com/us/developer/docs/api/Content/sforce_api_quickstart_intro.htm</a></li>
</ul>
<h3>Description:</h3>
<p>SOAP stands for “Simple Object Access Protocol,” and that describes it pretty well. The idea behind these APIs is to somehow serialize your objects and then send them over the wire to someone else.</p>
<p>This is usually accomplished by downloading a WSDL file, which your IDE can then use to generate a whole ton of objects. You can then treat these as local, and the library will know how to make the remote magic happen.</p>
<p>These are much more common in the .NET world, and have fallen out of favor in startup land. Many larger businesses still use SOAP, though, due to tight integration with the IDE.</p>
<h2>“REST”</h2>
<h3>Synopsis:</h3>
<p>Ruby on Rails brought respect for HTTP into the developer world. A blending of RPC, SOAP, and hypermedia API types.</p>
<h3>Examples:</h3>
<ul>
<li><a href="http://developer.github.com/">http://developer.github.com/</a></li>
<li><a href="https://dev.twitter.com/">https://dev.twitter.com/</a></li>
<li><a href="http://developers.facebook.com/docs/reference/api/">http://developers.facebook.com/docs/reference/api/</a></li>
</ul>
<h3>Description:</h3>
<p>Originally, REST was synonymous with what is now called “Hypermedia APIs.” However, after large amounts of misunderstanding, REST advocates are rebranding REST to “Hypermedia APIs” and leaving REST to the RESTish folks. See ’<a href="/posts/2012-02-23-rest-is-over">REST is over’</a> for more.</p>
<p>REST is basically “RPC and/or SOAP that respects HTTP.” A large problem with RPC and SOAP APIs is that they tunnel everything through one endpoint, which means that they can’t take advantage of many features of HTTP, like auth and caching. RESTful APIs mitigate this disadvantage by adding lots of endpoints; one for each ‘resource.’ The SOAPish ones basically allow you to CRUD objects over HTTP by using tooling like ActiveResource, and the RPC ones let you perform more complicated actions, but always with different endpoints.</p>
<h2>Hypermedia</h2>
<h3>Synopsis:</h3>
<p>Hypermedia is used to drive clients through various business processes. The least understood and deployed API type, with one exception: the World Wide Web.</p>
<h3>Examples:</h3>
<ul>
<li><a href="http://www.twilio.com/docs/api/rest">http://www.twilio.com/docs/api/rest</a></li>
<li><a href="http://www.spire.io/docs/tutorials/rest-api.html">http://www.spire.io/docs/tutorials/rest-api.html</a></li>
<li><a href="http://kenai.com/projects/suncloudapis/pages/Home">http://kenai.com/projects/suncloudapis/pages/Home</a></li>
</ul>
<h3>Description:</h3>
<p>Originally called REST, Hypermedia APIs take full advantage of HTTP. They use hypermedia formats to drive business processes, providing the ultimate decoupling of clients and servers.</p>
<p>You can tell an API is Hypermedia by providing only one API endpoint, but which accepts requests at other endpoints that are provided by discovery. You navigate through the API by letting the server’s responses guide you.</p>
</content:encoded></item><item><title>Introducing metadown</title><link>https://steveklabnik.com/writing/introducing-metadown/</link><guid isPermaLink="true">https://steveklabnik.com/writing/introducing-metadown/</guid><pubDate>Mon, 23 Jan 2012 00:00:00 GMT</pubDate><content:encoded><p>Because I don’t have enough gems made already, I made another one last night: <a href="https://rubygems.org/gems/metadown">metadown</a>.</p>
<h2>What’s Metadown do?</h2>
<p>This blog originally used Jekyll. When I moved it to my own personal blog implementation, I noticed something: Markdown doesn’t actually have support for adding YAML at the top of your files, like Jekyll does. I always <em>knew</em> this, I just didn’t think about it before. And I love using Markdown, so I ended up extracting my own version of this trick into Metadown.</p>
<p>Basically, sometimes you need metadata about your markup file. YAML is a nice format for writing key/value pairs, lists, and other things… so I’ve let you smash the two of them together in one file by adding some <code>---</code>s at the top.</p>
<h2>Gimme an Example</h2>
<pre><code>require 'metadown'
data = Metadown.render("hello world")
data.output #=&gt; "&lt;p&gt;hello, world&lt;/p&gt;"
data.metadata #=&gt; "{}"
text = &lt;&lt;-MARKDOWN
---
key: "value"
---
hello world
MARKDOWN
data = Metadown.render(text)
data.output #=&gt; "&lt;p&gt;hello, world&lt;/p&gt;\n"
data.metadata #=&gt; {"key" =&gt; "value"}
</code></pre>
<h2>Where’s the code?</h2>
<p>It’s implemented using a custom parser for Redcarpet, has a test suite, and works on every Ruby that’s not JRuby. You can <a href="https://github.com/steveklabnik/metadown">check out the source here</a>. Issues and Pull Requests welcome. There’s at least one thing that I know I’ll need to add to it in the near future.</p>
</content:encoded></item><item><title>Moving from Sinatra to Rails</title><link>https://steveklabnik.com/writing/moving-from-sinatra-to-rails/</link><guid isPermaLink="true">https://steveklabnik.com/writing/moving-from-sinatra-to-rails/</guid><pubDate>Tue, 17 Jan 2012 00:00:00 GMT</pubDate><content:encoded><p>I love both Sinatra and Rails, for different reasons. I’ve heard a few different heuristics for which framework would be better for your application, but I’m not sure the answer is all that simple, really. Regardless of which is correct for your application, I haven’t seen a lot of people discussing how to move between the two frameworks.</p>
<h2>Step One: Evaluate your test coverage</h2>
<p>In <a href="http://www.amazon.com/gp/product/0131177052/ref=as_li_ss_tl?ie=UTF8&amp;tag=stesblo026-20&amp;linkCode=as2&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0131177052">Working Effectively with Legacy Code</a> (this is an affiliate link. Does this bother you?), Michael Feathers introduces a fantastic technique called “Lean on the Compiler.” It’s on page 315, for those of you playing along at home.</p>
<blockquote>
<p>The primary purpose of a compiler is to translate source code into some other form, but in statically typed languages, you can do much more with a compiler. You can take advantage of its type checking and use it to identify which changes you need to make. I call this practice Leaning on the Compiler.Lean on the Compiler is a powerful technique, but you have to know what its limits are; if you don’t, you can end up making some serious mistakes.</p>
</blockquote>
<p>Now, in a language like Ruby, we don’t have type checking. We do have something that double checks our code correctness: tests. Just like static type checks, tests cannot <em>ensure</em> that you’ve done your transformations, but they sure as hell can help. This becomes step one in the effort to move from one framework to another: evaluate your test coverage.</p>
<h3>Acceptance Tests are most important</h3>
<p>When looking at your tests, first check out your acceptance tests. These are the most important, for two reasons: they’re abstracted away from the framework itself, and their purpose in life is to make sure that major functionality is working. Do you have your happy paths covered? Is there any major functionality that’s <em>not</em> covered by acceptance tests?</p>
<p>While the happy path is a big deal, when moving to a new framework, we’re going to introduce a high chance of people encountering issues, so the sad path is also important. I personally tend to not write very many sad path acceptance tests and leave that for tests at the lower level. This is a good time to take stock of the worst of your worst paths: is there certain functionality that’s <em>absolutely</em> important to be handled 100% correctly? Then write some new acceptance tests. As with any kind of engineering project, there’s a tradeoff here: you can’t get coverage of every possible situation. Tests are supposed to give you confidence in your code, so liberally add coverage for any situation that makes you nervous.</p>
<h3>Integration tests are important too</h3>
<p>These tests are a bit more tricky, as they are tied into the framework that you’re using. I’m not sure that anyone writes many of these kinds of tests with Sinatra, outside of maybe model to model integration tests.</p>
<h3>Model and View tests: not a big deal</h3>
<p>These kinds of tests are much more of a sanity check than anything else for the purposes of this kind of move. It’s good that they pass, but really, they shouldn’t be too reliant on the framework you’re using. They’re better for making sure you’ve put things in the correct directories and haven’t forgotten anything in the move.</p>
<p>I’m not even sure what a ‘view test’ would be in Sinatra, and they tend to be not in favor with Rails projects anyway, so that’s more of a general ‘framework to framework’ bit of advice than anything else.</p>
<h3>Controllers don’t exist in Sinatra…</h3>
<p>… so you don’t really write those kinds of tests. Controller tests don’t seem to be that popular in Rails-land these days either.</p>
<h2>Step Two: Git Can help</h2>
<p>I’ll show you how I managed transitioning files over. However, I’m really interested in the idea of using a <a href="http://help.github.com/subtree-merge/">subtree merge</a> to keep being able to update the original Sinatra project while working on the new Rails project.</p>
<p>First step is, of course, make a new branch. This transition will probably take some time, and you may want to push hotfixes into production during that period. Using master is not advised.</p>
<p>Next, make a copy of everything and shove it in a temp directory. Then delete everything in the git repo (except the .git directory, of course) and commit that blank slate. Back up one directory above your project directory, run <code>rails new myproject</code> with your project’s name, and you’ll get a blank rails app. <code>cd myproject</code> and <code>mkdir sinatra</code>, then copy the backup you made from the temp directory into the sinatra directory. Finally, commit this.</p>
<p>Now you’ve got a new blank Rails app with all your old code in a directory, and you can start moving things over.</p>
<h2>Step Three: Set up your test harness</h2>
<p>Since we’re going to allow our tests to guide our move, it pays to get tests up and running first! Depending on what testing framework you use, get it going with Rails. In our case, we were using minitest to test our code. This took a little bit of effort to get working with Rails, but there weren’t a ton of problems.</p>
<p>As always: red, green, refactor. I’d make a simple test that doesn’t test anything, <code>assert true</code>. Make sure that your <code>rake test</code> or <code>rspec .</code> or whatever you’ll use to run the tests works and then remove the dummy test.</p>
<p>There are two strategies for moving tests over: you can move a chunk at a time or move one at a time. Chunks are easier, but then you get commits where the build is broken. It’s really up to you and your team’s tastes: a huge test suite with a number of failures can show you how close you are to being done with the first 90% of the work. And since you’re not planning on releasing in this half-finished state, having a broken build is not <em>that</em> terrible… As always, you’re the professional: make the call.</p>
<h2>Step Four: Move your models</h2>
<p>Since models are the simplest to move, I like to do them first. You <em>should</em> just be able to copy over each test file and model file, though because you were using Sinatra, your models may be all in one file. This step should be largely painless, though: you’re not really relying on any framework-specific things.</p>
<p>Model tests can still suss out problems with your environment, though, like incorrect database settings, certain environment variables…</p>
<p>The idea to begin with the easiest thing comes from <a href="http://www.daveramsey.com/home/">Dave Ramsey</a>, oddly enough. My aunt works at a bank, and when I went to college, she bought me a few of his books to help me learn about personal finance. Dave is a big fan of the ‘get rid of all debt’ school of thought, and so a large portion of his work is strategies for getting out of debt. Dave contends that paying off loans with the highest interest, while mathematically the fastest way to pay off your obligations, is not actually the best strategy. People like to see progress and are heartened by seeing it. By paying off the smallest loan first, one is much more likely to feel more positive about the headway one is making, and so it’s a much better strategy in the end.</p>
<p>The same logic applies with this move: why start with the hard stuff? Knock out some quick wins to keep your spirits high.</p>
<h2>Step Five: Convert your controllers</h2>
<p>Next up: get your controllers created. This is a pretty manual process:</p>
<pre><code>get "foos" do
@foos = Foo.all
render :"foos/index"
end
</code></pre>
<p>becomes</p>
<pre><code>class FooController &lt; ApplicationController
def index
@foos = Foo.all
end
end
</code></pre>
<p>The biggest issue with moving all of this stuff is that Sinatra and Rails use <code>redirect</code> and <code>redirect_to</code>, respectively. So you’ll have to convert that stuff. However, I wouldn’t recommend changing things like <code>redirect "/foos/#{id}"</code> to <code>redirect_to foo_path(foo)</code> just yet. When dealing with legacy code, you want to change as little as possible with each step so that you know when you have introduced an error. If you try to convert things to a Rails style as well, you run the risk of introducing errors. Therefore, in all of these moves, leave the hell enough alone as much as possible. Once your code is up and running, you can gladly refactor. Just don’t do it now.</p>
<p>Don’t forget to generate your routes, too. Sinatra’s DSL is like a combination of Rails’ routing and controllers. Set those up in this step as well.</p>
<p>Since we don’t have tests, this part is very error-prone. Luckily, our acceptance tests will catch these issues in step seven. So give it your best shot, but don’t worry about being 100% perfect. Focus on getting the basic structural changes in place.</p>
<p>Having tests is so nice. :/</p>
<h2>Step Six: Move your views</h2>
<p>This should be as simple as the models: put your views in the correct directory, and things should be golden. If you were using inline views with Sinatra, well, you shouldn’t have that many of them, so breaking them out should be pretty easy.</p>
<h2>Step Seven: Listen to your tests</h2>
<p>Okay! Those last moves were really awkward since we didn’t have tests to check our work. This is where the acceptance tests come in. You can move these over in batches or one at a time, but the acceptance tests will tell you where you forgot a route, if your view is missing, or if you left some little bit of Sinatra-ness in your Rails somewhere.</p>
<p>You’re almost home! Once you get all your acceptance tests working, you should pat yourself on the back! You’re not done yet, but you’ve made a lot of progress.</p>
<h2>Step Eight: Track your exceptions!</h2>
<p>You should have been doing this anyway, but if you weren’t, set up some sort of mechanism to catch and handle exceptions. <a href="http://airbrakeapp.com/pages/home">Airbrake</a> is one such service, but even just recording them yourself would be fine. You <em>need</em> to have this set up in some form, as you’re likely to generate errors, and examining your exceptions is the best way to track down actual problems.</p>
<h2>Step Nine: Plan out deployment strategies</h2>
<p>Of course, now that you’re done developing, it’s time to get the app out to users. This’ll require a comprehensive plan for actually making the jump. A little bit of foresight can go a long way here.</p>
<p>My strategy was to roll out the new Rails app under a new domain: http://beta.myapp.com/. I then pinged my users with a message along the lines of “We’ve made some infrastructure updates, if you’d like to help us test them out, visit the beta site.”</p>
<p>This approach does have some sticky points, however. The first one is the database. In my case, I had a high level of confidence in my code, and this application wasn’t for anything that was considered mission-critical. We also had a decent backup strategy in place. Therefore, I connected the beta site up to the production data store. This meant there were no migration issues later. However, this may make you uncomfortable, but there are other options, too. You can treat the beta as a sandbox, and tell people that the data will not be persisted after the beta, or you can migrate people’s data back to the production data store afterwards.</p>
<p>Another approach is to automatically migrate a portion of your users over to the new code base. Ideally, nobody notices: your code base shouldn’t have changed in looks or functionality.</p>
<h2>Step Ten: Push to production!</h2>
<p>Congrats! Execute on the plan you created in step nine, and get ready to answer tons of emails about why your app is broken! Just kidding, I hope! I really recommend the beta strategy for this reason. No plan survives first contact with the enemy. Your users <em>will</em> find things you didn’t, no matter how much manual testing you did along the way.</p>
<h2>Another strategy: mounting</h2>
<p>I haven’t actually tried this myself, but I’ve been thinking about another way to make this happen: mount your Sinatra app inside of a Rails app, and use that to move things over slowly. <a href="http://stackoverflow.com/a/6972706">Here’s an example</a> of how to make this happen technically. If you had a really big Sinatra application, I could see how this might get you deploying faster; just carve out one vertical chunk of your app, move it over, and keep the rest of your app running in Sinatra.</p>
</content:encoded></item><item><title>Implementing HATEOS with presenters</title><link>https://steveklabnik.com/writing/implementing-hateoas-with-presenters/</link><guid isPermaLink="true">https://steveklabnik.com/writing/implementing-hateoas-with-presenters/</guid><pubDate>Fri, 06 Jan 2012 00:00:00 GMT</pubDate><content:encoded><p>I’m a big fan of using the presenter pattern to help separate logic from presentation. There’s a great gem named <a href="https://github.com/jcasimir/draper">Draper</a> that can help facilitate this pattern in your Rails apps. When doing research for <a href="http://designinghypermediaapis.com/">my book about REST</a>, I realized that the presenter pattern is a great way to create responses that comply with the hypermedia constraint, a.k.a. HATEOAS. I wanted to share with you a little bit about how to do this.</p>
<p>Please note that ’<a href="/posts/2012-02-23-rest-is-over">REST is over’</a>.</p>
<p>Note: We’ll be creating HTML5 responses in this example, as HTML is a hypermedia format, and is therefore conducive to HATEOAS. JSON and XML don’t cut it.</p>
<h2>First, some setup</h2>
<p>I fired up a brand new Rails app by doing this:</p>
<pre><code>$ rails new hateoas_example
$ cd hateoas_example
$ cat &gt;&gt; Gemfile
gem "draper"
^D
$ bundle
$ rails g resource post title:string body:text
$ rake db:migrate
$ rails g draper:decorator Post
</code></pre>
<p>Okay, now we should be all set up. We’ve got a Rails app, it’s got draper in the Gemfile, we have a Post resource, and our PostDecorator.</p>
<h2>The View</h2>
<p>I like to do the view first, to drive our what we need elsewhere. Here it is:</p>
<pre><code>&lt;h2&gt;Title&lt;/h2&gt;
&lt;p&gt;&lt;%= @post.title %&gt;&lt;/p&gt;
&lt;h2&gt;Body&lt;/h2&gt;
&lt;p&gt;&lt;%= @post.body %&gt;&lt;/p&gt;
&lt;h2&gt;Links&lt;/h2&gt;
&lt;ul&gt;
&lt;% @post.links.each do |link| %&gt;
&lt;li&gt;&lt;%= link_to link.text, link.href, :rel =&gt; link.rel %&gt;&lt;/li&gt;
&lt;% end %&gt;
&lt;/ul&gt;
</code></pre>
<p>We’re displaying our title and body, but we also want to spit out some links. These links should have a few attributes we need. I might even (shhhhhhh) extract this link thing out into a helper to add the rel stuff every time. It just depends. For this example, I didn’t feel like it.</p>
<h2>The Controller</h2>
<p>Well, we know we’re gonna need a <code>@post</code> variable set, so let’s get that going in our controller:</p>
<pre><code>class PostsController &lt; ApplicationController
def show
@post = PostDecorator.find(params[:id])
end
end
</code></pre>
<p>Super simple. Yay Draper!</p>
<h2>The Presenter</h2>
<p>We know we need a <code>links</code> method that returns some links, and those links need to have rel, href, and text attributes. No problem!</p>
<pre><code>class Link &lt; Struct.new(:rel, :href, :text)
end
class PostDecorator &lt; ApplicationDecorator
decorates :post
def links
[self_link, all_posts_link]
end
def all_posts_link
Link.new("index", h.posts_url, "All posts")
end
def self_link
Link.new("self", h.post_url(post), "This post")
end
end
</code></pre>
<p>Now, we could have just returned an array of three-element arrays, but I really like to use the Struct.new trick to give us an actual class. It makes error messages quite a bit better, and reminds us that we don’t happen to have an array, we have a Link.</p>
<p>We construct those links by taking advantage of the ‘index’ and ‘self’ rel attributes that are <a href="http://www.iana.org/assignments/link-relations/link-relations.xml">defined in the registry</a>.</p>
<h2>The output</h2>
<p>That gives us this:</p>
<pre><code>&lt;!DOCTYPE html&gt;
&lt;html&gt;
&lt;head&gt;
&lt;title&gt;HateoasSample&lt;/title&gt;
&lt;link href="/assets/application.css?body=1" media="screen" rel="stylesheet" type="text/css" /&gt;
&lt;link href="/assets/posts.css?body=1" media="screen" rel="stylesheet" type="text/css" /&gt;
&lt;script src="/assets/jquery.js?body=1" type="text/javascript"&gt;&lt;/script&gt;
&lt;script src="/assets/jquery_ujs.js?body=1" type="text/javascript"&gt;&lt;/script&gt;
&lt;script src="/assets/posts.js?body=1" type="text/javascript"&gt;&lt;/script&gt;
&lt;script src="/assets/application.js?body=1" type="text/javascript"&gt;&lt;/script&gt;
&lt;meta content="authenticity_token" name="csrf-param" /&gt;
&lt;meta content="0k+SQVv6yr0d12tGWYx7KNXUWaf6f+wgUUNITsAOnHI=" name="csrf-token" /&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;h2&gt;Title&lt;/h2&gt;
&lt;p&gt;A post, woo hoo!&lt;/p&gt;
&lt;h2&gt;Body&lt;/h2&gt;
&lt;p&gt;this is some text that's the body of this post&lt;/p&gt;
&lt;h2&gt;Links&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href="http://localhost:3000/posts/1" rel="self"&gt;This post&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="http://localhost:3000/posts" rel="index"&gt;All posts&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;/body&gt;
&lt;/html&gt;
</code></pre>
<p>You’ll probably want to make a layout that ignores all of the JS stuff, but for this example, I just left it as-is. It’s just that easy. Happy linking!</p>
</content:encoded></item><item><title>ActiveRecord considered harmful</title><link>https://steveklabnik.com/writing/active-record-considered-harmful/</link><guid isPermaLink="true">https://steveklabnik.com/writing/active-record-considered-harmful/</guid><pubDate>Fri, 30 Dec 2011 00:00:00 GMT</pubDate><content:encoded><blockquote>
<p>It is practically impossible to teach OO design to students that have had a prior exposure to Rails: as potential programmers they are mentally mutilated beyond hope of regeneration.Edsger W. Dijkstra (paraphrased)</p>
</blockquote>
<p>I love ActiveRecord. It was the first ORM I’d ever interacted with. My first technical employer had commissioned a DBA, and so of course, we wrote all our own queries. Which was fine; I know my way around a JOIN or two. The problem came when it was time to make a new class; time to write “SELECT * FROM ‘tableName’ WHERE ‘id’=‘%’”… for each class. “Just copy one of the other small classes, hack out everything, and change the table names,” were my instructions. Fine. Whatever. But I knew there had to be a better way…</p>
<p>Along comes Rails. Holy crap, ActiveRecord is <em>awesome</em>! It writes the <em>exact</em> SQL I would have written myself! I don’t need to do <em>anything</em>. This is the best thing since sliced bread. But years later, not everything is rainbows and sunshine. I’ve written a <em>lot</em> of crappy code, and most of it is due to following Rails ‘best practices.’ Which is totally fine! Rails is kinda getting old these days. It’s no longer the new and shiny. But while Rails developers have gotten really good at busting out little CRUD apps, we haven’t moved web application design <em>forward</em> in a really, really long time.</p>
<p>And that’s what Rails did, really. That fifteen minute blog video was shocking. I know several people that threw away two or three week old projects and re-built their stuff in Rails within moments of watching it. And Rails has continued to lead the way in improving how we build rich applications on the web; the (not so but kinda whatever nobody but me cares) RESTful routing was great. The asset pipeline, though it has some bugs, has been great. The obsession with DRY has been great. The test obsession has been great. But Rails has also not been great in aiding us in writing maintainable software. Many of Rails’ design decisions make it difficult to write <em>great</em> software. It helps us write good software, but that isn’t enough anymore.</p>
<p>The real problem is that to truly move forward, Rails will have to re-invent itself, and I’m not sure that it can.</p>
<h2>ActiveRecord is the problem</h2>
<p>I’m not going to talk about all of the problems Rails has today, but I’d like to show you the biggest, most central one: ActiveRecord.</p>
<p>ActiveRecord’s greatest strength is also its problem: Tying class names to table names. This means that it’s impossible to de-couple your persistence mechanism from your domain logic. You can manage it through a combination of <code>set_table_name</code>, making a bunch of <code>Repository</code> classes, and careful coding… but then you might as well be using DataMapper. Except the Ruby library of the same name doesn’t really implement the DataMapper pattern all that well either, having the same issue of tying it all together.</p>
<p>This has another interesting effect: it led directly to the ‘fat model’ recommendation. While ‘get your stuff out of the controller’ is correct, it’s lead Rails developers to build huge, monolithic models that are hard to test, and violate SRP. It took me two and a half years to realize that Ruby classes in the models folder don’t have to inherit from <code>ActiveRecord::Base</code>. That is a problem.</p>
<p>We’ve gotten into a situation with a local minimum: our quest for great software has lead us to DRY ourselves into a corner. Now we have code that’s incredibly tightly coupled. Our ‘single responsibility’ is ‘anything and everything that tangentially relates to a Post.’</p>
<h2>ActionController is the problem</h2>
<p>ActionController relies on instance variables to pass information from the controller to the view. Have you ever seen a 200 line long controller method? I have. Good luck teasing out which instance variables actually get set over the course of all those nested <code>if</code>s.</p>
<p>The whole idea is kinda crazy: Yeah, it looks nice, but we literally just say ‘increase the scope of variables to pass data around.’ If I wrote a post saying “Don’t pass arguments to methods, just promote your data to a global” I’d be crucified. Yet we do the same thing (albeit on a smaller scale) every time we write a Rails application.</p>
<h2>ActionView is the problem</h2>
<p>The whole idea of logic in templates leads to all kinds of problems. They’re hard to test, they’re hard to read, and it’s not just a slippery slope, but a steep one. Things go downhill <em>rapidly</em>.</p>
<p>What I’d really like to see is Rails adopting a ‘ViewModel + templates’ system, with logic-less templates and presenter-esque models that represent the views. The differences between Django’s idea of ‘views’ and Rails’ idea of ‘views’ are interesting here.</p>
<h2>MVC is the problem</h2>
<p>If you’ll notice, I basically have said that models are a problem. Controllers are a problem. Views are a problem. MVC has served the web well, even if it isn’t the GUI style MVC that named the pattern. But I think we’re reaching its limits; the impedance mismatch between HTTP and MVC, for example, is pretty huge. There are other ways to build web applications; I’m particularly excited about <a href="http://rubyconf-webmachine.heroku.com/">WebMachine</a>. I don’t have a constructive alternative to offer here, I just know there’s a problem. I’m still mulling this one over.</p>
<h2>It’s still good, even with problems</h2>
<p>I love Rails. I build software with it daily. Even with its flaws, it’s been a massive success. But because I love Rails, I feel like I can give it straightforward criticism: it’s easier to trash something you love. The real issue is that changing these things would require some really serious changes. It’d involve re-architecting large portions of things that people classically identify with Rails, and I’m not sure that Rails wants to or can do that.</p>
<p>This post is light on examples. I want this to be the <em>starting point</em> of a discussion, not the end of it. Expect more, in detail, from me in the future. What do you think? Are these problems pain points for you? Are they worth fixing? Are they actually problems?</p>
</content:encoded></item><item><title>Devise: ActionController::RoutingError (No Route Matches [GET] /users/sign_out)</title><link>https://steveklabnik.com/writing/devise-actioncontroller-routingerror-no-route-matches-get-slash-users-slash-sign-out/</link><guid isPermaLink="true">https://steveklabnik.com/writing/devise-actioncontroller-routingerror-no-route-matches-get-slash-users-slash-sign-out/</guid><pubDate>Thu, 22 Dec 2011 00:00:00 GMT</pubDate><content:encoded><p>Just a quick note about Devise, and its RESTful implications. I ran across this error today, and thought I’d share.</p>
<p>I was trying to log out, so I hit the normal route for such things with my browser. Here’s the error:</p>
<pre><code>Devise: ActionController::RoutingError (No Route Matches [GET] /users/sign_out)
</code></pre>
<p>Uhhhhh what? I run rake routes…</p>
<pre><code>$ rake routes | grep destroy_user
destroy_user_session DELETE /users/sign_out(.:format) {:action=&gt;"destroy", :controller=&gt;"devise/sessions"}
</code></pre>
<p>So, wtf? Well, there is that pesky <code>DELETE</code>… Googling for this error led me to <a href="https://github.com/plataformatec/devise/commit/f3385e96abf50e80d2ae282e1fb9bdad87a83d3c">this commit</a>. Looks like they changed some behavior, you can’t just go to that page anymore and log out. Bummer. But why?</p>
<h2>HTTP verbs: transfer semantics, not state</h2>
<p>As I’ve been doing my research for <a href="http://designinghypermediaapis.com/">my book on REST</a>, I’ve been doing a lot of reading about HTTP. And I kept coming across these kinds of curious comments in the spec. For example, <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6">sec 9.6: PUT</a>:</p>
<blockquote>
<p>HTTP/1.1 does not define how a PUT method affects the state of an origin server.</p>
</blockquote>
<p>Uhhh… what? Don’t we all know that PUT means that we should be updating a resource? And that we have to send the whole representation?</p>
<p>When trying to get to the bottom of this, I came across <a href="http://www.imc.org/atom-protocol/mail-archive/msg05425.html">this comment from Fielding</a>:</p>
<blockquote>
<p>FWIW, PUT does not mean store. I must have repeated that a million times in webdav and related lists. HTTP defines the intended semantics of the communication – the expectations of each party. The protocol does not define how either side fulfills those expectations, and it makes damn sure it doesn’t prevent a server from having absolute authority over its own resources. Also, resources are known to change over time, so if a server accepts an invalid Atom entry via PUT one second and then immediately thereafter decides to change it to a valid entry for later GETs, life is grand.</p>
</blockquote>
<p>Soooooooo wtf?</p>
<p>Let’s take a look again at what <code>PUT</code> does:</p>
<blockquote>
<p>The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.</p>
</blockquote>
<p>It says “store,” Roy says “I don’t mean store.” Uhhhh…</p>
<p>Here’s my ‘translated for the laymen’ version of that quote:</p>
<blockquote>
<p>PUT means ‘I’d like to later GET something at this URI.’ If something is already there, update it. If there isn’t anything there, then create it.</p>
</blockquote>
<p>That’s it. It’s talking about the semantics of what goes on: create or update. It doesn’t actually say anything about how this is implemented. But if you PUT something to a URI, a GET needs to 200 afterwards. So what’s the difference between PUT and POST? <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5">HTTP sec 9.5: POST</a>:</p>
<blockquote>
<p>The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line.The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database.The action performed by the POST method might not result in a resource that can be identified by a URI.</p>
</blockquote>
<p>Again, it’s really vague about what it <em>does</em>. With POST, it basically says “You have no idea what a POST does.” What you <em>do</em> know is the semantics of the action, POST ‘requests a new subordinate, but it might not create something.’</p>
<p>The <em>main</em> difference is actually mentioned <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.2">here</a>:</p>
<blockquote>
<p>Methods can also have the property of “idempotence” in that (aside from error or expiration issues) the side-effects of N &gt; 0 identical requests is the same as for a single request. The methods GET, HEAD, PUT and DELETE share this property.</p>
</blockquote>
<p>Semantics again. PUT is <em>idempotent</em>, and POST is not. They could both be used for creation, they could both be used for updating. With POST, you don’t need a URI, and PUT specifies a specfic one. That’s it. Nowhere in those two sentences states ‘store in the database,’ nowhere does it says ‘full representation,’ nowhere does it say ‘POST is create and PUT is update.’ However you fulfill these semantics are up to you, but the semantics are what’s important.</p>
<h2>So wtf does this have to do with Devise?</h2>
<p>The issue is that Devise’s old semantics were wrong. A GET to <code>/users/sign_out</code> shouldn’t modify state: <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.1">HTTP sec 9.1.1: Safe Methods</a>:</p>
<blockquote>
<p>In particular, the convention has been established that the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered “safe”.</p>
</blockquote>
<p>When Devise used GETs to log you out, that was a violation of the semantics of GET. Here’s the interesting part, though: Since POST is a super generic ‘unsafe action’ method, you could also use POST to represent a logging out. POST also has unsafe, non-idempotent semantics. DELETE specifically says delete, and POST says ‘any action,’ and ‘delete’ is a subset of ‘any action.’ So DELETE is <em>better</em>, but POST is not <em>wrong</em>, categorically.</p>
<h2>Fixing the ‘bug’</h2>
<p>So how do we take care of this? Personally, I did the pragmatic thing. In <code>/config/initializers/devise.rb</code></p>
<pre><code>config.sign_out_via = :delete
config.sign_out_via = :get if Rails.env.test?
</code></pre>
<p>Now, during normal operations, we have our usual DELETE semantics, but in our test environment, we can just hit it with GET. This way we don’t have to hit a page, use Javascript to make a link with DELETE, and hit the page. This keeps my test times down, means I can run my tests with rack-test and not Selenium, and still gives me a high level of confidence that my tests work properly, even though it’s technically not the exact same thing as production.</p>
<h2>In conclusion</h2>
<p>The HTTP spec defines <em>semantics</em> but not implementation details. Semantics should be obeyed. But in testing, obeying them 100% may not be worth it.</p>
</content:encoded></item><item><title>Redcarpet is awesome</title><link>https://steveklabnik.com/writing/redcarpet-is-awesome/</link><guid isPermaLink="true">https://steveklabnik.com/writing/redcarpet-is-awesome/</guid><pubDate>Wed, 21 Dec 2011 00:00:00 GMT</pubDate><content:encoded><p>It’s true.</p>
<p>If you haven’t used it yet, <a href="https://github.com/tanoku/redcarpet">Redcarpet</a> is the Markdown parser that <a href="https://github.com/blog/832-rolling-out-the-redcarpet">GitHub uses</a> to work all that magic on their site. So of course, it’s awesome.</p>
<h2>You can use it and abuse it</h2>
<p>What makes it <em>really</em> awesome is the custom renderers feature. Here’s the one from the documentation:</p>
<pre><code># create a custom renderer that allows highlighting of code blocks
class HTMLwithAlbino &lt; Redcarpet::Render::HTML
def block_code(code, language)
Albino.safe_colorize(code, language)
end
end
markdown = Redcarpet::Markdown.new(HTMLwithAlbino)
</code></pre>
<p>You can guess what that does: uses some library called Albino to render code blocks. There are a whole bunch of hooks you can use to make a custom renderer. They make it super easy to do something neat with markdown.</p>
<p>So I did.</p>
<h2>Check it: outline generation</h2>
<p>I’m working on… this project. And it needs to render a bunch of articles that are Markdown files. There will be a sidebar. I want to have links to each section. But I’m a good programmer, which means I’m lazy. So why bother making my own sidebar? Especially when I can abuse Redcarpet to do it.</p>
<p>Oh, and disclaimer: normally, I’m all about super clean code. Last night, that wasn’t the mood I was in. This code is probably terrible. That’s part of the fun! Please feel free to {clean up,obfuscate,golf,unit test} this code, <a href="mailto:[email protected]">email me</a>, and tell me how awesome you are.</p>
<p>With that disclaimer out of the way, round one:</p>
<pre><code>class OutlineRenderer &lt; Redcarpet::Render::HTML
attr_accessor :outline
def initialize
@outline = []
super
end
def header(text, header_level)
text_slug = text.gsub(/\W/, "_").downcase
self.outline &lt;&lt; [header_level, "&lt;a href='##{text_slug}'&gt;#{text}&lt;/a&gt;"]
"&lt;h#{header_level} id='#{text_slug}'&gt;#{text}&lt;/h#{header_level}&gt;"
end
end
</code></pre>
<p>Every time we hit a header, Redcarpet gives us the text and the header level. We grab the text, turn it into a slug, and then append a two-element array to our outline. It keeps track of the level of this header, and makes a link from the slug. Then, we spit out a header tag, adding on an <code>id</code> element that we linked to in our link.</p>
<p>Next up, rendering:</p>
<pre><code>renderer = OutlineRenderer.new
r = Redcarpet::Markdown.new(renderer)
content = r.render(some_content)
outline = renderer.outline
</code></pre>
<p>This instantiates our Renderer, creates our Markdown parser, and renders the content. We also grab the outline we made. Sweet.</p>
<p>Finally, rendering the outline:</p>
<pre><code>def format_outline(outline)
prev_level = 2
"&lt;ul&gt;" + outline.inject("") do |html, data|
level, link = data
if prev_level &lt; level
html += "&lt;ul&gt;"
elsif prev_level &gt; level
html += "&lt;/ul&gt;"
end
prev_level = level
html += "&lt;li&gt;#{link}&lt;/li&gt;"
html
end + "&lt;/ul&gt;"
end
</code></pre>
<p>This… is amazing. And terrible. Don’t drink scotch and code, kids. Or do, whatever. This takes our <code>Array</code> of <code>Array</code>s that we made with our renderer and runs <code>inject</code> over it. If our previous level is less than the new level, we indent with a <code>&lt;ul&gt;</code>, and if it’s greater, we outdent with a <code>&lt;/ul&gt;</code>. Then we render our link, and wrap the whole thing in its own set of <code>&lt;ul&gt;</code>s. Done!</p>
<p>We can call this helper method in our layout, and bam! We get automatic outlines generated for our Markdown files. Like I said, this is quick and dirty, but for a first pass, I don’t feel too bad about it. Get it done, and then get it done right.</p>
</content:encoded></item><item><title>Write better cukes with the rel attribute</title><link>https://steveklabnik.com/writing/write-better-cukes-with-the-rel-attribute/</link><guid isPermaLink="true">https://steveklabnik.com/writing/write-better-cukes-with-the-rel-attribute/</guid><pubDate>Tue, 20 Dec 2011 00:00:00 GMT</pubDate><content:encoded><p>The other day, I was working on some Cucumber features for a project, and I discovered a neat technique that helps you to write better Cucumber steps.</p>
<p>Nobody wants to be <a href="http://elabs.se/blog/15-you-re-cuking-it-wrong">cuking it wrong</a>, but what does that really mean? Here’s Jonas’ prescription:</p>
<blockquote>
<p>A step description should never contain regexen, CSS or XPath selectors, any kind of code or data structure. It should be easily understood just by reading the description.</p>
</blockquote>
<p>Great. Let’s <a href="http://www.theregister.co.uk/2007/06/25/thoughtworks_req_manage/">pop the why stack</a> a few times, shall we?</p>
<p><strong>Q1</strong>: Why do we want to have descriptions not use regexen, CSS selectors, or code? <strong>A1</strong>: To give it a simpler language.</p>
<p><strong>Q2</strong>: Why do we want it to be in a simpler language? <strong>A2</strong>: So that it’s easily understandable for stakeholders.</p>
<p><strong>Q3</strong>: Why do we want it to be easily understandable for stakeholders? <strong>A3</strong>: Because then we can share a <a href="http://domaindrivendesign.org/node/132">common language</a>.</p>
<p><strong>Q4</strong>: Why is a common language important? <strong>A4</strong>: A shared language assists in making sure our model matches the desires of our stakeholders.</p>
<p><strong>Q5</strong>: Why do we want to match the desires of our stakeholders? <strong>A5</strong>: That’s the whole reason we’re on this project in the first place!</p>
<p>Anyway, that’s what it’s really all about: developing that common language. Cukes should be written in that common language so that we can make sure we’re on track. So fine: common language. Awesome. Let’s do this.</p>
<h2>Write some cukes in common language</h2>
<p>Time to write a cuke:</p>
<pre><code>Scenario: Editing the home page
Given I'm logged in as an administrator
When I go to the home page
And I choose to edit the article
And I fill in some content
And I save it
Then I should see that content
</code></pre>
<p>Basic CMS style stuff. I’m not going to argue that this is the best cuke in the world, but it’s pretty good. What I want to do is examine some of these steps in more detail. How would you implement these steps?</p>
<pre><code>When /^I choose to edit the article$/ do
pending
end
When /^I fill in some content$/ do
pending
end
When /^I save it$/ do
pending
end
Then /^I should see that content$/ do
pending
end
</code></pre>
<p>Go ahead. Write them down somewhere. I’ll wait.</p>
<p>… done yet?</p>
<h2>Implementing a step</h2>
<p>Done? Okay! Before I show you my implementation, let’s talk about this step:</p>
<pre><code>When /^I choose to edit the article$/
</code></pre>
<p>When writing this step, I realized something. When trying to write steps like these, there’s a danger in tying them too closely to your specific HTML. It’s why many people don’t write view tests: they’re brittle. I actually like view tests, but that’s another blog post. Point is this: we know we’re going to follow a link, and we know that we want that link to go somewhere that will let us edit the article. We don’t really care <em>where</em> it is in the DOM, just that somewhere, we’ve got an ‘edit article’ link. How to pull this off?</p>
<h3>First idea: id attribute</h3>
<p>You might be thinking “I’ll give it an id attribute!” Here’s the problem with that: ids have to be unique, per page. With article editing, that might not be a problem, but it’s certainly not a general solution. So that’s out.</p>
<h3>Second time: class attribute</h3>
<p>“Okay, then just use a class. Your blog sucks.” Well, let’s check out what <a href="http://www.w3.org/TR/html5/elements.html#classes">the HTML5 spec says about classes</a>.</p>
<blockquote>
<p>Basically nothing about semantics.</p>
</blockquote>
<p>Okay, so that’s paraphrased. But still, the spec basically says some stuff about the details of implementing classes, but absolutely nothing about the semantics of a class. In practice, classes are largely used for styling purposes. We don’t want to conflate our styling with our data, so overloading class for this purpose might work, but feels kinda wrong.</p>
<h3>What about the text?</h3>
<p>We could match on the text of the link. After all, that’s what people use to determine what links to click on. The link with the text “Edit this article” lets us know that that link will let us edit a article.</p>
<p>Matching on the text is brittle, though. What happens when marketing comes through and changes the text to read “Edit my article”? Our tests break. Ugh.</p>
<p>There’s got to be a better way. Otherwise, I wouldn’t be writing this blog post.</p>
<h3>The best way: the rel attribute</h3>
<p>When doing research for <a href="http://designinghypermediaapis.com/">my book on REST</a>, I’ve been doing a lot of digging into various standards documents. And one of the most important attributes from a REST perspective is one that nobody ever talks about or uses: the <code>rel</code> attribute. From <a href="http://www.w3.org/TR/html5/links.html#attr-hyperlink-rel">the HTML5 spec</a>:</p>
<blockquote>
<p>The rel attribute on a and area elements controls what kinds of links the elements create. The attribue’s value must be a set of space-separated tokens. The allowed keywords and their meanings are defined below.</p>
</blockquote>
<p>Below? That’s <a href="http://www.w3.org/TR/html5/links.html#linkTypes">here</a>:</p>
<blockquote>
<p>The following table summarizes the link types that are defined by this specification. This table is non-normative; the actual definitions for the link types are given in the next few sections.alternate: Gives alternate representations of the current document. author: Gives a link to the current document’s author. bookmark: Gives the permalink for the nearest ancestor section.</p>
</blockquote>
<p>Hey now! Seems like we’re on to something. There’s also <a href="http://tools.ietf.org/html/rfc5988">RFC 5988</a>, “Web Linking”. <a href="http://tools.ietf.org/html/rfc5988#section-4">Section four</a> talks about Link Relation Types:</p>
<blockquote>
<p>In the simplest case, a link relation type identifies the semantics of a link. For example, a link with the relation type “copyright” indicates that the resource identified by the target IRI is a statement of the copyright terms applying to the current context IRI.Link relation types can also be used to indicate that the target resource has particular attributes, or exhibits particular behaviours; for example, a “service” link implies that the identified resource is part of a defined protocol (in this case, a service description).</p>
</blockquote>
<p>Bam! Awesome! This is exactly what we want!</p>
<h2>So how do we use rel attributes?</h2>
<p>I’ll be going into more depth about these kinds of topics in <a href="http://designinghypermediaapis.com/">my book</a>, but here’s the TL;DR:</p>
<ol>
<li>There are a set of official types. Try to use those if they’re applicable, but they’re quite general, so that’s often not the case.</li>
<li>You can put whatever else you want. Space delineated. *</li>
<li>The best way is to use a URI and then make a resource at that URI that documents the relation’s semantics.</li>
</ol>
<p>We’ll go with option two for now, for simplicity. In a real application, make it a URI.</p>
<p>(*) Technically, this isn’t true. Extension relations are <em>required</em> to be URIs, or something that can be serialized to a URI. Again, details are outside of the scope of this post.</p>
<h2>Making our link, with semantics.</h2>
<p>Here’s what a link with our newly minted relation looks like:</p>
<pre><code>&lt;a href="/articles/1/edit" rel="edit-article"&gt;Edit this article&lt;/a&gt;
</code></pre>
<p>Super simple. Just that one little attribute. Now we can write a step to match:</p>
<pre><code>When /^I choose to edit the article$/ do
find("//a[@rel='edit-article']").click
end
</code></pre>
<p>This code matches what we’d do as a person really well. “Find the link that edits an article, and click on it.” We’ve not only made the title of our step match our idea of what a person would do, but the code has followed suit. Awesome. We can move this link anywhere on the page, our test doesn’t break. We can change the text of the link, and our test doesn’t break. So cool.</p>
<h2>What about stuff that’s not links? What about data attributes?</h2>
<p>That’s what <a href="http://designinghypermediaapis.com/">my book</a> is going to talk about, sorry. These kinds of practical examples are one of the reasons I decided to write it in the first place, and I don’t want to publish all the content on my blog…</p>
<h2>Better tests through web standards</h2>
<p>Turns out that diving around in standards has some practical benefits after all, eh? Think about the relationship between your cukes, your tests, and your API clients: Cucumber, through Selenium, is an automated agent that interacts with your web service. API clients are automated agents that interact with your web service. Hmmmm…</p>
<p>If you want to know more about this, that’s what <a href="http://designinghypermediaapis.com/">my book</a> is for. I’ll be covering topics like this in depth, and explaining standards in simple language.</p>
<p>Seriously. Did you sign up for <a href="http://designinghypermediaapis.com/">my book</a> yet? ;)</p>
</content:encoded></item><item><title>Marx, anarchism, and web standards</title><link>https://steveklabnik.com/writing/marx-anarchism-and-web-standards/</link><guid isPermaLink="true">https://steveklabnik.com/writing/marx-anarchism-and-web-standards/</guid><pubDate>Thu, 15 Dec 2011 00:00:00 GMT</pubDate><content:encoded><h3>An aside for software devs</h3>
<p>You might not care about anarchism, and I can almost guarantee that you don’t care about Marx, but please bear with me. I think my point is best made by putting the web stuff at the end, so please, just read it. ;) It’ll be good for you, I swear. I’ll explain better at the end.</p>
<h2>Domain Specific Languages and information density</h2>
<p>I’ve been noticing an effect lately that’s certainly not new; it’s just come up frequently. When working within a domain that’s new to them, most people tend to not respect the density of the information being given to them, and it causes them to draw incorrect inferences.</p>
<p>For example, this tweet earlier:</p>
<blockquote>
<p>Does anybody else’s head explode when they read an “unless” statement? What good is readability if comprehension goes out the window?</p>
</blockquote>
<p>— Christopher Deutsch (@cdeutsch) December 15, 2011</p>
<blockquote></blockquote>
<p>And this response:</p>
<blockquote>
<p>@cdeutsch There are weirder idioms. @users.collect{|u| u.email} vs @users.collect(&amp;:email) :) That should weird you out more.</p>
</blockquote>
<p>— Brian P. Hogan (@bphogan) December 15, 2011</p>
<blockquote></blockquote>
<p>Now, I’m not saying that Christopher is in the wrong here, in fact, he agrees with what I’m about to say:</p>
<blockquote>
<p>@philcrissman @bphogan I’m sure I’ll get used to the “unless” pattern someday. But from a Ruby outsiders perspective it’s hard to comprehend</p>
</blockquote>
<p>— Christopher Deutsch (@cdeutsch) December 15, 2011</p>
<blockquote></blockquote>
<p>From someone new to Ruby, <code>Symbol#to_proc</code> or <code>unless</code> are hard to understand initially, and that’s because they’ve increased the density of information being conveyed. <code>unless</code> is the same as <code>if not</code> and <code>&amp;:foo</code> is the same as <code>{|a| a.foo }</code>. Both of these constructs condense something more complicated into something that is simpler, but denser.</p>
<p>You’ll note that I said ‘simpler,’ but by a certain measure, <code>&amp;:foo</code> is actually more complex. When I say <code>&amp;:foo</code> is simpler, I’m meaning for someone who’s well versed in Ruby, functional programming, or first-class functions. I have this exact background, and so for me, <code>collection.map &amp;:foo</code> is simpler and more readable than <code>collection.map {|a| a.foo }</code>. When I read the first example, I say in my head “Map foo over the collection.” You have to grok what map really is to get that sentence or the corresponding code. Whereas what (I imagine) someone who does not have this kind of background thinks when they see the second example is “Okay, so map is like an each, and for each thing in the collection, we’re going to call foo on it.” This is a totally valid interpretation, but notice how much longer it is, and how much more involved in the details it is. That’s cool, but to someone who groks map, it has a much larger amount of mental overhead, in the same way that my concise explanation causes much more thinking for someone who doesn’t grok it.</p>
<p>This happens often in education. DHH made <a href="http://news.ycombinator.com/item?id=3328427">a comment</a> about this recently that illustrates this principle, and he couches it in terms of “learnability” vs. “readability”:</p>
<blockquote>
<p>If you optimize a framework for beginners, you’re optimizing for learnability. That’s great for the first few days or even weeks of learning. But once you’re past that, it’s not so great. What you care more about is the usability of the thing once you know it. There are plenty of cases where learnability and usability are in conflict. Letting learnability win is a short-term relief.If you on the other hand optimize for usability, for making things simple and easy to use for someone who understands the basics, you’ll often end up with something that has great learnability as well. Maybe not as much as could be achieved as if that was your only goal, but plenty still.</p>
</blockquote>
<p>I think that ‘learnability’ is a pretty good shortening for ‘light density’ and ‘usability’ is decent for ‘heavy density.’ It’s the same effect, though. For the rest of this essay, I’ll be using ‘learnable’ and ‘usable’ to mean this particular usage.</p>
<p>Any time you encounter experts in a particular domain, they’ll often have fairly specific language that corresponds to that domain. This language is usually designed to be usable, not learnable. This is because they’ve already done the learning; learnability holds no utility for them. However, usability is incredibly… useful. To them. They’re working at a higher level of abstraction, and don’t want to get bogged down in details they already know well. Using learnable language would cause them to take twice as long to say things; to return to the density analogy, dense information is transferred from one person to another more quickly. If you can read a sentence a second, but that sentence is dense, you acquire much more information per second than if it was lighter.</p>
<p>This tendency means that most language that’s specific to a domain will generally trend towards the usable at the expense of the learnable. The impact this has on individuals new to the domain, however, is that of a wall. An impediment. Overcoming this obstacle requires a bit of good faith on the part of the beginner; to cross quickly over the chasm between beginner and expert, they must recognize and respect this aspect of the conversations they will invariably become a part of. When faced with a term that is used in a strange way, beginners should ask for clarification, and not start arguments over semantics they don’t yet even understand. Experts will recognize these arguments as coming from a place where concepts are not yet fully understood, and while they may recognize the need to help educate, if the newbie is being belligerent, they may just ignore them instead. Nobody wins; the signal/noise ratio has been decreased, the beginner doesn’t learn, and everyone’s time is wasted.</p>
<p>Here’s three other situations where I’ve seen this happen lately:</p>
<h2>Marx and the labor theory of value</h2>
<p>Philosophy writing is generally a great example of text that is very much usable, and not at all learnable. Some writers can still be learnable, but most are not, in my experience. One of the reasons this happens is that they introduce concepts early in a text and then utilize them later without referring back to the earlier definition. This isn’t a problem for anyone who’s thinking about taking off the training wheels or anyone who reads the entire text. The danger comes in when someone <em>not</em> versed in the entire text attempts to take portions of it out of its context.</p>
<p>Consider Marx, and <em>Capital</em>. The meaning of ‘value’ is a central concern of his writing, and indeed, entire critique of the political economy. It’s so important that the first few chapters are devoted to an (excruciatingly, frankly) detailed explanation of his thoughts on the true meaning of value. The rest of <em>Capital</em> is built on top of this: at least in my understanding of Marx, it all boils back down to that one question. And when having discussions between people who are devotees of Marx and those who come from other schools of economics, this kind of language gets in the way.</p>
<p>It also causes ideas to be misrepresented: the first time I was ever introduced to the labor theory of value, it was by a close friend who’s very libertarian. This was a few years ago, so it’s an obvious paraphrase, but he summarized it thusly: “Yeah, I mean, the labor theory of value basically says that people should be paid for however much work they put into things, so if I take six hours to make a widget and you take four, the price of a widget from you should be six yet mine should only be four, and ideas like ‘market value’ should be disregarded. It’s totally irrelevant if I suck at making widgets, I should get paid more than you anyway.” Which, to put it lightly, is a misrepresentation. While explaining the labor theory of value is outside of the scope of this post, what I will say is that to Marx, ‘value’ is something intrinsic to an object; it’s the ‘socially necessary abstract labor’ inherent to it. Talk about dense language! What a capitalist would call ‘value,’ a Marxist would call ‘price.’</p>
<p>As you can see, even just one little word, ‘value,’ can be quite dense! Can you imagine a discussion intended to be ‘learnable’ to outsiders about what’s meant? Imagine the expansion: ‘value’ -&gt; ‘socially necessary abstract labor’ -&gt; … Marx is already long enough; Capital would be thousands of pages! Yet to a beginner who flips to Chapter 12, they’ll read a sentence that contains ‘value’ and draw poor conclusions! They wouldn’t even realize they’re making a mistake, I mean, how could five letters be misinterpreted?</p>
<p>Furthermore, people who haven’t read Marx don’t generally draw distinctions between his critique of capitalism and his solution: communism. This is annoying when trying to explain to people that I love his critique, but am critical of his answers to its problems; they perceive this weakness in his answer as a weakness in his description of the problem. Furthermore, they then say “but I thought you call yourself a communist?” and I respond with “sure; the issues I have are with the dictatorship of the proletariat, not with the general idea of communism” and then their eyes glaze over and they change the subject. Information density claims another victim…</p>
<p>Oh, and a great example of Marxist economics in a usable form <a href="http://people.wku.edu/jan.garrett/303/marxecon.htm">is here</a>.</p>
<h2>Anarchism and ‘anarcho’-capitalists</h2>
<p>Arguments often boil down to these kinds of questions of definition, but one place where I see it happen almost <em>constantly</em> is amongst anarchists. I mean, from the outset, anarchists have to battle against the general definition of ‘chaos and disorder’ versus the domain-specific ‘without rulers.’ Within that, ‘rulers’ in anarchism is fraught with the same sort of questions that ‘value’ has for Marx. The prime example of this are the terribly misguided ‘anarcho’-capitalists, better described as ‘voluntaryists.’</p>
<p>Here’s the deal: ancaps lack an understanding of the vast majority of historical anarchist thought, and so try to appropriate the term ‘anarchism’ to describe their philosophy which is decidedly not anarchist. The ones who do have started using ‘voluntaryist’ to describe themselves, which is a great example of using information density to mislead, but that’s a whole separate rant. Here’s the 411, from <a href="http://infoshop.org/page/AnarchistFAQSectionF1">the Anarchist FAQ</a>, which has its own things to say about density when it comes to the language specific to political theory discussions:</p>
<blockquote>
<p>“Anarcho”-capitalists claim to be anarchists because they say that they oppose government. As noted in the last section, they use a dictionary definition of anarchism. However, this fails to appreciate that anarchism is a political theory. As dictionaries are rarely politically sophisticated things, this means that they fail to recognise that anarchism is more than just opposition to government, it is also marked a opposition to capitalism (i.e. exploitation and private property). Thus, opposition to government is a necessary but not sufficient condition for being an anarchist – you also need to be opposed to exploitation and capitalist private property. As “anarcho”-capitalists do not consider interest, rent and profits (i.e. capitalism) to be exploitative nor oppose capitalist property rights, they are not anarchists.Part of the problem is that Marxists, like many academics, also tend to assert that anarchists are simply against the state. It is significant that both Marxists and “anarcho”-capitalists tend to define anarchism as purely opposition to government. This is no co-incidence, as both seek to exclude anarchism from its place in the wider socialist movement. This makes perfect sense from the Marxist perspective as it allows them to present their ideology as the only serious anti-capitalist one around (not to mention associating anarchism with “anarcho”-capitalism is an excellent way of discrediting our ideas in the wider radical movement). It should go without saying that this is an obvious and serious misrepresentation of the anarchist position as even a superficial glance at anarchist theory and history shows that no anarchist limited their critique of society simply at the state. So while academics and Marxists seem aware of the anarchist opposition to the state, they usually fail to grasp the anarchist critique applies to all other authoritarian social institutions and how it fits into the overall anarchist analysis and struggle. They seem to think the anarchist condemnation of capitalist private property, patriarchy and so forth are somehow superfluous additions rather than a logical position which reflects the core of anarchism.</p>
</blockquote>
<p>Part of the problem with the second half of this quote is that I’m such an ‘expert’ on this kind of language that I don’t even know if it’ll make sense to you; without the kind of background reading in socialist political philosophies, it might just be gibberish. At least, the first paragraph should be pretty straightforward, and you can take the second as an example of this kind of language.</p>
<p>Giving you a short explanation of why anarchists are against Capitalism is a great example in and of itself of domain specific language and density. Here:</p>
<blockquote>
<p>Property is theft.</p>
</blockquote>
<p>This is a quote by <a href="http://en.wikipedia.org/wiki/Pierre-Joseph_Proudhon">Proudhon</a>, the first person to call himself an anarchist. Let’s unpack the first few layers of this statement:</p>
<p>Property. Here’s <a href="http://en.wikipedia.org/wiki/Private_property">Wikipedia’s explanation</a>:</p>
<blockquote>
<p>Private property is the right of persons and firms to obtain, own, control, employ, dispose of, and bequeath land, capital, and other forms of property. Private property is distinguishable from public property, which refers to assets owned by a state, community or government rather than by individuals or a business entity. Private property emerged as the dominant form of property in the means of production and land during the Industrial Revolution in the early 18th century, displacing feudal property, guilds, cottage industry and craft production, which were based on ownership of the tools for production by individual laborers or guilds of craftspeople.Marxists and socialists distinguish between “private property” and “personal property”, defining the former as the means of production in reference to private enterprise based on socialized production and wage labor; and the latter as consumer goods or goods produced by an individual.</p>
</blockquote>
<p>Whew! There’s a few things to note here: Captialists like to pretend that capitalism is synonymous with ‘trade,’ and not something that started in the 1800s. Likewise, that private property rights are something that has always existed. However, as this alludes to, there are many different kinds of property rights that have existed at different places and times.</p>
<p>So in ‘property is theft,’ Proudhon is referring to private ownership of the ‘means of production.’ Let’s expand that. Again, <a href="http://en.wikipedia.org/wiki/Means_of_production">Wikipedia</a>:</p>
<blockquote>
<p>Means of production refers to physical, non-human inputs used in production—the factories, machines, and tools used to produce wealth — along with both infrastructural capital and natural capital. This includes the classical factors of production minus financial capital and minus human capital. They include two broad categories of objects: instruments of labour (tools, factories, infrastructure, etc.) and subjects of labour (natural resources and raw materials). People operate on the subjects of labour, using the instruments of labour, to create a product; or, stated another way, labour acting on the means of production creates a product. When used in the broad sense, the “means of production” includes the “means of distribution” which includes stores, banks, and railroads. The term can be simply and picturesquely described in an agrarian society as the soil and the shovel; in an industrial society, the mines and the factories.</p>
</blockquote>
<p>We could continue to discuss how to distinguish between this ‘private property’ and ‘possessions,’ which anarchists are <em>not</em> against, but I’m just trying to demonstrate that this word ‘property’ is incredibly complex.</p>
<p>Okay, so Proudhon claims that ‘owning the physical inputs used in factories used to produce wealth is theft.’ I could expand on ‘theft,’ but really, I think my point about density is made. For more on this, see <a href="http://anarchism.pageabode.com/afaq/secB3.html">Why are anarchists against private property?</a>.</p>
<h2>Web standards</h2>
<p>Web standards are another great example of a domain that has a lot of very specific language. And one that people often think they can grab random chunks out of and quote without fully understanding the rest of the context.</p>
<p>This is going on right now <a href="https://github.com/rails/rails/pull/505">with Rails</a>. There’s a discussion about if the <code>PATCH</code> HTTP verb should get support, and if it should be the verb that matches to the <code>update</code> action or not. It’s ended up in a discussion about the semantics of <code>PUT</code>, which has resulted in a lot of random quoting of standards documents by myself and others. Here’s some running commentary on some of the comments. It’s impossible to do this without it becoming semi-personal, so let me just say upfront that I think everyone is participating honestly in this discussion, but I think it’s a great example of people who aren’t familiar with a domain jumping in and drawing incorrect conclusions.</p>
<p>First up, benatkin comments:</p>
<blockquote>
<p>I googled for HTTP verbs and clicked the first result and PATCH isn’t listed.http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.htmlWhere is it?</p>
</blockquote>
<p>Anyone participating in a discussion about <code>PATCH</code> should be reasonably familiar with <code>PATCH</code>. Learning where <code>PATCH</code> is defined is as simple as <a href="https://www.google.com/#hl=en&amp;q=http+patch">Googling HTTP PATCH</a>, which shows it being defined in <a href="http://tools.ietf.org/html/rfc5789">RFC 5879</a>. With that said, this is a good example of asking for clarification, and not immediately progressing into an argumentatitive “It’s not in HTTP 1.1, so it’s bullshit!” style of learning where <code>PATCH</code> is being defined.</p>
<p>Of course, the thread starts to dissolve later, when @stevegraham mentions:</p>
<blockquote>
<p>i must be the only person in the world that disagrees with “PUT requires a complete replacement”, as per RFC2616 “HTTP/1.1 does not define how a PUT method affects the state of an origin server”</p>
</blockquote>
<p>Now, Steve is an awesome guy, but he’s a bit misguided in this case. This is a great example of drawing an incorrect conclusion based on one sentence out of context. He’s not <em>wrong</em> in a strict sense, <a href="http://www.ietf.org/rfc/rfc2616.txt">RFC2616</a> does contain that line. However, this is because <a href="http://www.imc.org/atom-protocol/mail-archive/msg05425.html">HTTP defines the semantics of communication</a>, and the semantics are ‘idempotent creation or replacement.’ The fact that HTTP does not define how PUT affects state is irrelevant, an entire representation is needed for idempotency reasons. <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6">PUT’s definition</a> also says pretty clearly:</p>
<blockquote>
<p>The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.</p>
</blockquote>
<p>“The enclosed entity be stored” is pretty straightforward: it needs an entity, not a portion of an entity. Furthermore, how is a server going to create a resource without a complete representation if the resource didn’t already exist?</p>
<p>In this case, the standard’s organization also doesn’t help: if you just read the section titled <code>PUT</code>, you wouldn’t get the full understanding, since the fact that it’s safe and idempotent is mentioned above in the section regarding those two things. I’m not sure why those aspects aren’t in each definition, and are in a different section above, but the point is that you need to consider the full document in its entirety to understand the semantics of <code>PUT</code>. Steve is only reading one section and then extrapolating from there.</p>
<p>There’s a lot more in that pull request, but one more example: Konstantin points out that Rails supports a lot of things that aren’t standards:</p>
<blockquote>
<p>You mean Rails should not support proposed standards like, say, Cookies?</p>
</blockquote>
<p>Yep. Did you know that? Cookies aren’t actually a standard, just a proposed one.</p>
<p>Anyway, I also want to say this: I am not a perfect interpreter of these things either. I often change my opinions after learning more things, and I think this is a good thing. There’s nothing the matter with being wrong; it’s how you handle it that matters. The discussion in this thread continues. RFCs are also not perfect, and do have wiggle-room; but it’s important that agreements are followed. Amongst people who discuss REST and HTTP, the fact that PUT requires a full representation is not controversial; it’s simply understood as true.</p>
<h2>It’s good for you!</h2>
<p>I don’t want to turn anyone off from learning new things; exploring new domains is a great path towards personal growth. I’ve said a few times that I think more programmers should read Marx, and it’s because I think this experience of jumping into the deep end of a new set of language that you don’t fully understand is a tremendous educational experience. But to truly learn, an open mind and open ears are crucial. Making arguments on semantics doesn’t work if you don’t understand the context and semantics of words, as words change significantly when placed in different surroundings.</p>
</content:encoded></item><item><title>Fast Rails tests with CanCan</title><link>https://steveklabnik.com/writing/fast-rails-tests-with-cancan/</link><guid isPermaLink="true">https://steveklabnik.com/writing/fast-rails-tests-with-cancan/</guid><pubDate>Mon, 12 Dec 2011 00:00:00 GMT</pubDate><content:encoded><p>If you haven’t used it, <a href="https://github.com/ryanb/cancan">CanCan</a> is a great library for Rails that handles authorization for you. Its calling card is simplicity; just do this:</p>
<pre><code>class Ability
include CanCan::Ability
def initialize(user)
user ||= User.new
if user.is? :paying_customer
can :show, Article
else
can :show, Article, :free =&gt; true
cannot :show, Article, :free =&gt; false
end
end
end
</code></pre>
<p>And then in a controller:</p>
<pre><code>def show
@article = Article.find(params[:id])
authorize! :read, @article
end
</code></pre>
<p>Super simple! However, as simple as CanCan is, if you want to keep your test times super low, there’s a few things that you should know about.</p>
<h2>Incidental coupling: the silent killer</h2>
<p>The biggest problem with CanCan from the Fast Tests perspective is that we’ve got coupling with class names. In order to test our Ability class, we also have to load User and Article. Let’s try it without concern for how isolated our tests are:</p>
<pre><code>require "cancan/matchers"
require "spec_helper"
describe Ability do
let(:subject) { Ability.new(user) }
let(:free_article) { Article.new(:free =&gt; true) }
let(:paid_article) { Article.new(:free =&gt; false) }
context "random people" do
let(:user) { nil }
it "can see free Articles" do
subject.can?(:show, free_article).should be_true
end
it "cannot see non-free Articles" do
subject.can?(:show, paid_article).should be_false
end
end
context "paying users" do
let(:user) do
User.new.tap do |u|
u.roles &lt;&lt; :paying_customer
end
end
it "can see free Articles" do
subject.can?(:show, free_article).should be_true
end
it "can see non-free Articles" do
subject.can?(:show, paid_article).should be_true
end
end
end
</code></pre>
<p>A few notes about testing with CanCan: I like to organize them with contexts for the different kinds of users, and then what their abilities are below. It gives you a really nice way of setting up those <code>let</code> blocks, and our tests read real nicely.</p>
<p>Anyway, so yeah. This loads up all of our models, connects to the DB, etc. This is unfortunate. It’d be nice if we didn’t have to load up <code>User</code> and <code>Article</code>. Unfortunately, there’s no real way around it in our <code>Ability</code>, I mean, if you want them to read all Articles, you have to pass in <code>Article</code>… hmm. What about this?</p>
<pre><code>let(:free_article) { double(:class =&gt; Article, :free =&gt; true) }
let(:paid_article) { double(:class =&gt; Article, :free =&gt; false) }
</code></pre>
<p>Turns out this works just as well. CanCan reflects on the class that you pass in, so if we just give our double the right class, it’s all gravy. Now we’re not loading <code>Article</code>, but what about <code>User</code>?</p>
<p>The key is in the <code>||=</code>. If we pass in a <code>User</code>, we won’t call <code>User.new</code>. So let’s stub that:</p>
<pre><code>let(:user) { double(:is? =&gt; false) }
let(:user) do
double.tap do |u| #tap not strictly needed in this example, but if you want multiple roles...
u.stub(:is?).with(:paying_customer).and_return(true)
end
end
</code></pre>
<p>Sweet! The first one is a random user: all their <code>is?</code> calls should be false. The second is our user who’s paid up; we need to stub out their role properly, but that’s about it.</p>
<h2>It’s that easy</h2>
<p>With some dillegence, isolating your tests isn’t hard, and works really well. I have a few more tests in the example I took this from, and 14 examples still take about a third of a second. Not too shabby!</p>
<p>Happy authorizing!</p>
<p>Oh, and the final spec, for reference:</p>
<pre><code>require "cancan/matchers"
require "app/models/ability"
describe Ability do
let(:subject) { Ability.new(user) }
let(:free_article) { double(:class =&gt; Article, :free =&gt; true ) }
let(:paid_article) { double(:class =&gt; Article, :free =&gt; false) }
context "random people" do
let(:user) { double(:is? =&gt; false) }
it "can see free Articles" do
subject.can?(:show, free_article).should be_true
end
it "cannot see non-free Articles" do
subject.can?(:show, paid_article).should be_false
end
end
context "users" do
let(:user) do
double.tap do |u|
u.stub(:is?).with(:user).and_return(true)
end
end
it "can see free Articles" do
subject.can?(:show, free_article).should be_true
end
it "can see non-free Articles" do
subject.can?(:show, paid_article).should be_true
end
end
end
</code></pre>
</content:encoded></item><item><title>Book review: New Programmer's Survival Manual</title><link>https://steveklabnik.com/writing/book-review-new-programmers-survival-manual/</link><guid isPermaLink="true">https://steveklabnik.com/writing/book-review-new-programmers-survival-manual/</guid><pubDate>Mon, 28 Nov 2011 00:00:00 GMT</pubDate><content:encoded><p>Disclaimer: I do have a small credit in this book. A long time ago, the author mentioned that he was looking for people to help out, so I stepped up. It was only participating in discussions like “What do you think about version control’s use in industry?”; I didn’t help with any actual writing, and yesterday was my first time reading virtually all the final content. No monetary interest, of course.</p>
<h3>Lots of new books lately</h3>
<p>I’ve been doing a lot of reading over the last few weeks, some of it technical, some of it non. I’ve been meaning to type up a few book reviews, and yesterday I finished reading my copy of Josh Carter’s <a href="http://pragprog.com/book/jcdeg/new-programmer-s-survival-manual">“New Programmer’s Survival Manual,”</a> published by PragProg. Since that’s a bit lighter material than Spinoza, I figured I’d write it up first.</p>
<p>The <em>Survival Manual</em> is intended to be the book you should buy after you graduate from college, but before you get a job. It’s a bunch of tips, arranged by topic, that give you a taste of what ‘real world’ programming is like, as well as what working life is about. I think that it accomplishes this goal wonderfully, but if you’d like to hear specifics, read on.</p>
<h2>The Good</h2>
<p>One of my roommates is not particularly technical. She picked the book up, and read the introduction and the first tip. She then said something that, after reading all of it, I agree with 100%. “I like this guy’s writing style. It’s very… readable.” Josh’s prose feels… comfortable. Like you’re talking to a down-to-earth friend, or co-worker. Which is great, given the topic. I really enjoyed the actual <em>reading</em> of this book. Especially in technical books, this is a great quality to have, but I guess this book isn’t exactly technical; it’s much more about soft skills than hard.</p>
<p>I also really enjoyed the cookbook-style organization. Basically, the book is one big ball of tips that you can read individually. They’re grouped together with three or four others that are on the same topic, but each stands alone, and they cross-reference each other when appropriate. While I sat down and read it front to back, I’d imagine it’s pretty easy to cherry-pick certain things and read it in a more non-linear fashion.</p>
<p>I also feel that ‘pragmatic’ is a good description of most of the content. In particular, as much as we all love DVCS, the book frankly states that most places still use centralized version control, which is totally true. He makes sure to carefully not be particularly dogmatic about the specifics of anything, focusing on strategy more than tactics. For example, “Get good with your editor, make sure it can work effectively without using the mouse, for speed,” not “zomg emacs r teh suck, use vim.”</p>
<p>Finally, Josh links to copious other works, and provides references for all of it. Some stuff is just general internet culture, but the bibliography alone would assist any programmer in finding some useful primary materials to read. Here’s the first six: <em>SICP</em>, <em>Getting Things Done</em>, <em>Extreme Programming Explained</em>, <em>TDD By Example</em>, <em>The Mythical Man-Month</em>, and <em>Foundations of Object Oriented Languages: Types and Semantics</em>. Oh, and just because there’s some XP/Agile stuff in those particular books, don’t think it’s slanted towards that: waterfall is explained as well. Just so happens those books were first.</p>
<h2>The Bad</h2>
<p>There’s not that much bad that I can say about this book, really. There are a few small things, though:</p>
<p>There’s a ‘white belt brown belt black belt’ metaphor that’s supposed to indicate a difficulty of each tip. Given that there’s only two black belt tips, I didn’t feel that three levels were really needed. I also just thought that there wasn’t a huge difference between white and brown, either, so I’m not sure that it really adds any utility.</p>
<p>Because this book is fairly high-level, it might be hard to actually <em>apply</em> these lessons. This might be more of a problem with readers than with the actual book, but I can’t count how many times I (and others) have said “Of course, write descriptive variable names!!1” and then typed <code>x = 5</code>. It’s just the way things are. And since this book is essentially a collection of advice at that high level, I can see someone reading this book, and then just simply not following the advice. I’m not sure if this is something Josh could have fixed, or if it’s inherent in this kind of book. Also, it won’t affect a diligent reader. It is something to think about, though.</p>
<h2>Summary: If you just graduated, buy this book</h2>
<p>This book is totally worth it. The style reminds me of the seminal <em>Pragmatic Programmers</em>. If you’re a freshly minted graduate, put this book at the top of your reading list, then also grab a copy of all the books in the bibliography to read later.</p>
</content:encoded></item><item><title>Burnout</title><link>https://steveklabnik.com/writing/burnout/</link><guid isPermaLink="true">https://steveklabnik.com/writing/burnout/</guid><pubDate>Thu, 17 Nov 2011 00:00:00 GMT</pubDate><content:encoded><p>I have a confession to make: Other than one little stint at RubyC, I haven’t really written code in almost a month. And I haven’t really written a ‘real’ blog post in almost six weeks.</p>
<p>It happens. I was hitting it really hard there for a while, and the combination of stresses from travel, speaking, writing, conference parties, and everything else was really getting to me. I don’t think that I was burned out <em>yet</em>, but I decided to explicitly take the last two weeks off from doing almost anything to prevent myself from getting to that point.</p>
<p>This doesn’t mean that I’m going to <em>stop</em> doing any of these things. I’d be lying if I tried to say that I’d be reintroducing long-term balance into my life. That balance never really existed. I’ve always been one to throw myself at things 100%. Balance is something I don’t do particularly well.</p>
<p>One thing I am going to be working on, though, is this: Hackety and Shoes stress me out. Like a lot. This causes me to avoid working on them, which makes them not improve so quickly, which makes me stress about it worse. I’m not really sure how to fix that one… but at some point, I’m going to have to confront that.</p>
<p>Anyway, hopefully I’ll be back at it soon. Those of you waiting on some of my projects, my bad. Things will still be slow for a bit. I’m going to D.C. this weekend to teach kids with Hackety and Lego Mindstorms, so maybe that’ll get me out of this funk. Or maybe it’ll delay it another couple of days.</p>
<p>But I’ll be around again soon.</p>
</content:encoded></item><item><title>Your startup is not a platform</title><link>https://steveklabnik.com/writing/your-startup-is-not-a-platform/</link><guid isPermaLink="true">https://steveklabnik.com/writing/your-startup-is-not-a-platform/</guid><pubDate>Tue, 25 Oct 2011 00:00:00 GMT</pubDate><content:encoded><p>No, seriously. This is a public service announcment to all MBAs who want to do startups: ‘platform’ does not mean ‘web site.’</p>
<p>I know that ‘platform’ is a hot thing right now. But seriously, the term has become basically useless. It’s almost to the point that I just pretend you don’t even say it.</p>
<p>A platform is primarily a B2B play. It can be B2C too, maybe, but largely, to have a platform, you need to enable other people to build something. Usually, that’s another business. But if your website does not enable others to build something, or you’re not white labeling it in some way, you’re not a platform.</p>
<p>A platform basically needs an API. APIs are the primary way that two pieces of software connect to one another. If your web service doesn’t have an API, it’s not a platform. White labeling being an exception.</p>
<p>I’d make this rant longer, but I really need a good platform that addresses my market segment. Ideally, it’ll enable some win-win synergies with other channel partners and grow the pie for both of us. Until then, I really won’t find a good solution for this real pain I feel as a content producer.</p>
</content:encoded></item><item><title>For science: I'm killing my cell phone</title><link>https://steveklabnik.com/writing/for-science-im-killing-my-cell-phone/</link><guid isPermaLink="true">https://steveklabnik.com/writing/for-science-im-killing-my-cell-phone/</guid><pubDate>Wed, 19 Oct 2011 00:00:00 GMT</pubDate><content:encoded><p>Yep. Doin’ it. This morning I swung by the Sprint store and picked up a 4G hotspot. Gonna transfer my number to Google Voice this evening.</p>
<h2>lolwhut?</h2>
<p>Yep. I’m terribly addicted to my phone right now. This is fine, but I’m always interested in improving myself. I have a few friends who’ve done similar things, and really enjoyed it. So I’m giving it a solid 6 months. Sometimes, experiments work, and sometimes they don’t. I’m doing this for fun more than anything, and so it might not work out. Google Maps is damn handy.</p>
<h2>So I can’t call you?</h2>
<p>Naw, it just means there might be a tad more latency. I’m on my computer all the effing time anyway, so I doubt that you’ll really notice. If someone calls when I’m not at my computer, Google will just take a recording and email me the transcript. No biggie.</p>
<h2>Yeah, well, have fun with that!</h2>
<p>I will. And I’ll keep you all posted on how it goes. I plan to do a retrospective monthly.</p>
<p>EDIT: I had to change my number because Google Voice wouldn’t transfer my old one. So get a hold of me some other way, and I’ll give you the new one.</p>
</content:encoded></item><item><title>Matz is nice so we are nice</title><link>https://steveklabnik.com/writing/matz-is-nice-so-we-are-nice/</link><guid isPermaLink="true">https://steveklabnik.com/writing/matz-is-nice-so-we-are-nice/</guid><pubDate>Wed, 19 Oct 2011 00:00:00 GMT</pubDate><content:encoded><p>Rubyists have a saying, or at least, we used to. “Matz is nice so we are nice.” This has sort of been lost over the years… some people who have been around Ruby for a long time still say this, but it’s something that’s gone in and out of popularity.</p>
<h2>Nice should be the default</h2>
<p>Most of being nice, in a software context, is about being considerate. Not unnecessarily trash talking other people’s stuff. Not going out of your way to be a jerk. Try to be helpful. Treat others how you want to be treated.</p>
<h2>Sometimes, the truth hurts</h2>
<p>Being considerate of others, however, can only go so far. While a lot of what we do is subjective, a lot of is objective, too. If you’re building a new project, not saying “this is why we’re awesome” is silly. Old code is often bad. Other people’s code is often bad. The key in “not unnecessarily trash talking’ is the <em>neccesary</em> part. There’s a difference between”My library is better" and “My library is better and their library sucks.”</p>
<h2>The last thing I’ll say about RVM Drama</h2>
<p>After talking to a few people, I guess I committed a bit of the sin I was trying to rail against: I got some feedback that it was thought that I was villainizing Sam, and a little easy on Wayne. This is natural, because I’m friends with Wayne and I don’t really know Sam, but in the interest of being nice myself, I just want to say this: I think that all that drama last week was largely a perfect storm of a few different factors, and I don’t think that Sam set out to hurt anyone specifically. If and when I ever have an issue with RVM, I’m totally going to try out rbenv. I like having options. I like there being multiple options. TMTOWTDI. And Sam builds a ton of other software that I really enjoy, like Sprockets, and pow. There’s a big difference between “that guy is an asshole” and “I think that that situation could have been handled better.”</p>
<p>Let’s all build lots of cool software together.</p>
</content:encoded></item><item><title>Rubinius is awesome</title><link>https://steveklabnik.com/writing/rubinius-is-awesome/</link><guid isPermaLink="true">https://steveklabnik.com/writing/rubinius-is-awesome/</guid><pubDate>Tue, 04 Oct 2011 00:00:00 GMT</pubDate><content:encoded><p>You walk into work tomorrow morning, and your boss says this:</p>
<blockquote>
<p>Boss: Hey, we’re gonna need to go ahead and have you implement require_relative in rubinius. We have some new servers coming in, and we’ll be running it in production, but we use some 1.9.2 features of MRI that haven’t been implemented yet. So if you could just go ahead and get to implementing that, that would be terrific, OK?</p>
</blockquote>
<p>Wat do?</p>
<p>(Disregard that your boss would never say this. I kinda wish some of mine would…)</p>
<blockquote>
<p>You: Excuse me, I believe you have my stapler…</p>
</blockquote>
<p>Err, sorry.</p>
<blockquote>
<p>You: No problem, boss. Rubinius is mostly just Ruby, and it has a really helpful core team. I’m sure it won’t be a big deal.</p>
</blockquote>
<p>Would you say this? Well, you should. I’m not gonna lie and say that <em>every</em> feature is super simple to implement, or rbx-2.0 would be out already, but for a language interpreter, Rubinius is one of the easiest I’ve ever tried to get into. I actually did implement require_relative last night, and it took me about an hour and a half.</p>
<p>This is my story.</p>
<p>(note: dramatazation. Actual wording made unneccesarily silly to protect the innocent. please do try this at home)</p>
<pre><code>$ ssh stevessecretserver
$ tmux attach
&lt;brings up irssi, I'm already in #rubinius on Freenode&gt;
#rubinius: steveklabnik: hey, I need require\_relative to run my app. How hard
would you think that is to implement?
#rubinius: evan: Naw, man, it'd take like fifteen minutes. Go check out
http://rubygems.org/gems/rbx-require-relative, it's alredy basically got what
you want
#rubinius: steveklabnik: sweet.
#rubinius: brixen: just grep for 'def require'
#rubinius: brixen: and run bin/mspec -tx19 core/kernel/require\_relative to
run the RubySpecs for it.
</code></pre>
<p>Armed with this knowledge, I found where require is defined in rbx:</p>
<pre><code>def require(name)
Rubinius::CodeLoader.require name
end
module_function :require
</code></pre>
<p>Of course, I had to look at CodeLoader:</p>
<pre><code># Searches for and loads Ruby source files and shared library extension
# files. See CodeLoader.require for the rest of Kernel#require
# functionality.
def require
Rubinius.synchronize(self) do
return false unless resolve_require_path
return false if CodeLoader.loading? @load_path
if @type == :ruby
CodeLoader.loading @load_path
end
end
if @type == :ruby
load_file
else
load_library
end
return @type
end
</code></pre>
<p>Hmm, see <code>CodeLoader.require</code>. There are codeloader18 and codeloader19 files…</p>
<pre><code>module Rubinius
class CodeLoader
# Searches $LOAD_PATH for a file named +name+. Does not append any file
# extension to +name+ while searching. Used by #load to resolve the name
# to a full path to load. Also used by #require when the file extension is
# provided.
def search_load_path(name, loading)
$LOAD_PATH.each do |dir|
path = "#{dir}/#{name}"
return path if loadable? path
end
return name if loading and loadable? "./#{name}"
return nil
end
end
end
</code></pre>
<p>Okay. So we define specific stuff into the 19 file. There’s also a kernel19. So I worked this up:</p>
<pre><code># Attempt to load the given file, returning true if successful. Works just
# like Kernel#require, except that it searches relative to the current
# directory.
#
def require_relative(name)
Rubinius::CodeLoader.require_relative(name)
end
module_function :require_relative
</code></pre>
<pre><code># requires files relative to the current directory. We do one interesting
# check to make sure it's not called inside of an eval.
def self.require_relative(name)
scope = Rubinius::StaticScope.of_sender
script = scope.current_script
if script
require File.join(File.dirname(script.data_path), name)
else
raise LoadError.new "Something is wrong in trying to get relative path"
end
end
</code></pre>
<p>But then… running the specs gives me bunches of errors. What gives?</p>
<pre><code>&lt;back in irc&gt;
#rubinius: steveklabnik: yo, my specs are failing, and I'm a n00b. Halp
pl0x?
#rubinius: brixen: oh, yeah, well, the StaticScope is the current scope,
and you're down another method, so that's _totally_ gonna screw up
your paths. Just get it above and pass it down. Oh, and File.join is
dumb, File.expand_path that shit!
#rubinius: steveklabnik: words.
</code></pre>
<p>So then I made this one little change:</p>
<pre><code># Attempt to load the given file, returning true if successful. Works just
# like Kernel#require, except that it searches relative to the current
# directory.
#
def require_relative(name)
scope = Rubinius::StaticScope.of_sender
Rubinius::CodeLoader.require_relative(name, scope)
end
module_function :require_relative
</code></pre>
<pre><code># requires files relative to the current directory. We do one interesting
# check to make sure it's not called inside of an eval.
def self.require_relative(name, scope)
script = scope.current_script
if script
require File.expand_path(name, File.dirname(script.data_path))
else
raise LoadError.new "Something is wrong in trying to get relative path"
end
end
</code></pre>
<pre><code>$ bin/mspec -tx19 core/kernel/require_relativerubinius 2.0.0dev (1.9.2 cade3517 yyyy-mm-dd JI) [x86_64-apple-darwin10.8.0].....................Finished in 0.191995 seconds1 file, 20 examples, 47 expectations, 0 failures, 0 errors
</code></pre>
<p>One more pull request, and I’ve got a patch in on rbx. I had one a few months ago, but this one was way cooler.</p>
<h2>In conclusion</h2>
<p>Even if I didn’t have some code to look at, because Rubinius is just Ruby, it was really easy to dive in and check out the code, because it’s all Ruby. Seriously. Rubinius is just Ruby. Just Ruby. Ruby.</p>
<p>So dive in and add some features today! If having an awesome alternate Ruby implementation isn’t enough for you, the rbx team will bribe you with special stickers for committers, and sweet black-on-black tshirts for people with 10 commits or more!</p>
</content:encoded></item><item><title>GitHub is anarchy for programmers</title><link>https://steveklabnik.com/writing/github-is-anarchy-for-programmers/</link><guid isPermaLink="true">https://steveklabnik.com/writing/github-is-anarchy-for-programmers/</guid><pubDate>Sat, 01 Oct 2011 00:00:00 GMT</pubDate><content:encoded><p>I finally got to see Zach Holman give his talk, <a href="http://speakerdeck.com/u/holman/p/how-github-uses-github-to-build-github">How GitHub Uses GitHub to Build GitHub</a>. It was awesome. Zach is a great speaker, and the slides don’t do it justice. I highly recommend catching it on video or in person sometime.</p>
<p>Anyway, there’s something that I found really interesting about GitHub’s ‘process’: it’s basically anarchism, applied to software development.</p>
<p>If you’re not familliar with anarchism, check out <a href="http://truth-reason-liberty.blogspot.com/p/what-i-believe-in.html">“What I believe in”</a>. It’s a pretty decent introduction to a complicated topic. Ultimately, anarchism is fairly simple, but it requires dismantling a <em>lot</em> of beliefs that Americans in particular hold about society, power, and control. We’re quite happy in this country to be slaves to the rich, but that’s another blog post. ;)</p>
<p>Another excellent post on that blog is <a href="http://truth-reason-liberty.blogspot.com/2010/11/digression-on-what-it-means-to-be.html">"A Digression On What It Means to Be An Anarchist</a>. I’m going to use it as a framework to point out my comparison to GitHub. The first sentence is stolen from TFA, but the second sentence is my GitHub comparison.</p>
<h2>What anarchists oppose</h2>
<ul>
<li>Hierarchy – anarchists oppose domination of one person or group of people by another as detrimental to human society. GitHub has built a very flat process, where “some days the CEO ships more code than I do.”</li>
<li>Authority – all forms of authority must bear a heavy burden of proof in order to demonstrate their legitimacy and necessity. GitHub has hired people that have built awesome things, and have shown themselves to be a trusted authority in whatever kind of software that they build. Nobody needs to question if anyone is in the position to deploy or review some code.</li>
<li>The state – centralised rule of a set geographical area (country) or people (nation) by a government of elites is inherently illegitimate. Okay, this has nothing to do with GitHub all together. All abstractions are leaky here. :) The same goes for Capitalism and State Socialism too.</li>
<li>Nationalism and fascism – these are but the worst forms of the state, gaining the loyalty of the people with strong, often brutal discipline and by developing an almost religious, fevered love of the state and the rulers in the form of patriotism. GitHub actively uses multiple programming languages in production, and while it came out of the Ruby world, they don’t really run around pitching “Ruby is awesome and the only thing you should ever use.”</li>
<li>Discrimination – nobody should be excluded or discriminated against based on nothing more than their gender, ethnicity, sexuality, background, or beliefs. I don’t know everyone at GitHub, but it’s a pretty diverse crew overall. This is hard for any tech company, but they’re doing pretty well on the race/sex thing.</li>
</ul>
<h2>What anarchists stand for</h2>
<ul>
<li>Liberty – all people should be free to live their life as they see fit, without rules and laws passed from above that serve no purpose other than control and domination, as long as they are not infringing the right of anybody else to the same. See Homan’s talk. That’s pretty much how they work.</li>
<li>Equality – as stated above, nobody should face discrimination because of their gender, ethnicity, sexuality, background, or beliefs. See above comment about GitHub’s varied devs.</li>
<li>Community – human civilisation evolved, from its primitive roots, through the priciple of Mutual Aid. GitHub has a lot of elements of mutal aid in their process, where they mention some people in some pull requests, work together on features, and generally high five until the code is running in production.</li>
<li>Solidarity – humanity is divided only between the rulers and the ruled. Other divisions, those which bring about sexism, racism, heterosexism, and other bigotries, are promoted by the ruling class in order to divide their subjects and keep them under control. GitHub holds tons of events, buys everyone lots of beer, and just generally encourages lots of friendly interactions between everyone. CodeConf was a great group of diverse people hanging out and having a good time.</li>
</ul>
<h2>How anarchy would work</h2>
<ul>
<li>Self-management – groups, such as workforces or local communities, would be free to operate and govern themselves free of any higher authority. GitHub has no deadlines, no prioritization meetings, and 100% “do what you want” time.</li>
<li>Free association – all individuals would be free to live where they wanted and associate with who they chose. Zach mentioned in his talk that different people work on different parts of the site, and it’s really based on what people are interested in and what they’ve done in the past.</li>
<li>Voluntary federation – instead of the state, where indivudal communities and groups of people are bound together by the coercive force of a central authority, local communities and workers collectives can choose for themselves which other communities or collectives to associate with. Sort of like the free association comment, people get together and work on what they want to, and these groups are organic.</li>
<li>Direct democracy – unlike in parliamentary democracy, these spokespeople would be just that, elected not to a position of authority but to voice decisions that remain in the hands of the people, as in trade union and workers council structures. Anybody can comment on a pull request, add some commits, put in their two cents, and be heard.</li>
<li>Mutual Aid – in participatory communities and workers collectives, Mutual Aid is a central principle. Easily summed up with the phrase “from each according to his ability, to each according to his need,” this boils down to voluntary cooperation, fair distribution of resources, and help and support to those who need it within a local community. As mentioned before, everybody helps each other out at GitHub to get things done.</li>
<li>Free, fair trade – for the sharing of resources between different communities and individuals who opt out of Mutual Aid, anarchy would see the emergence of a truly free market. This doesn’t really fit with software, I think. Same with ‘individual liberty.’</li>
<li>Collective defence – this is not to say that anarchist society will contain “perfect people,” and there will certainly be acts of aggression, oppression, and violence – albeit on a lesser scale than is commonplace in today’s world. I’d liken this to things like “the site is down.” Everyone at GitHub cares about GitHub being up, and so they don’t need someone who’s responsible for ‘defense,’ as if shit hits the fan, everyone chips in and takes care of it.</li>
<li>Justice, not vengeance – courts would be elected for each individual case, rather than appointed and given unnecessary authority, with the aim to establish guilt or innocence, negotiate reparations, and organise rehabilitation rather than to support the oppressive prison systems which only make matters worse by serving as little more than universities of crime. I’m not sure this fits well here.</li>
</ul>
<h2>All abstractions are leaky</h2>
<p>Just like when ‘lean’ was a concept that applied manufacturing techniques to software development, I don’t think that ‘anarchism’ is a 100% fit with GitHub’s process. I do think that it’s close enough to hold a lot of value, however, even to the point that Zach had to write a blog post about <a href="http://zachholman.com/posts/scaling-github-employees/">scaling</a>, which is something that <em>always</em> gets leveled at people talking about anarchism. I see GitHub in the same way that I see <a href="http://en.wikipedia.org/wiki/Anarchism_in_Spain">anarchist Catalonia</a> or the <a href="http://en.wikipedia.org/wiki/Free_Territory">Free Territory</a> in the Ukraine. It’s a place where ideals are actually working in practice. Let’s just hope that the <a href="http://en.wikipedia.org/wiki/Anarchist_Catalonia#Clashes_with_the_Communists">communists don’t betray them when the fascists show up</a>.</p>
<p>Damn leaky abstractions.</p>
</content:encoded></item><item><title>Real, modern Ruby development</title><link>https://steveklabnik.com/writing/real-modern-ruby-development/</link><guid isPermaLink="true">https://steveklabnik.com/writing/real-modern-ruby-development/</guid><pubDate>Wed, 28 Sep 2011 00:00:00 GMT</pubDate><content:encoded><p>I came across a blog post the other day titled <a href="http://ascarter.net/2011/09/25/modern-ruby-development.html">Modern Ruby Development</a>. While it’s a perfectly fine blog post (other than the digs at rvm…) it really should have been titled something more along the lines of “My default tooling to build a Rails application.” I thought of this yesterday, as I was invited to show up at <a href="http://www.vertigo.com/">Vertigo</a> and show them how we do things in the Open Source world. Vertigo builds lots of cool things, but they tend to be very .NET centric, and it was neat for both of us to see how the other half lives.</p>
<p>Shortly before my ‘presentation,’ we decided that I’d basicaly just build a little web app in front of them. Live coding to the max! We decided for maximum awesome, and since I was in town for the Twilio conference, that I’d make a Twilio app that they could text and it’d text you ’hello world: you said #{msg}" back. I thought I’d share that process with you, too. I ended up throwing away the code, though, because I committed my account credentials into the git repo to simplify things, so I’ll just be describing it to you. Check my other posts for code stuff. ;)</p>
<p>Oh, and one more thing: lots of this stuff is about getting ‘iteration zero’ out of the way: it’s setup work that you have to do to set the tone for everything else. In its own way, it’s also a tooling overview, I’ve just focused on the tools that help the process rather than the language stuff. I’ve already written a more ‘dev proccess’ style post over at <a href="http://timelessrepo.com/bdd-with-rspec-and-steak">Timeless</a>, about BDD.</p>
<h2>rvm –rvmrc –create 1.9.2@hello_text</h2>
<p>First up: decide what Ruby you’re using. I talked breifly about MRI, Rubinius, and JRuby, and some pros and cons for each. I still tend to use MRI for my production code at the moment, but soon, rbx will become my default.</p>
<p>I use rvm because it does exactly what I want.</p>
<h2>mvim README.md</h2>
<p>I really enjoy <a href="http://tom.preston-werner.com/2010/08/23/readme-driven-development.html">README driven development</a>. I didn’t actually demo this due to time constraints, but I generally write at least a minimal README.md before doing anything else. During Windy City Rails, Tom shared something that he likes doing: <code>mv README.md SPEC.md</code>. That way, you can keep your actual README in sync with what the code does right now, but have the final one that you want to work towards too.</p>
<h2>git push origin</h2>
<p>Now that we have a file, we can push it up to GitHub. A few clicks on the web, a <code>git add remote</code> and <code>git push -u origin master</code> and we’re off to the races.</p>
<h2>mvim Gemfile &amp;&amp; bundle</h2>
<p>Next up we talked about Bundler, Gemfiles, and managing gems. I talked about the ability to run your own private gem server for internal gems, and how easy it is to build and publish a gem.</p>
<h2>mkdir spec</h2>
<p>I then generally type this. I would be lying to you if I told you that I do TDD 100% of the time. Sometimes, I get lazy. But I try to. And since testing is such a huge part of Ruby culture, and we have world-class testing tools, this is where I started.</p>
<p>I wrote specs for a model class called SmsGateway. This class would handle sending a message to a phone via Twilio. I was able to show off rspec, talk about DSLs, mocking, and TDD.</p>
<h2>mvim something.rb</h2>
<p>In this case it’s sms_gateway.rb, but you know what I mean. Write some code, <code>rspec spec/sms_gateway_spec.rb</code>, repeat. Red, Green, Refactor. Wheee!</p>
<h2>mvim .travis.yml</h2>
<p>I wanted to show Travis off, so in my demo I did this before implementing the gateway class, but normally I’d do this after the first spec passes. Continuous Integration and even Continuous Deployment are awesome. Travis makes it super easy.</p>
<h2>Sign up for external services</h2>
<p>I went over to Twilio’s site, registered a new account, and got my credentials. I used their docs to figure out where to put them, and how to get it all going inside my gateway class (I just mocked it out before), and talked about how needing to Google for docs is a weakness of Ruby.</p>
<h2>heroku create &amp;&amp; git push heroku master</h2>
<p>Then I showed them how to get going with Heroku, and how easy it was to start giving them hundreds of dollars per month when you need to scale. .NET people are used to paying for everything. ;)</p>
<p>I also showed off one of my latest projects, which is an IRC bot that lets me deploy rstat.us into production. I have some serious Hubot envy. :D</p>
<h2>… repeat!</h2>
<p>I’d be lying if I said it worked the first time, I accidentally made a loop where the app texted itself. Whoops! But that was cool, and showing off my debugging process (I’m a <code>puts/throw params</code> debugger) was fun, as well. After realizing what I did wrong, we had the app working.</p>
<h2>It’s not quite perfect</h2>
<p>This dev process isn’t perfect yet. Nothing ever is. I’m always trying to improve upon it, but I’m pretty happy with how quickly I’m able to turn out new projects with a reasonable amount of quality. What’s your workflow? How does mine suck? <a href="mailto:[email protected]">Email me</a>, I’d love to hear about it.</p>
</content:encoded></item><item><title>I'm deleting my Facebook tonight</title><link>https://steveklabnik.com/writing/im-deleting-my-facebook-tonight/</link><guid isPermaLink="true">https://steveklabnik.com/writing/im-deleting-my-facebook-tonight/</guid><pubDate>Mon, 26 Sep 2011 00:00:00 GMT</pubDate><content:encoded><p>Well, <a href="http://adrianshort.co.uk/2011/09/25/its-the-end-of-the-web-as-we-know-it/">it’s the end of the web as we know it</a>. I’d already been thinking about this all weekend, and last night, I decided. I just can’t deal with Facebook’s crazy privacy stuff anymore. I mean, I already basically <a href="http://rstat.us/">forked Twitter</a> over their ToS, and Facebook has been a lot more evil for a lot longer. Frictionless sharing is pretty much the last straw.</p>
<p>I’ll be using this as an excuse to keep helping out with the federated web. There’s a lot of cool stuff in this area, but it needs help. If you’re a developer, consider helping out with the <a href="https://github.com/LockerProject/Locker">Locker Project</a> or <a href="https://github.com/homesteading">the Homesteading Project</a>. The only way that we can prevent people from becoming digial sharecroppers is by helping them to ‘own their own land.’</p>
<p>If we’re Facebook friends, you can always find out how to reach me at <a href="http://steveklabnik.com/">http://steveklabnik.com</a>. Please be careful, and consider if Facebook’s privacy settings are really worth it to you. They may be, and they may not. I’m under no illusions that the vast majority of people will give up Facebook, but I figured I’d warn you anyways.</p>
</content:encoded></item><item><title>More rstat.us refactoring</title><link>https://steveklabnik.com/writing/more-rstat-dot-us-refactoring/</link><guid isPermaLink="true">https://steveklabnik.com/writing/more-rstat-dot-us-refactoring/</guid><pubDate>Fri, 23 Sep 2011 00:00:00 GMT</pubDate><content:encoded><p>Hey everyone! I just wanted to share One More Thing with you about this rstat.us refactoring.</p>
<p>The main thrust of the last article I posted was to show you a technique for extracting a class, getting it under some kind of test, and then refactoring it a bit. Refactoring is always an iterative process, and Ryan Bates from the always awesome <a href="http://railscasts.com/">Railscasts</a> asked me why I made the Salmon class into a module, especially given my recent <a href="/2011/09/06/the-secret-to-rails-oo-design.html">rant</a> against modules. The answer was, ‘because it’s simpler, and the first thing I thought of.’ He shared with me an alternate implementation that I like too, and I wanted to share with you. Check it:</p>
<pre><code>class SalmonNotifier
def initialize(user, feed)
@user = user
@feed = feed
end
def mention(update)
deliver OStatus::Salmon.new(update.to_atom("http://#{@user.author.domain}/"))
end
def follow
deliver OStatus::Salmon.from_follow(@user.author.to_atom, @feed.author.to_atom)
end
def unfollow
deliver OStatus::Salmon.from_unfollow(@user.author.to_atom, @feed.author.to_atom)
end
protected
def deliver(salmon)
send_envelope_to_salmon_endpoint salmon.to_xml(@user.to_rsa_keypair)
end
def send_envelope_to_salmon_endpoint(envelope)
uri = URI.parse(@feed.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<p>This follows a more OO style, and is a bit more well-factored. Here’s his description of what he did:</p>
<blockquote>
<ol>
<li>Switched to a class and gave it a name that is a noun (maybe name should be different though).</li>
</ol>
</blockquote>
<ol>
<li>
<p>Moved common method params into instance variables.</p>
</li>
<li>
<p>This simplified the mention/follow/unfollow methods enough to be on one line.</p>
</li>
<li>
<p>Renamed "<code>send_to_salmon_endpoint</code>" to "<code>deliver</code>" because it feels nicer, the "salmon endpoint" can be assumed due to the name of the class. I generally don't like to put the class name in the method name.</p>
</li>
<li>
<p>Extracted commented out into its own method with the same name. I don't know if this is really necessary though (same reason as above).</p>
</li>
</ol>
<p>The only thing that bothers me now is the constant access of <code>@user.author</code> and <code>@feed.author</code>. This makes me think it should be interacting directly with authors. However you still need access to <code>@user.to_rsa_keypair</code> but maybe that method could be moved elsewhere more accessible.</p>
<blockquote></blockquote>
<p>All of these changes are pretty good. Ryan’s suggestions for moving forward are pretty good, as well, but I’d add one more thing: we’re starting to collect a bunch of methods all related to delivering more generic HTTP requests in the protected section. This also might be worth pulling out, too. Protected/private in general is a minor smell that indicates there’s code being used multiple times that’s not part of the main objective of the class. It might not be worth it yet, but then again, it might. Speaking of protected methods, you might wonder why I ended up mocking out my protected method in the last post, as well. There are no more tests that actually test the method is working properly. This is true, and it’s because I was leading into thinking of something like this. Mocked tests are almost always unit tests, and two methods are two separate units, so I had just ended up doing it out of habit. This further shows that maybe an extraction of another class is worthwhile.</p>
</content:encoded></item><item><title>Extracting domain models: a practical example</title><link>https://steveklabnik.com/writing/extracting-domain-models-a-practical-example/</link><guid isPermaLink="true">https://steveklabnik.com/writing/extracting-domain-models-a-practical-example/</guid><pubDate>Thu, 22 Sep 2011 00:00:00 GMT</pubDate><content:encoded><p>Hey everyone! We’ve been doing a lot of refactoring on rstat.us lately, and I wanted to share with you a refactoring that I did. It’s a real-world example of doing the domain models concept that I’ve been talking about lately.</p>
<h2>Step one: check the tests</h2>
<blockquote>
<p>I don’t know how much more emphasized step 1 of refactoring could be: don’t touch anything that doesn’t have coverage. Otherwise, you’re not refactoring; you’re just changing shit. - Hamlet D’Arcy</p>
</blockquote>
<p>One of the best reasons to extract extra domain models is that they’re often much easier to test. They have less dependencies, less code, and are much simpler than the Mega Models that happen if you have a 1-1 model to table ratio.</p>
<p>Let’s check out the code. It’s in our <a href="https://github.com/hotsh/rstat.us/blob/362cb38031/app/models/user.rb#L209">User model</a>:</p>
<pre><code># Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
f = Feed.first :id =&gt; to_feed_id
u = Update.first :id =&gt; update_id
base_uri = "http://#{author.domain}/"
salmon = OStatus::Salmon.new(u.to_atom(base_uri))
envelope = salmon.to_xml self.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(f.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
</code></pre>
<p>You’re probably saying “this is pretty reasonable.” Prepare to get shocked…</p>
<p>This code has very little to do with the user, other than using some of the User’s attributes. This really could be factored out into a domain object whose job it is to push Salmon notifications. But first, let’s see if we can get some tests in place so we know <em>everything</em> isn’t broken. Examining this method, we need two things: an <code>update_id</code> and a <code>feed_id</code>. Here’s my first stab at an integration test:</p>
<pre><code>describe "salmon update" do
it "integration regression test" do
feed = Feed.create
update = Update.create
salmon = double(:to_xml =&gt; "&lt;xml&gt;&lt;/xml&gt;")
uri = double(:host =&gt; "localhost", :port =&gt; "9001", :path =&gt; "/")
Ostatus::Salmon.should_receive(:new).and_return(salmon)
Uri.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(mock(:post =&gt; true))
user = User.create
user.send_mention_notification(update.id, feed.id)
end
end
</code></pre>
<p>This is an integration test, we’re obviously testing stuff with a ton of models. After running <code>ruby test/models/notify_via_salmon.rb</code> about a zillion times, I ended up with this running test:</p>
<pre><code>require 'minitest/autorun'
require 'rspec/mocks'
require 'mongo_mapper'
require 'whatlanguage'
MongoMapper.connection = Mongo::Connection.new("localhost")
MongoMapper.database = 'rstatus-test'
require_relative '../../app/models/feed'
require_relative '../../app/models/update'
require_relative '../../app/models/author'
require_relative '../../app/models/authorization'
$:.unshift("lib")
require_relative '../../app/models/user'
require 'ostatus'
describe "salmon update" do
before :each do
User.all.each(&amp;:destroy)
RSpec::Mocks::setup(self)
end
it "integration regression test" do
author = Author.create
user = User.create!(:username =&gt; "steve", :author =&gt; author)
feed = user.feed
update = Update.create!(:feed_id =&gt; feed.id, :author_id =&gt; author.id, :text =&gt; "hello world")
salmon = double(:to_xml =&gt; "&lt;xml&gt;&lt;/xml&gt;")
uri = double(:host =&gt; "localhost", :port =&gt; "9001", :path =&gt; "/")
OStatus::Salmon.should_receive(:new).and_return(salmon)
URI.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(mock(:post =&gt; true))
user.send_mention_notification(update.id, feed.id)
end
end
</code></pre>
<p>Holy. Fuck. Seriously. Let me just quote <a href="http://avdi.org/devblog/2011/04/07/rspec-is-for-the-literate/">Avdi Grimm</a>:</p>
<blockquote>
<p>Mocks, and to some degree RSpec, are like the Hydrogen Peroxide of programming: they fizz up where they encounter subtle technical debt.</p>
</blockquote>
<p>This test class is insane. It runs faster than many people’s integration tests, clocking in at about 0.4 seconds. But that’s still an order of magnitude slower than I’d like. A suite with 200 tests would still take over a minute. Also, look at all this junk that we have to do for setup.</p>
<p>All of this pain and effort doesn’t mean that testing sucks: it means that our implementation is terrible. So let’s use that to guide us. We’ll try to fix this code by eliminating all of this junk.</p>
<h2>Step two: simple extraction</h2>
<p>First, let’s extract this out to a domain model. Here’s the new code:</p>
<pre><code># Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
NotifyViaSalmon.mention(update_id, to_feed_id)
end
</code></pre>
<pre><code>module NotifyViaSalmon
extend self
def mention(update_id, to_feed_id)
f = Feed.first :id =&gt; to_feed_id
u = Update.first :id =&gt; update_id
base_uri = "http://#{author.domain}/"
salmon = OStatus::Salmon.new(u.to_atom(base_uri))
envelope = salmon.to_xml self.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(f.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<p>Now, when we run the test (via <code>ruby test/models/notify_via_salmon_test.rb</code>) we see an error:</p>
<pre><code>1) Error:
test_0001_integration_regression_test(SalmonUpdateSpec):
NameError: undefined local variable or method `author' for NotifyViaSalmon:Module
/Users/steveklabnik/src/rstat.us/app/models/notify_via_salmon.rb:8:in `mention'
</code></pre>
<p>This is a form of Lean On The Compiler. This test is now failing because it’s relying on stuff that used to be internal to the User, and we don’t have that stuff now. After doing this a few times, we’re left with this:</p>
<pre><code># Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
NotifyViaSalmon.mention(self, update_id, to_feed_id)
end
</code></pre>
<pre><code>module NotifyViaSalmon
extend self
def mention(user, update_id, to_feed_id)
f = Feed.first :id =&gt; to_feed_id
u = Update.first :id =&gt; update_id
base_uri = "http://#{user.author.domain}/"
salmon = OStatus::Salmon.new(u.to_atom(base_uri))
envelope = salmon.to_xml user.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(f.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<p>Okay. Now we’re cooking. We have a test that’s isolated, yet still does what we thought it did before.</p>
<h2>Step three: break dependencies</h2>
<p>Next step: let’s break the hard dependency this method has on both Feed and Update. We can do this by moving the finds up into the User method instead of keeping them in the mention method:</p>
<pre><code># Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
feed = Feed.first :id =&gt; to_feed_id
update = Update.first :id =&gt; update_id
NotifyViaSalmon.mention(self, update, feed)
end
</code></pre>
<pre><code>module NotifyViaSalmon
extend self
def mention(user, update, feed)
base_uri = "http://#{user.author.domain}/"
salmon = OStatus::Salmon.new(update.to_atom(base_uri))
envelope = salmon.to_xml user.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(feed.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<p>Okay. Tests passing. Sweet. Now we can try testing <em>just</em> the mention method. Let’s try it by killing most of that crap that was in our test:</p>
<pre><code>require 'minitest/autorun'
require 'rspec/mocks'
require_relative '../../app/models/notify_via_salmon'
describe "salmon update" do
before :each do
RSpec::Mocks::setup(self)
end
it "integration regression test" do
NotifyViaSalmon.mention(stub, stub, stub)
end
end
</code></pre>
<p>Let’s try running this and seeing what happens:</p>
<pre><code>$ ruby test/models/notify_via_salmon_test.rb
Loaded suite test/models/notify_via_salmon_test
Started
E
Finished in 0.000992 seconds.
1) Error:
test_0001_integration_regression_test(SalmonUpdateSpec):
RSpec::Mocks::MockExpectationError: Stub received unexpected message :author with (no args)
</code></pre>
<p>First of all, daaaaamn. 0.001 seconds. Nice. Secondly, okay, we are getting some messages sent to our stubs. Let’s flesh them out to make things pass:</p>
<pre><code>require 'minitest/autorun'
require 'rspec/mocks'
require 'ostatus'
require_relative '../../app/models/notify_via_salmon'
describe "salmon update" do
before :each do
RSpec::Mocks::setup(self)
end
it "integration regression test" do
user = stub(:author =&gt; stub(:domain =&gt; "foo"), :to_rsa_keypair =&gt; stub)
update = stub(:to_atom =&gt; "")
feed = stub(:author =&gt; stub(:salmon_url =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:new).and_return(salmon)
uri = stub(:host =&gt; "", :port =&gt; "", :path =&gt; "")
URI.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(stub(:post =&gt; ""))
NotifyViaSalmon.mention(user, update, feed)
end
end
</code></pre>
<p>Okay! Note how similar this looks to the previous test. We’re still mocking a bunch of ‘external’ stuff. But other than that, our test is pretty simple: we make three stubs, and we call our mention method.</p>
<h3>Step four: repeat</h3>
<p>We could do some more work on this method. There are a few things that are a bit smelly: the nested stub of User is not great. The fact that we’re stubbing out three external dependencies isn’t great. But before we get to that, let’s check out another method that looks similar: <code>send_{follow,unfollow}_notification</code>. Here’s <a href="https://github.com/hotsh/rstat.us/blob/362cb38031/app/models/user.rb#L167">some code</a>:</p>
<pre><code># Send Salmon notification so that the remote user
# knows this user is following them
def send_follow_notification to_feed_id
f = Feed.first :id =&gt; to_feed_id
salmon = OStatus::Salmon.from_follow(author.to_atom, f.author.to_atom)
envelope = salmon.to_xml self.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(f.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
# Send Salmon notification so that the remote user
# knows this user has stopped following them
def send_unfollow_notification to_feed_id
f = Feed.first :id =&gt; to_feed_id
salmon = OStatus::Salmon.from_unfollow(author.to_atom, f.author.to_atom)
envelope = salmon.to_xml self.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(f.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
</code></pre>
<p>Look familliar? Yep! 90% the same code! We’ll do the same process to these methods, just like the other one. Check out this code and test:</p>
<pre><code># Send Salmon notification so that the remote user
# knows this user is following them
def send_follow_notification to_feed_id
feed = Feed.first :id =&gt; to_feed_id
NotifyViaSalmon.follow(self, feed)
end
# Send Salmon notification so that the remote user
# knows this user has stopped following them
def send_unfollow_notification to_feed_id
feed = Feed.first :id =&gt; to_feed_id
NotifyViaSalmon.unfollow(self, feed)
end
# Send an update to a remote user as a Salmon notification
def send_mention_notification update_id, to_feed_id
feed = Feed.first :id =&gt; to_feed_id
update = Update.first :id =&gt; update_id
NotifyViaSalmon.mention(self, update, feed)
end
</code></pre>
<pre><code>module NotifyViaSalmon
extend self
def mention(user, update, feed)
base_uri = "http://#{user.author.domain}/"
salmon = OStatus::Salmon.new(update.to_atom(base_uri))
envelope = salmon.to_xml user.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(feed.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
def follow(user, feed)
salmon = OStatus::Salmon.from_follow(user.author.to_atom, feed.author.to_atom)
envelope = salmon.to_xml user.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(feed.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
def unfollow(user, feed)
salmon = OStatus::Salmon.from_unfollow(user.author.to_atom, feed.author.to_atom)
envelope = salmon.to_xml user.to_rsa_keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(feed.author.salmon_url)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<pre><code>require 'minitest/autorun'
require 'rspec/mocks'
require 'ostatus'
require_relative '../../app/models/notify_via_salmon'
describe NotifyViaSalmon do
before :each do
RSpec::Mocks::setup(self)
end
it ".mention" do
user = stub(:author =&gt; stub(:domain =&gt; "foo"), :to_rsa_keypair =&gt; stub)
update = stub(:to_atom =&gt; "")
feed = stub(:author =&gt; stub(:salmon_url =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:new).and_return(salmon)
uri = stub(:host =&gt; "", :port =&gt; "", :path =&gt; "")
URI.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(stub(:post =&gt; ""))
NotifyViaSalmon.mention(user, update, feed)
end
it ".follow" do
user = stub(:author =&gt; stub(:domain =&gt; "foo", :to_atom =&gt; ""), :to_rsa_keypair =&gt; stub)
feed = stub(:author =&gt; stub(:salmon_url =&gt; "", :to_atom =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:from_follow).and_return(salmon)
uri = stub(:host =&gt; "", :port =&gt; "", :path =&gt; "")
URI.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(stub(:post =&gt; ""))
NotifyViaSalmon.follow(user, feed)
end
it ".unfollow" do
user = stub(:author =&gt; stub(:domain =&gt; "foo", :to_atom =&gt; ""), :to_rsa_keypair =&gt; stub)
feed = stub(:author =&gt; stub(:salmon_url =&gt; "", :to_atom =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:from_unfollow).and_return(salmon)
uri = stub(:host =&gt; "", :port =&gt; "", :path =&gt; "")
URI.should_receive(:parse).and_return(uri)
Net::HTTP.should_receive(:new).and_return(stub(:post =&gt; ""))
NotifyViaSalmon.unfollow(user, feed)
end
end
</code></pre>
<p>Now we can see some of these patterns start to emerge, eh? All of this stuff is starting to come together. Let’s get rid of the URI and Net::HTTP stuff, as it’s just straight up identical in all three. Pretty basic Extract Method:</p>
<pre><code>module NotifyViaSalmon
extend self
def mention(user, update, feed)
base_uri = "http://#{user.author.domain}/"
salmon = OStatus::Salmon.new(update.to_atom(base_uri))
send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)
end
def follow(user, feed)
salmon = OStatus::Salmon.from_follow(user.author.to_atom, feed.author.to_atom)
send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)
end
def unfollow(user, feed)
salmon = OStatus::Salmon.from_unfollow(user.author.to_atom, feed.author.to_atom)
send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)
end
protected
def send_to_salmon_endpoint(salmon, keypair, uri)
envelope = salmon.to_xml keypair
# Send envelope to Author's Salmon endpoint
uri = URI.parse(uri)
http = Net::HTTP.new(uri.host, uri.port)
res = http.post(uri.path, envelope, {"Content-Type" =&gt; "application/magic-envelope+xml"})
end
end
</code></pre>
<p>We run our tests, it still all works. Now we can just mock out the endpoint method, and all of our tests on the individual methods become much, much simpler:</p>
<pre><code>require 'minitest/autorun'
require 'rspec/mocks'
require 'ostatus'
require_relative '../../app/models/notify_via_salmon'
describe NotifyViaSalmon do
before :each do
RSpec::Mocks::setup(self)
end
it ".mention" do
user = stub(:author =&gt; stub(:domain =&gt; "foo"), :to_rsa_keypair =&gt; stub)
update = stub(:to_atom =&gt; "")
feed = stub(:author =&gt; stub(:salmon_url =&gt; ""))
NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)
NotifyViaSalmon.mention(user, update, feed)
end
it ".follow" do
user = stub(:author =&gt; stub(:domain =&gt; "foo", :to_atom =&gt; ""), :to_rsa_keypair =&gt; stub)
feed = stub(:author =&gt; stub(:salmon_url =&gt; "", :to_atom =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:from_follow).and_return(salmon)
NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)
NotifyViaSalmon.follow(user, feed)
end
it ".unfollow" do
user = stub(:author =&gt; stub(:domain =&gt; "foo", :to_atom =&gt; ""), :to_rsa_keypair =&gt; stub)
feed = stub(:author =&gt; stub(:salmon_url =&gt; "", :to_atom =&gt; ""))
salmon = stub(:to_xml =&gt; "")
OStatus::Salmon.should_receive(:from_unfollow).and_return(salmon)
NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)
NotifyViaSalmon.unfollow(user, feed)
end
end
</code></pre>
<p>We’ve managed to get rid of all of the mocking of the URI stuff. It’s all details now. We also have great information about what is actually needed for this test: we know exactly what our objects expect the caller to pass. We’re only depending on ourselves and some external libraries. Our tests are fast. Overall, I’m feeling much better about this code than I was before.</p>
<h2>In conclusion</h2>
<p>I hope you’ll take a few things away from this post.</p>
<p><strong>Start with tests first!</strong> If you don’t do this, it’s very easy to introduce simple errors. Don’t be that guy. You know, the one that has to make <a href="https://github.com/hotsh/rstat.us/pull/398/files">pull requests like this…</a> I suck.</p>
<p><strong>Don’t be afraid to change the tests!</strong> As soon as you’ve verified that you’ve transcribed the code correctly, don’t be afraid to just nuke things and start again. Especially if you have integration level tests that confirm that your features actually <em>work</em>, your unit tests are expendable. If they’re not useful, kill them!</p>
<p><strong>Mock like a motherfucker.</strong> When done correctly, mocks allow you to help design your code better, make your tests fast, and truly isolate the unit under test. It’s pretty much all upside, unless you suck at them.</p>
<p><strong>Extract, extract, extract!</strong> Adele Goldberg once said, “In Smalltalk, everything happens somewhere else.” Make tons of little methods. They help to provide seams that are useful for testing. Smaller methods are also easier to understand.</p>
<p><strong>Refactoring is a marathon.</strong> Even with all this work, this code isn’t the best. The tests could even use a little TLC. But it’s still better than it was before. Those who ship, win. Make things a little better, <code>git commit &amp;&amp; git push</code>, repeat.</p>
</content:encoded></item><item><title>We forget that open source is made of people</title><link>https://steveklabnik.com/writing/we-forget-that-open-source-is-made-of-people/</link><guid isPermaLink="true">https://steveklabnik.com/writing/we-forget-that-open-source-is-made-of-people/</guid><pubDate>Mon, 12 Sep 2011 00:00:00 GMT</pubDate><content:encoded><p>Programmers like to think that software is the purest form of meritocracy. We like to consider ourselves scientists; what we do is entirely impersonal. I’m at Lone Star Ruby right now, and the first two talks are entirely about software as a science.</p>
<p>But this is a fallacy. There’s a human element to open source that’s often totally forgotten, and it drives people away.</p>
<h2>Software Celebrity and Hero Worship</h2>
<p>We sort of acknowledge this, in a way. There are most certainly software celebrities, even if their celebrity is localized to the community in which they operate. I heard someone say this today:</p>
<blockquote>
<p>If I can go to one Ruby conference and not hear anyone mention Zed Shaw or _why, I can die happy.</p>
</blockquote>
<p>If I say “jnicklas” you say “Capybara.” If I say “DHH” you say “Rails.” If I say “matz” you say “Ruby.” We have our heroes, villains, and foils, just like anybody else. But we forget that in every celebrity, there is a real person in there. Actual people, who have real thoughts, feelings, ideas, emotions, and that spend all of their time working for free just to make your lives better out of the goodness of their heart. Often <em>after</em> working at day jobs, often <em>at the expense</em> of a personal life, often <em>straining relationships</em> with people they care about.</p>
<h2>Competition Means Somebody Loses</h2>
<p>I used to think that Git was great because it made forks trivial. It changed what we mean by ‘fork.’ This is largely a good thing. But ‘the eff word’ used to be a last resort. The nuclear option. You’re fed up, can’t take it anymore, and so this is what they’ve driven you to do. Forking was Serious Business.</p>
<p>Now, GitHub has made it so easy to fork a project and distribute it to tons of people, people just fork things willy-nilly. And, from the outside, competition is good: in theory, it means that the good ideas work and the bad ideas don’t.</p>
<p>(A small aside: forking in git is different than ‘forking’ in svn. This overloading of terminology is unfortunate. The fork button is actually one of my favorite features of GitHub, because it makes contribution easier. But as it’s easier to publish new software, it’s easier to ‘fork’ in the svn sense. It used to be a pain to start a new project, and GitHub’s made that so easy that it happens more often than I think it should.)</p>
<p>But guess what: competition sucks. Cooperation is much better. One of the biggest problems with competition is that it means that somebody has to lose. For every winner, someone is a loser. That person is going to be hurting. In the case of open source, someone has spent (possibly) years of their life working on something, and now, all of that is gone. That shit stings.</p>
<p>Sometimes, forks are necessary. Sometimes, there are approaches that are fundamentally different. Forking is still an option, even if it’s a last resort. I’m also not saying that feelings should totally trump technical merit. All I’m saying is this: consider the blood, sweat, and tears of others before you act. Just in general.</p>
<p>It’s also not a zero-sum game, either. The pie can be grown. But that also doesn’t mean that it’s grown in a way that makes anyone feel good about it. There’s a difference between “We represent two sides of some coin” and “Those fuckers make shitty software.” Both approaches can grow the pie. One is far better than the other.</p>
<p>What’s the human cost of rapid competition between software projects? Is this cost worth it?</p>
<h2>Open Source: All take, no give</h2>
<p>The Pareto Principle: 80% of the effects come from 20% of the causes. This is absolutely true in OSS, and it might be more imbalanced. A few people contribute, and lots of people use. Sometimes, this is a good thing: it’s actually what draws me to open source. I can help people in a scalable way. My code is currently being used by tens of thousands of people. This is awesome. Some documentation that I’ve written is being read by millions of people. This is way more awesome.</p>
<p>But I rarely get any help with my projects. It’s really easy to let that get to a person.</p>
<p>Do you know how many people have told me that what I do is important? I got given a Ruby Hero award for my work on Hackety and Shoes. Do you know how many commits I have in on them? Probably less than 200. Do you know why? I’ve spent weeks of my life trying to track down bugs, but some of them are too hard for me, at my current stage of programming-fu. And it gets depressing. And everyone that’s ever offered to help has flaked on me. And all the recognition for something that was largely done by someone else gets to me, really deeply. And I get tons of people screaming about Lion support, and ‘this doesn’t work,’ and all sorts of things… and no patches. It’s why the ‘patches accepted’ saying was coined: this shit grinds on you after a while.</p>
<h2>All of my Heroes are Dying</h2>
<p>Today, this situation with rbenv and rvm made me realize something: everyone that I truly, deeply respect in open source has either burned out or left. I’ve said once before that there are four people that I truly respect in software: Zed Shaw, _why, Yehuda Katz, and (more recently) Wayne Seguin (and also Aaron Patterson. Dammit. This list is getting long). All of these developers are brilliant, work hard, and are tirelessly supporting their communities. That is, until they were driven away. This isn’t totally applicable to Yehuda, but it happened to Zed. It happened to _why. And I can see it happening, right now, to Aaron and Wayne. And Yehuda isn’t really doing Ruby…</p>
<p>But seeing this happen makes me deeply sad. For them personally, for my own selfish reasons, and that that’s the state of our communities.</p>
<h2>Where Does This Leave Us?</h2>
<p>I don’t have any answers. I only have a bunch of questions:</p>
<ul>
<li>Why do we drive people away from Open Source?</li>
<li>Can we learn to work together, instead of competing?</li>
<li>How can we better recognize this human element?</li>
</ul>
<p>If you don’t contribute to Open Source: please consider thanking someone that does open source, in a project that you love. Consider helping out, in even the smallest of ways.</p>
<p>Thanks.</p>
<p><a href="/2011/08/19/matz-is-nice-so-we-are-nice.html">My next post is a bit of a follow-up to this</a>.</p>
</content:encoded></item><item><title>Better Ruby Presenters</title><link>https://steveklabnik.com/writing/better-ruby-presenters/</link><guid isPermaLink="true">https://steveklabnik.com/writing/better-ruby-presenters/</guid><pubDate>Fri, 09 Sep 2011 00:00:00 GMT</pubDate><content:encoded><p>My <a href="/2011/09/06/the-secret-to-rails-oo-design.html">last blog post</a> caused a bit of a stir in some circles. I got a bunch of emails. Apparently, I need to expand on a few things. So here we go. Let’s rap about the Presenter pattern, shall we?</p>
<h2>No seriously, helpers suck</h2>
<blockquote>
<p>In Ruby, everything is an object. Every bit of information and code can be given their own properties and actions. - ruby-lang.org/about</p>
</blockquote>
<p>… except helpers. Why is it in Ruby that everything is an object, even integers, yet as soon as we need to format a date, we all turn into Dijkstra and bust out structured programming? Actually, come to think of it, Dijkstra wouldn’t even write any code, because it’s beneath him, but you get the idea. (if you don’t get this, it’s a joke. About proofs, telescopes, and CS…) Helpers are like a compliment that you don’t want to give, but feel obligated to: “Yeah, I mean, well, you tried, and that’s what counts, right?”</p>
<p>This is the topic of a future post, but when programming in a language, you want to work with its primary methods of abstraction. In Ruby, that’s objects, and there’s a good reason for that: functions don’t provide a sufficient amount of power to tackle hard problems. I don’t want to get into this either, but there’s a reason that objects exist, and that nobody’s making procedural languages anymore. No biggie. C has a special place in my heart.</p>
<p>But I digress: objects &gt; functions. At least in the context of getting stuff done in Ruby. This pretty much applies to most of the rest of the points in this post, so just keep that in the back of your brain.</p>
<h2>Why not a class method?</h2>
<p>Well, first of all, <a href="http://nicksda.apotomo.de/2011/07/are-class-methods-evil/">class methods also suck</a>. Here’s the issue: Can you tell me the difference between these two methods?</p>
<pre><code>def foo
"hello"
end
class Bar
def self.foo
"hello"
end
end
foo
Bar.foo
</code></pre>
<p>Yep, that’s right. You can see it clearly from the last two lines: class methods are functions that happen to be namespaced. That means they’re slightly better, but really, <code>DictionaryPresenter.as_dictionary(something)</code> might as well just be a helper. At least <code>as_dictionary(something)</code> is shorter.</p>
<p>Here’s another reason: I didn’t make that method be a class method, even though there was no state, because a really real presenter (I mean, that was a real world example, but…) generally works a little different. Usually, I make presenters that actually stand in for the objects they’re presenting. Check this out. Here’s the presenter I showed you:</p>
<pre><code>class DictionaryPresenter
def initialize(collection)
@collection = collection
end
def as_dictionary
dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
@collection.each do |p|
dictionary[p.title[0]] &lt;&lt; p
end
dictionary
end
end
</code></pre>
<p>The real-world presenter that I used this for looked like this:</p>
<pre><code>class DictionaryPresenter
include Enumerable
def initialize(collection)
@dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
collection.each do |p|
@dictionary[p.title[0]] &lt;&lt; p
end
end
def each &amp;blk
@dictionary.each &amp;blk
end
end
</code></pre>
<p>… or close to this. There was an ‘other’ category, and a few other things… but you get the idea. Now, instead of this:</p>
<pre><code>@posts = DictionaryPresenter.new(Post.all).as_dictionary
</code></pre>
<p>You do this:</p>
<pre><code>@posts = DictionaryPresenter.new(Post.all)
</code></pre>
<p>And the presenter actually stands in for the hash. A subtle but important difference. This gives you more options, because you have a real, live object instead of just some random processing. With this simplistic presenter, you might not see a huge benefit, but often, you’ll want something a bit more complicated. And since this was what I had in my head, I didn’t make <code>as_dictionary</code> be a static method.</p>
<p>Oh, and presenters that decorate a single object rather than a collection often implement <code>method_missing</code>, so that you only <em>add</em> methods rather than hide most of them. Some don’t. It depends.</p>
<h2>Blocks vs. Policy</h2>
<p>I also showed off a Ruby version of a design pattern known as the Policy pattern, or sometimes the Strategy pattern. Turns out that Policy is often a bit heavyweight for Ruby, especially in an example like this, so I wanted to show you an alternate version of it, taking advantage of a Ruby feature: blocks.</p>
<p>For reference, the old code, put into the new code:</p>
<pre><code>class DictionaryPresenter
include Enumerable
def initialize(policy, collection)
@dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
collection.each do |p|
@dictionary[policy.category_for(p)] &lt;&lt; p
end
end
def each &amp;blk
@dictionary.each &amp;blk
end
end
class UserCategorizationPolicy
def self.category_for(user)
user.username[0]
end
end
</code></pre>
<p>We could use blocks instead:</p>
<pre><code>class DictionaryPresenter
include Enumerable
def initialize(collection, &amp;blk)
@dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
collection.each do |p|
@dictionary[blk.call(p)] &lt;&lt; p
end
end
def each &amp;blk
@dictionary.each &amp;blk
end
end
DictionaryPresenter.new(Post.all) do |item|
item.title[0]
end
</code></pre>
<p>While this is shorter, and a bit more Rubyish, it also means that we lose the <a href="http://en.wikipedia.org/wiki/Reification_(computer_science)">reification</a> of our concept of a policy. While this is <em>shorter</em>, I find it more confusing. Why am I giving this one line block to the presenter? It’s not really clear to me. And I can’t give a better name to item… and if I want to change the policy, it has to be done in every place we use the presenter…</p>
<p>This does get to the root of another thing that will end up being a follow-up: What’s the difference between closures and objects? If you don’t know, maybe this will get you thinking:</p>
<pre><code>#ugh, this is slightly odd, thanks Ruby!
def apply(proc=nil, &amp;blk)
if block_given?
blk.call
else
proc.call
end
end
proc = Proc.new do
puts "a proc!"
end
class MyProc
def call
puts "an object!"
end
end
my_proc = MyProc.new
apply do
puts "a block!"
end
apply proc
apply my_proc
</code></pre>
<p>Chew on that a while.</p>
<h2>This is too verbose</h2>
<p>Absolutely. <code>DictionaryPresenter.new(PostCategorizationPolicy, Post.all).as_dictionary</code> is too long. 100% there. Good thing that I’d <em>actually</em> write this:</p>
<pre><code>Dictionary.new(Post.all, ByTitle)
</code></pre>
<p>I switched the arguments around, because it reads better. When writing blog posts, you have to balance writing code samples that explain what you’re trying to say, and ones that are ‘real.’ I tend to be a bit more verbose with things like this, because it’s easier for someone who just jumps in to remember what all the parts are… then again, by <em>not</em> showing the ‘real code,’ people might end up implementing the crappy, verbose version. It’s always a trade-off, and I may or may not make the right choice. It happens.</p>
<p>And yes, even earlier in this blog post: I said <code>DictionaryPresenter</code> and I’d really just say <code>Dictionary</code>.</p>
<h2>Where does all of this go?</h2>
<p>You’ve got two options, and only one of them is good: You should make a directory in app for your presenters. In this case, I’d be running <code>touch app/presenters/dictionary.rb</code> pretty quick. The other case is to put things in lib. This whole debacle uncovers something that @avdi went on a rant about over the last few days: models don’t have to inherit from anything. Here’s <a href="http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller#Concepts">what Wikipedia says about models</a>:</p>
<blockquote>
<p>The model manages the behaviour and data of the application domain, responds to requests for information about its state (usually from the view), and responds to instructions to change state (usually from the controller). In event-driven systems, the model notifies observers (usually views) when the information changes so that they can react.</p>
</blockquote>
<p>See anything about “saves stuff to the database?” Okay, ‘manages… data’ could be, but the point is, it’s not <em>required</em>. Models are the reification of a concept that exists in your application domain. Persistance is not required for every concept.</p>
<p>Therefore, if it relates to your problem domain, it goes in app. If it doesn’t, it goes in lib. Something like <code>Dictionary</code>? Depends on how important you think it is. It’s probably pretty important; that’s why you made it a class.</p>
<h2>Design Patterns? Are you serious?</h2>
<p>Yes. Design patterns don’t suck, Design patterns <em>in Java</em> suck. I’m not even going to start this one here, expect more in this space soon.</p>
<p>Design patterns are great, and they’re totally useful in Ruby.</p>
<h2>In conclusion: A shout-out</h2>
<p>I hope you’ve enjoyed this small digression on Presenters. If you’re gonna use presenters in your application, I encourage you to write simple ones as simple Ruby classes, but if they get slightly more complicated, you need to check out <a href="https://github.com/jcasimir/draper">Draper</a> by my buddy Jeff Casimir. It lets you do stuff like this:</p>
<pre><code>class ArticleDecorator &lt; ApplicationDecorator
decorates :article
def author_name
model.author.first_name + " " + model.author.last_name
end
end
article = ArticleDecorator.find(1)
article.author_name
</code></pre>
<p>There’s a lot of features to it, so check it out on GitHub. Totally worthwhile.</p>
</content:encoded></item><item><title>The secret to Rails OO design</title><link>https://steveklabnik.com/writing/the-secret-to-rails-oo-design/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-secret-to-rails-oo-design/</guid><pubDate>Tue, 06 Sep 2011 00:00:00 GMT</pubDate><content:encoded><p>UPDATE: I also have a follow-up <a href="/2011/09/09/better-ruby-presenters.html">here</a>.</p>
<p>I often tell people that I learned Ruby via Rails. This is pretty much the worst way to do it, but I’d learned so many programming languages by then that it didn’t hinder me too much. The one thing that it did do, however, was give me a slightly twisted sense of how to properly design the classes needed in a Rails app. Luckily, I obsessively read other people’s code, and I’ve noticed that there’s one big thing that is common in most of the code that’s written by people whose design chops I respect.</p>
<p>This particular thing is also seemingly unique to those people. It’s not something that people who don’t write good code attempt, but do badly. It’s like a flag, or signal. Now, when I see someone employ this, I instantly think “they get it.” Maybe I’m giving it too much credit, but this advanced design technique offers a ton of interconnected benefits throughout your Rails app, is easy to imploy, and speeds up your tests by an order of magnitude or more. Unfortunately, to many beginning Rails devs, it’s non-obvious, but I want <em>you</em> to write better code, and so I’m here to ‘break the secret,’ if you will, and share this awesome, powerful technique with you.</p>
<p>It’s called the ‘Plain Old Ruby Domain Object.’</p>
<p>Yep, that’s right. A Ruby class that inherets from nothing. It’s so simple that it hides in plain sight. Loved by those who’ve mastered Rails, Plain Old Ruby Objects, or “POROs” as some like to call them, are a hidden weapon against complexity. Here’s what I mean. Examine this ‘simple’ model:</p>
<pre><code>class Post &lt; ActiveRecord::Base
def self.as_dictionary
dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
Post.all.each do |p|
dictionary[p.title[0]] &lt;&lt; p
end
dictionary
end
end
</code></pre>
<p>We want to display an index page of all our posts, and do it by first letter. So we build up a dictionary, and then put our posts in it. I’m assuming we’re not paginating this, so don’t get caught up in querying for all Posts. The important thing is the idea: we can now display our posts by title:</p>
<pre><code>- Post.as_dictionary do |letter, list|
%p= letter
%ul
- list.each do |post|
%li= link_to post
</code></pre>
<p>Sure. And in one way, this code isn’t <em>bad</em>. It’s also not good: We’ve mixed a presentational concern into our model, which is supposed to represent buisness logic. So let’s fix that, via a Presenter:</p>
<pre><code>class DictionaryPresenter
def initialize(collection)
@collection = collection
end
def as_dictionary
dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
@collection.each do |p|
dictionary[p.title[0]] &lt;&lt; p
end
dictionary
end
end
</code></pre>
<p>We can use it via <code>DictionaryPresenter.new(Post.all).as_dictionary</code>. This has tons of benefits: we’ve moved presentation logic out of the model. We’ve <em>already</em> added a new feature: any collection can now be displayed as a dictionary. We can easily write isolated tests for this presenter, and they will be <em>fast</em>.</p>
<p>This post isn’t about the Presenter pattern, though, as much as I love it. This sort of concept appears in other places, too, “this domain concept deserves its own class.” Before we move to a different example, let’s expand on this further: if we want to sort our Posts by title, this class will work, but if we want to display, say, a User, it won’t, because Users don’t have titles. Furthermore, we’ll end up with a lot of Posts under “A,” because the word “a” is pretty common at the beginning of Posts, so we really want to take the second word in that case. We can make two kinds of presenters, but now we lose that generality, and the concept of ‘display by dictionary’ has two representations in our system again. You guessed it: POROs to the rescue!</p>
<p>Let’s change our presenter slightly, to also accept an organizational policy object:</p>
<pre><code>class DictionaryPresenter
def initialize(policy, collection)
@policy = policy
@collection = collection
end
def as_dictionary
dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}
@collection.each do |p|
dictionary[@policy.category_for(p)] &lt;&lt; p
end
dictionary
end
end
</code></pre>
<p>Now, we can inject a policy, and have them be different:</p>
<pre><code>class UserCategorizationPolicy
def self.category_for(user)
user.username[0]
end
end
class PostCategorizationPolicy
def self.category_for(post)
if post.starts_with?("A ")
post.title.split[1][0]
else
post.title[0]
end
end
end
</code></pre>
<p>Bam!</p>
<pre><code>DictionaryPresenter.new(PostCategorizationPolicy, Post.all).as_dictionary
</code></pre>
<p>Yeah, so that’s getting a bit long. It happens. :) You can see that now each concept has one representation in our system. The presenter doesn’t care how things are organized, and the policies only dictate how things are organized. In fact, my names sorta suck, maybe it should be “UsernamePolicy” and “TitlePolicy”, actually. We don’t even care what class they are!</p>
<p>It goes further than that in other directions, too. Combining the flexibility of Ruby with one of my favorite patterns from “Working Effectively with Legacy Code,” we can take complex computations and turn them into objects. Look at this code:</p>
<pre><code>class Quote &lt; ActiveRecord::Base
#&lt;snip&gt;
def pretty_turnaround
return "" if turnaround.nil?
if purchased_at
offset = purchased_at
days_from_today = ((Time.now - purchased_at.to_time) / 60 / 60 / 24).floor + 1
else
offset = Time.now
days_from_today = turnaround + 1
end
time = offset + (turnaround * 60 * 60 * 24)
if(time.strftime("%a") == "Sat")
time += 2 * 60 * 60 * 24
elsif(time.strftime("%a") == "Sun")
time += 1 * 60 * 60 * 24
end
"#{time.strftime("%A %d %B")} (#{days_from_today} business days from today)"
end
end
</code></pre>
<p>Yikes! This method prints a turnaround time, but as you can see, it’s a complex calculation. We’d be able to understand this much more easily of we used Extract Method a few times to break it up, but then we risk polluting our Quote class with more stuff that’s only relevant to pretty turnaround calculation. Also, please ignore that this is also presentation on the model; we just care that it’s a complex bit of code for this example.</p>
<p>Okay, so here’s the first step of this refactoring, which Feathers calls “Break Out Method Object.” You can open your copy of “Working Effectively With Legacy Code” and turn to page 330 to read more. If you don’t have a copy, get one. Anyway, I digress. Here’s the plan of attack:</p>
<ol>
<li>Create a new class for the computation</li>
<li>Define a method on that class to do the new work.</li>
<li>Copy the body of the old method over, and change variable references to instance variables.</li>
<li>Give it an initialize method that takes arguments to set the instance variables used in step 3.</li>
<li>Make the old method delegate to the new class and method.</li>
</ol>
<p>I’ve changed this slightly for Ruby, since we can’t Lean On The Compiler, and a few of Feathers’ steps are about doing this. Anyway, let’s try this on that code. Step 1:</p>
<pre><code>class Quote &lt; ActiveRecord::Base
def pretty_turnaround
#snip
end
class TurnaroundCalculator
end
end
</code></pre>
<p>Two:</p>
<pre><code>class TurnaroundCalculator
def calculate
end
end
</code></pre>
<p>Three:</p>
<pre><code>class TurnaroundCalculator
def calculate
return "" if @turnaround.nil?
if @purchased_at
offset = @purchased_at
days_from_today = ((Time.now - purchased_at.to_time) / 60 / 60 / 24).floor + 1
else
offset = Time.now
days_from_today = @turnaround + 1
end
time = offset + (@turnaround * 60 * 60 * 24)
if(time.strftime("%a") == "Sat")
time += 2 * 60 * 60 * 24
elsif(time.strftime("%a") == "Sun")
time += 1 * 60 * 60 * 24
end
"#{time.strftime("%A %d %B")} (#{days_from_today} business days from today)"
end
end
</code></pre>
<p>I like to give it a generic name at first, and then give it a better one in step 5, after we see what it really does. often our code will inform us of a good name.</p>
<p>Four:</p>
<pre><code>class TurnaroundCalculator
def initialize(purchased_at, turnaround)
@purchased_at = purchased_at
@turnaround = turnaround
end
def calculate
#snip
end
end
</code></pre>
<p>Five:</p>
<pre><code>class Quote &lt; ActiveRecord::Base
def pretty_turnaround
TurnaroundCalculator.new(purchased_at, turnaround).calculate
end
end
</code></pre>
<p>Done! We should be able to run our tests and see them pass. Even if ‘run our tests’ consists of manually checking it out…</p>
<p>So what’s the advantage here? Well, we now can start the refactoring process, but we’re in our own little clean room. We can extract methods into our TurnaroundCalcuator class without polluting Quote, we can write speedy tests for just the Calculator, and we’ve split out the idea of calculation into one place, where it can easily be changed later. Here’s our class, a few refactorings later:</p>
<pre><code>class TurnaroundCalculator
def calculate
return "" if @turnaround.nil?
"#{arrival_date} (#{days_from_today} business days from today)"
end
protected
def arrival_date
real_turnaround_time.strftime("%A %d %B")
end
def real_turnaround_time
adjust_time_for_weekends(start_time + turnaround_in_seconds)
end
def adjust_time_for_weekends(time)
if saturday?(time)
time + 2 * 60 * 60 * 24
elsif sunday?(time)
time + 1 * 60 * 60 * 24
else
time
end
end
def saturday?(time)
time.strftime("%a") == "Sat"
end
def sunday?(time)
time.strftime("%a") == "Sun"
end
def turnaround_in_seconds
@turnaround * 60 * 60 * 24
end
def start_time
@purchased_at or Time.now
end
def days_from_today
if @purchased_at
((Time.now - @purchased_at.to_time) / 60 / 60 / 24).floor + 1
else
@turnaround + 1
end
end
end
</code></pre>
<p>Wow. This code I wrote three years ago isn’t perfect, but it’s almost understandable now. And each of the bits makes sense. This is after two or three waves of refactoring, which maybe I’ll cover in a separate post, becuase this was more illustrative than I thought… anyway, you get the idea. This is what I mean when I say that I shoot for roughly five-line methods in Ruby; if your code is well-factored, you can often get there.</p>
<p>This idea of extracting domain objects that are pure Ruby is even in Rails itself. Check out this route:</p>
<pre><code>root :to =&gt; 'dashboard#index', :constraints =&gt; LoggedInConstraint
</code></pre>
<p>Huh? LoggedInConstraint?</p>
<pre><code>class LoggedInConstraint
def self.matches?(request)
current_user
end
end
</code></pre>
<p>Whoah. Yep. A domain object that describes our routing policy. Awesome. Also, validations, blatantly stolen from <a href="http://omgbloglol.com/post/392895742/improved-validations-in-rails-3">omgbloglol</a>:</p>
<pre><code>def SomeClass &lt; ActiveRecord::Base
validate :category_id, :proper_category =&gt; true
end
class ProperCategoryValidator &lt; ActiveModel::EachValidator
def validate_each(record, attribute, value)
unless record.user.category_ids.include?(value)
record.errors.add attribute, 'has bad category.'
end
end
end
</code></pre>
<p>This isn’t a plain Ruby class, but you get the idea.</p>
<p>Now, you might be thinking, “Steve: This isn’t just for Rails! You’ve lied!” Why yes, actually, you’ve caught me: this isn’t the secret to Rails OO, it’s more of a general OO design guideline. But there’s something special about Rails which seems to lure you into the trap of never breaking classes down. Maybe it’s that <code>lib/</code> feels like such a junk drawer. Maybe it’s that the fifteen minute examples only ever include ActiveRecord models. Maybe it’s that more Rails apps than not are (WARNING: UNSUBSTANTIATED CLAIM ALERT) closed source than open, so we don’t have as many good examples to draw upon. (I have this hunch since Rails is often used to build sites for companies. Gems? Sure? My web app? Not so much. I have no stats to back this up, though.)</p>
<p>In summary: Extracting domain objects is good. They keep your tests fast, your code small, and make it easier to change things later. I have some more to say about this, specifically the “keeps test fast” part, but I’m already pushing it for length here. Until next time!</p>
</content:encoded></item><item><title>The self improvement pomodoro</title><link>https://steveklabnik.com/writing/the-self-improvement-pomodoro/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-self-improvement-pomodoro/</guid><pubDate>Sun, 04 Sep 2011 00:00:00 GMT</pubDate><content:encoded><p>By now, the <a href="http://www.pomodorotechnique.com/">pomodoro technique</a> is pretty well-known amongst programmers. I forget where I heard of it first, but if you haven’t, here it is: Break your day up into 30 minute chunks. Work for 25 minutes, break for 5. Repeat. It’s easy to stay concentrated when you know a break is coming up soon, the fact that you can fit two in an hour fits with my natural sense of order in the world, and once you get in the rhythm, you Get Shit Done. Heck, I liked the idea so much that I <a href="http://rubygems.org/gems/pomodoro">made a Ruby gem</a>.</p>
<p>That said, I hadn’t been following the technique for a while, because it’s easy to lose discipline. Happens to the best of us. Recently, though, it’s crept back into my workflow, largely thanks to Jeff Casimir. I was even lazy enough to not use my gem, and try something else. So I grabbed the software he uses from the App Store, and fired it up. It presented me with an interesting option to give my pomodoro a name before starting it. Pretty much without thinking, I typed ‘rstat.us rails3 port,’ since that’s what I was working on, and away I went.</p>
<p>The more I think about it, however, the more interesting the naming aspect becomes. I can see how long and how often I’m working on various things, and over the past few weeks, it’s been neat to examine. When doing some reading this morning, my groggy brain combined a few different things together:</p>
<ul>
<li>Programmers often don’t have the time to practice their craft</li>
<li>I work on a lot of random software projects</li>
<li>These projects are often of varying quality, especially regarding things like documentation and tests</li>
<li>Almost none of the gems that have been released are at a 1.0 release (can anyone find this blog post for me? I can’t. It’s something about “Just Release It”…)</li>
</ul>
<p>So, with an aim of combining all these things, I’m going to give something a try, and you should too: I’m going to name at least one pomodoro a day “self-improvement,” and use that time to go back over my random projects on GitHub and clean them up. I’m going to use that time to start programming in a totally new language. I’m going to use that time to do that refactoring I’ve been avoiding. I’m going to use that time to read a book.</p>
<p>Basically, I’m going to force myself to invest in myself. Make sure to find time to invest in you, too.</p>
</content:encoded></item><item><title>I'm making it dead simple to contribute to Ruby's documentation</title><link>https://steveklabnik.com/writing/im-making-it-dead-simple-to-contribute-to-ruby-s-documentation/</link><guid isPermaLink="true">https://steveklabnik.com/writing/im-making-it-dead-simple-to-contribute-to-ruby-s-documentation/</guid><pubDate>Mon, 22 Aug 2011 00:00:00 GMT</pubDate><content:encoded><p>Okay! So, if you’d read <a href="/2011/05/10/contributing-to-ruby-s-documentation.html">my previous article on this</a>, you’d know how easy it is to contribute to Ruby’s Documentaiton.</p>
<blockquote>
<p>But Steve, I’m still kinda scared.</p>
</blockquote>
<p>Okay, so here we go: I’m making it even easier on you.</p>
<p>Send me what you want changed, and how, and I’ll make a patch and submit it on your behalf.</p>
<p>No, seriously, I will. I already did once. <a href="http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/38873">This guy</a> posted something about how it was too hard, <a href="http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/38875">so I made a patch for him</a>. And now Ruby is better because of it.</p>
<h2>Patches don’t have to be patches</h2>
<p>Seriously, I don’t even mean diff outputs. I just got this email:</p>
<pre><code>&gt; I'm eager to contribute docs but I don't know where to start: I don't
&gt; mean pulling down the repo and all that, I mean, I see this link
&gt; here:http://segment7.net/projects/ruby/documentation\_coverage.txt but
&gt; all I see is a ton of "# is documented" comments. Where does a
&gt; relative n00b begin? Poking around in the source code is, well,
&gt; intimidating. I'm completely new to this but I take direction well.
&gt; Just point me in the right direction. Just give me something to
&gt; document, please!
</code></pre>
<p>I sent this back:</p>
<pre><code>&gt; No code diving needed! That's part of it. Let's do this: Find the documentation for one of your favorite methods. Here, I'll pick one: http://ruby-doc.org/core/classes/Array.html#M000278 okay, that looks okay, but look at compact!:
&gt;
&gt; &gt; Removes nil elements from the array. Returns nil if no changes were made, otherwise returns &lt;/i&gt;ary&lt;/i&gt;.
&gt;
&gt; Why's that &lt;/i&gt; there? Total screw-up. For example. So, send me this email:
&gt;
&gt; "Hey Steve-
&gt;
&gt; Check out the docs for Array#compact!, here: http://ruby-doc.org/core/classes/Array.html#M000279 . There's an extra &lt;/i&gt; that's screwing things up.
&gt;
&gt; -Jonathan"
&gt;
&gt; Done! I'll go from there. How about this one:
&gt;
&gt; "Hey Steve-
&gt;
&gt; I checked out the docs for Time._load, here: http://www.ruby-doc.org/core/classes/Time.html#M000394
&gt;
&gt; These docs kind of suck. What if I don't know what Marshal is? There should at _least_ be a link to Marshall, here: http://ruby-doc.org/core/classes/Marshal.html And it should probably say something like "You can get a dumped Time object by using _dump: http://www.ruby-doc.org/core/classes/Time.html#M000393
&gt;
&gt; - Jonathan"
&gt;
&gt; I'm sure you can find something that's formatted wrong, or worded incorrectly, or anything else.
&gt;
&gt; -Steve
</code></pre>
<p>Now, the <em>closer</em> it is to an actual patch, the faster I’ll be able to do it. A vague “this documentation is confusing, but I’m not sure why” is helpful, but will take longer. I’d rather get that email than not. If you’re not sure, hit send.</p>
<h2>Just Do It</h2>
<p>And so I’ll do it again. Scared to deal with doc patches? I’ll make them up. I’m serious. I’ll do it. Send them to me.</p>
<p>That’s all.</p>
<p>Send me patches. <a href="mailto:[email protected]">[email protected]</a>. Do it.</p>
</content:encoded></item><item><title>Some people understand REST and HTTP</title><link>https://steveklabnik.com/writing/some-people-understand-rest-and-http/</link><guid isPermaLink="true">https://steveklabnik.com/writing/some-people-understand-rest-and-http/</guid><pubDate>Sun, 07 Aug 2011 00:00:00 GMT</pubDate><content:encoded><p>This is a follow-up post to my post <a href="/2011/07/03/nobody-understands-rest-or-http.html">here</a>. You probably want to read that first.</p>
<p>UPDATE: Please note that ‘<a href="/posts/2012-02-23-rest-is-over">REST is over’</a>. ’Hypermedia API’ is the proper term now.</p>
<h2>A few words on standards versus pragmatism</h2>
<p>When I wrote my first post on this topic, I tried to take a stance that would be somewhat soft, yet forceful. Engineering is the art of making the proper trade-offs, and there are times when following specifications is simply not the correct decision. With that said, my motivation for both of these posts is to eradicate some of the ignorance that some developers have about certain areas of the HTTP spec and Fielding’s REST paper. If you understand the correct way, yet choose to do something else for an informed reason, that’s absolutely, 100% okay. There’s no use throwing out the baby with the bathwater. But ignorance is never a good thing, and most developers are ignorant when it comes to the details of REST.</p>
<p>Secondly, while I think that REST is the best way to develop APIs, there are other valid architectural patterns, too. Yet calling non-REST APIs ‘RESTful’ continues to confuse developers as to what “RESTful” means. I’m not sure what exactly we should call “RESTish” APIs (hey, there we go, hmmm…) but I’m also not under the illusion that I personally will be able to make a huge dent in this. Hopefully you, humble reader, will remember this when dealing with APIs in the future, and I’ll have made a tiny dent, though.</p>
<h2>So who <em>does</em> understand REST?</h2>
<p>As it turns out, there are two companies that you’ve probably heard of who have APIs that are much more RESTful than many others: <a href="http://www.twilio.com/docs/api/rest/">Twilio</a> and <a href="http://developer.github.com/">GitHub</a>. Let’s take a look at GitHub first.</p>
<h3>GitHub: logically awesome</h3>
<p>GitHub’s developer resources are not only beautiful, but thorough. In addition, they make use of lots more of REST.</p>
<h3>The good</h3>
<p>GitHub uses <a href="http://developer.github.com/v3/mime/">custom MIME</a> types for all of their responses. They’re using the vendor extensions that I talked about in my post, too. For example:</p>
<p>application/vnd.github-issue.text+json</p>
<p>Super cool.</p>
<p>Their <a href="http://developer.github.com/v3/#authentication">authentication</a> works in three ways: HTTP Basic, OAuth via an Authentication Header, or via a parameter. This allows for a maximum amount of compatibility across user agents, and gives the user some amount of choice.</p>
<p>Their <a href="http://developer.github.com/v3/#pagination">Pagination</a> uses a header I didn’t discuss in part I: the Link header. <a href="http://tools.ietf.org/html/rfc5988">Here</a>’s a link to the reference. Basically, Link headers enable HATEOAS for media types which aren’t hypertext. This is important, especially regarding JSON, since JSON isn’t hypermedia. More on this at the end of the post. Anyway, so pagination on GitHub:</p>
<pre><code>$ curl -I "https://api.github.com/users/steveklabnik/gists"
HTTP/1.1 200 OK
Server: nginx/1.0.4
Date: Sun, 07 Aug 2011 16:34:48 GMT
Content-Type: application/json
Connection: keep-alive
Status: 200 OK
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4994
Link: &lt;https://api.github.com/users/steveklabnik/gists?page=2&gt;; rel="next", &lt;https://api.github.com/users/steveklabnik/gists?page=33333&gt;; rel="last"
Content-Length: 29841
</code></pre>
<p>The Link header there shows you how to get to the next page of results. You don’t need to know how to construct the URL, you just have to parse the header and follow it. This, for example, is a great way to connect a resource that’s not text-based, such as a PNG, to other resources.</p>
<h3>The bad</h3>
<p>There’s really only one place that GitHub doesn’t knock it out of the park with their new API, and that’s HATEOAS. GitHub’s API isn’t discoverable, because there’s no information at the root:</p>
<pre><code>$ curl -I https://api.github.com/
HTTP/1.1 302 Found
Server: nginx/1.0.4
Date: Sun, 07 Aug 2011 16:44:02 GMT
Content-Type: text/html;charset=utf-8
Connection: keep-alive
Status: 302 Found
X-RateLimit-Limit: 5000
Location: http://developer.github.com
X-RateLimit-Remaining: 4993
Content-Length: 0
</code></pre>
<p>Well, at least, this is how they present it. If you ask for JSON:</p>
<pre><code>$ curl -I https://api.github.com/ -H "Accept: application/json"
HTTP/1.1 204 No Content
Server: nginx/1.0.4
Date: Sun, 07 Aug 2011 16:45:32 GMT
Connection: keep-alive
Status: 204 No Content
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4991
Link: &lt;users/{user}&gt;; rel="user", &lt;repos/{user}/{repo}&gt;; rel="repo", &lt;gists&gt;; rel="gists"
</code></pre>
<p>You do get Links, but you have to construct things yourself. As a user, you get the same thing. It doesn’t change the links to point to your repos, it doesn’t give you links to anything else that you can do with the API.</p>
<p>Instead, the root should give you a link to the particular resources that you can actually view. Maybe something like this:</p>
<pre><code>$ curl -I https://api.github.com/ -H "Accept: application/json" -u "username:password"
HTTP/1.1 204 No Content
Server: nginx/1.0.4
Date: Sun, 07 Aug 2011 16:45:32 GMT
Connection: keep-alive
Status: 204 No Content
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4991
Link: &lt;/gists/public&gt;; rel="public_gists", &lt;/user/repos&gt;; rel="repos", &lt;gists&gt;; rel="gists"
</code></pre>
<p>And a bunch more, for all of the other resources that are available. This would make the API truly discoverable, and you wouldn’t be forced to read their gorgeous documentation. :)</p>
<h3>Twilio</h3>
<p>I’ve always really enjoyed Twilio. Their API is incredibly simple to use. I once hooked up a little “Text me when someone orders something from my site” script, and it took me about fifteen minutes. Good stuff.</p>
<h3>The good</h3>
<p>Twilio has got the HATEOAS thing down. Check it out, their home page says that the base URL is “https://api.twilio.com/2010-04-01”. Without looking at any of the rest of their docs, (I glanced at a page or two, but I didn’t really read them fully yet), I did this:</p>
<pre><code>$ curl https://api.twilio.com/2010-04-01
&lt;?xml version="1.0"?&gt;
&lt;TwilioResponse&gt;
&lt;Version&gt;
&lt;Name&gt;2010-04-01&lt;/Name&gt;
&lt;Uri&gt;/2010-04-01&lt;/Uri&gt;
&lt;SubresourceUris&gt;
&lt;Accounts&gt;/2010-04-01/Accounts&lt;/Accounts&gt;
&lt;/SubresourceUris&gt;
&lt;/Version&gt;
&lt;/TwilioResponse&gt;
</code></pre>
<p>I introduced some formatting. Hmm, okay, Accounts. Let’s check this out:</p>
<pre><code>$ curl https://api.twilio.com/2010-04-01/Accounts&lt;?xml version="1.0"?&gt;
&lt;TwilioResponse&gt;&lt;RestException&gt;&lt;Status&gt;401&lt;/Status&gt;&lt;Message&gt;Authenticate&lt;/Message&gt;&lt;Code&gt;20003&lt;/Code&gt;&lt;MoreInfo&gt;http://www.twilio.com/docs/errors/20003&lt;/MoreInfo&gt;&lt;/RestException&gt;&lt;/TwilioResponse&gt;
</code></pre>
<p>Okay, so I have to be authenticated. If I was, I’d get something like this:</p>
<pre><code>&lt;TwilioResponse&gt;
&lt;Account&gt;
&lt;Sid&gt;ACba8bc05eacf94afdae398e642c9cc32d&lt;/Sid&gt;
&lt;FriendlyName&gt;Do you like my friendly name?&lt;/FriendlyName&gt;
&lt;Type&gt;Full&lt;/Type&gt;
&lt;Status&gt;active&lt;/Status&gt;
&lt;DateCreated&gt;Wed, 04 Aug 2010 21:37:41 +0000&lt;/DateCreated&gt;
&lt;DateUpdated&gt;Fri, 06 Aug 2010 01:15:02 +0000&lt;/DateUpdated&gt;
&lt;AuthToken&gt;redacted&lt;/AuthToken&gt;
&lt;Uri&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d&lt;/Uri&gt;
&lt;SubresourceUris&gt;
&lt;AvailablePhoneNumbers&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/AvailablePhoneNumbers&lt;/AvailablePhoneNumbers&gt;
&lt;Calls&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Calls&lt;/Calls&gt;
&lt;Conferences&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Conferences&lt;/Conferences&gt;
&lt;IncomingPhoneNumbers&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/IncomingPhoneNumbers&lt;/IncomingPhoneNumbers&gt;
&lt;Notifications&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Notifications&lt;/Notifications&gt;
&lt;OutgoingCallerIds&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/OutgoingCallerIds&lt;/OutgoingCallerIds&gt;
&lt;Recordings&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Recordings&lt;/Recordings&gt;
&lt;Sandbox&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Sandbox&lt;/Sandbox&gt;
&lt;SMSMessages&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/SMS/Messages&lt;/SMSMessages&gt;
&lt;Transcriptions&gt;/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Transcriptions&lt;/Transcriptions&gt;
&lt;/SubresourceUris&gt;
&lt;/Account&gt;
&lt;/TwilioResponse&gt;
</code></pre>
<p>Awesome. I can see my all of the other resources that I can interact with. Other than knowing how to authenticate, I can follow the links from the endpoint, and discover their entire API. Rock. This is the way things are supposed to be.</p>
<h3>The bad</h3>
<p>This:</p>
<pre><code>$ curl https://api.twilio.com/2010-04-01/Accounts -H "Accept: application/json"
&lt;?xml version="1.0"?&gt;
&lt;TwilioResponse&gt;&lt;RestException&gt;&lt;Status&gt;401&lt;/Status&gt;&lt;Message&gt;Authenticate&lt;/Message&gt;&lt;Code&gt;20003&lt;/Code&gt;&lt;MoreInfo&gt;http://www.twilio.com/docs/errors/20003&lt;/MoreInfo&gt;&lt;/RestException&gt;&lt;/TwilioResponse&gt;
</code></pre>
<p>Versus this:</p>
<pre><code>$ curl https://api.twilio.com/2010-04-01/Accounts.json
{"status":401,"message":"Authenticate","code":20003,"more_info":"http:\/\/www.twilio.com\/docs\/errors\/20003"}
</code></pre>
<p>:/ Returning JSON when your resource ends with ‘.json’ isn’t bad, but not respecting the Accept header, even when you return the right MIME type, is just unfortunate.</p>
<h2>… and a little Announcement</h2>
<p>It seems that this is a topic that people are really interested in. Part I of this article was pretty well-received, and I got lots of great email and feedback from people. It was also made pretty clear by <a href="http://twitter.com/#!/wayneeseguin/status/97733413611638784">a few</a> people that they want more content from me on this topic.</p>
<p>So I decided to write a book about it. You can check out the site for “<a href="http://designinghypermediaapis.com/">Get Some REST</a>”, and put in your email address. Then you’ll get updated when I start pre-release sales.</p>
<p>So what’s in “Get Some REST”? It’s going to be a full description of how to build RESTful web applications, from the ground up. Designing your resources, laying out an API, all the details. I’m going to try to keep most of the content language-agnostic, but provide code samples in Rails 3.1, as well.</p>
<p>I plan on writing a bunch of content, and then releasing the book at half-price in beta. Early adopters will be able to get their two cents in, and I’ll cover things they still have questions on. It’ll be available under a liberal license, in PDF, ePub, all that good stuff.</p>
<p>I’ve also set up a Twitter account at <a href="http://twitter.com/hypermediaapis">@hypermediaapis</a>. I’ll be tweeting updates about the book, and also other good content related to RESTful design.</p>
</content:encoded></item><item><title>Twitter is too important to be owned by Twitter</title><link>https://steveklabnik.com/writing/twitter-is-to-important-to-be-owned-by-twitter/</link><guid isPermaLink="true">https://steveklabnik.com/writing/twitter-is-to-important-to-be-owned-by-twitter/</guid><pubDate>Sun, 24 Jul 2011 00:00:00 GMT</pubDate><content:encoded><p>When I spoke on a panel at <a href="http://therealtimereport.com/">RTLM NY ’11</a>, I got a little rise out of the crowd when I said “Twitter is too important to be owned by Twitter.” This is still absolutely true, and yesterday was a great example of that: the #fuckyouwashington debacle.</p>
<p>If you haven’t heard, <a href="http://twitter.com/#!/jeffjarvis">Jeff Jarvis</a> started something on Twitter last night: <code>#fuckyouwashington</code>.</p>
<p>As of right this second, when I <a href="http://twitter.com/#!/search/realtime/%23fuckyouwashington">search for #fuckyouwashington on Twitter</a>, they’re coming in at two or three a second. It’s crazy. And it’s really interesting to read what people have to say, pro and con, picking their own issues, all of that stuff. Yet, it’s been removed from the trending topics.</p>
<p>Now, maybe this particular trend was simply blocked because of the profanity. Sure, whatever. I don’t like that, but some might. And Twitter certainly allows all kinds of trends to stick around that would also be ‘offensive,’ for example, #babymakingsong, #turnoff, and #itsalwaystheuglypeople are currently trending. Don’t even get me started on trends like #whatblackgirlslike #whatwhitegirls like and the other racially charged topics. Sure, fine, it’s their site, they can do whatever they want.</p>
<p>That, however, is the root of the issue: they can do whatever they want. And we’re all beholden to them. Nothing we can do. Maybe I’m not sensitive to four letter words, and I <em>do</em> want to hear about things like this. Too bad, Twitter says no. Maybe I don’t want <a href="http://venturebeat.com/2011/07/08/twitter-adding-promoted-tweets-to-your-stream-this-summer/">promoted tweets added to my stream</a>. Too bad, Twitter says yes.</p>
<p>And that’s why I’m feeling worse and worse about Twitter every day. It sucks, because I really like Twiter, the website. But Twitter, the company, really bums me out. They promised all of us nerds long ago that they wouldn’t do this kind of thing, and they’re not keeping that promise. Which means I’m not really getting emotionally invested in Twitter any more. It’s still valuable enough to me to keep using, but I can’t care about it the way I used to.</p>
<p>This is why we’re still hacking away at <a href="http://rstat.us/">rstat.us</a>. If you haven’t heard about it before, you can <a href="http://blog.steveklabnik.com/2011/03/23/announcing-rstat-us.html">read my release announcement</a>. If you’re lazier than that, here’s the summary: rstat.us is a distributed, open-source Twitter. You can use our public node at the website, or get your own copy of the code and run your own, make whatever customizations you’d like, and you’re still on equal footing with everyone else in the network. We’re on the cusp of a 1.0 announcement, and I’d love to have been able to make that in this post, but we’re still working out some last kinks. If you’ve seen it before, you won’t find much differences right now, but we have 100% standards compliance, a complete visual refresh, and some other fun goodies waiting in the wings, ready to come out shortly.</p>
<p>Just remember: if you use a service like Twitter that’s controlled by a company like Twitter, you have to play by their rules. And sometimes, they’ll change the rules, and it’ll be time to move on.</p>
</content:encoded></item><item><title>Nobody understands REST or HTTP</title><link>https://steveklabnik.com/writing/nobody-understands-rest-or-http/</link><guid isPermaLink="true">https://steveklabnik.com/writing/nobody-understands-rest-or-http/</guid><pubDate>Sun, 03 Jul 2011 00:00:00 GMT</pubDate><content:encoded><h2><strong>HI <a href="http://news.ycombinator.com/item?id=3635085">HN</a>, PLEASE READ THIS!!!</strong></h2>
<p>Since I’ve posted this, I’ve refined a few of my positions on things. Everyone learns and grows, and while I still stand by most of what I said, I specifically don’t agree that versioning the media type is how to properly version APIs. Hypermedia APIs should not actually use explicit versioning, but I’d rather see a version in the URI with HATEOAS than no HATEOAS and versioned media types. I’ve been meaning to update this post and write more, but alas, my work on <a href="http://designinghypermediaapis.com/">Get some REST</a> has taken priority. I don’t have a HN account, so feel free to <a href="mailto:[email protected]">email me</a> with any thoughts or questions!</p>
<p>Furthermore, everything in engineering is ALWAYS a trade-off. I primarily wish that more people understood the tools that HTTP provides them with, and made an informed choice, rather than cargo-culting what they’ve seen others do.</p>
<p>Update: Part II of this post is <a href="/2011/08/07/some-people-understand-rest-and-http.html">here</a>. Check it out, and there’s an announcement at the end!</p>
<p>Update: Please note that <a href="/posts/2012-02-23-rest-is-over">REST is over. Hypermedia API is the new nomenclature.</a></p>
<p>The more that I’ve learned about web development, the more that I’ve come to appreciate the thoroughness and thoughtfulness of the authors of the HTTP RFC and Roy Fielding’s dissertation. It seems like the answers to most problems come down to “There’s a section of the spec for that.” Now, obviously, they’re not infallible, and I’m not saying that there’s zero room for improvement. But it really disappoints me when people don’t understand the way that a given issue is supposed to be solved, and so they make up a partial solution that solves their given case but doesn’t jive well with the way that everything else works. There are valid criticisms of the specs, but they have to come from an informed place about what the spec says in the first place.</p>
<p>Let’s talk about a few cases where either REST or HTTP (which is clearly RESTful in its design) solves a common web development problem.</p>
<h3>I need to design my API</h3>
<p>This one is a bit more general, but the others build off of it, so bear with me.</p>
<p>The core idea of REST is right there in the name: “Representational State Transfer” It’s about transferring representations of the state… of resources. Okay, so one part isn’t in the name. But still, let’s break this down.</p>
<h3>Resources</h3>
<p>From <a href="http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_2_1_1">Fielding’s dissertation</a>:</p>
<blockquote>
<p>The key abstraction of information in REST is a resource. Any information that can be named can be a resource: a document or image, a temporal service (e.g. “today’s weather in Los Angeles”), a collection of other resources, a non-virtual object (e.g. a person), and so on. In other words, any concept that might be the target of an author’s hypertext reference must fit within the definition of a resource. A resource is a conceptual mapping to a set of entities, not the entity that corresponds to the mapping at any particular point in time.</p>
</blockquote>
<p>When we interact with a RESTful system, we’re interacting with a set of resources. Clients request resources from the server in a variety of ways. But the key thing here is that resources are <em>nouns</em>. So a RESTful API consists of a set of URIs that map entities in your system to endpoints, and then you use HTTP itself for the verbs. If your URLs have action words in them, you’re doing it wrong. Let’s look at an example of this, from the early days of Rails. When Rails first started messing around with REST, the URLs looked like this:</p>
<pre><code>/posts/show/1
</code></pre>
<p>If you use Rails today, you’ll note that the corresponding URL is this:</p>
<pre><code>/posts/1
</code></pre>
<p>Why? Well, it’s because the ‘show’ is unnecessary; you’re performing a GET request, and that demonstrates that you want to show that resource. It doesn’t need to be in the URL.</p>
<h3>A digression about actions</h3>
<p>Sometimes, you need to perform some sort of action, though. Verbs are useful. So how’s this fit in? Let’s consider the example of transferring money from one Account to another. You might decided to build a URI like this:</p>
<pre><code>POST /accounts/1/transfer/500.00/to/2
</code></pre>
<p>to transfer $500 from Account 1 to Account 2. But this is wrong! What you really need to do is consider the nouns. You’re not transferring money, you’re creating a Transaction resource:</p>
<pre><code>POST /transactions HTTP/1.1
Host: &lt;snip, and all other headers&gt;
from=1&amp;to=2&amp;amount=500.00
</code></pre>
<p>Got it? So then, it returns the URI for your new Transaction:</p>
<pre><code>HTTP/1.1 201 OK
Date: Sun, 3 Jul 2011 23:59:59 GMT
Content-Type: application/json
Content-Length: 12345
Location: http://foo.com/transactions/1
{"transaction":{"id":1,"uri":"/transactions/1","type":"transfer"}}
</code></pre>
<p>Whoah, <a href="http://timelessrepo.com/haters-gonna-hateoas">HATEOS</a>! Also, it may or may not be a good idea to return this JSON as the body; the important thing is that we have the Location header which tells us where our new resource is. If we give a client the ID, they might try to construct their own URL, and the URI is a little redundant, since we have one in the Location. Regardless, I’m leaving that JSON there, because that’s the way I typed it first. I’d love to <a href="mailto:[email protected]">hear your thoughts on this</a> if you feel strongly one way or the other.</p>
<p>EDIT: I’ve since decided that yes, including the URI is a bad idea. The Location header makes much more sense. More on this in Part ii, yet to come.</p>
<p>Anyway, so now we can GET our Transaction:</p>
<pre><code>GET /transactions/1 HTTP/1.1
Accept: application/json
</code></pre>
<p>and the response:</p>
<pre><code>HTTP/1.1 blah blah blah
{"id":1,"type":"transfer","status":"in-progress"}
</code></pre>
<p>So we know it’s working. We can continue to poll the URI and see when our transaction is finished, or if it failed, or whatever. Easy! But it’s about manipulating those nouns.</p>
<h3>Representations</h3>
<p>You’ll notice a pair of headers in the above HTTP requests and responses: Accept and Content-Type. These describe the different ‘representation’ of any given resource. From <a href="http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_2_1_2">Fielding</a>:</p>
<blockquote>
<p>REST components perform actions on a resource by using a representation to capture the current or intended state of that resource and transferring that representation between components. A representation is a sequence of bytes, plus representation metadata to describe those bytes. Other commonly used but less precise names for a representation include: document, file, and HTTP message entity, instance, or variant.A representation consists of data, metadata describing the data, and, on occasion, metadata to describe the metadata (usually for the purpose of verifying message integrity).</p>
</blockquote>
<p>So <code>/accounts/1</code> represents a resource. But it doesn’t include the form that the resource takes. That’s what these two headers are for.</p>
<p>This is also why adding <code>.html</code> to the end of your URLs is kinda silly. If I request <code>/accounts/1.html</code> with an <code>Accept</code> header of <code>application/json</code>, then I’ll get JSON. The <code>Content-Type</code> header is the server telling us what kind of representation it’s sending back as well. The important thing, though, is that a given resource can have many different representations. Ideally, there should be one unambiguous source of information in a system, and you can get different representations using <code>Accept</code>.</p>
<h3>State and Transfer</h3>
<p>This is more about the way HTTP is designed, so I’ll just keep this short: Requests are designed to be stateless, and the server holds all of the state for its resources. This is important for caching and a few other things, but it’s sort of out of the scope of this post.</p>
<p>Okay. With all of that out of the way, let’s talk about some more specific problems that REST/HTTP solve.</p>
<h3>I want my API to be versioned</h3>
<p>The first thing that people do when they want a versioned API is to shove a /v1/ in the URL. <em>THIS IS BAD!!!!!1</em>. <code>Accept</code> solves this problem. What you’re really asking for is “I’d like the version two representation of this resource.” So use accept!</p>
<p>Here’s an example:</p>
<pre><code>GET /accounts/1 HTTP/1.1
Accept: application/vnd.steveklabnik-v2+json
</code></pre>
<p>You’ll notice a few things: we have a + in our MIME type, and before it is a bunch of junk that wasn’t there before. It breaks down into three things: <code>vnd</code>, my name, and <code>v2</code>. You can guess what v2 means, but what about <code>vnd</code>. It’s a <a href="http://tools.ietf.org/html/rfc4288#section-3.2">Vendor MIME Type</a>. After all, we don’t really want just any old JSON, we want my specific form of JSON. This lets us still have our one URL to represent our resource, yet version everything appropriately.</p>
<p>I got a comment from <a href="http://avdi.org/">Avdi Grimm</a> about this, too:</p>
<blockquote>
<p>Here’s an article you might find interesting: http://www.informit.com/articles/article.aspx?p=1566460The author points out that MIMETypes can have parameters, which means you can actually have a mimetype that looks like this:vnd.example-com.foo+json; version=1.0Sadly, Rails does not (yet) understand this format.</p>
</blockquote>
<h3>I’d like my content to be displayed in multiple languages</h3>
<p>This is related, but a little different. What about pages in different languages? Again, we have a question of representation, not one of content. /en/whatever is not appropriate here. Turns out, <a href="http://tools.ietf.org/html/rfc2616#section-14.4">there’s a header for that: Accept-Language</a>. Respect the headers, and everything works out.</p>
<p>Oh, and I should say this, too: this doesn’t solve the problem of “I’d like to read this article in Spanish, even though I usually browse in English.” Giving your users the option to view your content in different ways is a good thing. Personally, I’d consider this to fall out in two ways:</p>
<ul>
<li>It’s temporary. Stick this option in the session, and if they have the option set, it trumps the header. You’re still respecting their usual preferences, but allowing them to override it.</li>
<li>It’s more permanent. Make it some aspect of their account, and it trumps a specific header. Same deal.</li>
</ul>
<h3>I’d like my content to have a mobile view</h3>
<p>Sounds like I’m beating a dead horse, but again: it’s a representation question. In this case, you’d like to vary the response by the User-Agent: give one that’s mobile-friendly. There’s a whole list of <a href="http://www.w3.org/TR/mobile-bp/">mobile best practices</a> that the w3c recommends, but the short of it is this: the User-Agent should let you know that you’re dealing with a mobile device. For example, here’s the first iPhone UA:</p>
<pre><code>Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A543a Safari/419.3
</code></pre>
<p>Then, once detecting you have a mobile User-Agent, you’d give back a mobile version of the site. Hosting it on a subdomain is a minor sin, but really, like I said above, this is really a question of representation, and so having two URLs that point to the same resource is kinda awkward.</p>
<p>Whatever you do, for the love of users, please don’t detect these headers, then redirect your users to m.whatever.com, at the root. One of my local news websites does this, and it means that every time I try to follow a link from Twitter in my mobile browser, I don’t see their article, I see their homepage. It’s infuriating.</p>
<h3>I’d like to hide some of my content</h3>
<p>Every once in a while, you see a story like this: <a href="http://www.boingboing.net/2010/10/25/local-newspaper-boas.html">Local paper boasts ultimate passive-agressive paywall policy</a>. Now, I find paywalls distasteful, but this is not the way to do it. There are technological means to limit content on the web: making users be logged-in to read things, for example.</p>
<p>When this was discussed on Hacker News, <a href="http://news.ycombinator.com/item?id=1834075">here’s</a> what I had to say:</p>
<p>nkurz:</p>
<blockquote>
<p>I presume if I had an unattended roadside vegetable stand with a cash-box, that I’d be able to prosecute someone who took vegetables without paying, certainly if they also made off with the cash-box. Why is this different on the web? And if a written prohibition has no legal standing, why do so many companies pay lawyers to write click-through “terms of service” agreements?</p>
</blockquote>
<p>me:</p>
<blockquote>
<p>Why is this different on the web?Let’s go through what happens when I visit a web site. I type a URL in my bar, and hit enter. My web browser makes a request via http to a server, and the server inspects the request, determines if I should see the content or not, and returns either a 200 if I am allowed, and a 403 if I’m not. So, by viewing their pages, I’m literally asking permission, and being allowed.It sounds to me like a misconfiguration of their server; it’s not doing what they want it to.</p>
</blockquote>
<h3>I’d like to do some crazy ajax, yet have permalinks</h3>
<p>This is an example of where the spec is obviously deficient, and so something had to be done.</p>
<p>As the web grew, AJAXy ‘web applications’ started to become more and more the norm. And so applications wanted to provide deep-linking capabilities to users, but there’s a problem: they couldn’t manipulate the URL with Javascript without causing a redirect. They <em>could</em> manipulate the anchor, though. You know, that part after the #. So, Google came up with a convention: <a href="http://code.google.com/web/ajaxcrawling/docs/getting-started.html">Ajax Fragments</a>. This fixed the problem in the short term, but then the spec got fixed in the long term: <a href="http://dev.w3.org/html5/spec-author-view/history.html">pushState</a>. This lets you still provide a nice deep URL to your users, but not have that awkward #!.</p>
<p>In this case, there was a legitimate technical issue with the spec, and so it’s valid to invent something. But then the standard improved, and so people should stop using #! as HTML5 gains browser support.</p>
<h3>In conclusion</h3>
<p>Seriously, most of the problems that you’re solving are social, not technical. The web is decades old at this point, most people have considered these kinds of problems in the past. That doesn’t mean that they always have the right answer, but they usually do have an answer, and it’d behoove you to know what it is before you invent something on your own.</p>
</content:encoded></item><item><title>Single text, many masters</title><link>https://steveklabnik.com/writing/single-text-many-masters/</link><guid isPermaLink="true">https://steveklabnik.com/writing/single-text-many-masters/</guid><pubDate>Sat, 21 May 2011 00:00:00 GMT</pubDate><content:encoded><p>Hey everyone. Here’s a draft of an essay I’ve been working on. I’d love to hear your feedback.</p>
<p>The word ‘engineering’ has a deep connection to the word ‘trade-offs’ in my mind. Most engineering decisions come down to evaluating a few differing alternatives, and often multiple factors end up being negatively correlated. You can make something stronger, but then it will be heavier. It can be made faster, but then it’s significantly more expensive. A good engineer is able to take all of these factors into account, and design a system such that it maximizes its effectiveness across the sum of all of the relevant constraints. No matter if you consider the act of writing software an art, science, or engineering, its indisputable that designing complex software systems is identical in this respect. There are dozens of different metrics that system architects take into consideration while crafting a plan of attack, but but there’s a deeper question of balance here that’s significantly different than these more traditional engineering issues.</p>
<p>Text, in the form of source code, presents unique challenges of composition. These difficulties all stem from the same root: source code is a singular text, but must be intelligible to multiple, simultaneous audiences. More traditional forms of authorship still take audience into consideration, of course, but the decision to engage a diverse group of people is the choice of the writer. While mass appeal may be something that certain authors strive to attain, it’s not an inherent property of their chosen form of expression. Source code, while text, inhabits a multiplicity of forms, and software developers are confronted with this inherent multi-faceted complexity when composing any particular software work. Some of these forms suit certain audiences better than others, and so it falls to the programmer to manage which form they are currently working in, consider which audience they are attempting to write for, and arrange all of these forms amongst one another in such a way that any particular audience is able to navigate and extract the proper information from the source without confusion.</p>
<p>In this post, I’ll expand on the concept of audiences for code, and in a future post, I’ll explore the simultaneous forms that code takes.</p>
<h2>The multitude of audiences</h2>
<h3>The default audience: the computer</h3>
<blockquote>
<p>Science is what we understand well enough to explain to a computer. Art is everything else we do.Don Knuth</p>
</blockquote>
<p>This may seem obvious, but the when considering the question of “Who are programs written for?”, many would say “The computer. Duh.” In many ways, a computer is the primary reader of a particular piece of source code. The code that’s given to a computer will be executed billions of times per second, repeated, recalculated, and re-interpreted over and over and over again.</p>
<p>The computer’s native understanding of software comes from machine code. Machine code are the binary numbers that the CPU loads into memory and processes directly. For example, here’s a line of machine code for an x86 machine that puts the value ‘97’ into the AL register, graciously stolen <a href="http://en.wikipedia.org/wiki/Assembly_language#Assembly_language">from Wikipedia</a>:</p>
<pre><code>10110000 01100001
</code></pre>
<p>While most would consider this unintelligible, computers were actually programmed this way at one time. My uncle actually did this by flipping switches to set the binary and pushed a button to store it in memory. Unfortunately, what’s good for the computer isn’t good for the human programmer. This is why assembly language was created. Assembly language has a 1 to 1 mapping to machine code, but is much easier for humans to understand. Here’s that same line in assembly:</p>
<pre><code>MOV AL, 61h ; Load AL with 97 decimal (61 hex)
</code></pre>
<p>The <code>MOV</code> corresponds with <code>10110</code>, <code>AL</code> maps to <code>000</code>, and 61 hex is <code>01100001</code>. <code>MOV</code> is short for ‘move,’ though, and this mnemonic is just a bit easier to understand than <code>10110</code>. This is the most basic example of a compositional trade-off. It’s not a true trade-off, because they map perfectly to one another, but it illustrates the difference between composing in a language that the computer understands and one that’s more natural for the programmer. Another important concept comes into play, that of <em>compilation</em>. Virtually every work of composition created in software is automatically translated to another form before it is executed. We’ll address this concept more fully when we discuss form later.</p>
<p>If that’s where assembly stopped, it would remain a 1 to 1 mapping. However, virtually every assembly language also offers macros, and this moves the code further away from the machine and destroys the synchrony of the two forms. Here’s an example:</p>
<pre><code>MOV EAX, [EBX]
</code></pre>
<p>The <code>[]</code> characters change the meaning of <code>EBX</code>, rather than be the value stored in that particular register, they imply that the value is a memory address, and we want to move the contents of that address to <code>EAX</code>. The generated machine code could now be processed into multiple valid assembly forms, and so the transformation is only perfect in one direction, even if it’s possible to ‘decompile’ it into one of those possible encodings. This is considered to be an acceptable trade-off for human readability; we very rarely want to turn machine code back into assembly.</p>
<p>There’s also a jump between higher level languages, as well. Here’s the assembly statements that adds 2 and 3 together:</p>
<pre><code>MOV EAX, 2
ADD EAX, 3
</code></pre>
<p><code>ADD</code>, of course, is the statement that adds a number to the register that’s given. Now <code>EAX</code> has the value 5. Here’s the same code, but in C:</p>
<pre><code>int x = 2;
x = x + 3;
</code></pre>
<p>Pretty simple. You can see how the C is much easier to understand; we say what type <code>x</code> is (an integer), and it’s a bit more explicit. <code>x</code> is equal to <code>x</code> + three. However, since the C is divorced from the machine, and is written for the person, we can change our compiler to make assembly code that works on a different kind of processor architecture. If we were to compile the above C for x86_64, a 64 bit version of x86, we might get some assembly that’d look like this:</p>
<pre><code>MOVL RAX, 2
ADDL RAX, 3
</code></pre>
<p>While this code looks similar to the above, it is quite different. This uses the native 64 bit types, rather than the 32 bit types above. The other important thing is that by writing code that’s divorced from the machine, and written for people, we’re able to translate it into the code for multiple machines. If we had written the assembly above, when moving to another architecture, it would have required a total re-write. And while this particular sample looks very similar, a more complex piece of code will be significantly divergent, but I don’t want to go into the details of two kinds of assembly code. Because we can define the languages for humans, and the language of computers is somewhat beholden to the physical machine itself, it’s significantly easier to do the translation from C to the two kinds of machines, rather than trying to translate from one machine to another. What we lose in this kind of translation, though, is efficiency. Code that was hand-crafted for each machine would be more performant, and better represent each individual platform.</p>
<p>Even though we may choose to use a language that’s more understandable to people, it still has to be understood by the computer in some form. This translation will introduce some amount of penalty, and so it’s important that this gets taken into consideration. Sometimes, code must be written in a way that’s not easy for a person to read, because it’s easier for the computer to be efficient with a more opaque implementation.</p>
<h3>The reflexive audience: the programmer himself</h3>
<blockquote>
<p>Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.Brian Kernighan</p>
</blockquote>
<p>Everyone who writes code has experienced this at some time or another. You write a whole bunch of code, and then a few months goes by, and you take a look at it, and it’s absolutely unintelligible. This happens because at the time of inception, the author of a particular piece of code has an incredibly close relationship to it. As it was just written, the code is obvious to the author. They’re in the proper mindset to understand the intention that was drawn upon to necessitate bringing those lines into the world, and so no extra explanation is necessary. As time goes on, however, the author becomes more close to the third audience, other programmers. It’s important for coders to recognize this fact, and take preventative steps to ameliorate this confusion.</p>
<p>Even though the author will approach the position of the other audience eventually, this audience is distinct because there is a certain level of explanation that sits between undocumented, inexpressive code and code that’s well explained, and this is the position most code is in. An explanation that’s helpful to those who understand the code, but not to those who don’t is better than nothing. This sort of code may be overly contextual, and could use some added information to improve its clarity.</p>
<h3>The other audience: colleagues and coworkers</h3>
<blockquote>
<p>Always code as if the guy who ends up maintaining your code is a violent psychopath who knows where you live.Martin Golding</p>
</blockquote>
<p>As I touched on earlier, there’s a similarity between the ‘other’ audience and the reflexive. The primary distinction is drawn around the proximity to the source. The other does not have the advantage of having authored the code, and therefore doesn’t have the native’s understanding of the underlying logical organization of the source. This disadvantage can be overcome via composing in such a manner that the meaning is emergent from the design. Even if it’s too complex to be obvious, good documentation can address this particular deficiency.</p>
<p>Ultimately, much of software design is about modeling. Software that solves a particular problem should emulate the nature of the challenge it’s attempting to address. If this can be achieved, it’s significantly easier for those who understand the problem to figure out how the software works. Therefore, good design can help improve the effectiveness of a given piece of source to communicate its intent. Along a similar vector, if the design is similar to code that solves a particular issue, it’s easier to understand. As an example, a friend recently asked for feedback about an interface that he’d designed. It loaded a save game file for StarCraft 2. This is what he came up with:</p>
<pre><code>replay_file = File.new("spec/fixtures/1v1-game1.sc2replay")
@replay = SC2Refinery::Parser.parse(replay_file)
</code></pre>
<p>However, Ruby already has several kinds of code in its standard library that loads some information from disk and parses it into some kind of data structure that you can use in your program. The JSON, YAML, and Marshall classes already use a set of methods to import and export data, and they’re named <code>load</code> and <code>dump</code>, and they’re part of the class directly. Also, in this case, the user of the code shouldn’t need to deal with the creation of a file, since it’s unreasonable to assume that a game replay would come from any other source. Therefore, after some discussion, he adopted the following interface instead:</p>
<pre><code>@replay = SC2Refinery.load("spec/fixtures/1v1-game1.sc2replay")
</code></pre>
<p>This is much nicer to use, and is much simpler. While it may not seem like a whole lot, when rules like this are applied across an entire codebase, they can significantly increase understanding. Multiple reductions of mental overhead add up quickly.</p>
<p>My new favorite trick for adding a little bit of modeling that significantly reduces overhead for the user is the Presenter Pattern. Jeff Casimir demonstrated this very clearly in his presentation at RailsConf 2011, “<a href="http://dl.dropbox.com/u/69001/Fat%20Models%20Aren%27t%20Enough%20-%20RailsConf.pdf">Fat Models Aren’t Enough</a>”. Here’s a slightly modified example. Imagine that we have a system that manages students, and we’d like to display a report card for them. We might start with some code that looks like this:</p>
<pre><code>student = Student.find(options[:student_id])
term = Term.find(options[:term_id])
report_type = ReportType.find(options[:report_type])
puts "#{student.last_name}, #{student.first_name}"
puts "#{report_type.report_title} for #{term.start_date} to #{term.end_date}"
student.courses.each do |course|
course_report = student.report_data_for_course(course)
puts course_report.to_s
end
</code></pre>
<p>Honestly, until this past week, this is the exact code that I would have written. But it turns out that we can do better. Basically, we’re displaying some information that comes from a combination of three different objects. If we think about it some more, we’re really trying to display a report card. So let’s make an object that represents the card, and delegates to its sub-objects. It will then know how to display itself.</p>
<pre><code>class ReportCard
delegate :start_date, :end_date, :to =&gt; :term
delegate :first_name, :last_name, :courses, :report_data_for_course, :to =&gt; :student
delegate :report_title, :to =&gt; :report_type
def initialize(params)
@student = Student.find params[:student_id]
@term = Term.find params[:term_id]
@report_type = ReportType.find params[:report_type_id]
end
def student_name
[last_name, first_name].join(", ")
end
def title
"#{report_title} for #{start_date} to #{end_date}"
end
def course_reports
out = ""
courses.each do |course|
out += report_data_for_course(course)
end
out
end
end
</code></pre>
<p>Now, this is a lot of code. However, as you can see, it’s all focused on composing the information and exposing an interface that makes sense for a report card. Using it is super easy:</p>
<pre><code>report = ReportCard.new(options)
puts report.student_name
puts report.title
puts report.course_reports
</code></pre>
<p>Bam! It’s incredibly obvious. This code is much more clear than before. We’ll see if I’m still as hot on this pattern as I am now in a few months, but I feel the extra object adds a significant amount of clarity.</p>
<p>If the model is too hard to create, or if additional clarity is needed, documentation in the form of comments can also help to improve the understanding of the ‘other’ audience. Comments can be a difficult form of prose to write, because they need to be written at the correct level of abstraction. If they simply repeat what the code does, they’re useless, and if they’re too high-level, certain details and semantics may not be made clear.</p>
<p>Individual bits of code can also be made more clear by developing a narrative within any particular method that’s being written. Telling a story with code may not be something you’ve considered before, but it’s really about maintaining a proper flow in the actions your code is taking. For example, if there’s a bunch of error handling strewn about inside of a method, it’s less clear than bunching all of the error handling near the end. Most code should be an act in three parts: input, processing, and output. If these three parts are mixed together, it can appear much more complicated.</p>
<h3>The forgotten audience: end-users</h3>
<blockquote>
<p>If I asked my customers what they wanted, they’d have told me, “A faster horse.”Henry Ford</p>
</blockquote>
<p>In the end, all software is used by someone. Use-value is the driving force of virtually all code. Code that doesn’t do anything may be making some kind of important philosophical statement, but isn’t really the sort that I’m talking about.</p>
<p>The introduction of a user imposes significant restrictions upon the way that code is composed. End-users do not need to understand the code itself, but they do need to be able to understand its external interfaces. These needs place an imposition on the way that the code needs to be written, because it <em>must</em> address this issue of interface. Sometimes, interface requirements can create a burden on the internal implementation. Needing to support certain behaviors and forms can create complexity for an implementor.</p>
<p>Documentation created for end users must be completely different than that which is created for those inspecting the code itself. Most end-users will not be literate in the arts of software development, and so approach the software object in an entirely different way than those who write code do. Yet, the same semantics must be passed on to them, but at a higher level. There’s a strong movement within the community to start designing software with this kind of end-user documentation in mind, called <a href="http://tom.preston-werner.com/2010/08/23/readme-driven-development.html">README driven development</a>. There are advantages to thinking on this level when beginning, but a nice effect of doing it first is that you can ensure it gets done. A surprising amount of software has poor documentation for its users, because it’s created after the software is finished, and at that time there’s intense pressure to ship it out the door. Writing down information for the end user first ensures that it’s done properly, that all development works in accordance with the documentation, and that all of the use-cases for an end-user have been thought of and are being addressed.</p>
</content:encoded></item><item><title>Contributing to Ruby's documentation</title><link>https://steveklabnik.com/writing/contributing-to-ruby-s-documentation/</link><guid isPermaLink="true">https://steveklabnik.com/writing/contributing-to-ruby-s-documentation/</guid><pubDate>Fri, 20 May 2011 00:00:00 GMT</pubDate><content:encoded><h3>Update!</h3>
<p>I’m now making it even easier than this: <a href="/2011/08/22/im-making-it-dead-simple-to-contribute-to-ruby-s-documentation.html">Read my new post about how I’ll do this for you</a>. That said, if you want to do it yourself, the following will tell you how.</p>
<h3>Original Article</h3>
<p>Ruby 1.9.3 is coming out soon! <a href="http://blog.segment7.net/2011/05/09/ruby-1-9-3-documentation-challenge">drbrain has challenged the Ruby community to improve its documentation</a>, but some people were asking about how to do so. So I made a video!</p>
<p>Some small errata: drbrain has informed me that he should edit the Changelog, not me. So don’t do that. :)</p>
<p><a href="http://vimeo.com/23522731">How to contribute to Ruby’s documentation.</a> from <a href="http://vimeo.com/steveklabnik">Steve Klabnik</a> on <a href="http://vimeo.com/">Vimeo</a>.</p>
<p>If you don’t want to watch me talk about it, here’s the same info, in text:</p>
<p>Getting the Ruby source is pretty easy. You can find it on GitHub, here: <a href="http://github.com/ruby/ruby">http://github.com/ruby/ruby</a> . Click the “fork” button and clone down your own fork:</p>
<pre><code>$ git clone [email protected]:YOURUSERNAME/ruby.git
</code></pre>
<p>After that’s done, type <code>cd ruby</code> and add the main project as an upstream. This will let you keep up-to-date with the latest changes:</p>
<pre><code>$ git remote add upstream https://github.com/ruby/ruby.git
$ git fetch upstream
</code></pre>
<p>Okay! Now that you’re all set up, poke around and find something that needs documented. I like to just look through the source, but you can also look <a href="http://segment7.net/projects/ruby/documentation_coverage.txt">here</a> for a list of things that have no docs. Documentation is written in rdoc, and I’d check the recent commits that drbrain has been making to guide you in style. <a href="https://github.com/ruby/ruby/commit/071a678a156dde974d8e470b659c89cb02b07b3b">This commit</a>, for example, is a pretty good template. You can also check out the formatting guides <a href="http://rdoc.rubyforge.org/RDoc/Markup.html">here</a>. There’s also <a href="http://rdoc.rubyforge.org/RDoc/Parser/Ruby.html">this</a> which explains some directives for .rb files and <a href="http://rdoc.rubyforge.org/RDoc/Parser/C.html">this</a> which handles directives for .c files.</p>
<p>Now that you’ve made a change to the documentation, you can regenerate the docs by using rdoc. First, grab the latest version from rubygems:</p>
<p>$ gem install rdoc</p>
<p>Always best to have the latest tools. Now do this to generate the docs:</p>
<pre><code>$ rdoc --o tmpdoc lib/rss*
</code></pre>
<p>I’m passing it in an output directory with op, since the doc directory is not an rdoc directory. rdoc will complain and refuse to overwrite those files, which is a good thing. I’m also passing in a pattern of what to compile documentation for, compiling all of it takes a few minutes! In this case, I chose to document the rss library.</p>
<p>Now you have a website in rdocout. Open up its index.html, and poke around for what you’ve changed. If it all looks good, you’re ready to make a patch!</p>
<pre><code>$ rm -r tmpdoc
$ git add .
$ git commit -m "adding documentation for $SOMETHING"
</code></pre>
<p>Now, you have two options here. One is to simply push the change up to GitHub, and make a pull request.</p>
<pre><code>$ git push origin
</code></pre>
<p>… aaand pull request. The core Ruby development doesn’t really happen on GitHub though, and so your patch may take a while to get included. If you really want to do it right, submit a patch to RedMine. We’ll use git to make this patch:</p>
<pre><code>$ git format-patch HEAD~1
</code></pre>
<p>This says “make a patch out of the last commit.” It’ll tell you a file name, it should start with 000.</p>
<p>Now, sign up for the Ruby RedMine <a href="http://bugs.ruby-lang.org/account/register">here</a>. Once you’ve clicked the confirmation email, <a href="http://bugs.ruby-lang.org/projects/ruby-trunk/issues/new">open a new ticket</a>, and assign it to Eric Hodel, category DOC, and give it your Ruby version, even though it’s not a big deal here. Click ‘choose file’ and pick your patch, then ‘create and continue’ and BAM! You’re done!</p>
<p>Let’s all pitch in and make this the best documented Ruby release ever! In writing documentation, you might even find some things that you’d like to help improve. ;)</p>
</content:encoded></item><item><title>The next iteration of my blog</title><link>https://steveklabnik.com/writing/the-next-iteration-of-my-blog/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-next-iteration-of-my-blog/</guid><pubDate>Thu, 12 May 2011 00:00:00 GMT</pubDate><content:encoded><p>Well well well, some things never change. Long ago, I hosted my own blog, and here we are again. I’ve moved everything over to Jekyll, if that means anything to you. You can now <a href="https://github.com/steveklabnik/blog">look at my blog on GitHub</a>. Big thanks to <a href="http://www.jstorimer.com/2009/12/29/jekyll-on-heroku.html">this article</a> for lots of help and pointers with final deployment stuff, and <a href="http://mikeferrier.ca/2011/04/29/blogging-with-jekyll-haml-sass-and-jammit/">this one</a> for haml and sass and stuff. Yay!</p>
<h2>New-ish beginnings</h2>
<p>So why’d I do this? Well, this is just part of moving more towards hosting all of my own services, and not relying on services that others provide. This isn’t some sort of argument over individualism vs. collectivism, it’s more about <a href="http://nomoresharecropping.org/">No More Sharecropping</a>.</p>
<p>I’m still relying on people for hosting, of course. Nobody is 100% removed, but with my own backup of everything, I’m in total control here. It’s really minimal work to keep this going, and only a few hours of setup to move everything over.</p>
<h2>A small change in focus, too</h2>
<p>I’m also going to stop bitching about politics so often, and try to be more constructive in general. I have a lot to say about this new humanities angle that my life has taken as of late. Hence, I’ve changed the title of my blog. And you can expect more posts along those lines. More about books, more about art and code, less whining. One may think I’m maturing or something…</p>
</content:encoded></item><item><title>The first week of rstat.us: users, press, and scaling, oh my!</title><link>https://steveklabnik.com/writing/the-first-week-of-rstat-us-users-press-scaling-oh-my/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-first-week-of-rstat-us-users-press-scaling-oh-my/</guid><pubDate>Wed, 30 Mar 2011 00:00:00 GMT</pubDate><content:encoded><p>Hey everyone. A lot of people have been asking me about <a href="http://rstat.us/">rstat.us</a> lately, so I figured I’d tell a little bit of the story as it’s gone down so far.</p>
<h2>Stats</h2>
<p>First, here’s some numbers:</p>
<ul>
<li>Users: 4553</li>
<li>Uniques: 25,000</li>
<li>Pageviews: 119,385</li>
<li>Pages/visit: 4.77</li>
<li>Statuses posted: 9387</li>
</ul>
<p>Here’s some fun info: stats on statuses by language:</p>
<ul>
<li>german 118</li>
<li>russian 1355</li>
<li>none 97 &lt;- ha!</li>
<li>english 4836</li>
<li>spanish 1412</li>
<li>dutch 98</li>
<li>french 1155</li>
<li>portuguese 272</li>
<li>farsi 66</li>
<li>pinyin 1</li>
</ul>
<h2>Code</h2>
<p>Lines, by committer:</p>
<ul>
<li>28 AndrewVos</li>
<li>124 BRIMIL01</li>
<li>6 Blaine Cook</li>
<li>107 Brendan Taylor</li>
<li>924 Brian Miller</li>
<li>45 Caley Woods</li>
<li>162 Carol Nichols</li>
<li>20 Claudio Perez Gamayo</li>
<li>1347 Dominic Dagradi</li>
<li>4 James Larkby-Lahet</li>
<li>99 Jorge H. Cuadrado</li>
<li>258 Kat Hagan</li>
<li>10 Lauren Voswinkel</li>
<li>143 LindseyB</li>
<li>16 MenTaLguY</li>
<li>3 Michael Stevens</li>
<li>3 Murilo Santana</li>
<li>10 Nate Good</li>
<li>1 Peter Aronoff</li>
<li>24 Shebanian</li>
<li>44 Stephen Paul Weber</li>
<li>1409 Steve Klabnik</li>
<li>8 Tony Arcieri</li>
<li>478 Zachary Scott</li>
<li>104 burningTyger</li>
<li>28 james cook</li>
<li>56 kat</li>
<li>200 wilkie</li>
<li>10 wolfwood</li>
</ul>
<p>Also, wilkie has a lot more code in the 3 gems that we build off of, for our ostatus implementation.</p>
<p>That makes for 29 unique committers. These stats were generated by git-blame.</p>
<p>Just over 200 pull requests</p>
<p>872 commits</p>
<h2>The deal with Heroku</h2>
<p>Crazy! Such good stuff. I’m really excited as to how it’s going, but there were a few rocky points along the way as well. Some people saw me tweeting in frustration with Heroku, so here’s the skinny:</p>
<p>On Sunday, for some reason, our 3 line app.js file, as well as our favicon.ico, started to take 60 seconds to respond. This meant that a dyno was getting tied up, and since Heroku changed the way their mesh works, they kept routing requests to these screwed up dynos. This caused a rash of 500 errors, and I frantically filed a support ticket. This was the first time I’d had a paid app on Heroku, and they’ve always been fantastic. The Hackety Hack site was on the free account, and performed beautifully, with similar traffic.</p>
<p>What it boils down to is this, though: Heroku took about 22 hours to respond to my support ticket, in any way. Not like “we’re looking at it, but it’s Sunday, so it’ll take a while.” Crickets. Meanwhile, the errors kept happening. Eventually, I heard something from them, where they pretty much said ‘I see other errors, try fixing those.’ with no response on the actual root cause. I got an email from a Level2 person who promised to investigate and escalate, but by now, I still haven’t heard.</p>
<p>I also had a smaller issue with MongoHQ. To their credit, I heard back Sunday night, and the issue with them was an annoyance, not a showstopper. I ended up hearing from the CEO, who ended up following up with me multiple times, and in a pretty timely fashion.</p>
<p>So, we moved our hosting away from Heroku and onto Duostack. Now, full disclosure: I’m friends with the guys from DuoStack. That said, they took care of me. I found a small bug in their platform when deploying the app, but everything’s gone really well. Bunches of users have commented on how fast the site is now, and I’ve been really happy with it so far. We’ll see how it goes.</p>
<h2>The Future</h2>
<p>Anyway, here’s what’s next for <a href="http://rstat.us/">rstat.us</a>: The biggest thing is getting to 100% ostatus compliance. We spent lots of time this week getting screwed over by a bug in Google’s hub implementation, but now that it’s all taken care of now. While we work on that, we’re also getting ready to deploy a statusnet compliant API, so people can use statusnet clients on mobile and desktop. This is huge for me, as I’m really missing notifications.</p>
<p>Other than that, just keep using it! Tell your friends! If you code, come join us on the IRC (#rstatus on freenode), and help file bugs and patch them!</p>
</content:encoded></item><item><title>Announcing rstat.us</title><link>https://steveklabnik.com/writing/announcing-rstat-us/</link><guid isPermaLink="true">https://steveklabnik.com/writing/announcing-rstat-us/</guid><pubDate>Wed, 23 Mar 2011 00:00:00 GMT</pubDate><content:encoded><p>Whew.</p>
<p>If you’ve been <a href="http://twitter.com/steveklabnik">following me on Twitter</a> at all lately, you’ll know that I’ve been working hard on a new project lately. Tonight, even though it’s late, I’m finally getting it out there. Please welcome <a href="http://rstat.us/">http://rstat.us/</a> to the world! (as soon as the DNS updates. ;) )</p>
<p><a href="http://rstat.us/">rstat.us</a> is a micro-blogging service, just like Twitter. There’s one big difference, though: it is distributed. Oh, and it’s 100% open source. So two things. It’s also integrated with Facebook and Twitter, so you can just log in with either of those accounts, and it’ll syndicate your posts. Well, just to Twitter right now, but we’re getting there.</p>
<p>Here’s the other cool thing: Want to own your data? You can actually <a href="http://github.com/hotsh/rstat.us">get the code</a> and run a copy yourself! But you won’t get left out: you can follow anyone else that uses the <a href="http://ostatus.org/">ostatus</a> protocol, like <a href="http://identi.ca/">http://identi.ca/</a>, and it’ll Just Work. You just have to grab the URL to their feed, put it in, and it’s all good! But you probably shouldn’t do that yet. We’re still working on some features.</p>
<p><a href="http://rstat.us/">rstat.us</a> is still very much alpha quality. 7 or 8 of my best friends and I have been fighting off sleep, food, and tests for the last two weeks to put this together. Please <a href="http://rstat.us/">give it a try</a> and let me know what you think, tell me if you can break it, or anything else that’s particularly interesting. If you’re the coding type, please check out the <a href="http://github.com/hotsh/rstat.us/wiki">wiki</a> for a big old list of stuff we’d like to do.</p>
<p>Oh, and of course, you should <a href="http://rstat.us/users/steveklabnik">follow me on rstat.us</a>.</p>
</content:encoded></item><item><title>Did you hear? I'm Ruby Mendicant University's first visiting teacher</title><link>https://steveklabnik.com/writing/id-you-hear-i-m-ruby-mendicant-university-s-first-visiting-teacher/</link><guid isPermaLink="true">https://steveklabnik.com/writing/id-you-hear-i-m-ruby-mendicant-university-s-first-visiting-teacher/</guid><pubDate>Fri, 18 Mar 2011 00:00:00 GMT</pubDate><content:encoded><p>I’ve heard <a href="http://university.rubymendicant.com/">Ruby Mendicant University</a> (also known as RMU, not to confuse you Robert Morris alums!) described as “the best kept secret of the Ruby community.” High praise, indeed. If you haven’t heard about RMU before, check out this presentation by Andrew Cox:</p>
<p>TL;DR: RMU is a free, online ‘unversity’ from Gregory Brown, author of ‘Ruby Best Practices.’ You enter an intermediate Rubyist, and you exit an expert.</p>
<p>Anyway, RMU recently announced that <a href="http://university.rubymendicant.com/changelog/visiting-teacher-2011-t2-steve-klabnik">I’ll be their first visiting teacher</a>. Basically, I’ll be hanging out, giving advice, and just generally being even more accessible to RMU students than I already am. Greg and I have talked a lot about education, Ruby, and a lot of other things… between Hackety’s ‘nothing to intermediate’ and RMU’s ‘intermediate to expert’ approaches to learning, anyone wanting to get up to speed with Ruby should have no shortage of resources at their disposal.</p>
<p>If you’re feeling like you’ve got the hang of Ruby, but want to take it to the next level, <a href="http://university.rubymendicant.com/admissions">applications are open</a>! Hopefully I’ll see you there.</p>
</content:encoded></item><item><title>I'm quitting Hacker News</title><link>https://steveklabnik.com/writing/i-m-quitting-hacker-news/</link><guid isPermaLink="true">https://steveklabnik.com/writing/i-m-quitting-hacker-news/</guid><pubDate>Thu, 03 Mar 2011 00:00:00 GMT</pubDate><content:encoded><p>I’ve been thinking about this post for a while.</p>
<p>I freaking love Hacker News. It’s my favorite place on the Internet. It’s incredibly valuable. I have learned so much from my time there. With that said, I feel like it’s taken up too much of my time.</p>
<p>I’m trying to accomplish things. I got accepted to grad school. I’m trying to rebuild Shoes. I’m working on improving Hackety. I’m finding that I don’t have enough hours in the day to accomplish the things I want to accomplish. And arguing for the hundredth time about how artists will survive in a post-IP world, or how to get your first couple of users, or whatever it is that Google or Apple did this week hasn’t gotten boring yet, and that’s a problem.</p>
<p>The only respectable solution is to go cold-turkey for a while.</p>
<p>I’ve been doing good on some of my new years’ resolutions, and bad on others. I haven’t spent a lot of time writing lately, and I have only been doing okay at my writing. Hacker News is cutting into that. I feel like I’m using it in the same way some use TV. Yeah, it’s significantly deeper, but I really should be reading more books and less HN. At least, that’s what’s right for me right now.</p>
<p>I won’t be gone forever. I just need to spend more time elsewhere.</p>
<p>With that said, I’ve been spending a <em>lot</em> of time on Shoes. I’ll just leave this here: <a href="https://github.com/shoes/shoes/tree/feature/gemify">https://github.com/shoes/shoes/tree/feature/gemify</a></p>
</content:encoded></item><item><title>Living in the cloud</title><link>https://steveklabnik.com/writing/living-in-the-cloud/</link><guid isPermaLink="true">https://steveklabnik.com/writing/living-in-the-cloud/</guid><pubDate>Thu, 24 Feb 2011 00:00:00 GMT</pubDate><content:encoded><p>I’ve been joking for a while that I’d like to “move to the Internet.” Generally I cite my familiarity and love with online culture as opposed to American culture. I don’t really care what’s going on in “the real world,” as I feel it’s kind of played out. The whole cycle of “fight wars, reduce freedoms, sit in front of your TV” has gotten really boring quickly. I find Internet culture to be more complex, entertaining, and useful. But I’m getting way off topic, here.</p>
<p>As a programmer, one of the key skills I rely on daily is the ability to recognize abstractions. Writing great code requires the ability to see similarities in things, even when they may not be immediately apparent. Lately, I’ve been giving a lot of thought to the idea of “living in the cloud.” Primarily, cloud computing is about providing services on-demand, and in the amount you need. Dynamic allocation. It’s generally great stuff, and even though “the cloud” is a bit buzzwordy, it’s an awesome concept.</p>
<p>So what happens when we apply that to meatspace? What does “cloud living” look like? “Cloud working”?</p>
<p>Living in the cloud would look a lot like the world Tim Ferriss encourages with his whole lifestyle design concept, or at least the lifestyle he’s designed for himself. Move around from place to place, live where you find things interesting, reallocate yourself to a better place when the feeling dictates. The concept of being a “digital nomad” certainly makes me think of “living in the cloud.” Doing this right means giving up most material possessions, as they impede freedom of movement.</p>
<p>Another important aspect of cloud living would be finding a way to make a living while being location independent. There was a great article making the rounds a while back called “<a href="http://romansnitko.posterous.com/jobs-dont-scale">Jobs don’t scale</a>.” It’s only tangentially related, but his point about working for other people certainly relates to freedom of movement. If you have to convince your boss to let you work remotely, it’s much harder than just doing it yourself.</p>
<p>In any case, I’m very attracted to all of this as an idea. I’d love to travel the world, working from a different place every day, seeing new sights and exploring new places. It requires re-thinking a lot of things about what it means to work, how relationships work, friendships. Can you have kids if you keep moving around from place to place? How can you stay connected to friends if you never see them, because you’re halfway across the globe? Can your significant other do the same thing? Can you get month to month leases easily?</p>
<p>I’m still thinking about this as a concept, but I thought I’d share my thoughts so far. I’m not moving around the world yet, but I am trying to increase my own mobility as much as possible. I think I’m finally at a place where I don’t care much for most of my possessions anymore…</p>
</content:encoded></item><item><title>A few notes on Foucault</title><link>https://steveklabnik.com/writing/a-few-notes-on-foucault/</link><guid isPermaLink="true">https://steveklabnik.com/writing/a-few-notes-on-foucault/</guid><pubDate>Sat, 22 Jan 2011 00:00:00 GMT</pubDate><content:encoded><p>I’ve almost always been in a habit of reading. When I was younger, I spent a lot of time in the library. I’d just grab an entire series of novels, and read them all, right in a row. Sometimes I’d end up reading non-fiction too. I ended up with a lot of books on software.</p>
<p>Then my local library got Internet access. Slowly, I started to spend less time in the stacks and more in front of a terminal. This wasn’t a bad thing; I still kept reading. What’s more, what was previously a one way street turned into two: I didn’t just read the Internet, I wrote it. I spent hours and hours discussing the things I’d read with others.</p>
<p>In any case, as the years flew by, the things that I’ve been reading have become less and less substantial. Current events are fine and all, and pictures of cute cats are nice, but I feel like the volume of what I’ve been reading has gone up, but the quality has gone down. It happens. I can’t really be snide about not owning a TV while being subscribed to <a href="http://www.reddit.com/r/fffffffuuuuuuuuuuuu">/r/fffffffuuuuuuuuuuuu</a>. Reddit somehow has morphed into a place where you can feel superior, but it’s really just the same exact thing in different clothing. Combine this general unease with my aspirations for grad school in the fall, as well as my renewed interest in political philosophy, and my need to hit the books has become readily apparent.</p>
<p>Luckily, I have an ally in this quest, and Jamie has given me a reading list. Since reading and writing are two sides of a single coin, I’ll be writing about the things that I read here. Like many other things I’ve written about in the past, I’m sure that putting my thoughts down on paper (?) will help to gel my understanding and thoughts. First up: Michel Foucault’s “Discipline and Punish.”</p>
<p>I’ll get right to it: I really enjoyed reading this book. It’s partially because it took a lot of random things that I kind of knew and tied them to some experiences that I’ve had in a meaningful way. It’s also partially because I have a substantial infatuation with the conspiratorial; I re-read “1984” every year, and I often think about its all-knowing, all-seeing telescreen imagery when discussing anything vaguely political. “Discipline and Punish” gave me the same sort of images, but they were fewer, and more firmly rooted in history and reality. The book opens with a section named “Torture,” and the scene of Robert-François Damiens’ punishments for attempted regicide. I’m sure that before the release of “Saw,” these images were even more shocking:</p>
<p>… he was to be ‘taken and conveyed in a cart, wearing nothing but a shirt, holding a torch of burningwax weighing two pounds’; then, ’in the said cart, to the Place de Greve, where, on a scaffold that will be erected there, the flesh will be torn from his breasts, arms, thighs, and calves with red-hot pincers, his right hand, holding the knife with which he committed the said parricide, burnt with sulphur, and, on those places where the flesh will be torn away, poured molten lead, boiling oil, burning resin, wax, and sulphur melted together and then his body drawn and quartered by four horses and his limbs and body consumed by fire, reduced to ashes, and his ashes thrown to the winds.</p>
<p>Let it never be said that the people of 1757 were not thorough. Regardless, by January of 1840, we’d transitioned to a prison system that looks pretty much the same as it does now. The book’s primary theme is discussing how we got from point A to point B, and then examining the ‘why’ to explain how modern society has a new sovereign institution: the ‘carceral system.’</p>
<p>Before we can examine that question, though, we need to ask why we used torture as a form of punishment in the first place. The reasoning is actually straightforward: during the period of monarchy, everything revolves around the monarch. He is sovereign in a more absolute way than we even initially think of; the pop-culture image of a king has more to do with something of a popularity contest or that he’s simply the guy on the top of the pyramid, but the nature of a monarch’s power runs more deeply than that. It was called ‘divine right’ for a reason, the physical body of the sovereign was the representation of God himself, and since the entire world belongs to God, thus it belongs to and is a part of the monarch. It reminds me of the kind of doublethink necessary to grasp the Catholic conception of the Holy Trinity, in this case God the Father, the king his son, and the world rather than a Holy Ghost. All one, yet three at the same time. In any case, if the land itself is the literal body of the king, then any transgression is an act of defiance not only of the rule of the monarch, but is making war upon God himself. And since damage has been done to the body of God, so must an equivalent exchange be made with the body of the aggressor. Torture also has an element of the theatrical to it, and therefore demonstrates to all of those watching that they must also comply with the rule of law or face the consequences.</p>
<p>However, eventually, torture became socially inconvenient. Basically, it was a case of the Streisand Effect: when you place that much attention on someone, you create a forum for sympathizers to create romantic images of their fallen hero. There’s a great historical example of this in the Christian mythos: consider the polarizing effect that Christ’s torture on the cross maintains to this day. History is littered with the songs of fallen heros, and a call to follow in their stead. Eventually, whenever a new convict was to be strung up at a gallows, there’d be a state of disarray. Foucault describes several images of rioters throwing stones and even in some cases killing the executioner.</p>
<p>As a result of this, the nature of punishment slowly changed. Reformists argued that punishment was metered out unevenly, and inconsistently. Thus in the same way that monarchy gave way to democracy, the absolute right of the king to punish became distributed as well. However, centralized and distributed systems are quite different, and require different constructs to operate properly. Therefore, a distributed form of the right to punish would need some mechanism by which to operate. This mechanism is termed “discipline” by Foucault. Discipline creates a certain order all by itself, and he uses a great example of monks and monasteries to illustrate the concept of discipline. Think about all of these things that we consider virtuous:</p>
<ul>
<li>Studying is a disciplined form of reading</li>
<li>Dieting is discipline applied to eating</li>
<li>The image of a soldier is almost entirely one of discipline</li>
<li>Morality is discipline applied to all of life</li>
<li>Exercise is disciplined form of the body</li>
</ul>
<p>But discipline has even greater roots in our society. Think about Taylor’s Scientific Management, for example: it’s a means of imposing discipline on workers to maximize their production. Schooling is a way of giving children a ‘structured environment’ (structure seems to be synonymous with discipline in many cases) to develop in. Churches are places for the soul to become disciplined.</p>
<p>Submitting to discipline has deeper psychological effects as well. It creates the idea of a division: there’s those who follow the rules, and those that disregard them. And since we’ve established that those who follow the rules are virtuous, those who don’t must not be. Since those that don’t follow the rules are doing bad things, they should be subject to punishment, so that they can remain disciplined. And thus the system of rules can be used as the distributed form of this right to punish, replacing the absolute right of the monarch. Submitting to this mentality makes people into ‘docile bodies’ that perfectly fit into this worldview.</p>
<p>As an example of how far this disciplinary approach has gone, Foucault presents the Panopticon, which was a prison shaped like a pie, with a tower in the middle. Prisoners would be able to observe others directly across from themselves, and the guard who may or may not be in the tower would be able to watch all of the prisoners at once. Since you couldn’t tell if the guard was in the tower or not, discipline would become internalized, since you always had to assume that Big Brother is watching you… This way of thinking about society ends up creating an all-encompassing ‘carceral system’ that we now live in.</p>
<p>It’s also important to note that this carceral system is total and absolutely permeating every aspect of society, yet they aren’t presented as such. Foucault specifically mentions that it’s important to consider that the delinquent are still a part of the system, and not outside of it. Yet we’re constantly presented with images that serve to present the idea that there’s a ‘criminal underworld,’ that those who lead a life of crime are part of a shadowy alternate universe. Foucault refers to this idea as ‘enclosure’: “Discipline sometimes requires enclosure, the specification of a place heterogeneous to all others and closed in upon itself. It is the protected space of disciplinary monotony.” The enclosure embodies this separation, since there’s a space both within and outside of the enclosure. A self and an Other.</p>
<p>… so yeah. That’s my summary. I’m still digesting a lot of this stuff, and so I may have more to say about it later.</p>
</content:encoded></item><item><title>Goals for 2011</title><link>https://steveklabnik.com/writing/goals-for-2011/</link><guid isPermaLink="true">https://steveklabnik.com/writing/goals-for-2011/</guid><pubDate>Sun, 02 Jan 2011 00:00:00 GMT</pubDate><content:encoded><p>This year I’m happy to report that I have much more upbeat plans than last year’s kind of sad post. With that said, here’s my 2011 goals:</p>
<ul>
<li>I want to work for other people 2 days per week on average for the year.</li>
<li>I want to write something every day</li>
<li>I want to keep building on the initial success that Hackety Hack’s 1.0 launch has brought. I’d like to get a new release out every month or so, with a 2.0 for Christmas 2011.</li>
<li>I want to start exercising again. My current lifestyle is still too sedentary for my liking.</li>
<li>I want to eat better.</li>
<li>I want to start logging my activities every day.</li>
</ul>
<p>Most of these are kinda standard. I used to go to the gym all the time, but I’ve never eaten healthily. Both of these things are going to become more important as I age, however, and so I’m starting to actually give them serious consideration.</p>
<p>Hackety Hack had an impressive 1.0 launch, but I have lots of work to do. Please check out the <a href="http://blog.hackety-hack.com/">Hackety Hack blog</a> tomorrow for a report on that. Basically, I now have a sizable userbase to attend to, and they’ve found a bunch of bugs that I need to iron out before starting to build more stuff.</p>
<p>The logging is an idea that I got from <a href="http://www.sebastianmarshall.com/">Sebastian Marshall</a>. We’ve had some good conversation recently, and we talked briefly about the importance of measuring things. You can only improve things if you know what’s actually wrong… I’m going to be tracking the time that I do various activities, and seeing if I can improve. If I feel this is working, I’ll write about it in a month or so.</p>
<p>I’ve already made some progress on all of these, actually. Obviously, most of them can only truly be realized in a year’s time, but here’s what I’ve got going:</p>
<ul>
<li>I’ve got my logging set up going. I’m trying to figure out how to make it as unobtrusive as possible, but I think I have a good first run at it.</li>
<li>I’ve got a set up for the work situation, and if all goes well, that should be accomplished. I originally thought I’d have to front-load a bit, but it seems like I’ll be doing this most of the year.</li>
<li>Working that little will enable me to really accomplish my Hackety goals.</li>
<li>Now that Maria is graduated, she’s got some time to cook things, and she’s also interested in eating better, and her habits are better than mine already. Hopefully this will lead to success.</li>
<li>Logging should help keep my eyes on the prize for my exercise and writing goals.</li>
</ul>
<p>So, that’s it! Good luck with your goals in the new year!</p>
</content:encoded></item><item><title>Hackety Hack v1.0</title><link>https://steveklabnik.com/writing/hackety-hack-v1-0/</link><guid isPermaLink="true">https://steveklabnik.com/writing/hackety-hack-v1-0/</guid><pubDate>Sat, 25 Dec 2010 00:00:00 GMT</pubDate><content:encoded><p>It’s been a long journey. I have a lot to say about this, but it’ll have to wait until I’m not in the middle of nowhere spending some time with Maria’s family. I just wanted to tell you all Merry Christmas, and point you at the <a href="http://blog.hackety-hack.com/post/2457961910/merry-christmas-and-a-1-0-announcement">Hackety Hack v1.0 release announcement</a> and the new <a href="http://hackety-hack.com/">Hackety Hack website</a>.</p>
<p>Merry Christmas!</p>
</content:encoded></item><item><title>The hardest decision I've ever made</title><link>https://steveklabnik.com/writing/the-hardest-decision-i-ve-ever-made/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-hardest-decision-i-ve-ever-made/</guid><pubDate>Wed, 17 Nov 2010 00:00:00 GMT</pubDate><content:encoded><p>I’ll cut to the chase: I’ve stepped down as CTO of CloudFab. I don’t want to discuss details, but life is too short to not be doing what you want with it. And, after a period of careful thought and consideration, it’s no longer the right thing for me. It was an amicable parting, and my replacements are already in place, and doing cool things. I’m happy with what I’ve accomplished, but it’s time for me to move on.</p>
<p>This raises the question: what am I moving on to? Here’s the answer: Hackety Hack. What I truly care about is teaching, writing, and discussing software. If I could do anything, I’d hack on Hackety, share what I know with others, and keep learning and playing with new things all day long.</p>
<p>However, there’s a certain question of sustainability here: I’m not made of money, and I have lots of bills to pay. However, doing the ‘obvious’ thing would be really, truly wrong: Hackety Hack is not and will never be a startup. It’s a social venture, if anything: it would be absolutely unquestioningly wrong for me to ever ask anyone for a single cent for it. It’d also be counterproductive: how could I share programming with everyone if I excluded those without disposable income?</p>
<p>So to this end, Hackety cannot be all that I do. And while I really want to teach, I also have a lot to learn. Luckily, I have some great allies in this area: I’ve been working alongside two teachers at the University of Pittsburgh over the past month or so: one in the English department, and one in the Computer Science department. In doing so, I’ve come to realize that my place in life is actually in the middle: while I love the theoretical side of CS, I’m actually much more interested in the application. And teachers need to be effective communicators, so investing in writing is absolutely worthwhile for me. At the same time, I’ve really enjoyed every chance I’ve had to write: there’s a reason I keep three blogs now! I’m also incredibly interested in the parallels that exist between computer and human languages. I like the comparison between refactoring and composition. I think the two fields have a lot to learn from one another. It’s something I’d like to explore more.</p>
<p>To do so, I’ll have to finish off my undergrad degree. Luckily, I’m very close, so I’ll be doing that in the spring. Afterward, it’s my intention to go to grad school in the English department, so I’m furiously working on my application, with the help of the aforementioned teacher. At the same time, I’ll possibly help teach or TA classes in the CS department as well. I’m hoping to combine my solid foundations in the hard sciences with the new perspective and improved analysis that the humanities can offer.</p>
<p>I don’t think that academia is where I want to stay forever, but I think it’s a good place for me to grow for now. It’s time for me to invest in myself a little. I think I’ll end up a much better person in the end.</p>
</content:encoded></item><item><title>Trolling is a art</title><link>https://steveklabnik.com/writing/trolling-is-a-art/</link><guid isPermaLink="true">https://steveklabnik.com/writing/trolling-is-a-art/</guid><pubDate>Fri, 24 Sep 2010 00:00:00 GMT</pubDate><content:encoded><p>I find it incredibly hard not to be judgmental.</p>
<p>I’m not sure what part of my personality makes this happen, but even when I try to curb tearing down other people, I end up doing it anyway. I’m not sure if it’s just me, but I think part of the problem is that these kinds of things are rewarded.</p>
<p>One of the reasons that I use my real name everywhere is to try to improve the civility of my discourse. When you’re anonymous, it’s really easy to be callous. And when I used a handle, I was even worse. But even now, I can think of three times when I accidentally was mean for basically no reason: That tweet, the Diaspora incident, and the only Hacker News comment I’ve ever deleted.</p>
<p>In all three of these instances, it’s not so much that I was wrong. It’s that I ended up presenting my criticisms poorly. I think I did a decent job with the Diaspora one, but people still took it the wrong way, so I failed to communicate. I’m certainly not infallible. I guarantee you there are n00b errors and security holes in my own codebases. So I really shouldn’t be so quick to judge. That’s all I have to say about that. I’ll just have to do better going forward.</p>
</content:encoded></item><item><title>80% of success is showing up</title><link>https://steveklabnik.com/writing/80-of-success-is-showing-up/</link><guid isPermaLink="true">https://steveklabnik.com/writing/80-of-success-is-showing-up/</guid><pubDate>Tue, 21 Sep 2010 00:00:00 GMT</pubDate><content:encoded><p>It’s funny how true this quote is.</p>
<p>There are three people that I really admire: _why, Zed Shaw, and Yehuda Katz. And to simplify my tenses a bit, I’ll speak of _why in the present tense. What do these three guys have in common? They all churn out badass software like nobody’s business. _why started so many projects that it just adds to his mythical qualities; Zed has been cranking out software and blogs left and right, and Yehuda’s “<a href="http://yehudakatz.com/2010/09/14/heres-to-the-next-3-years/">I’m quitting Engine Yard</a>” post basically said “I’m pretty much done rebuilding the server side of software in my own image, now it’s time for me to focus on the client side of the equation.” Not with that much hubris, of course, but it’s pretty much true.</p>
<p>Now, I don’t mean to imply that they’ve ‘just shown up,’ but it seems to me that productivity is directly related to just getting started on things. I’ve had people tell me that they think I’m superhumanly productive, which is funny to me: I’m almost constantly fretting about how much time I waste every day. And while I do have a ton of projects, unlike all those guys, I tend to not fully finish them. It’s something that I’ve been trying to work on lately, because I have so many things that I want to accomplish.</p>
<p>For me, the biggest hold on cranking out even more stuff is just simply starting to work. Often, if i slack off for half an hour, I slack off all evening. Now that there’s a TV with Netflix in my house again, it’s difficult to not watch all of the amazing documentaries that are on there. But if I manage to start working on stuff, I find that I keep doing it. It’s like I get to pick one or the other: projects or slacking. I can’t quite seem to do both. And so I’ve been starting some minor experimentation to see if I can’t work on projects more often than not.</p>
<p>One thing that I’ve sort of half-tried is the <a href="http://en.wikipedia.org/wiki/Pomodoro_Technique">Pomodoro Technique</a>. It’s pretty sweet: basically, you set a timer for 25 minutes, and start working. You can take a break when it goes off. Surely you can wait to refresh Hacker News for 25 minutes, right? That’s something else I’ve done: on Friday, I put Reddit and HN in my /etc/hosts file. They’re totally blocked, for now. After <a href="http://blog.steveklabnik.com/trouble-with-diaspora">last week’s situation</a>, I spent waaaay too much freaking time on HN, even for me. I love it, and I’m learning a ton every day, but it’s becoming an issue for my productivity. So it’s gone for a bit. It’s been rough, but I’ve been getting lots done, and I think when I start reading it again, I’ll be able to keep it at reasonable levels.</p>
<p>Anyway, so that’s what I’m doing to keep productive. Anyone else have any thoughts on this? How do you keep working on side projects? What works for you?</p>
</content:encoded></item><item><title>Trouble with Diaspora</title><link>https://steveklabnik.com/writing/trouble-with-diaspora/</link><guid isPermaLink="true">https://steveklabnik.com/writing/trouble-with-diaspora/</guid><pubDate>Fri, 17 Sep 2010 00:00:00 GMT</pubDate><content:encoded><p>So, Wednesday, Diaspora was released.</p>
<p>If you’re not familiar, a few months ago everyone was up in arms about the latest Facebook privacy change. So four college kids started <a href="http://www.kickstarter.com/projects/196017994/diaspora-the-personally-controlled-do-it-all-distr">a Kickstarter project</a> with a dream: let’s make a distributed, private Facebook. They asked for $10,000. They got about $200,000.</p>
<p>They worked on the code all summer, in secret. Wednesday, a ‘developer preview’ came out. They’d promised something by the end of the summer, and the 15th of September is the last day. So <a href="http://github.com/diaspora/diaspora">they put it up on GitHub</a>.</p>
<p>Oh boy.</p>
<p>Basically, the code is really, really bad. I don’t mean to rain on anyone’s parade, but <a href="http://www.theregister.co.uk/2010/09/16/diaspora_pre_alpha_landmines/">there are really, really bad security holes</a>. And they’re there due to things that any professional programmer would never dream of leaving out of their code. I don’t want to disclose too many details, but you can see the code yourself on GitHub.</p>
<p>At first, I found one. So I tried to patch it. And I did, and it got accepted into master. Awesome. But then, the more I read, the more bad things I found. They’re going to need a complete overhaul to fix this. Go over every last piece of code. And don’t even get me started on their encryption code.</p>
<p>But basically, please heed their own warning from <a href="http://www.joindiaspora.com/2010/09/15/developer-release.html">the announcement</a>:</p>
<blockquote>
<p>Feel free to try to get it running on your machines and use it, but we give no guarantees. We know there are security holes and bugs, and your data is not yet fully exportable. If you do find something, be sure to log it in our bugtracker, and we would love screenshots and browser info.</p>
</blockquote>
<p>If you find one of the many, many nodes that people are publicly posting, please don’t use your real passwords, or information.</p>
<p>And if you’re a developer, consider helping out. It’s going to be rough going: the mailing list and issue trackers are full of people that have no idea what they’re doing. Be prepared to wade through tons of crap. But they could really, really use the help. I’ll be submitting another patch or two, but it needs much, much more than I can give.</p>
<p>EDIT: Please check out <a href="http://blog.steveklabnik.com/this-is-why-new-media-hates-old-media">my follow up post</a>, where I talk about being misquoted regarding this post.</p>
<p>EDIT 2: Patrick has a post up describing the exploits, now that there have been some patches applied. If you’re a technically inclined individual, you might be interested. You can find it on <a href="http://www.kalzumeus.com/2010/09/22/security-lessons-learned-from-the-diaspora-launch/">his blog</a>.</p>
</content:encoded></item><item><title>A word about _why, Whyday, and Hackety Hack</title><link>https://steveklabnik.com/writing/a-word-about-why-whyday-and-hackety-hack/</link><guid isPermaLink="true">https://steveklabnik.com/writing/a-word-about-why-whyday-and-hackety-hack/</guid><pubDate>Thu, 19 Aug 2010 00:00:00 GMT</pubDate><content:encoded><p>Today is Whyday. A programmer’s holiday. Today is the day that we don’t forget that programming isn’t all Agile and driven by behavior and Serious Business. Inside each cubicle, the hacker spirit is trying to burst free. Today, it will.</p>
<p>Today, I’m also releasing a prerelease version of Hackety Hack 1.0. It’s been a long time coming. The announcement of Whyday got me thinking about the past year of a world without _why, and the ‘your domain is about to expire’ notices got me thinking about how I’ve now been the head of the Hackety Hack project for almost 12 months, too. I don’t like thinking about this, because I almost let _why down.</p>
<p>I never met _why. I can’t really claim to know him, or know what his intentions were, but I think about him a lot. ‘_why’ the persona, really embodies everything that I love about software. The world owes _why a great debt, and I’d like to think that I’m doing what I can to repay him.</p>
<p>At least, I am now. I don’t like to think about roughly 6 or 8 months of the last year of Hackety Hack; I did an exceptionally poor job of being a steward. Being alone, having no help, and having a bit of an identity crisis is an excuse; you can decide if it’s a good or poor one. But things have been looking up, and we’ve got some momentum, I owe a great debt to Fela for helping make this summer happen.</p>
<p>I really do feel that the Little Coder still has a predicament, and I’m almost to the point where they can find some solace. I still feel that Hackety Hack could become an absolutely invauable project, I’ve just got to keep going. I can, I will, and I am. But I wouldn’t be able to if so many people didn’t care, say a nice world, or lend a hand.</p>
<p>So thanks. And as ashbb would say, “Let’s all have fun with Hackety Hack!”</p>
<p>Happy Whyday.</p>
<p>EDIT: The release is now out. See the announcement <a href="http://hackety-hack.com/posts/happy_whyday">here</a>, and get it <a href="http://hackety-hack.com/">here</a>.</p>
<p>Thanks!</p>
</content:encoded></item><item><title>A blip in time</title><link>https://steveklabnik.com/writing/a-blip-in-time/</link><guid isPermaLink="true">https://steveklabnik.com/writing/a-blip-in-time/</guid><pubDate>Sat, 24 Jul 2010 00:00:00 GMT</pubDate><content:encoded><p>Sometimes, insight comes from very unlikely places.</p>
<p>I’m keenly interested in the music industry, and how it will evolve, or possibly die. I care deeply about music; it’s had a profound effect on my life over the years. Even though the entire industry has been screaming that they’re dying, I truly believe that music will never die. It’s bigger than money. I’m not exactly sure about the details, even though I have some ideas.</p>
<p>In any case, I was reading about this the other day, and I came across <a href="http://sometimesright.com/2010/05/sir-mick-on-music-profits/">this quote from Mick Jaggar</a>:</p>
<blockquote>
<p>But I have a take on that – people only made money out of records for a very, very small time. When The Rolling Stones started out, we didn’t make any money out of records because record companies wouldn’t pay you! They didn’t pay anyone!Then, there was a small period from 1970 to 1997, where people did get paid, and they got paid very handsomely and everyone made money. But now that period has gone.So if you look at the history of recorded music from 1900 to now, there was a 25 year period where artists did very well, but the rest of the time they didn’t.</p>
</blockquote>
<p>An interesting bit of history, and an interesting perspective from someone who has some skin in the game, to say the least. But it’s the last sentence of that blog post that gives the true insight:</p>
<blockquote>
<p>Don’t stop at 1900, though. If you think of the entire history of the world, the notion that you could make an outsized return on making music is a complete aberration.</p>
</blockquote>
<p>When I first read this sentence, something clicked inside my head, and I had a realization: does this apply to everything?</p>
<p>We always think that we’re moving forward. It’s called ‘progress’ for a reason. But it’s kind of obvious why we’d think this: we’re the ones living it! Of course we’d do know wrong!</p>
<p>But what if we are?</p>
<p>What if we’re not progressing, but regressing? What if the latest and greatest isn’t where things are going… it’s just a small blip in the graph?</p>
<p>Maybe we’re actually going down the wrong path, building towards deflation, and need to turn around and revise our position. Maybe we’ve misstepped. What if we’re doing what we think is right, but it’s actually wrong? What if we’re just on the upswing of some local minima?</p>
<p>I don’t necessarily mean that, for example, the music industry will cause the total collapse of everything. But maybe it was a temporary peak. Things will return back to some even level again, and over the grand sum of history, such a small deviation will be forgotten.</p>
<p>Then I start to apply this line of reasoning to everything. How can we really know if we’re doing better than yesterday?</p>
</content:encoded></item><item><title>A case study in being excellent: Divvy</title><link>https://steveklabnik.com/writing/case-study-in-being-excellent-divvy/</link><guid isPermaLink="true">https://steveklabnik.com/writing/case-study-in-being-excellent-divvy/</guid><pubDate>Wed, 30 Jun 2010 00:00:00 GMT</pubDate><content:encoded><p>The images in this post have been lost to time. Oh well. Original post below, images removed.</p>
<hr />
<p>The other day I wrote a post about <a href="http://blog.steveklabnik.com/being-excellent-to-each-other">being excellent to each other</a>. Today, I’ve got a great example of that for you: Divvy.</p>
<p>What would you do if you received an email like this?</p>
<p>Here’s what happened:</p>
<p>Here’s a <a href="http://www.reddit.com/r/programming/comments/ckcbr/mizage_a_small_development_company_that_made/">link to the thread</a>. Check out some of these comments:</p>
<p>… okay, maybe that last one is a bit much. But still, they’ve obviously been repaid many times over for something that <em>didn’t even cost them money</em>. This <a href="http://news.ycombinator.com/item?id=1473770">comment on HN</a> really says it all:</p>
<p>If you’d also like to support a company for being awesome, here’s a link to <a href="http://www.mizage.com/divvy/">Divvy</a>.</p>
<p>EDIT: They’ve posted the sales afterward on Reddit <a href="http://www.reddit.com/r/programming/comments/cmmfg/hey_reddit_a_week_ago_someone_posted_about_divvy/">here</a>.</p>
</content:encoded></item><item><title>Productivity Waves</title><link>https://steveklabnik.com/writing/productivity-waves/</link><guid isPermaLink="true">https://steveklabnik.com/writing/productivity-waves/</guid><pubDate>Tue, 08 Jun 2010 00:00:00 GMT</pubDate><content:encoded><p>Right now, I’m having a bit of a ‘darkness before the dawn’ kind of moment. I feel like I might soon start to become productive again.</p>
<p>It’s sort of strange, how these things travel in waves. Just a month or two ago, I was super-ultra-crazy productive. I was working on Hackety, coding Bindlr, writing three days a week on the blog, and more. This doesn’t even count the more-than-a-full-time CloudFab. But lately, all I’ve wanted to do was hang out and play video games.</p>
<p>It seems like this is a cyclic kind of thing. I think 80 hour weeks, even of things that you love, is kind of unsustainable. It was lots of fun, and I was feeling pretty good while I was doing it, but now that I’m off that horse, it’s really hard to get back on.</p>
<p>In any case, here’s the status of some of the stuff I’ve been up to, if you’ve been wondering:</p>
<ul>
<li>Blog: I’ve got a bunch of stuff half-written. I’ll be starting to finish those off and getting them up here. I’ve had lots to say, just no motivation to say it.</li>
<li>Watch.steve: I want a better design, so I’m not doing anything with the site ’till that’s done. I don’t want to mess with Ryan’s look and feel. But I’ve got episodes written up, they just need to be recorded.</li>
<li>Hackety Hack: Fela is making great progress, and I’m going to start reaching out to Mac development communities to see if I can get some help on the issue that’s ruining my ability to fix things.</li>
<li>Bindlr: Up, running, and working. Needs more marketing/outreach.</li>
<li>Twitter: started using it again, you read that post.</li>
<li>Selling my stuff: most of it is in boxes or thrown away. Time to start selling the things that were worth money.</li>
<li>Maria: currently missing her.</li>
<li>Friends: started actually making it back to XOmB meetings on Saturdays, and actually going out with some people I haven’t talked to in a while. I was really really bad at this in some cases.</li>
</ul>
<p>This post is kind of random, and I apologize. I’ve got a good mix of technical things, book reviews, and social commentary waiting in the wings.</p>
<p>I’m excited for what June will bring.</p>
</content:encoded></item><item><title>A Declaration of the Independence of Cyberspace</title><link>https://steveklabnik.com/writing/you-are-terrified-of-your-own-children-since-they-are-natives-in-a-world-where-you-will-always-be-immigrants/</link><guid isPermaLink="true">https://steveklabnik.com/writing/you-are-terrified-of-your-own-children-since-they-are-natives-in-a-world-where-you-will-always-be-immigrants/</guid><pubDate>Sat, 01 May 2010 00:00:00 GMT</pubDate><content:encoded><p>I just saw this today. It kind of goes along with my “Move to the Internet” post from a few days ago.</p>
<p>Enjoy.</p>
<p><a href="https://www.eff.org/cyberspace-independence">A declaration of the Independence of Cyberspace</a></p>
</content:encoded></item><item><title>A break with the past</title><link>https://steveklabnik.com/writing/a-break-with-the-past/</link><guid isPermaLink="true">https://steveklabnik.com/writing/a-break-with-the-past/</guid><pubDate>Tue, 27 Apr 2010 00:00:00 GMT</pubDate><content:encoded><p><a href="http://www.countdowntooauth.com/">Pretty soon</a>, Twitter is going to turn off Basic Authentication and switch entirely to OAuth. <a href="http://www.scripting.com/stories/2010/04/26/theToxicCoralReef.html">People are upset</a>. It’s natural. If apps aren’t updated, they’ll stop working, entirely. This could be bad.</p>
<p>But in the long run, it won’t be.</p>
<p>In any sort of long-term endeavor, things can go sour, due to accumulating cruft over time. You see this in married couples that argue every day, you see this in old-guard companies that have six layers of management, you see this in software of all kinds. You don’t want to break old stuff, so you support it. Rocking the boat may cause some problems. It could bring up old wounds. You might have to ask someone to improve. Things won’t ever be the same, and you’re gambling with the outcome, hoping that in the end, everything will be better.</p>
<p>Sometimes, this gamble pays off big.</p>
<p>Apple’s managed to do this three times in its history, in a huge way. They’ve reinvented themselves a few times, but there were three huge, backward- compatibility breaking changes that could have killed the company. The first was the transition from the 68k architecture to the PowerPC architecture, the eventual change from PowerPC to x86, and the transition from OS9 to OSX.</p>
<p>Apple managed to make all of these transitions successfully, and it’s been one of the greatest benefits to the company. A huge part of why Windows is so horrible is that it goes above and beyond the call of duty with backwards compatibility; Microsoft never wants to rock the boat. When they try to, they get punished. Look at how poorly the transition to Vista went. So what made it go so right for Apple, and so wrong for Microsoft?</p>
<p>The first thing that Apple did right with all of these transitions was being open about them. People were given ample time to move over to the new platform. I was young when the 68k change happened, so my remembering of that time might be fuzzy, but the OSX announcement certainly was made with ample time. An entire year elapsed between the public beta and the release of 10.1, which was widely considered the first version of OSX that was worth using. I remember buying a copy of 10.0, and Apple gave you an upgrade to 10.1 for $30, similar to the Snow Leopard upgrade. 10.0 was just too buggy to be considered a proper replacement. Anyway, that was a year, and that’s not counting the time between the announcement and the beta release. For the transition to Intel, the initial announcement was made in 2005, and Steve Jobs said the transition would happen over the next two years. That gave everyone plenty of time to get their ducks in a row.</p>
<p>The second thing that Apple did correctly was provide ample tools for dealing with the transition. For the first switch, they provided a 68k emulator, and kept it going all the way up until the Intel transition. This meant that people didn’t have to re-write their apps from scratch, and gave them a big window for re-writing apps. Rosetta fulfills the same role for the PowerPC/Intel change. And during the OS9/OSX changeover, Apple not only let you emulate OS9, but also created the Carbon framework to bridge the gap between System 9 and Cocoa.</p>
<p>Finally, Apple made these changes boldly, and didn’t back down. With any kind of massive change like this, unless someone forces the change through, people will stay with the old system, and the transition fails. You can see this happen with both the Python move to Python 3000, the move from Ruby 1.8 to 1.9, and the move from XP to Vista. There’s a fine line between giving people leeway to make the transition and actually forcing them to do it. Apple did a poor job with this when it came to Adobe, who’s being an incredible laggard with Photoshop. It’s a massive application, sure, but Apple announced Carbon’s end of life ages ago, they really should have gotten their shit together. Generally, Apple’s not afraid to end-of-life products after the transition time is over, though.</p>
<p>In any case, those seem to be the three things that make a transitional period work: transparency, tools, and finality. If Twitter wants to survive the Oauthcalypse (as if they won’t), they need to follow the same path. And they have been. They announced the move to Oauth over a year ago (I think), they’ve had libraries and examples out there for ages, and they’ve picked a date and they’re sticking to it.</p>
<p>I’m glad they’re doing it. OAuth is an important part of Twitter’s future, and it’s good that they’re embracing it fully.</p>
</content:encoded></item><item><title>Book review: Rework</title><link>https://steveklabnik.com/writing/book-review-rework/</link><guid isPermaLink="true">https://steveklabnik.com/writing/book-review-rework/</guid><pubDate>Wed, 10 Mar 2010 00:00:00 GMT</pubDate><content:encoded><p>I’ve been waiting for this book for a while. “Rework” is the new book by Jason Fried and David Heinemeier Hansson from 37signals. It hit stores on Tuesday. Here’s a (non-affiliate) link to <a href="http://www.amazon.com/gp/product/0307463745/ref=s9_simh_gw_p14_t1?pf_rd_m=ATVPDKIKX0DER&amp;pf_rd_s=center-2&amp;pf_rd_r=1RRQJN39HSJ4SMVA2EM4&amp;pf_rd_t=101&amp;pf_rd_p=470938631&amp;pf_rd_i=507846">Rework on Amazon</a>.</p>
<p>For those of you who don’t know, I’m down in Houston, Texas. I’m here for a few days before heading over to Austin for SXSW. There’s a lot of really cool start up stuff, as well as 3D people, and a friend of mine. It’s been a good trip so far. But even with waking up at 3am, connecting flights, and a few hours of driving, I still managed to find a spare moment to head over to a Borders and grab a copy of Rework. And even though I’m running all over town, Nick is driving, so I’ve been able to read all of Rework in between lunches and networking events.</p>
<p>Rework is interesting. I described it earlier today <a href="http://twitter.com/steveklabnik/status/10281523422">as a philosophy text</a>, and I feel that description is absolutely apt. It’s 37signals in its purest, most potent form. If you’ve read <a href="http://gettingreal.37signals.com/">Getting Real</a>, this territory will be familiar. In fact, a lot of it is basically the same. I’d be lying if I told you otherwise. The real truth is that it doesn’t matter. People who don’t already know and love 37signals won’t have read Getting Real, and so this stuff will be novel to them. People who do won’t mind re-reading this information again, as they’ve bought into the philosophy. And an update isn’t a bad thing, either. What makes Rework interesting is how it’s different from Getting Real, not what’s the same.</p>
<p>I thought it’d be most interesting to talk about Rework in the context of it’s own philosophy. I think there are three points in particular in the book itself that point out just why this book is so good. The first is an old 37signals standby, the other two are new.</p>
<h3>Build half a product, not a half-assed product</h3>
<p>This got <a href="http://gettingreal.37signals.com/ch05_Half_Not_Half_Assed.php">a section devoted to it in Getting Real</a>. Here’s the core idea:</p>
<blockquote>
<p>Throw in every decent idea that comes along and you’ll just wind up with a half-assed version of your product. What you really want to do is build half a product that kicks ass.</p>
</blockquote>
<p>They mention that Rework used to be twice as big. The next to last draft was 57,000 words, and the final draft was 27,000. This is the biggest difference between the two books. It’s the most pure dosage of Kool-Aid I’ve ever read. Each section feels finely honed. They’re all either one, two, or three pages, and an accompanying picture. This book is about what’s worked for the company so far over its lifetime, and this refinement process is clearly demonstrated here.</p>
<p>It’s always easy to ask for more. I’m really curious about the things that were cut. Were there more sections? Was each section twice as long? A little of both? At the same time, this exactly exemplifies the thinking this section is railing against. If the book was twice as long, would I have learned twice as much? Probably not. YAGNI.</p>
<h3>Decommoditize your product</h3>
<blockquote>
<p>Make you part of your product or service. Inject what’s unique about the way you think into what you sell.</p>
</blockquote>
<p>This is something that these guys do really well. It’s part of having such strong opinions, and sharing them with the world. Everyone knows who 37signals is and what they stand for. If I wrote each chapter of Rework into a blog post, you’d still recognize it as their philosophy. It also comes through in the writing. They mention in the notes that Matthew Linderman helped them pull their distinct styles of writing into a more cohesive whole. He did a good job, and didn’t let the voice get lost in the editing.</p>
<h3>Out-teach your competition</h3>
<blockquote>
<p>Teaching is something individuals and small companies can do that big companies can’t.</p>
</blockquote>
<p>Teaching is a topic I’ve been getting more and more into lately. Hackety Hack is about teaching; when I speak, I’m teaching; this blog is about teaching. Rework is about teaching the lessons 37signals have learned about business to the world. A lot of Signal vs. Noise is about teaching. It’s a great way to get people to recognize you, and a great way to give back. The world can always use more great teachers.</p>
<h2>Wrapping up</h2>
<p>There’s a reason I’m up late, writing this review. I couldn’t put Rework down. I enjoyed revisiting the old topics; the new ones are still tumbling around in my brain. I’m sure this book is going to spawn a bunch of posts on this blog in the future, as I add my own thoughts to the stuff I’ve learned. I’d recommend this book to anyone who’s working in the business world or doing a startup, even if they hate that term. It’ll give you a lot of interesting thoughts to chew on.</p>
</content:encoded></item><item><title>Create a more compelling experience for your users through game mechanics</title><link>https://steveklabnik.com/writing/create-a-more-compelling-experience-for-your-users-through-game-mechanics/</link><guid isPermaLink="true">https://steveklabnik.com/writing/create-a-more-compelling-experience-for-your-users-through-game-mechanics/</guid><pubDate>Mon, 08 Mar 2010 00:00:00 GMT</pubDate><content:encoded><p>Ever wonder why some websites are so addictive? Certain sites always keep you going back, time after time after time. Well, I can’t speak for all of them, but there’s a subtle reason that some sites draw your attention on such a repeated basis: They’re actually games.</p>
<h2>Wait, games?</h2>
<p>Try a little thought experiment: If I say, “Yeah, he’s a ______ addict,” what are the first few things that pop into your mind? For me, top two are “heroin” and “World of Warcraft.” I’m not sure what that says about me as a person, but ignore that for now. What makes these two things so addicting? Why are they basically synonymous with the word “addict”? Lots of people smoke pot. Lots of people play Call of Duty. Lots do both, and in copious amounts. So why don’t they get the same label?</p>
<h2>Heroin: it’s a hell of a drug</h2>
<p>Yeah, that reference is to cocaine, another famously addictive substance. Oh well.</p>
<p>Heroin is the poster child for addiction because it’s got a built-in viral loop. That sentence sounds ridiculous, but it’s true. It’s very easy to start out with, as it’s snorted. No scary needles or anything. You get high really quickly, due to its chemical properties combined with the fact that your nose is already close to your brain. It gives a really intense high that is also fairly short. As you do it, you develop both a psychological addiction as well as a tolerance. You simultaneously develop a deep desire for more of the drug as you need a larger quantity of the drug to get the same high. Eventually, it becomes more and more difficult, but you’re so addicted that you get over your fear of needles and start mainlining.</p>
<p>World of Warcraft works the same way. It’s easy to try, as there are mechanisms to invite your friends, and the system requirements are fairly low for a video game. The first few quests are super easy, and so you hit that quick reward. You get addicted to “Ding!” but it takes longer and longer every time you do it. Eventually, you max out on levels and have to start doing other things to get your fix. It may sound funny, but it’s absolutely true. People talk about “relapsing.” They speak of “craving.” That’s why WoW has so many subscribers.</p>
<h2>How to replicate this success</h2>
<p>I can’t guarantee that you’ll be able to make your site as addictive as heroin is, but many sites use the same basic psychology to keep you coming back. Game mechanics are one of the tools they use to develop that psychological addiction. This is something we’ve been seeing more and more of lately, but it isn’t really being talked about explicitly as a major trend. I really think that this stuff is really important and useful.</p>
<p>There are a couple of different mechanisms that web sites can incorporate that fall under the realm of “game mechanics:”</p>
<ul>
<li>Collectibles: <em>Any sort of item you can accumulate. Sometimes comes in “sets,” which are finite lists.</em></li>
<li>Points: A concrete number that lets you compare two people.</li>
<li>Levels: A target number of points, you gain the “level” when you go over that number.</li>
<li>Trophies: A special kind of level that’s unrelated to points. You get it for some other arbitrary reason.</li>
<li>Rankings: A place where you can go to see how many points, levels, and trophies others have</li>
<li>Tournaments: A competition between people.</li>
</ul>
<p>We’ve all heard these terms used in games. But in web sites? Okay, let’s try those things again:</p>
<ul>
<li>Collectibles: Gowalla items. Facebook “Gifts”</li>
<li>Points: Twitter followers. Facebook friends. Number of feedbacks. Reddit Karma.</li>
<li>Levels: eBay “Power Sellers.” Foursquare “Super Users.”</li>
<li>Trophies: Badges, of any kind. “Achievements”</li>
<li>Rankings: FourSquare’s Leaderboard. Klout. Listorious. Hacker News’ top list.</li>
<li>Tournaments: I actually can’t come up with a good example of this. Thoughts?</li>
</ul>
<p>The same feedback loop happens on these websites. You say something interesting on Twitter, you gain another follower or two. You say something else, another follower. You check in, oh look, you’re the mayor! You sell an extra hundred things and get your Power Seller discount.</p>
<p>That’s the hard stuff. It’ll get you hooked, and coming back for more.</p>
<h2>Where’s all of this going?</h2>
<p>This is the current stuff that’s being done with game mechanics. But where could we go, in the future?</p>
<p>A while back, there was a huge debacle over ReadWriteWeb and Facebook connect. To give you the basic idea, <a href="http://www.readwriteweb.com/">ReadWriteWeb</a> is a blog that talks about everything Web2.0. They wrote an article entitled “<a href="http://www.readwriteweb.com/archives/facebook_wants_to_be_your_one_true_login.php">Facebook Wants to be your One True Login</a>.” Read the comments. Notice something funny? Due to some Google magic, if you were to Google “Facebook login” the day that was posted, that article would appear at the top under the “Google News” results. Now, RWW uses Facebook Connect for their commenting system, and a ton of people apparently don’t know how to use the Internet. So when they said, “Hey, I think I’ll go to Facebook today,” they Googled “facebook login,” clicked the news story, and went to RWW. They then ignored that RWW is a blog completely covered in red that looks nothing like Facebook, scrolled until they found the Facebook icon, clicked it, logged in, and then said “wtf, this isn’t my facebook? Why’d they change the interface again???” This happened a week after a middle-sized interface upgrade on Facebook, for extra hilarity.</p>
<p>Now, I won’t comment on those people or that situation directly. But one of my favorite Hacker News posters, <a href="http://news.ycombinator.com/user?id=patio11">patio11</a>, posted <a href="http://news.ycombinator.com/item?id=1119186">a really interesting comment</a> about the situation. I’m linking to the person he’s responding to, for context:</p>
<blockquote>
<p>Pyre: Facebook can’t improve their interface to make users not type “facebook login” into Google as a way of accessing their site.</p>
</blockquote>
<blockquote>
<p>patio11: That is a failure of the imagination. They certainly could – whether it is worth doing or not is another question, but hey, that is what God gave us A/B testing to figure out.</p>
</blockquote>
<blockquote>
<p>“Hey user, it looks like you came to us today from Google searching for [Facebook login]. Did you know that there is a better way? Type facebook.com into [blah blah blah]. Try it now and we’ll give you 5 free credits for [without loss of generality: FarmVille]!”</p>
</blockquote>
<blockquote>
<p>Great job! You should do that every time. If you do that to log into Facebook the next five days you use the service, we’ll award you a Facebook Diploma and give you another 10 free credits for [without loss of generality: FarmVille]!"</p>
</blockquote>
<blockquote>
<p>On the back end, you show the above prompts to N% of your users who you detect coming to the login page from Google search results (this is trivial – check the referer). You then compare any user metric you want for the “Was Shown Facebook Login Course” population and “Complete Facebook Login Course” population with the population at large. Kill the test if it hurts your metrics, deploy it sitewide if it helps them.</p>
</blockquote>
<p>How cool would that be? Now the game mechanics aren’t being used just to increase engagement, but to actually teach people how to use your site or service. It’s classical conditioning; reward people for doing the right thing, and they’ll keep doing the right thing.</p>
<h2>Game mechanics are your MVP</h2>
<p>So how’s this stuff relevant to your startup? Well, I think this idea ties in really well with the concept of a Minimum Viable Product. Here’s the idea: Build your MVP, and then build game mechanics in. Unlock new features based on game mechanics. This gives you a few advantages:</p>
<ol>
<li>Your new users only get the most basic experience, which is still useful. It’s a simplified, streamlined experience.</li>
<li>Users only get the new features added that are relevant to how they use the site itself.</li>
<li>You can “fake it till you make it” by implementing the features that are most useful to your users. Is everyone getting Badge A and not Badge B? Implement Feature A Level 2 first!</li>
</ol>
<p>I think that this makes for a really strong experience, if done right. Foursquare kind of does this already in a crude way with their Super User features. But I think it could be taken to a whole new level.</p>
<p>Think about this: Facebook, where you can only friend people, update your profile, and send messages at first. Soon you unlock the ability to use applications. Then the ability to create pages and groups. The interface slowly unfolds in front of you. What about Reddit, where posting comments is all you can do at first? A hundred upvotes gives you the ability to downvote. Ten comments lets you post stories. (Hacker News sort of does this already, with a minimum karma before downvoting is enabled.)</p>
<p>If you could pull it off, I think it’d make for a really compelling user experience. It does bring one extra design skill that many people may not have, though: balance. Game designers are used to this already, but your potential “Power Users” might not like having to wait to get more advanced features. Then again, this might also solve some issues, like spam. If you had to have 100 positively moderated comments before posting a story on Digg, it’d be much harder to just sign up for spam accounts to submit bogus stories.</p>
<p>This idea can be taken in a lot of different directions. I’m sure I’m only barely scratching the surface with this idea, but I think it’ll go really far. What do you think? Any interesting mechanics I’ve missed? Any really interesting thoughts for how services can incorporate game mechanics? I’ve decided to re-open comments, but if nobody uses them, I’ll just shut them off again. Let me know what you think.</p>
<p>This post has been featured on the Startup Pittsburgh blog, <a href="http://startuppittsburgh.com/2010/04/create-a-more-compelling-experience-for-your-users-through-game-mechanics/">here</a>.</p>
</content:encoded></item><item><title>Writing a su feature with Authlogic</title><link>https://steveklabnik.com/writing/writing-a-su-feature-with-authlogic/</link><guid isPermaLink="true">https://steveklabnik.com/writing/writing-a-su-feature-with-authlogic/</guid><pubDate>Fri, 05 Mar 2010 00:00:00 GMT</pubDate><content:encoded><p>Sometimes, when responding to a support request, it’s nice to see what your users see. At the same time, you don’t want to ask your users for their passwords, out of respect for their privacy. So what do you do?</p>
<p>Well, *NIX systems have a program called su. Here’s what man su has to say:</p>
<pre><code>NAME
su - run a shell with substitute user and group IDs
SYNOPSIS
su [OPTION]... [-] [USER [ARG]...]
DESCRIPTION
Change the effective user id and group id to that of USER.
</code></pre>
<p>su can be thought of as “substitute user” or “switch user.” It’s a command system administrators use to assume the identity of one of their users, or a way for someone with the root password on the system to switch to the root account itself. So how can we incorporate this into a web application?</p>
<p>Well, we want to first log ourselves out, and then log in as the user we’re su-ing to. That’s it. The tricky part, however, comes in when we’re logging in: as we said before, we don’t want to ask for their password. Luckily, Authlogic provides a way to create our UserSession object directly from a User object by just passing it to create.</p>
<p>This lets us write a controller method to do this pretty easily:</p>
<pre><code>def su
@user = User.find params[:id]
current_user_session.destroy
UserSession.create!(@user)
flash[:notice] = "You've been su-d to that user."
redirect_to dashboard_path
end
</code></pre>
<p>Add in a route:</p>
<pre><code>map.admin_su "/admin/su/:id", :controller =&gt; "admin", :action =&gt; "su"
</code></pre>
<p>And to a view somewhere in your administrative tools:</p>
<pre><code>&lt;%= link_to "log in as this user", admin_su_path(@user) %&gt;
</code></pre>
<p>And we’re good to go!</p>
<p>One last thing about this, though: You don’t want to let anyone who’s not an administrator do this, for obvious reasons. My administrative controllers always include a block like this:</p>
<pre><code>access_control do
allow :admin
end
</code></pre>
<p>acl9 makes this really easy, but it’s really important.</p>
<p>So there you have it. Easy as pie.</p>
<p>EDIT: This post made the Rails subreddit, and <a href="http://www.reddit.com/r/rails/comments/cb0da/writing_a_su_feature_with_authlogic/c0rf26w">brettbender posted his code</a> to get you back to admin.</p>
</content:encoded></item><item><title>Reddit drama: an interesting look at community values</title><link>https://steveklabnik.com/writing/reddit-drama-an-interesting-look-at-community-values/</link><guid isPermaLink="true">https://steveklabnik.com/writing/reddit-drama-an-interesting-look-at-community-values/</guid><pubDate>Mon, 01 Mar 2010 00:00:00 GMT</pubDate><content:encoded><p>I bet social psychologists are having a field day with the Internet. I know nothing about the field, so I don’t know if there are thousands of papers about it already written or not, but there should be. I can’t image that we’ve ever had a period in history when more communities are being formed; they’re also being formed out in the open.</p>
<p>I’ve been involved in many different online communities over the years. I’ve already written before about the seeming decline of Reddit’s content… but this past week, something really interesting has happened. Reddit has had two major events occur that have caused it to be increasingly self-reflexive on the values and norms that the community desires for itself. This is particularly interesting because normally, cultural norms have to be observed, not codified. But here, we’re seeing completely open discussion about “The community says this is not okay.” It’s really interesting stuff.</p>
<h2>Speed Issues</h2>
<p>I won’t comment on the technical details involved, but Reddit has gotten significantly slower over the past few months. This is normal, as the community is growing. But it’s caused quite a bit of a stir lately. Many users are seeing increased loading times, error messages, missing posts, and various other problems.</p>
<p>What’s a user to do in this situation? Post about it. See the <a href="http://www.reddit.com/search?q=fix+search&amp;sort=hot&amp;t=month">posts about the search feature</a>, or posts about the slowdown.</p>
<p>For example, as I look for the links to these posts, I get this:</p>
<blockquote>
<p>Our search machines are under too much load to handle your request right now. :( Sorry for the inconvenience.</p>
</blockquote>
<p>Amusing. But unfortunate. The interesting part, though, is the backlash against the complaints. See this thread, “<a href="http://www.reddit.com/r/reddit.com/comments/b7jnr/reddit_fix_your_fucking_users_they_are_unbearably/">Reddit, fix your fucking users. They are unbearably bitchy.</a>” (The cursing is emulating the posts made complaining about Reddit’s lack of speed.)</p>
<p>There’s a huge discussion about what it means to be part of a free online community. There’s a contingent of people who say that people aren’t entitled to complain, because Reddit is free. Another points out that complaining is better than silently leaving the site, and that while people don’t pay for Reddit accounts, the eyeballs Reddit users provide enable the site to make money.</p>
<p>Some choice comments:</p>
<blockquote>
<p>Hey, my incredibly expensive Reddit subscription fees entitle me to 100% perfect service! LEAVE ME ALONE!</p>
</blockquote>
<blockquote>
<p>Man, I’ve been loving this particular strawman for over a decade now. Thanks for breaking out the classics.</p>
</blockquote>
<blockquote>
<p>The reality is that Reddit is a business. The owners get paid because the users keep coming back to the site. Having users bitch and moan is not a problem for the admins - this isn’t their personal blog, they’re not mopy teenagers who are gonna cut themselves when they see how mean people are being to them on the interwebs.</p>
</blockquote>
<blockquote>
<p>The problem for the admins is when users stop visiting Reddit altogether because of constant technical problems, or fishy administrative decisions about deleting threads, or too many spammers, or etc. They would much rather have us bitch about these things on the front page, create giant threads about them that keep thousands of posters and tens of thousands of readers engaged and on the site, and provide catharsis and a public forum to address the issues.</p>
</blockquote>
<blockquote>
<p>tl;dr: Reddit is a business, they’d rather have us complain than have us leave.</p>
</blockquote>
<blockquote>
<p>Meta-bitching about bitching time</p>
</blockquote>
<blockquote>
<p>The OP posted a well thought out comment simply suggesting that Reddit’s users should be polite and respectful when addressing each other and the people that make using this site possible, and 90% of the responses here just prove his point. It seems like people are more concerned with gaining comment karma by posting bad one-liners than actually participating in intelligent conversation.</p>
</blockquote>
<blockquote>
<p>Is anyone else absolutely done with the reddit user base? I mean… I was told that reddit was this great intellectual news aggregator that through the use of ‘karma’ was able to bad submissions off the front page.</p>
</blockquote>
<blockquote>
<p>Over the past few months I’ve been here all that has been up on the front page has been Glen Beck, Conan O’Brian, Tiger Woods, weekly “Reddit is slow” posts and now this Saydrah BS.</p>
</blockquote>
<blockquote>
<p>I’ll say what I said when Conan O’Brian was on the front page for a week longer then Haiti ever was: I DON’T CARE. The Reddit admins don’t think Saydrah has abused her power as a moderator. At the very least (I personally don’t think she is a spammer) go flame some of the other 1000000000000000 spammers on reddit FFS. This is boring uneducated dribble.</p>
</blockquote>
<blockquote>
<p>KEEP THIS TRASH OFF THE FRONT PAGE.</p>
</blockquote>
<p>And more. Interesting stuff, overall. There’s still quite a few trash comments, though.</p>
<h2>The Saydrah Situation</h2>
<p>The other event involves a user named Saydrah. She’s been accused of promoting links on Reddit for money, and abusing her moderator powers in conjunction with those offenses. This has spawned a huge amount of discussion on if these actions were inappropriate or not. Here’s <a href="http://www.reddit.com/r/reddit.com/comments/b7e25/today_i_learned_that_one_of_reddits_most_active/">the first big thread</a>. Then <a href="http://www.reddit.com/r/IAmA/comments/b7hpb/re_the_alleged_conflict_of_interest_on_reddit/">an AMA with moderators about their opinions</a>. Lastly, <a href="http://www.reddit.com/r/IAmA/comments/b7tew/fine_here_saydrah_ama_it_couldnt_get_much_worse/">Saydrah does an AMA</a> herself and explains her side of the story.</p>
<p>I won’t show you a bunch of comments, only one. And it echoes my opinion on the matter:</p>
<blockquote>
<p>Two things: Frankly, I don’t care if people are paid to submit links…if they’re interesting, I upvote. If not, I ignore them.</p>
</blockquote>
<blockquote>
<p>Second, I have lurked around AR and RA a fair amount, and consistently find myself thinking, “That’s a helpful and well-reasoned response! Oh, it’s Saydrah again!” Whatever else people may say, I feel that you at least do try to contribute positively to this site.</p>
</blockquote>
<blockquote>
<p>…I guess I don’t have a question, so I’ll just leave my un-asked-for opinion here.</p>
</blockquote>
<p>There’s a lot more, on both sides of the conflict. Regardless, people are hashing it out.</p>
<h2>Growth Hurts</h2>
<p>There’s a reason they’re called “growing pains.” Change is always a painful time in any organism’s life, and even though there’s a lot of drama, the current Reddit staff can take pride that they’ve created something that’s so important to people that they feel the need to scream about it for hours. It’s unfortunate that their baby is being embroiled in a flamewar, but these things happen.</p>
<p>We’ll see what the community ends up deciding is acceptable. I’ve managed to not get involved in these particular conflicts, but it sure is interesting to watch!</p>
<p>Edit: Jen added some interesting links on my facebook feed: <a href="http://j.mp/baRqdy">http://j.mp/baRqdy</a></p>
</content:encoded></item><item><title>The Hackety Manifesto</title><link>https://steveklabnik.com/writing/the-hackety-mainfesto/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-hackety-mainfesto/</guid><pubDate>Thu, 31 Dec 2009 00:00:00 GMT</pubDate><content:encoded><p>For those of you that don’t know, I’ve taken over a project by someone named _why. He dropped off of the face of the Internet back in March, and the community has picked up his works. Because I’ll be writing about the project in the future, and because I feel these words are still relevant, I’m reprinting the two of his essays that led up to the creation of the project, Hackety Hack, here on my blog. Here’s the second one.</p>
<p>Enjoy.</p>
<h1>The Hackety Manifesto</h1>
<p>Nearly four years ago, I wrote an essay called The Little Coder’s Predicament. It’s not too unusual. Lots of others like it have been written. The point is: programming just isn’t available to people like it was with the Commodore 64. I also outlined my requirements for a new cross-platform programming environment for beginners.</p>
<p>The essay was widely linked on Slashdot, Reddit, Lambda, etc. I got lots of mail from people who both agreed and disagreed. Great. Nice. Good people all of them! And yet, nothing has changed. Not really!</p>
<p>I’ve been told of the Xbox development kit and possible programming of Wii channels. The Playstation actually had a language. But none if it has met my criteria for a proper coding platform.</p>
<h2>An Aside: We Care, But Not Enough</h2>
<p>So, why has nothing been done about this? Maybe our interpreters and IDEs (ewww…) are good enough? Lots of people wrote in to say that HTML and JavaScript are the new BASIC. NO!!! You can’t be serious!!! So people have to write two languages now, which are intertwined in an almost inexplicable and unfathomable way? This doesn’t do it.</p>
<p>Hello world should be one line.</p>
<p>In fact, downloading an MP3 should be one line!!</p>
<p>We just don’t care right now, do we? Programmers have a paid gig. So business is happily slurping them up. Look at our books. Look at the programming sites. Programming is tightly coupled to business. Often the first example is an e-commerce site! Our books are like FIFTY DOLLARS!! For crying out loud.</p>
<p>This diatribe isn’t about business being bad. Of course you need to feed your family and drive an Audi.</p>
<p>This diatribe is about adding some balance to the world of programming. Okay, so, let’s take things into our own hands and bring hacking to the young folks.</p>
<h2>The Bylaws of Hackety</h2>
<p>Here are the rules by which Hackety Hack was established:</p>
<ul>
<li>Beginners should be greeted to Hackety Hack by a cartoon character. (For the sake of argument, let’s call this character: Hacky Mouse.)</li>
<li>Also, helpful sentences. Preferably short and with a period.</li>
<li>Hackety Hack is fundamentally a browser and a programming language. For now, Gecko and Ruby.</li>
<li>I’m only using Ruby because I know it. Hopefully, more languages can be added!</li>
<li>Again, this isn’t about Ruby, it’s about simply offering a place for plainspeople to tinker with code.</li>
<li>IDEs are a disaster. Newbs should see only one non-scary window free of tree controls and pinned windows and toolbars.</li>
<li>As such, we want to stay away from project files and makefiles, the trappings of an IDE.</li>
<li>Hackety Hack also adds simple libraries for common things.</li>
<li>Common things are one-liners.</li>
<li>Keep args and options to a minimum.</li>
<li>In Ruby, blocks should be used to open up a method to more advanced possibilities.</li>
<li>Help files are clean, short, simple. Lots of short examples. No frames.</li>
<li>While all bug tickets are helpful and great, I just value tickets from beginners to a greater degree.</li>
<li>Hackety Hack is free and will remain free henceforth.</li>
</ul>
<p>Beyond that… anything which makes life easy and fun for Hackety Hackers is definitely encouraged, even demanded.</p>
</content:encoded></item><item><title>The Little Coder's Predicament</title><link>https://steveklabnik.com/writing/the-little-coders-predicament/</link><guid isPermaLink="true">https://steveklabnik.com/writing/the-little-coders-predicament/</guid><pubDate>Mon, 28 Dec 2009 00:00:00 GMT</pubDate><content:encoded><p>For those of you that don’t know, I’ve taken over a project by someone named _why. He dropped off of the face of the Internet back in March, and the community has picked up his works. Because I’ll be writing about the project in the future, and because I feel these words are still relevant, I’m reprinting the two of his essays that led up to the creation of the project, Hackety Hack, here on my blog.</p>
<p>Enjoy.</p>
<h1>The Little Coder’s Predicament</h1>
<p>Okay, then, children of the modern age (where we live in a world so tied together with wires that Pangaea ain’t goin’ nowhere!), you tell me if this is a predicament or not.</p>
<p>In the 1980s, you could look up from your <a href="http://www.lemon64.com/">Commodore 64</a>, hours after purchasing it, with a glossy feeling of empowerment, achieved by the pattern of notes spewing from the speaker grille in an endless loop. You were part of the movement to help machines sing! You were a programmer! The <a href="http://kl.net/atari/">Atari 800</a> people had BASIC. They know what I’m talking about. And the <a href="http://web.archive.org/web/20070612155724/www.guidry.org/ti994a/systempics/4asystem-2.jpg">TI-994A</a> guys don’t need to say a word, because the TI could <a href="http://www.obsoletecomputermuseum.org/ti99/ti99_syn.jpg">say it</a> for them!</p>
<p>The old machines don’t compare to the desktops of today, or to the consoles of today. But, sadly, current versions of Windows have no immediately accessible programming languages. And what’s a kid going to do with Visual Basic? Build a modal dialog? Forget coding for XBox. Requires registration in the XBox Developer Program. Otherwise, you gotta crack the sucker open. GameCube? GameBoy? Playstation 2?</p>
<h2>Coding Just Isn’t Accessible</h2>
<p>Yes, there are burgeoning <a href="http://openxdk.sourceforge.net/">free</a> <a href="http://gbdk.sourceforge.net/">SDKs</a> for many of these platforms. But they are obscure and most children have no means of actually deploying or executing the code on their own hardware! This is obvious to us all and likely doesn’t seem such a big deal. But ask yourself what might have happened had you not had access to a programming language on an Atari 800 or a Commodore. You tell me if this is a predicament.</p>
<p>It turns out, most of the kids in my neighborhood are exposed to coding through the <a href="http://www.ticalc.org/">TI calculator</a>. A handful of <a href="http://www.ticalc.org/programming/columns/">languages</a> are available on the TI and its processor is interesting enough to evoke some curiousity. But this hasn’t spread to its PDA big brothers, where young people could have more exposure to programming. And undoubtedly the utility of a language on the Palm, Pocket PC and others would be useful to many.</p>
<p>So what’s the problem here? We have no shortage of new languages, but they become increasingly distanced from the populace. Are the companies behind these platforms weary of placing the power of a programming language in the hands of users? Is there not a demand any longer? It’s got to be some kind of greed, power, money thing, right?</p>
<p>Perhaps this is just another reason to push Linux and BSD on consumer systems. Still, are scripting languages easily accessible to beginners on those systems? OSX has made several scripting languages available (including Ruby and Python), but most users are unaware of their presence.</p>
<p>I should mention that Windows is equipped with its own scripting host for developing in JScript and VBScript. But the use of the scripting host is (I believe) under-documented and limited for beginners. Try doing something useful in a script without using Server.CreateObject. Let’s not let kids touch the COM objects, please!</p>
<h2>The Christmas List</h2>
<p>I’m thinking a toy language for consoles and desktops alike could be monumental. I’m ot saying it needs to be cross-platform. A language for GameCube that took advantage of platform-specific features could be more appealing to GameCube users than a language that used a reduced featureset, but could execute on a handheld. Really, we live in a world where both choices should be available.</p>
<p>As for essential features:</p>
<h3>1. Transportable code.</h3>
<p>On my TI-994A, I could make a little, animated Optimus Prime from pixels. Insert cassette. Record. Pass around to friends. Receive high fives from friends. Put on wraparound shades. Thank you, TI! Thank you, Optimus Prime!</p>
<p>A little language for the consoles could be wildly popular if combined with the good ature of sharing code. This could be done by trading memory cards, but would be more effective if code could be easily obtained and posted on the Web. Learning would accelerate and collaborative development could take place.</p>
<p>A suitable language should give coders access to I/O devices, to allow experimentation with network devices and the ability to enhance one’s connectivity with others. For the consoles, games could provide hooks for user mods. This has long proven a successful staple of the desktop gaming world.</p>
<h3>2. Simplicity.</h3>
<p>You’ve got to be able to write a single line of code and see a result. We need some instant results to give absolute beginners confidence. Simple methods for sending an e-mail, reading a web page, playing music. Demonstrable in a one- liner.</p>
<p>Admittedly, as our systems have grown complex, it is difficult to balance simplicity and capability. Most users will be unimpressed by code that emits beeps and bloops from a PlayStation 2. If Ruby were available on the PS2, then I would hope that I could hear rich symphonic sounds from a wee bit of code.</p>
<p><code>Orchestra.play( "A:2", "C:4", "E:1", "G:1" )</code></p>
<p>Access to the graphic engine might require more complex code. But simple drawing methods could be provided for beginners. Or images could be stored alongside code and accessed programmatically.</p>
<p><code>ImageLibrary.load( "GolfingOldMan" ).drawAt( 12, 10 )</code></p>
<p>The trick would be to uncover what small applications might entice novices and still provide the ability to write large applications that would drive developers to master the language and not limit their growth.</p>
<h3>3. Sensible environment.</h3>
<p>Considering that many won’t want to purchase a keyboard for their gaming unit, let’s make sure that a reasonable environment is provided for entry of text. Controllers could be worked like the <a href="http://www.handykey.com/">Twiddler</a>. Or code could be transferred via IR, TCP/IP. (Dare I say cassette? :D)</p>
<h3>4. Give it away!</h3>
<p>It used to be that programming was practically an inalienable right for users. Include a language with the system, situated in a friendly spot. Each of the game consoles I’ve mentioned has launchers. (With the exception of Game Boy and its successors.) Provide a development prompt from the launcher. From desktop software, provide shortcuts for both the command prompt and a development prompt.</p>
<p>Remember, we’re looking for a language that requires no system hacks. No obscure links. No warranty violation. We’ve become so used to these techniques that it seems to be an essential part of getting our way.</p>
<p>And in many ways it is essential. Tinkering with hardware is learning. Lobotomizing and renovating is meaningful, magical. On behalf of those who prefer to code, I make these wishes. Not to take away jobs from the Phillips screwdriver.</p>
<h2>The Ultimatum</h2>
<p>My challenge is to Sony, Nintendo, Microsoft, Apple, and to those who manufacture and develop our interactive technology. Let us interact with these machines more deeply. Provide us a channel for having a dialogue with the entertainment boxes we nurture and care for. I swear to you, the relationship between the public and your product will assuredly blossom. That box will become more of a chest for our personal works.</p>
<p>In addition, if your developers start putting out crap, then you have a whole world of people to pick up the slack.</p>
<p>My challenge is for you to bundle a useful programming language with your product. Ruby, Squeak, REBOL, Python. Take your pick. It will be inexpensive to add any of these languages to your systems. And people will seriously pray to you. You know how geeks get when they pledge allegiance to something. But, yes, Ruby is preferable.</p>
</content:encoded></item><item><title>About Crows</title><link>https://steveklabnik.com/writing/about-crows/</link><guid isPermaLink="true">https://steveklabnik.com/writing/about-crows/</guid><pubDate>Wed, 02 Dec 2009 00:00:00 GMT</pubDate><content:encoded><p>A short time ago, the G20 came right here to Pittsburgh. With an event of such worldwide importance happening in our backyards, several people decided to document it via social media. The Mattress Factory did a site called “MyG20” and it did pretty well. Some smaller groups did their own thing. But IndyMedia did one better: <a href="http://indypgh.org/g20/">http://indypgh.org/g20/</a></p>
<p>The <a href="http://thecaucus.blogs.nytimes.com/2009/09/24/getting-out-the-anti-globalization-message/">New York Times</a> called it “One of the best all-purpose sites with updates from the streets.” It got quite a lot of attention and hits from all over. And it was put together by a good friend of mine, <a href="http://www.twitter.com/edfilo">@edfilo</a> I helped a teeny tiny bit, but he (with the help of LibSyn) really did an awesome job.</p>
<p>So why should this matter to you? The G20 is old news by now.</p>
<p>He’s releasing the code for the site, making it available to anyone who wants to cover events. It’s called “Crows,” and it lives at <a href="http://crowsne.st/">http://crowsne.st/</a> . With Crows, you can quickly and easily build a site, pulling in content from Flickr, Twitter, and elsewhere, and posting it all on a geotagged Google Map.</p>
<p>It’s still in its rough infancy, but after the awesome role that it played during the G20, several other organizations had contacted IndyMedia about getting access to the code, and so it’s already gaining traction in the space. It’s undergoing rapid development, with new features being added quite fast. Ed really wants to make this useful for people, and I’m going to be contributing to the project in what little spare time I can as well. If you’re a coder too, you can <a href="http://github.com/edfilo/crows">fork crows on GitHub</a> to contribute back to the project.</p>
<p>Check it out, and you can follow <a href="http://twitter.com/crowsource">@crowsource</a> on Twitter for updates, too.</p>
</content:encoded></item></channel></rss>
Raw headers
{
"access-control-allow-origin": "*",
"age": "8378",
"cache-control": "public, max-age=0, must-revalidate",
"cf-cache-status": "DYNAMIC",
"cf-ray": "929b692ac6a9c94c-IAD",
"connection": "keep-alive",
"content-disposition": "inline; filename=\"feed.xml\"",
"content-type": "application/xml",
"date": "Tue, 01 Apr 2025 22:09:26 GMT",
"etag": "W/\"b922f3bdd181b344e55e459232dc0b3c\"",
"last-modified": "Tue, 01 Apr 2025 19:49:47 GMT",
"server": "cloudflare",
"strict-transport-security": "max-age=63072000",
"transfer-encoding": "chunked",
"vary": "accept-encoding",
"x-vercel-cache": "HIT",
"x-vercel-id": "iad1::l8697-1743545366294-af92453e1cbe"
}
Parsed with @rowanmanning/feed-parser
{
"meta": {
"type": "rss",
"version": "2.0"
},
"language": null,
"title": "Steve Klabnik",
"description": "Steve Klabniks blog",
"copyright": null,
"url": "https://steveklabnik.com/",
"self": null,
"published": null,
"updated": null,
"generator": null,
"image": null,
"authors": [],
"categories": [],
"items": [
{
"id": "https://steveklabnik.com/writing/does-unsafe-undermine-rusts-guarantees/",
"title": "Does unsafe undermine Rust's guarantees?",
"description": null,
"url": "https://steveklabnik.com/writing/does-unsafe-undermine-rusts-guarantees/",
"published": "2025-03-17T00:00:00.000Z",
"updated": "2025-03-17T00:00:00.000Z",
"content": "<p>import BlueskyPost from \"../../components/BlueskyPost.astro\";</p>\n<p>When people first hear about <code>unsafe</code> in Rust, they often have questions. A very\nnormal thing to ask is, \"wait a minute, doesn't this defeat the purpose?\" And while\nit's a perfectly reasonable question, the answer is both straightforward and has a\nlot of nuance. So let's talk about it.</p>\n<p>(The straightforward answer is \"no\", by the way.)</p>\n<h2>Memory safe languages and their implementations</h2>\n<p>The first way to think about this sort of thing is to remember that programming\nlanguages and their implementations are two different things. This is sometimes\ndifficult to remember, since many programming languages only have one\nimplementation, but this general principle applies even in those cases. That is,\nthere are the desired semantics of the language you are implementing, and then\nthe codebase that makes programs in that language do those things.</p>\n<p>Let's examine a \"real\" programming language to talk about this: <a href=\"https://en.wikipedia.org/wiki/Brainfuck\">Brainfuck</a>.\nBrainfuck programs are made up of eight different characters, each one performing\none operation. Brainfuck programs can only do those eight things, and nothing\nmore.</p>\n<p>However, this simplicity leads to two interesting observations about the\ndifferences between languages in the abstract and their implementations.</p>\n<h3>Unsafety in implementation</h3>\n<p>The first is that properties of the language itself are distinct from properties\nof an implementation of the language. The semantics of the Brainfuck language\ndon't provide an opportunity to call some sort of re-usable function\nabstraction, for example. Yet, when we write a Brainfuck interpreter or\ncompiler, we can use a programming language that supports functions. That\ndoesn't change that Brainfuck programs themselves can't define or call\nfunctions.</p>\n<p>And in fact, we often want to implement a language in another language that has\nmore abilities than the language we're implementing. For example, while it's\npossible to implement Brainfuck in Brainfuck, it's much easier to do so in a\nlanguage that lets you define and call functions. Another classic example is\ninteracting with hardware. Brainfuck has the <code>.</code> instruction that allows you to\nproduce output. But in order to actually produce output to say, a terminal, you\ninteract with your platform's API to do so. But inside of that API, eventually\nyou'll hit a layer where there are no abstractions: you need to talk to hardware\ndirectly. Doing so is not statically able to be guaranteed to be memory safe,\nbecause the hardware/software interface often boils down to \"put some\ninformation at this arbitrary memory location and the hardware will take it from\nthere.\" Our Brainfuck program is operating in an environment just like any\nother; at the end of the day it has to coordinate with the underlying operating\nsystem or hardware at some point. But just because our implementation must do\nso, doesn't mean that our Brainfuck programs do so directly. The semantics of\nprograms written in Brainfuck aren't changed by the fact that the implementation\ncan (and must) do things that are outside of those semantics.</p>\n<p>Let's take Ruby as a more realistic example than Brainfuck. Ruby does not let us modify\narbitrary memory addresses from within Ruby itself. This means that pure Ruby\nprograms should not be able to produce a segmentation fault. But in the real\nworld, Ruby programs can segfault. This is possible if there's a bug in the Ruby\ninterpreter's code. Sure, it's our Ruby code that's making the call to a\nfunction that results in a segfault, but the real fault lies with the code\noutside of Ruby's purview.</p>\n<p>That this can happen doesn't mean that Ruby's guarantees around memory\nmanipulation is somehow useless, or suffers segmentation faults at the same rate\nas programs in a language that allow you to manipulate arbitrary memory as a\nmatter of course. But it does also mean that we don't need to look at our entire\nprogram to figure out where the problem comes from: it instead comes from our\nnon-Ruby code. Ruby's guarantees have helped us eliminate a lot of suspects when\nfiguring out whodunit.</p>\n<h3>Extending languages and unsafety</h3>\n<p>The second property is that certain implementations may extend the language in\narbitrary ways. For example, I could write a Brainfuck interpreter and say that\nI support a ninth instruction, <code>@</code>, that terminates the program when invoked.\nThis would be a non-portable extension, and Brainfuck programs written in my\nvariant wouldn't work in other interpreters, but sometimes, this technique is\nuseful.</p>\n<p>Many languages find that these sorts of extensions are useful, and so offer a\nfeature called a \"foreign function interface\" that allows for your program to\ninvoke code in a different language. This provides the ability to do things that\nare outside of the domain of the language itself, which can be very useful.\nBrainfuck does not offer an FFI, but if it did, you could imagine that <code>@</code> could\nbe implemented in terms of it, making programs that use <code>@</code> portable again, as\nlong as the extended functionality was included somehow, often as a library of\nsome kind.</p>\n<p>Just like our implementation has the ability to do things outside of our\nlanguages' semantics, FFI and similar extension mechanisms also give us the\nability to do arbitrary things. I can write an extension for Ruby that writes to\narbitrary memory. And I can cause a segfault. But we're in the same place that\nwe were with our implementation issues; we know that if we get a segfault, the\nblame lies not with our Ruby code, but instead, with either the implementation\nor our FFI.</p>\n<h3>So how is this memory safe?</h3>\n<p>It may seem contradictory that we can call a language \"memory safe\" if\nreal-world programs have the ability to cause memory problems. But the thing is,\nit isn't <em>really</em> programs in that language that caused the issue: it was FFI,\nwhich is in a different language entirely, or it was implementation issues, and\nimplementations must do memory unsafe things, thanks to a need to interact with\nthe OS or hardware. And so the definition of \"memory safe language\" is commonly\nunderstood to refer to languages and their implementations in the absence of\neither implementation bugs or FFI bugs. In practice, these bugs occur at such a\nlow rate compared to languages that are clearly not memory safe that this\npractical definition serves a good purpose, even if you may feel a bit\nuncomfortable with these \"exceptions.\"</p>\n<p>But there's actually a deeper reason why these exceptions are acceptable, and\nthat's due to how we understand properties of programs and programming languages\nin the first place. That is, this isn't just a practical \"well these exceptions\nseem fine\" sort of thing, they're actually okay on a deeper level.</p>\n<h2>Programs, properties, and proofs</h2>\n<p>So how do we know anything about how programs and programming languages work at\nall?</p>\n<blockquote>\n<p>Computer science is no more about computers than astronomy is about\ntelescopes.</p>\n<ul>\n<li>Edsger Wybe Dijkstra</li>\n</ul>\n</blockquote>\n<p>A fun thing about computer science is that it's closely related to other\ndisciplines, including math! And so there's a long history of using math\ntechniques to understand computers and their programs.</p>\n<p>One technique of building knowledge is the idea of proofs. If you get a\ndegree in computer science, you'll engage with proofs quite a lot. I even\ntook a class on logic in the philosophy department as part of my degree.</p>\n<p>I don't intend to give you a full introduction to the idea of proofs in this\nblog post, but there's some high-level concepts that are useful to make sure\nwe're in the same page about before we go forward.</p>\n<h3>Aristotelian syllogisms</h3>\n<p>Here is a very classic example of a form of reasoning called a \"syllogism\",\ngiven by Aristotle in 350 BCE:</p>\n<ol>\n<li>All men are mortal.</li>\n<li>Socrates is a man.</li>\n<li>Therefore, Socrates is mortal.</li>\n</ol>\n<p>These first two lines are called \"propositions,\" and the third is a conclusion.\nWe can base our conclusion on a logical relationship between the information given\nto us by the propositions.</p>\n<p>But how do we know propositions are true? Are all men mortal? What's important\nhere is that we, for the purposes of our proof, assume that propositions are\ntrue. And we do this because, on some level, we cannot know everything. And so\nto begin, we have to start somewhere, and make some assumptions about what we\nknow. It's true that later, we may discover a fact that disproves our\nproposition, and now our proof no longer works. But that's just the way that\nthe world works. It doesn't prevent these sorts of proofs from being useful to\nhelp us gain knowledge about the world and how it works, as best as we can tell\nat the current time.</p>\n<p>So on some level, this is also why Ruby is a memory safe language even though a\nC extension can segfault: there's always some kind of things that we have to\nassume are true. Memory safe languages are ones where the amount of code we have\nto assume is memory safe, rather than is guaranteed to be by the semantics, is\nsmall, and preferably indicated in the code itself. Put another way, the amount\nof code we need to trust is memory safe is large in a memory unsafe language,\nand small in a memory safe language, rather than zero.</p>\n<h3>Logic and programs</h3>\n<p>As time went on, so did our understanding of logic. And, of course, we even have\ncompeting logics! And then this gets fun, because terms can mean something\nslightly different. For example, in more recent logics, we'd call something like\n\"All men are mortal\" to be an <em>axiom</em>, rather than a proposition. Same idea:\nit's something that we accept without proof.</p>\n<p>As computers appeared, people sought to apply the rules for mathematical logic\nonto them. We even call circuits that carry out classic logical operations\n\"logic gates.\" Number theory and logic were foundational to making computers\nwork. And so, once high level languages appeared on the scene, there was\ninterest in applying these mathematical tools to understanding programs as well.\nThis discipline is called \"formal verification,\" and the general idea is to\ndescribe various properties we wish a system to have, and then use formal\nmethods from mathematics to demonstrate that this is true.</p>\n<p>This area of study is very deep, and I don't plan to cover the vast majority of\nit here. However, I do want to pursue one particular thread in this area.</p>\n<h3>Hoare Logic</h3>\n<p><a href=\"https://dl.acm.org/doi/10.1145/363235.363259\">\"Hoare Logic\"</a> is, well:</p>\n<blockquote>\n<p>In this paper an attempt is made to explore the logical foundations of\ncomputer programming by use of techniques which were first applied in the\nstudy of geometry and have later been extended to other branches of\nmathematics. This involves the elucidation of sets of axioms and rules of\ninference which can be used in proofs of the properties of computer programs.\nExamples are given of such axioms and rules, and a formal proof of a simple\ntheorem is displayed. Finally, it is argued that important advantage, both\ntheoretical and practical, may follow from a pursuance of these topics.</p>\n</blockquote>\n<p>Incidentally, C. A. R. Hoare, the author of this paper, and Graydon Hoare, the\ncreator of Rust, are unrelated.</p>\n<p>How does Hoare logic work? Yet again, not going to cover all of it, but the\ngeneral idea is this: In order to figure out if a program does what it is\nsupposed to do, we need to be able to reason about the state of the program\nafter execution happens. And so we need to be able to describe the state before,\nand its relationship to the state after. And so you get this notation:</p>\n<pre><code>P { Q } R\n</code></pre>\n<p>P is a precondition, Q is a program, and R is the result of the program\nexecuting with those preconditions.</p>\n<p>But most programs have more than one statement. So how could we model this?\nHoare gives us the Rule of Composition:</p>\n<p>If <code>P { Q1 } R1</code> and <code>R1 { Q2 } R</code> then <code>P { Q1; Q2 } R</code>.</p>\n<p>This allows us to build up a program by proving each statement in turn.</p>\n<p>Hoare logic is very neat, and I've only scratched the surface here. People did a\nlot of work to extend Hoare logic to include more and more aspects of programs.\nBut then, something else happened.</p>\n<h3>Separation Logic</h3>\n<p>In 2002, <a href=\"https://www.cs.cmu.edu/~jcr/seplogic.pdf\">Separation Logic: A Logic for Shared Mutable Data Structures</a>\nwas published.</p>\n<blockquote>\n<p>In joint work with Peter O'Hearn and others, based on early ideas of Burstall,\nwe have developed an extension of Hoare logic that permits reasoning about\nlow-level imperative programs that use shared mutable data structure.</p>\n</blockquote>\n<p>Hmm, shared mutable data structures? Where have I heard that before...</p>\n<p>Let's see what they have to say:</p>\n<blockquote>\n<p>The problem faced by these approaches is that the correctness of a program\nthat mutates data structures usually depends upon complex restrictions on the\nsharing in these structures.</p>\n</blockquote>\n<p>For sure. Well, what are we to do about this?</p>\n<blockquote>\n<p>The key to avoiding this difficulty is to introduce a novel\nlogical operation <code>P * Q</code>, called separating conjunction (or\nsometimes independent or spatial conjunction), that asserts\nthat P and Q hold for disjoint portions of the addressable\nstorage.</p>\n</blockquote>\n<p>What disjoint portions of addressable storage might we care about?</p>\n<blockquote>\n<p>Our intent is to capture the low-level character of machine language. One can\nthink of the store as describing the contents of registers, and the heap as\ndescribing the contents of an addressable memory.</p>\n</blockquote>\n<p>Pretty useful!</p>\n<p>Before we talk a bit about how separation logic works, consider this paragraph\non why it's named as such:</p>\n<blockquote>\n<p>Since these logics are based on the idea that the structure of an assertion\ncan describe the separation of storage into disjoint components,we have come\nto use the term separation logics, both for the extension of predicate\ncalculus with the separation operators and the resulting extension of Hoare\nlogic. A more precise name might be storage separation logics, since it is\nbecoming apparent that the underlying idea can be generalized to describe the\nseparation of other kinds of resources.</p>\n</blockquote>\n<p>The plot thickens.</p>\n<p>Anyway, in Separation Logic, we use slightly different notation than Hoare Logic:</p>\n<pre><code>{ P } C { Q }\n</code></pre>\n<p>This says that we start with the precondition <code>P</code>, and if the program <code>C</code>\nexecutes, it will not have undefined behavior, and if it terminates, <code>Q</code> will\nhold.</p>\n<p>Furthermore, there is the \"frame rule\", which I am going to butcher the notation\nfor because I haven't bothered to install something to render math correctly\njust for this post:</p>\n<p>If <code>{ p } c { q }</code></p>\n<p>then <code>{ p * r } c { q * r }</code></p>\n<p>where no free variable in <code>r</code> is modified by <code>c</code>.</p>\n<p>Why is it interesting to add something to both sides of an equation, in a sense?\nWell what this gives us the ability to do is add any predicates about parts of\nthe program that <code>c</code> doesn't modify or mutate. You might think of <code>&mut T</code> or\neven just ownership in general: we can reason about just these individual parts\nof a program, separately from the rest of it. In other words, we have some\nfoundational ideas for ownership and even bits of borrowing, and while this original\npaper doesn't involve concurrency, eventually Concurrent Separation Logic would become\na thing as well.</p>\n<p>I think the paper explains why the frame rule matters better than I can:</p>\n<blockquote>\n<p>Every valid specification <code>{p} c {q}</code> is \"tight\" in the sense that every cell\nin its footprint must either be allocated by <code>c</code> or asserted to be active by\n<code>p</code>; \"locality\" is the opposite property that everything asserted to be active\nbelongs to the footprint. The role of the frame rule is to infer from a local\nspecification of a command the more global specification appropriate to the\nlarger footprint of an enclosing command.</p>\n</blockquote>\n<p>What this gives us is something called \"local reasoning,\" and local reasoning\nis <em>awesome</em>. Before I talk about that, I want to leave you with one other\nvery interesting paragraph:</p>\n<blockquote>\n<p>Since our logic permits programs to use unrestricted address arithmetic,there\nis little hope of constructing any general-purpose garbage collector. On the\nother hand, the situation for the older logic, in which addresses are disjoint\nfrom integers, is more hopeful. However, it is clear that this logic permits\none to make assertions, such as “The heap contains two elements” that might be\nfalsified by the execution of a garbage collector, even though, in any\nrealistic sense, such an execution is unobservable.</p>\n</blockquote>\n<p>Anyway. Let's talk about local vs global analysis.</p>\n<h2>Global vs local analysis</h2>\n<p>Proving things about programs isn't easy. But one thing that can make it even\nharder is that, for many properties of many programs you'd want to analyze, you\nneed to do a <em>global analysis</em>. As an example, let's use Ruby. Ruby is an\nincredibly dynamic programming language, which makes it fairly resistant to\nstatic analysis.</p>\n<h3>Global analysis</h3>\n<p>Here is a Ruby program. Do you know if this program executes successfully?</p>\n<pre><code>class Foo\n def self.bar\n \"baz\"\n end\nend\n\np Foo.bar\n</code></pre>\n<p>Yeah, it prints <code>baz</code>. But what about this Ruby program?</p>\n<pre><code>class Foo\n def self.bar\n \"baz\"\n end\nend\n\nrequire \"foo_ext.rb\"\n\np Foo.bar\n</code></pre>\n<p>We can't know. <code>foo_ext.rb</code> may contain no relevant code, but it also might\ninclude something like this:</p>\n<pre><code>Foo.class_eval { undef :bar }\n</code></pre>\n<p>In which case, when we try and call <code>Foo.bar</code>, it no longer exists:</p>\n<pre><code><anonymous>': eval:13:in `block in <main>': undefined method `bar' for class `Foo' (NameError)\neval:13:in `class_eval'\neval:13:in `<main>'\n-e:in `eval'\n</code></pre>\n<p>Ouch. So in this case, we're going to need the whole code of our program in\norder to be able to figure out what's going on here.</p>\n<p>Incidentally, <a href=\"https://sorbet.org/\">Sorbet</a> is a very cool project to add type checking to\nRuby. They do require access to the entire Ruby source code in order to do their\nanalysis. But they also made some decisions to help make it more tractable; if\nyou try Sorbet out on the web, the type checker is <em>fast</em>. What happens when you\ntry the above code with Sorbet?</p>\n<pre><code>editor.rb:5: Unsupported method: undef https://srb.help/3008\n 5 |Foo.class_eval { undef :bar }\n ^^^^^^^^^^\n</code></pre>\n<p>This is a very fair tradeoff! It's very common with various forms of analysis to\nchoose certain restrictions in order to make what they want to do tractable. Ruby\nhas so much evil in it, just not supporting some of the more obscure things is\ncompletely fair, in my opinion.</p>\n<h3>Local analysis</h3>\n<p>The opposite of global analysis is, well, local analysis. Let's consider a Rust\nequivalent of our Ruby:</p>\n<pre><code>struct Foo;\n\nimpl Foo {\n fn bar() -> String {\n String::from(\"baz\")\n }\n}\n\nfn main() {\n dbg!(Foo::bar());\n}\n</code></pre>\n<p>Can we know if this program works? Sure, everything is here. Now, if we tried the\nsame trick as the Ruby code, we know that it would work, because Rust doesn't have\nthe ability to remove the definition of <code>bar</code> like Ruby does. So let's try something\nelse:</p>\n<pre><code>struct Foo;\n\nimpl Foo {\n fn bar() -> String {\n // body hidden\n }\n}\n\nfn main() {\n dbg!(Foo::bar());\n}\n</code></pre>\n<p>Can we know if <code>main</code> is properly typed here? We can, even though we know\nnothing about the body. That's because <code>dbg!</code> takes any value that implements\nthe <code>Debug</code> trait, and we know that <code>Foo::bar()</code> returns <code>String</code>, which\nimplements <code>Debug</code>. Typechecking <code>main</code> is local to main's implementation, we\ndon't need to look into the bodies of any function it calls to determine if it's\nwell-typed. We only need to know their signatures. If we didn't require <code>bar</code> to\nhave a type signature, we'd have to peek into its body to figure out what it\nreturns, and so on for any function that <code>bar</code> calls in its body. Instead of\nthis arbitrarily deep process, we can just look at the signature and be done\nwith it.</p>\n<h2>Composability of proofs</h2>\n<p>So why is local analysis so helpful? The first reason is either speed or\nscalability, depending on how you want to look at it. If you are running global\nanalysis checks, they will get more expensive the larger your codebase, since\nthey will require checking the entire thing to work. Whereas local analysis only\nrequires a local context, it doesn't get any more expensive when you add more\ncode to your project, only when you change that local context. So when you're\ntrying to scale checks up to larger projects, local analysis is crucial.</p>\n<p>But I also like to personally think about it as a sort of abstraction. That is,\nglobal analysis is a leaky abstraction: changes in one part of the codebase can\ncascade into other parts. Remember this line about the frame rule?</p>\n<blockquote>\n<p>The role of the frame rule is to infer from a local specification of a command\nthe more global specification appropriate to the larger footprint of an\nenclosing command.</p>\n</blockquote>\n<p>If we have local reasoning, we can be sure that changes locally don't break out\nof the boundaries of those changes. As long as the types of our function don't\nchange, we can mess with the body as much as we want, and we know that the rest\nof the analysis of the program is still intact. This is really useful, in the\nsame way that abstractions are useful when building programs more generally.</p>\n<h2>Rustbelt</h2>\n<p>Okay, we have taken a real deep dive here. What's this all have to do with\nunsafe Rust?</p>\n<p>Well, seven years ago, <a href=\"https://plv.mpi-sws.org/rustbelt/popl18/paper.pdf\">RustBelt: Securing the Foundations of the Rust\nProgramming Language</a> by Ralf Jung, Jaques-Henri Jourdan, Robbert\nKrebbers, and Derek Dreyer, was published:</p>\n<blockquote>\n<p>In this paper, we give the first formal (and machine-checked) safety proof for\na language representing a realistic subset of Rust.</p>\n</blockquote>\n<p>But it's more exciting than that:</p>\n<blockquote>\n<p>Our proof is extensible in the sense that, for each new Rust library that uses\nunsafe features, we can say what verification condition it must satisfy in\norder for it to be deemed a safe extension to the language.</p>\n</blockquote>\n<p>Very good!</p>\n<p>The paper begins by talking about why verifying Rust is such a challenge:</p>\n<blockquote>\n<p>Consequently, to overcome this restriction, the implementations of Rust's\nstandard libraries make widespread use of unsafe operations, such as \"raw\npointer\" manipulations for which aliasing is not tracked. The developers of\nthese libraries claim that their uses of unsafe code have been properly\n\"encapsulated\", meaning that if programmers make use of the APIs exported by\nthese libraries but otherwise avoid the use of unsafe operations themselves,\nthen their programs should never exhibit any unsafe/undefined behaviors. In\neffect, these libraries extend the expressive power of Rust's type system by\nloosening its ownership discipline on aliased mutable state in a modular,\ncontrolled fashion</p>\n</blockquote>\n<p>This is basically the question asked at the start of this post: can you honestly\nsay that a function is safe if it contains <code>unsafe</code> code as part of its\nimplementation?</p>\n<p>They go on to describe the first challenge for the project: choosing the correct\nlogic to model Rust in (some information cut for clarity):</p>\n<blockquote>\n<p>Iris is a language-generic framework for higher-order concurrent separation\nlogic which in the past year has been equipped with tactical support for\nconducting machine-checked proofs of programs in Coq. By virtue of being a\nseparation logic, Iris comes with built-in support for reasoning modularly\nabout ownership. Moreover, the main selling point of Iris is its support for\nderiving custom program logics for different domains using only a small set of\nprimitive mechanisms. In the case of RustBelt, we used Iris to derive a novel\nlifetime logic, whose primary feature is a notion of borrow propositions that\nmirrors the “borrowing” mechanism for tracking aliasing in Rust.</p>\n</blockquote>\n<p>Separation logic!</p>\n<p>So how does this work? There are three parts to the proof. I'm going to\nsummarize them because I'm already quoting extensively, please go read the paper\nif you want the exact details, it is well written.</p>\n<ol>\n<li>Verify that the typing rules are sound when interpreted semantically.</li>\n<li>Verify that if a program is well typed semantically, it does not exhibit\nunsafe behavior.</li>\n<li>For any library that uses unsafe, check that it satisfies the semantics that\nits interface requires.</li>\n</ol>\n<p>If all three of these things are true, then the program is safe to execute. This\nis also exciting because, thanks to 3, when someone writes new unsafe code, they\ncan get RustBelt to let them know which properties they need to satisfy to make\nsure that their unsafe code doesn't cause problems.</p>\n<h3>Weaknesses</h3>\n<p>Of course, there is more to do following up on RustBelt:</p>\n<blockquote>\n<p>We do not model (1) more relaxed forms of atomic accesses, which Rust uses for\nefficiency in libraries like Arc; (2) Rust's trait objects (comparable to\ninterfaces in Java), which can pose safety issues due to their interactions\nwith lifetimes; or (3) stack unwinding when a panic occurs, which causes\nissues similar to exception safety in C++.</p>\n</blockquote>\n<p>However, the results were good enough to find some soundness bugs in the\nstandard library, and if I remember correctly, widen one of the APIs in a sound\nmanner as well. Another interesting thing about RustBelt is that it was\npublished before Rust gained Non-lexical lifetimes, however, it modeled them,\nsince folks knew they were coming down the pipeline.</p>\n<h2>The theoretical vs the empirical</h2>\n<p>So, we have some amount of actual proof that unsafe code in Rust doesn't\nundermine Rust's guarantees: what it does is allow us to extend Rust's\nsemantics, just like FFI would allow us to extend a program written in\na language that supports it. As long as our unsafe code is semantically\ncorrect, then we're all good.</p>\n<p>... but what if it's not semantically correct?</p>\n<h3>When things go wrong</h3>\n<p>At some point maybe I'll write a post about the specifics here, but this post is\nalready incredibly long. I'll cut to the chase: you can get undefined behavior,\nwhich means anything can happen. You don't have a real Rust program. It's bad.</p>\n<p>But at least it's scoped to some degree. However, many people get this scope a\nbit wrong. They'll say something like:</p>\n<blockquote>\n<p>You only need to verify the unsafe blocks.</p>\n</blockquote>\n<p>This is true, but also a bit misleading. Before we get into the unsafe stuff, I\nwant you to consider an example. Is this Rust code okay?</p>\n<pre><code>struct Foo {\n bar: usize,\n}\n\nimpl Foo {\n fn set_bar(&mut self, bar: usize) {\n self.bar = bar;\n }\n}\n</code></pre>\n<p>No unsafe shenanigans here. This code is perfectly safe, if a bit useless.</p>\n<p>Let's talk about unsafe. The canonical example of unsafe code being affected\noutside of unsafe itself is the implementation of <code>Vec<T></code>. Vecs look something\nlike this (the real code is different for reasons that don't really matter in\nthis context):</p>\n<pre><code>struct Vec<T> {\n ptr: *mut T,\n len: usize,\n cap: usize,\n}\n</code></pre>\n<p>The pointer is to a bunch of <code>T</code>s in a row, the length is the current number of <code>T</code>s\nthat are valid, and the capacity is the total number of <code>T</code>s. The length and the\ncapacity are different so that memory allocation is amortized; the capacity is\nalways greater than or equal to the length.</p>\n<p>That property is very important! If the length is greater than the capacity,\nwhen we try and index into the Vec, we'd be accessing random memory.</p>\n<p>So now, this function, which is the same as <code>Foo::set_bar</code>, is no longer okay:</p>\n<pre><code>impl<T> Vec<T> {\n fn set_len(&mut self, len: usize) {\n self.len = len;\n }\n}\n</code></pre>\n<p>This is because the unsafe code inside of other methods of <code>Vec<T></code> need to be\nable to rely on the fact that <code>len <= capacity</code>. And so you'll find that\n<code>Vec<T>::set_len</code> in Rust is marked as unsafe, even though it doesn't contain\nunsafe code.</p>\n<p>And this is why the module being the privacy boundary matters: the only way to\nset len directly in safe Rust code is code within the same privacy boundary as\nthe <code>Vec<T></code> itself. And so, that's the same module, or its children.</p>\n<p>This is still ultimately better than any line of code in the whole codebase, but\nit's not quite as small as you may think at first.</p>\n<h3>How can we check that things haven't gone wrong?</h3>\n<p>Okay, so while RustBelt could give you some idea if your code is correct, I doubt\nyou're about to jump into Coq and write some proofs. What can you do? Well,\nRust provides a tool, <code>miri</code>, that can interpret your Rust code and let you know\nif you've violated some unsafe rules. But it's not complete, that is, miri can\ntell you if your code is wrong, but it cannot tell you if your code is right. It's\nstill quite useful.</p>\n<p>To get miri, you can install it with Rust nightly:</p>\n<pre><code>$ rustup +nightly component add miri\n</code></pre>\n<p>And then run your test suite under miri with</p>\n<pre><code>$ cargo +nightly miri test\n</code></pre>\n<p>For more details, consult <a href=\"https://github.com/rust-lang/miri\">miri's documentation</a>.</p>\n<h3>Do things go wrong?</h3>\n<p>Do people write good unsafe code, or bad?</p>\n<p>Well, the obvious answer to the above question is \"yes.\" Anyone who says all\nunsafe code in Rust is sound is obviously lying. But the absolute isn't what's\ninteresting here, it's the shade of grey. Is unsafe pervasive? Does Rust have\nmemory related bugs at the same rate as other MSLs, or closer to memory-unsafe\nlanguages?</p>\n<p>It's still sort of early days here, but we do have some preliminary results that\nare helpful. In 2022, <a href=\"https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html\">Google reported</a> that:</p>\n<blockquote>\n<p>To date, there have been zero memory safety vulnerabilities discovered in\nAndroid's Rust code.</p>\n</blockquote>\n<p>I am very interested to see a more up to date report about how that's changed,\nbut as they say:</p>\n<blockquote>\n<p>We don't expect that number to stay zero forever, but given the volume of new\nRust code across two Android releases, and the security-sensitive components\nwhere it's being used, it's a significant result. It demonstrates that Rust is\nfulfilling its intended purpose of preventing Android's most common source of\nvulnerabilities. Historical vulnerability density is greater than 1/kLOC (1\nvulnerability per thousand lines of code) in many of Android's C/C++ components\n(e.g. media, Bluetooth, NFC, etc). Based on this historical vulnerability\ndensity, it's likely that using Rust has already prevented hundreds of\nvulnerabilities from reaching production.</p>\n</blockquote>\n<p>We'll see if these results replicate themselves, in the future, and also outside\nof Google. But it at least looks like in practice, most unsafe code has not led\nto problems. At least so far!</p>\n<hr />\n<p>Here's my post about this post on BlueSky:</p>\n<p><BlueskyPost uri=\"at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3lkqjlfxypk2d\" /></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/choosing-languages/",
"title": "Choosing Languages",
"description": null,
"url": "https://steveklabnik.com/writing/choosing-languages/",
"published": "2025-03-13T00:00:00.000Z",
"updated": "2025-03-13T00:00:00.000Z",
"content": "<p>import BlueskyPost from \"../../components/BlueskyPost.astro\";</p>\n<p>I tried to write this post yesterday, but I didn't like my approach there. Last\nnight, I had a conversation with one of my best friends about it, and he\nencouraged me to write this, but in a different way. I think this way is better,\nso thanks, dude.</p>\n<hr />\n<p>The other day, Microsoft announced <a href=\"https://devblogs.microsoft.com/typescript/typescript-native-port/\">that they are re-writing the TypeScript\ncompiler in Go</a>. A lot of people had a lot of feelings about this. I did\ntoo, but in a different way than many other people.</p>\n<p>The biggest questions people had were \"Why not C#?\" and \"Why not Rust?\". To\nbe clear, I do think that asking why someone chooses a programming language\ncan be valuable; as professionals, we need to make these sorts of decisions\non a regular basis, and seeing how others make those decisions can be useful,\nto get an idea of how other people think about these things.</p>\n<p>But it can also... I dunno. Sometimes, I think people have little imagination\nwhen it comes to choosing a particular technology stack.</p>\n<p><BlueskyPost uri=\"at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3lk6p4clry22p\" /></p>\n<p>It is true that sometimes, there are hard requirements that means that choosing\na particular technology is inappropriate. But I've found that often, these\nsorts of requirements are often more <em>contextually</em> requirements than ones that\nwould be the case in the abstract. For example, <a href=\"https://en.wikipedia.org/wiki/Lisp_machine\">you can write an OS in\nLisp</a> if you want. But in practice, you're probably not working with\nthis hardware, or willing to port the environment to bare metal.</p>\n<p>But it's also the case that you often don't know what others' contextual\nsituation actually is. I would have never guessed the main reason that Microsoft\nchose Go for, which is roughly \"we are porting the existing codebase, rather\nthan re-writing it, and our existing code looks kinda like Go, making it easy to\nport.\" That's a very good reason! I hadn't really thought about the differences\nbetween porting and re-writing in this light before, and it makes perfect sense\nto me.</p>\n<p>At the same time,</p>\n<p><BlueskyPost uri=\"at://did:plc:4eukmtg5kmyjmp6qw3xkpite/app.bsky.feed.post/3lk6t4a4xa22w\" /></p>\n<p>people are going <em>way</em> overboard here.</p>\n<p><BlueskyPost uri=\"at://did:plc:x2p6i7jmdhk4xj2ej3rflp7a/app.bsky.feed.post/3lk6was5n5227\" /></p>\n<p>There's some sort of balance here to be struck. But at the same time, I hate\n\"choose the right tool for the job\" as a suggestion. Other than art projects,\nhas anyone ever deliberately chosen the wrong tool? Often, something that\nseems like the wrong tool is chosen simply because the contextual requirements\nweren't obvious to you, the interlocutor.</p>\n<p>I think there's two other reasons why this situation is making me feel\ncomplicated things. The first is RIIR, and the second is a situation from a long\ntime ago that changed my life.</p>\n<p>Many of you have probably heard of the \"re-write it in Rust\" meme. The idea is\nthat there's this plague of programmers who are always suggesting that every\nsingle thing needs to be re-written in Rust. While I am sure this happens from\ntime to time, I have yet to see real evidence that it is actually widespread. I\nalmost even wrote a blog post about this, trying to actually quantify the effect\nhere. But on some level, perception is reality, and if people believe it's true,\nit's real on some level, even if that's not actually the case. Regardless of the\ntruth of the number of people who suggest this, on every thread about Rust,\npeople end up complaining about this effect, and so it has an effect on the\noverall discourse regardless. And while I don't believe that this effect is real\noverall, it absolutely has happened in this specific situation. A bunch of\npeople have been very aggressively suggesting that this should have been in\nRust, and not Go, and I get where they're coming from, but also: really? Go is\nan absolutely fine choice here. It's fine. Why do you all have to reinforce a\nstereotype? It's frustrating.</p>\n<p>The second one is something that's been on my mind for over a decade, but I've\nonly started talking about it in places other than conversations with close\nfriends. Long-time readers may remember <a href=\"/writing/node/\">that one time where I was an\nasshole</a>. I mean, I've been an asshole more than once in my life, but\nthis one was probably the time that affected me the most. The overall shape of\nthe situation is this:</p>\n<ol>\n<li>Someone wrote <code>grep</code>, but in Node.</li>\n<li>I made fun of it on the internet, because why would someone write a CLI tool in Node?</li>\n<li>The author found out that I did and felt bad about it.</li>\n</ol>\n<p>Looking back at my apology, I don't think it's necessarily the best one. It was\nbetter than the first draft or two of what I wrote, but I've gotten better at\napologizing since then. Regardless, this incident changed the trajectory of my\nlife, and if you happen to read this, Heather, I am truly, unequivocally, sorry.</p>\n<p>There was a couple of things that happened here. The first is just simply, in\n2013, I did not understand that the things I said had meaning. I hate talking\nabout this because it makes me seem more important than I am, but it's also\nimportant to acknowledge. I saw myself at the time as just Steve, some random\nguy. If I say something on the internet, it's like I'm talking to a friend in\nreal life, my words are just random words and I'm human and whatever. It is what\nit is.</p>\n<p>But at that time in my life, that wasn't actually the case. I was on the Rails\nteam, I was speaking at conferences, and people were reading my blog and tweets.\nI was an \"influencer,\" for better or worse. But I hadn't really internalized\nthat change in my life yet. And so I didn't really understand that if I\ncriticized something, it was something thousands of people would see. To me, I'm\njust Steve. This situation happened before we talked about \"cancel culture\" and\nall of that kind of stuff, but when the mob came for me, I realized: they were\nright, actually. I was being an asshole, and I should not have been. And I\nresolved myself to not being an asshole like that ever again.</p>\n<p>And to do that, I had to think long and hard about why I was like that. I love\nprogramming languages! I love people writing programs that they find\ninteresting! I don't think that it's stupid to write a tool in a language that\nI wouldn't expect it to be written in! So why did I feel the reason to make fun\nof this in that situation?</p>\n<p>The answer? A pretty classic situation: the people I was hanging out with were\naffecting me, and in ways that, when I examined it, I didn't like very much.</p>\n<p>You see, in the Rails world at the time, there was a huge <a href=\"https://blog.aurynn.com/2015/12/16-contempt-culture\">contempt\nculture</a> at the time, especially around JavaScript and Node. And, when\nyou're around people who talk shit all the time, you find yourself talking shit\ntoo. And once I realized that, well, I wanted to stop it. But there was a big\nproblem: my entire professional, and most of my social, life was built around\nRuby and Rails. So if I wanted to escape that culture... what should I do?</p>\n<p>If you clicked on the link to my apology above, you probably didn't make note of\nthe date: Jan 23 2013. Something had just happened to me, in December of 2012: I\nhad decided to check out this little programming language I had heard of called\nRust.</p>\n<p>One of the things that struck me about the \"Rust community\" at the time, which\nwas like forty people in an IRC room, was how nice they were. When I had trouble\ngetting Hello World to compile, and I typed <code>/join #rust</code>, I fully expected an\nRTFM and flames. But instead, I was welcomed with open arms. People were chill.\nAnd it was really nice.</p>\n<p>And so, a month and a day later, I had this realization about the direction my\nlife was heading, and how I wasn't really happy about it. And I thought about\nhow much I enjoyed the Rust stuff so far... and combined with a few other\nfactors, maybe of which I'll write about someday, this is when I decided that I\nwanted to make Rust a thing, and dedicated myself to achieving that.</p>\n<p>And part of that was being conscious about the culture, and how it developed,\nand the ways that that would evolve over time. I didn't want Rust to end up the\nway that Rails had ended up. And that meant a few things, but first of all,\ntruly internalizing that I had to change. And understanding that I had to be\nintentional with what I did and said. And what that led to, among other things,\nwas a \"we talk about Rust on its own merits, not by trash talking other languages\"\nculture. (That's also partially because of Nietzsche, but I'm already pretty far\nafield of the topic for this post, maybe someday.)</p>\n<p>Anyway, the Eternal September comes for us all. The \"Rust Community,\" if it ever\nwere a coherent concept, doesn't really make sense any more. And it is by no\nmeans perfect. I have a lot of criticism about how things turned out. But I do\nthink that it's not a coincidence that \"Rust makes you trans,\" for example, is\na meme. I am deeply proud of what we all built.</p>\n<p>So yeah, anyway: people choose programming languages for projects based on a\nvariety of reasons. And sure, there may be better or worse choices. But you\nprobably have a different context than someone else, and so when someone\nmakes a choice you don't agree with, there's no reason to be a jerk about it.</p>\n<hr />\n<p>Here's my post about this post on BlueSky:</p>\n<p><BlueskyPost uri=\"at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3lkblqoytls22\" /></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/a-happy-day-for-rust/",
"title": "A Happy Day for Rust",
"description": null,
"url": "https://steveklabnik.com/writing/a-happy-day-for-rust/",
"published": "2025-03-05T00:00:00.000Z",
"updated": "2025-03-05T00:00:00.000Z",
"content": "<p>import BlueskyPost from \"../../components/BlueskyPost.astro\";</p>\n<p>A while back, I wrote <a href=\"/writing/a-sad-day-for-rust/\">A sad day for Rust</a> about the Actix-web unsafe\ncontroversy. The short of it was, Actix-web was using some unsound unsafe code.\nWhen this was discovered, people responded pretty harshly, and the maintainer\nquit. Others picked up the work, but it still wasn't great:</p>\n<blockquote>\n<p>I’m not sure where we go from here, and I’m not sure what we could have done\nto prevent this from happening. But I feel like this is a failure, and it’s set\nRust back a bit, and I’m just plain sad.</p>\n</blockquote>\n<p>While the Rust community surely isn't perfect, today I'd like to share with you\na different story: one where there was some controvery, but it was handled in\nthe complete opposite way.</p>\n<p><a href=\"https://rustup.rs/\">Rustup</a> is the official installer and toolchain manager for Rust. It\nhas a much smaller team than Rust itself does, and releases on a much slower\nschedule.</p>\n<p>Back in August of last year, it was decided to <a href=\"https://github.com/rust-lang/rustup/pull/3985\">change some behavior</a>.\nThe details aren't super important, what matters is that it's a breaking change\nto a critical component of the ecosystem. I'm not here to say if this change is\ngood or bad; they had good motivations, there's also good reasons to keep the\nprevious behavior, it's one of those classic \"you can't make everyone happy\"\nsort of deals.</p>\n<p>The team knew that they'd need to let people know that this had happened:</p>\n<blockquote>\n<p>Yes, you've understood it correctly, and this breakage is intentional: #3635 (comment)</p>\n<p>Before the actual release, we'll definitely clearly communicate this change with\nthe community, and we'll use the beta phase to collect user feedback.</p>\n</blockquote>\n<p>And they did <a href=\"https://internals.rust-lang.org/t/seeking-beta-testers-for-rustup-v1-28-0/22060\">make a post on internals</a> about the change, seeking\nfeedback. This also landed in a This Week in Rust, in my understanding.</p>\n<p>The problem is, not everyone reads internals, and not everyone reads This Week\nin Rust. And so yesterday, some folks woke up to a broken CI, leading to <a href=\"https://github.com/rust-lang/rustup/issues/4211\">this\nbug report</a> among other upset posts elsewhere on the internet.</p>\n<p>I have been on the other side of this before. It's frustrating when you try and\ncommunicate something, and people don't hear about it, and then they find out\nlater and get mad at you about it. I also remember the Actix debacle, and many\nother controversies in the broader open source ecosystem.</p>\n<p><BlueskyPost uri=\"at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3ljiqodr7q22f\" /></p>\n<p>However, this time, things went differently. As you can see, the issue is very\nrespectful, and the comments are also generally respectful. The team responded\n<a href=\"https://github.com/rust-lang/rustup/issues/4211#issuecomment-2695620307\">graciously</a>, and decided to put in the work to <a href=\"https://blog.rust-lang.org/2025/03/04/Rustup-1.28.1.html\">release a new\nversion</a> to restore the previous behavior.</p>\n<p>I was kind of worried yesterday that this would end up being another Actix\nsituation. The Rust community has changed a lot over time, and while it's still\npretty good overall, sometimes things can go poorly, just like with any random\ngroup of thousands of people on the internet. But today, I'm grateful that\nfeedback was given constructively, it was taken with the spirit that it was\nintended, and everyone should end up happy.</p>\n<p>If only every controversy went this way, maybe I wouldn't be wondering about\ncoyotes.</p>\n<hr />\n<p>Here's my post about this post on BlueSky:</p>\n<p><BlueskyPost uri=\"at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3ljnctebzec25\" /></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/when-should-i-use-string-vs-str/",
"title": "When should I use String vs &str?",
"description": null,
"url": "https://steveklabnik.com/writing/when-should-i-use-string-vs-str/",
"published": "2024-10-16T00:00:00.000Z",
"updated": "2024-10-16T00:00:00.000Z",
"content": "<p>import BlueskyPost from \"../../components/BlueskyPost.astro\";</p>\n<p>Rust has two main string types: <code>String</code> and <code>&str</code>. Sometimes, people argue\nthat these two types make Rust code difficult to write, because you have to\nthink about which one you should be using in a given situation. My experience\nof writing Rust is that I don't really think about this very much, and this\npost is about some rules of thumb that you can use to be like me.</p>\n<h2>Level 1: Don't think about it at all</h2>\n<p>The very first thing you can do is follow the simplest rule:</p>\n<blockquote>\n<p>Always use <code>String</code>, never use <code>&str</code>.</p>\n</blockquote>\n<p>That looks like this:</p>\n<pre><code>struct Person {\n name: String,\n}\n\nfn first_word(words: String) -> String {\n words\n .split_whitespace()\n .next()\n .expect(\"words should not be empty\")\n .to_string()\n}\n</code></pre>\n<p>This style means you may need to add <code>.to_string()</code> or <code>.clone()</code>\nfor things to work sometimes:</p>\n<pre><code>fn main() {\n let sentence = \"Hello, world!\";\n\n println!(\"{}\", first_word(sentence.to_string()));\n\n let owned = String::from(\"A string\");\n\n // if we don't clone here, we can't use owned the second time\n println!(\"{}\", first_word(owned.clone()));\n println!(\"{}\", first_word(owned));\n}\n</code></pre>\n<p>But that's okay, the compiler will let you know when you need to:</p>\n<pre><code>error[E0382]: use of moved value: `owned`\n --> src/main.rs:21:31\n |\n18 | let owned = String::from(\"A string\");\n | ----- move occurs because `owned` has type `String`, which does not implement the `Copy` trait\n19 |\n20 | println!(\"{}\", first_word(owned));\n | ----- value moved here\n21 | println!(\"{}\", first_word(owned));\n | ^^^^^ value used here after move\n |\nnote: consider changing this parameter type in function `first_word` to borrow instead if owning the value isn't necessary\n --> src/main.rs:5:22\n |\n5 | fn first_word(words: String) -> String {\n | ---------- ^^^^^^ this parameter takes ownership of the value\n | |\n | in this function\nhelp: consider cloning the value if the performance cost is acceptable\n |\n20 | println!(\"{}\", first_word(owned.clone()));\n | ++++++++\n</code></pre>\n<p>Hey, compiler, that's a great idea. Let's move on to level 2.</p>\n<h2>Level 2: prefer <code>&str</code> for function parameters</h2>\n<p>A rule that's a little better is this one:</p>\n<blockquote>\n<p>Always use <code>String</code> in structs, and for functions, use <code>&str</code> for parameters\nand <code>String</code> types for return values.</p>\n</blockquote>\n<p>This is what the compiler error for level 1 was suggesting we do instead of <code>.clone</code>.</p>\n<p>That results in code that looks like this:</p>\n<pre><code>struct Person {\n name: String,\n}\n\nfn first_word(words: &str) -> String {\n words\n .split_whitespace()\n .next()\n .expect(\"words should not be empty\")\n .to_string()\n}\n\nfn main() {\n let sentence = \"Hello, world!\";\n\n println!(\"{}\", first_word(sentence));\n\n let owned = String::from(\"A string\");\n\n println!(\"{}\", first_word(&owned));\n println!(\"{}\", first_word(&owned));\n}\n</code></pre>\n<p>We're now doing much less copying. We do need to add a <code>&</code> on\nto <code>String</code> values that we wish to pass into <code>first_word</code>, but\nthat's not too bad, the compiler will help us when we forget:</p>\n<pre><code>error[E0308]: mismatched types\n --> src/main.rs:20:31\n |\n20 | println!(\"{}\", first_word(owned));\n | ---------- ^^^^^ expected `&str`, found `String`\n | |\n | arguments to this function are incorrect\n |\nnote: function defined here\n --> src/main.rs:5:4\n |\n5 | fn first_word(words: &str) -> String {\n | ^^^^^^^^^^ -----------\nhelp: consider borrowing here\n |\n20 | println!(\"{}\", first_word(&owned));\n | +\n</code></pre>\n<p>Following this rule will get you through 95% of situations successfully. Yes,\nthat number was found via the very scientific process of \"I made it up, but it\nfeels correct after writing Rust for the last twelve years.\"</p>\n<p>For 4% of that last 5%, we can go to level 3:</p>\n<h2>Level 3: return <code>&str</code> sometimes</h2>\n<p>Here's a slightly more advanced rule for certain circumstances:</p>\n<blockquote>\n<p>Always use <code>String</code> in structs, and for functions, use <code>&str</code> for parameters.\nIf the return type of your function is derived from an argument and isn’t\nmutated by the body, return <code>&str</code>. If you run into any trouble here, return\n<code>String</code> instead.</p>\n</blockquote>\n<p>That would look like this:</p>\n<pre><code>struct Person {\n name: String,\n}\n\n// we're returning a substring of words, so &str is appropriate\nfn first_word(words: &str) -> &str {\n words\n .split_whitespace()\n .next()\n .expect(\"words should not be empty\")\n}\n\nfn main() {\n let sentence = \"Hello, world!\";\n\n println!(\"{}\", first_word(sentence));\n\n let owned = String::from(\"A string\");\n\n println!(\"{}\", first_word(&owned));\n println!(\"{}\", first_word(&owned));\n}\n</code></pre>\n<p>This lets us remove a copy, we no longer have a <code>.to_string</code> in the body of\n<code>first_word</code>.</p>\n<p>Sometimes, we can't do that though:</p>\n<pre><code>// because we are going to uppercase the first word, our return type can't\n// be &str anymore, because we aren't actually returning a substring: we are\n// creating our own new string.\nfn first_word_uppercase(words: &str) -> String {\n words\n .split_whitespace()\n .next()\n .expect(\"words should not be empty\")\n .to_uppercase()\n}\n\nfn main() {\n let sentence = \"Hello, world!\";\n\n println!(\"{}\", first_word_uppercase(sentence));\n\n let owned = String::from(\"A string\");\n\n println!(\"{}\", first_word_uppercase(&owned));\n println!(\"{}\", first_word_uppercase(&owned));\n}\n</code></pre>\n<p>How do you know that this is the case? Well, in this specific case,\n<code>to_uppercase</code> already returns a <code>String</code>. So that's a great hint.\nIf we tried to return a <code>&str</code>, we'd get an error:</p>\n<pre><code>// this can't work\nfn first_word_uppercase(words: &str) -> &str {\n &words\n .split_whitespace()\n .next()\n .expect(\"words should not be empty\")\n .to_uppercase()\n}\n</code></pre>\n<p>would give us</p>\n<pre><code>error[E0515]: cannot return reference to temporary value\n --> src/main.rs:7:5\n |\n7 | &words\n | ______^-\n | | ______|\n8 | || .split_whitespace()\n9 | || .next()\n10 | || .expect(\"words should not be empty\")\n11 | || .to_uppercase()\n | || ^\n | ||_______________________|\n | |_______________________returns a reference to data owned by the current function\n | temporary value created here\n</code></pre>\n<p>And that's really it. Following this rule will get you through virtually every\nscenario where you need to wonder about <code>String</code> and <code>&str</code>. With some practice,\nyou'll internalize these rules, and when you feel comfortable with a level, you\ncan go up to the next one.</p>\n<p>What about that last 1% though? Well, there is a next level...</p>\n<h2>Level 4: When to use <code>&str</code> in a struct</h2>\n<p>Here's the rule for level 4:</p>\n<blockquote>\n<p>Should you use a <code>&str</code> in a struct? If you're asking that question, use\n<code>String</code>. When you need to use <code>&str</code> in a struct, you'll know.</p>\n</blockquote>\n<p>Storing references in structs is useful, for sure, and it's good that Rust\nsupports it. But you're only going to need it in fairly specific scenarios,\nand if you feel like you're worring about <code>String</code> vs <code>&str</code>, you're just\nnot in the position to be worrying about the complexity of storing a <code>&str</code>\nin a struct yet.</p>\n<p>In fact, some people believe in this rule so strongly that they're working\non a language where storing references in structs isn't even possible, and\nit's a language I've found very interesting lately:\n<a href=\"https://www.hylo-lang.org/\">Hylo</a>. They go a bit farther than that, even:\nin Hylo, you think of everything as being values, rather than having references\nat all. They think that you can write meaningful programs with this model.\nBut this isn't a post about Hylo. I'll write one of those eventually. My point\nis just that you can really, truly, get away with not storing <code>&str</code>s in structs\nfor a <em>lot</em> of useful Rust programs. So it's not really worth spending mental\nenergy on, until you determine that you must do so. That is the case when\nyou've profiled your program and determined that copying strings to and from\nyour struct is a big enough issue to bother with lifetimes.</p>\n<p>I hope you've found this helpful, and if you have any other rules of thumb\nlike this, I'd love to hear about them!</p>\n<hr />\n<p>Here's my post about this post on BlueSky:</p>\n<p><BlueskyPost uri=\"at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3l6nkx54sij2w\" /></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/against-names/",
"title": "Against Names",
"description": null,
"url": "https://steveklabnik.com/writing/against-names/",
"published": "2024-08-12T00:00:00.000Z",
"updated": "2024-08-12T00:00:00.000Z",
"content": "<p>import BlueskyPost from \"../../components/BlueskyPost.astro\";</p>\n<p>There's an old saying:</p>\n<blockquote>\n<p>There are only two hard things in Computer Science: cache invalidation and\nnaming things.</p>\n<p>― Phil Karlton</p>\n</blockquote>\n<p>I also appreciate the joke version that adds \"and off by one errors.\"</p>\n<p>Lately, I've been thinking about this saying, combined with another old joke:</p>\n<blockquote>\n<p>\"The patient says, \"Doctor, it hurts when I do this.\" The doctor says, \"Then\ndon't do that!\"</p>\n<p>― Henny Youngman</p>\n</blockquote>\n<p>Specifically, if naming things is so hard... why do we insist on doing it all\nthe time?</p>\n<p>Now, I am not actually claiming we should stop giving things names. But I have\nhad at least two situations recently where I previously found names to be kinda\ncritical, and then I changed to systems which didn't use names, and I think it\nimproved the situation.</p>\n<p>One of the most famous examples of not giving something a name,\nlambdas/closures, took some time to catch on. But many folks already recognize\nthat naming every single function isn't always neccesary. I wonder if there are\nmore circumstances where I've been naming things where I didn't actually have\nto.</p>\n<p>Anyway, here's my two recent examples:</p>\n<h2>Nameless Branches</h2>\n<p>I haven't written much about it on my blog yet, but I'm fully converted away\nfrom git to <a href=\"https://github.com/martinvonz/jj\">jj</a>. I'll say more about this in the future, but one major\ndifference between the two is that jj has anonymous branches. If, like me,\nyou are a huge fan of git, this sounds like a contradiction. After all, the\nwhole thing about branches are that they're a name for some point in the DAG.\nHow do you have a nameless name?</p>\n<p>Here's some output from <code>jj log</code>:</p>\n<p>Here's some sample output from jj log:</p>\n<pre><code> $ jj log --limit 5\n @ pzoqtwuv [email protected] 2024-03-01 15:06:59.000 -06:00 9353442b\n │ added some cool new feature\n │ ◉ xrslwzvq [email protected] 2024-02-29 23:06:23.000 -06:00 a70d464c\n ├─╯ create hello and goodbye functions\n │ ◉ yykpmnuq [email protected] 2024-02-29 23:03:22.000 -06:00 210283e8\n ├─╯ add better documentation\n ◉ ootnlvpt [email protected] 2024-02-28 23:26:44.000 -06:00 b5db7940\n │ only print hello world\n ◉ nmptruqn [email protected] 2024-02-28 23:09:11.000 -06:00 90a2e97f\n │ refactor printing\n</code></pre>\n<p>Here, we are working on change <code>pzoqtwuv</code>. (<code>@</code> means the working copy.) There\nare colors in the real CLI to make the differences more obvious, and to show you\nunique prefixes, so for example, you probably only need <code>p</code> or <code>pz</code> instead of\n<code>pzoqtwuv</code> to uniquely identify the change. I'll use the full IDs since there's\nno syntax highlighting here.</p>\n<p>We have two anonymous branches here. They have the change IDs of <code>xrslwzvq</code> and\n<code>yykpmnuq</code>. The log output shows the summary line of their messages, so we can\nsee \"create hello and goodbye functions\" on one branch, and \"add better\ndocumentation\" on the other.</p>\n<p>You don't need an additional branch name: the change ID is already there. If you\nwant to add even more better documentation, <code>jj new yykpmnuq</code> (or again, likely\n<code>jj new yy</code> in practice) and you're off to the races. (jj new makes a new change\noff of the parent you specify.)</p>\n<p>(And if you're in a larger repo with more outstanding branches, you can ask <code>jj log</code> to show specific subsets of commits. It has a powerful DSL that lets you do\nso. For example, say you only want to see your commits, <code>jj log -r 'mine()'</code> can\ndo that for you.)</p>\n<p>That's all there is to it. We already have the commit messages and IDs, giving\nan additional identifier doesn't help that much. In practice, I haven't missed\nnamed branches at all. And in fact, I kind of really appreciate not bothering to\ncome up with a name, and then eventually remembering to delete that name once\nthe PR lands, stuff like that. Life is easier.</p>\n<h2>Utility CSS</h2>\n<p>Another technology I have learned recently is <a href=\"https://tailwindcss.com/\">tailwind</a>. But Tailwind is just\none way of doing a technique that has a few names, I'm going to go with \"utility\nCSS\". The idea is in opposition to \"semantic CSS.\" To crib an example from a\n<a href=\"https://adamwathan.me/css-utility-classes-and-separation-of-concerns/\">blog post by the author of Tailwind</a> (which does a better job of thoroughly\nexplaining why doing utility CSS is a good thing, you should go read it),\nsemantic CSS is when you do this:</p>\n<pre><code><style>\n .greeting {\n text-align: center;\n }\n</style>\n\n<p class=\"greeting\">Hello there!</p>\n</code></pre>\n<p>Whereas, with Tailwind, you end up instead having something like this:</p>\n<pre><code><p class=\"text-center\">Hello there!</p>\n</code></pre>\n<p>We don't have a new semantic name <code>greeting</code>, but instead describe what\nwe want to be done to our element via a utility class.</p>\n<p>So the thing is, as a previous semantic CSS enjoyer, this feels like using\ninline styling. But there's a few significant differences. The first one is,\nyou're not writing plain CSS, you are re-using building blocks that are defined\nfor you. The abstraction is in building those utility classes. This means you're\nnot writing new CSS when you need to add functionality, which to me is a great\nsign that the abstraction is working.</p>\n<p>It's also that there is some sleight of hand going on here, as we do, on another\nlevel. An objection that gets raised to doing things this way is \"what happens\nwhen you need to update a bunch of similar styles?\" And the answer for that is\ncomponents. That is, it's not so much that utility CSS says that semantic names\nare bad, it's that semantic names <em>at the tag level</em> are the wrong level of\nabstraction to use names. To sort of mix metaphors, consider the lambda/closure\nexample. Here's a random function in Rust:</p>\n<pre><code>fn calculate(list: &[i32]) -> i32 {\n list.into_iter()\n .filter(|i| **i % 2 == 0)\n .sum()\n}\n</code></pre>\n<p>The <code>**</code> is unfortunate, but this function takes a list of numbers, selects for\nthe even ones, and then sums them. Here, we have a closure, the argument to\n<code>filter</code>, but it's inside a named function, <code>calculate</code>. This is what using\nTailwind feels like to me, we use names for higher level concepts (components),\nand then keep things semantically anonymous for some of the tasks inside of them\n(markup).</p>\n<p>Heck, even the most pro-semantic-styles folks don't advocate that you must give\nevery single element a class. Everyone recognizes the value of anonymous things\nsometimes, it's just a matter of what level of abstraction deserves to get\nnamed.</p>\n<hr />\n<p>Here's my post about this post on BlueSky:</p>\n<p><BlueskyPost uri=\"at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3kzkadmleqe2m\" /></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/how-does-bluesky-work/",
"title": "How Does BlueSky Work?",
"description": null,
"url": "https://steveklabnik.com/writing/how-does-bluesky-work/",
"published": "2024-02-24T00:00:00.000Z",
"updated": "2024-02-24T00:00:00.000Z",
"content": "<p>import BlueskyPost from \"../../components/BlueskyPost.astro\";</p>\n<p>One of the reasons I am enthusiastic about BlueSky is because of the way that\nit works. So in this post, I am going to lay out some of the design and the\nprinciples behind this design, as I understand them. I am not on the BlueSky\nteam, so these are my takes only.</p>\n<p>Let's begin.</p>\n<h2>Why does BlueSky exist?</h2>\n<p>Here's what <a href=\"https://bsky.social\">the BlueSky Website</a> says right now:</p>\n<blockquote>\n<p>Social media is too important to be controlled by a few corporations. We’re\nbuilding an open foundation for the social internet so that we can all shape\nits future.</p>\n</blockquote>\n<p>This is the big picture.</p>\n<p>Okay so that's a great idea, but like, what does that <em>mean</em>? Currently,\nBlueSky is a microblogging application, similar to Twitter and Mastodon. How\ndoes that fit into the big picture? Well, while it's true that BlueSky is a\nmicroblogging application, that's not the whole story: BlueSky is an initial\napplication to prove out the viability of <a href=\"https://atproto.com/\">the Authenicated Transfer\nProtocol</a>, known as AT, ATP, or \"atproto\" for short. BlueSky is the\n\"building\" and atproto is the \"open foundation for the social internet.\"</p>\n<p>An important thing to note: BlueSky is also a company. Some people look at a\ncompany saying \"hey we're building something that's too big to be controlled\nby companies!\" with skepticism. I think that's a healthy starting point, but\nthe answer for me is atproto.</p>\n<p>The interplay between these two things is important, but we're going to start\nby exploring atproto, and then talk about how BlueSky is built on top of it.</p>\n<h2>Is this a cryptocurrency?</h2>\n<p>The first thing we have to get out of the way: If you hear \"oh it's a\ndistributed network called 'something protocol'\" you may have a \"is this\na cryptocurrency?\" alarm bell going off in your head.</p>\n<p>Don't worry, it's not a cryptocurrency. It does use some technologies that\noriginated in the cryptocurrency space, but this isn't a blockchain, or a DAO,\nor NFTs, or any of that. Just some cryptography and merkle trees and the like.</p>\n<h2>What is the big picture with atproto?</h2>\n<p>Here's what <a href=\"https://atproto.com/guides/overview\">the AT Protocol Overview</a> says:</p>\n<blockquote>\n<p>The Authenticated Transfer Protocol, aka atproto, is a federated protocol for\nlarge-scale distributed social applications.</p>\n</blockquote>\n<p>Let's break that down:</p>\n<blockquote>\n<p>a federated protocol</p>\n</blockquote>\n<p>atproto is federated. This means that the various parts of the system can have\nmultiple people running them, and that they communicate with each other.</p>\n<p>Choosing federation is a big part of how atproto delivers on the \"can't be\ncontrolled by one organization\" promise. There are other parts too, but this\nis an important aspect of solving this.</p>\n<blockquote>\n<p>for large-scale</p>\n</blockquote>\n<p>If you want to scale, you have to design with scale in mind. atproto makes\nseveral interesting choices in order to distribute the load of running the\nsystem more onto the actors that can handle load, and less on those that can't.\nThis way, applications running on top of atproto can scale up to large userbases\nwithout issue.</p>\n<p>That's the hope, at least. Earlier this week, BlueSky hit five million users,\nand is far more stable than Twitter was in the early days. That's not as big\nas many social applications, but it's not nothing either. We'll see how this\nworks out in practice.</p>\n<blockquote>\n<p>distributed social applications</p>\n</blockquote>\n<p>atproto is for connecting to others, so it's focused on social applications.\nIt also is currently 100% public, there are no private messages or similar. The\nreasons for this is that achieving private things in a federated system is\nvery tricky, and they would rather get it right than ship something with serious\ncaveats. Best for now to only use this stuff for things you want to be public.</p>\n<p>These applications are \"distributed\" because running them involves running them\non the network directly. There's no \"BlueSky server,\" there's just servers\nrunning atproto distributing messages to each other, both BlueSky messages and\nwhatever other messages from whatever other applications people create.</p>\n<p>So that's the high level, but what does that mean concretely?</p>\n<p>In atproto, <em>users</em> create <em>records</em> that are cryptographically signed to\ndemonstrate authorship. Records have a schema called a <em>Lexicon</em>.</p>\n<p>Records are stored in <em>repositories</em>. Repositories run as a <em>service</em>, exposing\nHTTP and WebSockets. They then can then talk to each other and federate the\nrecords. These are often called PDSes, for \"Personal Data Server.\" Users\neither run their own PDS, or use one that someone else hosts for them.</p>\n<p>Applications can be built by looking at the various records stored in the\nnetwork, and doing things with them. These services all called <em>App Views</em>,\nbecause they are exposing a particular view of the information stored in the\nnetwork. This view is created via the Lexicon system: building an application\nmeans that you define a Lexicon, structuring the data that you want to deal with,\nand then look at records that use your lexicon, ignoring the rest.</p>\n<p>Now, if this were all there is, there would be pretty serious scaling issues.\nFor example, if every time I post a new update on BlueSky, if I had to send\nmy post to every single one of my followers' repositories, that would be\nextremely inefficent, and make running a popular repository very expensive to\nrun. To fix this, there's an additional kind of service, called a <em>relay</em>, that\naggregates information in the network, and exposes it as a firehose to others.\nSo in practice, App Views don't look at Repositories, but instead, look at\nRelays. When I make a post, my respository won't notify my followers'\nrepositories individually. My repository will notify a Relay, and my followers\nwill use an App View that filters the ouput of the Relay to show only the posts\nof people they're following. This does imply that Relays are often huge and\nexpensive to run, however you could imagine running a smaller relay that only\npropogates posts from a smaller subset of users too. They don't <em>have</em> to show\neverything on the network, though bigger ones will, of course.</p>\n<p>Here this is in ASCII art:</p>\n<pre><code> ┌─────┐ ┌──────────┐\n │ PDS ├───────┐ │ App View │\n └─────┘ │ └──────────┘\n ┌▼────────┐ ▲\n ┌─────┐ │ ├───────┘\n │ PDS ├──────► Relay │\n └─────┘ │ ├───────┐\n └▲────────┘ ▼\n ┌─────┐ │ ┌──────────┐\n │ PDS ├───────┘ │ App View │\n └─────┘ └──────────┘\n</code></pre>\n<p>This is all you really need to know to understand the core of atproto: people\ncreate data, it's shared in the network, and applications can interact with\nthat data.</p>\n<p>However, there are additional service types being introduced, with the\npossibility of more in the future. But before we talk about those, we have to\nexplain some ideological commitments to understand why things are shaped the way\nthey are.</p>\n<h2>What is \"speech vs reach\"?</h2>\n<p>Given that atproto is deliberately created to enable social applications, it\nneeds to consider not just connecting people, but also disconnecting people.\nModeration is a core component of any social application: \"no moderation\" is\nstill a moderation strategy. BlueSky handles these sorts of questions by\nacknowledging that different people will have different preferences when it\ncomes to moderation, and also that moderation at scale is difficult.</p>\n<p>As such, the protocol takes a \"speech vs reach\" approach to moderation. The\nstuff we've described so far falls under the \"speech\" layer. It is purely\nconcerned with replicating your content across the network, without caring\nwhat the semantic contents of that content is. Moderation tools fall under the\n\"reach\" layer: you take all of that speech, but provide a way to limit the\nreach of stuff you don't care to see yourself.</p>\n<p>Sometimes, people say that BlueSky is \"all about free speech\" or \"doesn't do\nmoderation.\" This is simply inaccurate. Moderation tooling is encoded into the\nprotocol itself, so that it can work with all content on the network, even\nnon-BlueSky applications. Moreover, it gives you the ability to choose your own\nmoderators, so that you aren't beholden to anyone else's choice of moderation or\nlack thereof. But I'm getting ahead of myself: let's talk about feed generators\nand labelers.</p>\n<h2>What are feed generators?</h2>\n<p>Most social applications have the concept of a \"feed\" of content. This is broken\nout into its own kind of service in atproto, called a <em>feed generator</em>. A classic\nexample of a feed is \"computer, show me the posts of the people I follow in\nreverse chronological order.\" Lately, algorithmic feeds have become popular with\nsocial networks, to the point of where some non-technical users refer to them\nas \"algorithms.\"</p>\n<p>Feed generators take the firehose produced by a relay, and then show you a list\nof content, filtered and ordered by whatever metric the feed generator desires.\nYou can then share these feeds with other users.</p>\n<p>As a practical example, one of my favorite feeds is the <a href=\"https://bsky.app/profile/did:plc:vpkhqolt662uhesyj6nxm7ys/feed/infreq\">Quiet\nPosters</a> feed. This feed shows posts by people who don't post\nvery often. This makes it so much easier to keep up with people who get drowned\nout of my main feed. There are feeds like <a href=\"https://bsky.app/profile/did:plc:vpkhqolt662uhesyj6nxm7ys/feed/followpics\">the 'Gram</a>, which shows\nonly posts that have pictures attatched. Or <a href=\"https://bsky.app/profile/did:plc:q6gjnaw2blty4crticxkmujt/feed/bangers\">My Bangers</a>, which shows\nyour most popular posts.</p>\n<p>This to me is one of the killer features of BlueSky over other microblogging\ntools: total user choice. If I want to make my own algorithm, I can do so.\nAnd I can share them easily with others. If you use BlueSky, you can visit\nany of those feeds and follow them too.</p>\n<p>Feeds are a recent addition to atproto, and therefore, while they do exist,\nthey may not be feature complete just yet, and may undergo some change in the\nfuture. We'll see. They're working just fine from my perspective, but I haven't\nbeen following the lower level technical details.</p>\n<h2>What are labelers?</h2>\n<p>A <em>Labeler</em> is a service that applies <em>labels</em> to content or accounts. As a user,\nyou can subscribe to a particular labeler, and then have your experience change\nbased on the labels on posts.</p>\n<p>A labeler can do this via whatever method it pleases: automatically by running\nsome sort of algorithm on posts, manually by having some human give a thumbs\nup or thumbs down, whatever method the person running the labeling service\nwants.</p>\n<p>An example of a labeling service would be a blocklist: a label on the posts\nauthored by people whose content you don't want to see. Another example is\nan NSFW filter, which may run some sort of algorithm over pictures in posts,\nand labeling them if they believe they contain NSFW content.</p>\n<p>Labeling exists, but I do not believe you can run your own labeler yet. BlueSky\nruns their own, but there hasn't been an external release that I am aware of.\nBut once they do, you can imagine communities running their own services, adding\nwhatever kind of labels they'd like.</p>\n<h2>How does moderation work in atproto?</h2>\n<p>Putting this all together, we can see how moderation works: Feeds may choose to\ntransform the feed based on labels, or App Views may take feeds and apply\ntransformations based on asking a Labeler about it. These can\nbe mixed and matched based on preference.</p>\n<p>This means you can choose your moderation experience, not just in applications,\nbut also within it. Want a SFW feed, but allow NSFW content in another? You\ncan do that. Want to produce a blocklist of people and share it with the\nworld? You can do that.</p>\n<p>Because these moderation tools work at the network level, rather than at the\napplication level, they actually go <em>further</em> than in other systems. If someone\nbuilds an Instagram clone on atproto, that could also use your blocklist\nlabeller, since your blocklist labeller works at the protocol level. Block\nsomeone in one place, and they can be blocked on every place, if you so choose.\nMaybe you subscribe to different moderation decisions in different applications.\nIt is 100% up to you.</p>\n<p>This model is significantly different from other federated systems, because\nyou don't really have an \"account\" on an \"instance,\" like in Mastodon. So a lot\nof people ask questions like \"what happens when my instance gets defederated\"\nwhich don't exactly make sense as stated. You can achieve the same goal, by\nblocking a set of users based on some criteria, maybe you dislike a certain\nPDS and want to ignore posts that come from a certain one, but that is <em>your</em>\nchoice and yours alone, it is not dictated by some \"server owner\" that your\naccount resides on.</p>\n<p>So if you don't have a home server, how does identity work?</p>\n<h2>How does identity and account portability work?</h2>\n<p>There are a LOT of details to how identity works, so I'm going to focus on the\nparts that I find important. I am also going to focus on the part that is\ncontroversial, because that is important to talk about.</p>\n<p>At its core, users have an identity number, called a \"Decentralized Identifier,\"\nor <em><a href=\"https://www.w3.org/TR/did-core/\">DID</a></em>. My DID looks like this: <code>did:plc:3danwc67lo7obz2fmdg6jxcr</code>.\nFeel free to follow me! Lol, of course that's not the interface that you'll see\nmost of the time. Identity also involves a <em>handle</em>, which is a domain name.\nMy handle is <code>steveklabnik.com</code>, unsurprisingly. You'll see my posts on BlueSky\nas coming from <code>@steveklabnik.com</code>. This system also works well for people who\ndon't own a domain; if you sign up for BlueSky, it'll give you the ability to\nchoose a name, and then your handle is <code>@username.bsky.social</code>. I started off\nmaking posts as <code>@steveklabnik.bsky.social</code>, and then moved to\n<code>@steveklabnik.com</code>. But because the DID is stable, there was no disruption to\nmy followers. They just saw the handle update in the UI.</p>\n<p>You can use a domain as your handle by getting the DID your PDS generated for\nyou, and then adding a <code>TXT</code> record in the DNS you use for that domain. If\nyou're not the kind of person who uses or even knows what DNS is, I envy you,\nbut you can also use BlueSky's partnership with NameCheap to register a domain\nand configure it to use as a handle without any technical knowledge necessary.\nYou can then log into applications with your domain as the handle, and\neverything works nicely.</p>\n<p>This is also how BlueSky delivers true \"account portability,\" partially because,\nwell, there isn't really a concept of an account. The person who uses a given\nDID uses cryptography to sign the content they create, and then that content\nis replicated across the network. \"Your account\" can't really be terminated,\nbecause that would mean someone forcibly stopping you from using keys that they\ndon't even have access to. If your PDS goes down, and you want to migrate to\na new one, there's a way to backfill the contents of the PDS from the network\nitself, and inform the network that your PDS has moved. It is real, meaningful\naccount portability, and that is radically different from any similar service\nrunning today.[^1]</p>\n<p>But.</p>\n<p>The devil is in the details, and I think this is one of the more meaningful\ncriticisms of BlueSky and atproto.</p>\n<p>You see, there are different \"methods\" of creating a DID. BlueSky supports\ntwo methods: <code>did:web</code>, which is based on domain names. There are some drawbacks\nwith this method that I don't personally fully understand well enough to describe,\nI'm sure I'll write something in-depth about DIDs in the future.</p>\n<p>So because of that weakness, BlueSky has implemented their own DID method,\ncalled <code>did:plc</code>. The <code>plc</code> stands for \"placeholder,\" because even though\nthey plan on supporting it indefinitely, it too has its weaknesses. And that\nweakness is that it involves asking a service that BlueSky runs in order to\nresolve the proper information. For example, <a href=\"https://plc.directory/did:plc:3danwc67lo7obz2fmdg6jxcr\">here is my lookup</a>.\nThis means that BlueSky can ban you in a more serious way than is otherwise\npossible thanks to the network design, which some people take to be a very\nserious issue.</p>\n<p>So, is the flaw fatal? I don't think so. The first reason is, if you really don't\nwant to engage with it, you can use <code>did:web</code>. Yes that isn't great for other\nreasons; that's why <code>did:plc</code> was created. But you do get around this issue.</p>\n<p>Another is that the BlueSky team has demonstrated, in my personal opinion,\nenough understanding and uncomfortableness with being in control here, and it's\ndesigned in such a way that if other, better systems develop, you can move\nto them. They've also indicated that moving governance of <code>did:plc</code> to some sort\nof consensus model in the future is possible. There are options. Also, others\ncould run a <code>did:plc</code> service and use that instead if they prefer, too.</p>\n<p>I personally see this as an example of pragmatically shipping something, others\nsee it as a nefarious plot. You'll have to decide for yourself.</p>\n<h2>How is BlueSky built on top of atproto?</h2>\n<p>So, now that we understand atproto, we can understand BlueSky. BlueSky is\nan application built on top of the atproto network. They run an App View, and\n<a href=\"https://bsky.app/\">a web application</a> that uses that App View to work. They also run a PDS\nfor users that sign up through the web app, as well as a relay that those PDSes\ncommunicate with.</p>\n<p>They publish two Lexicons, one as <code>com.atproto.*</code> and one as <code>app.bsky.*</code>. The\nformer are low level operations that any application on the network will need,\nand the ones specific to BlueSky are in the latter.</p>\n<p>But one nice thing about BlueSky in particular is that they've taken the product\ngoals that nobody should know any of this nerd shit to be able to use BlueSky.\nThe lack of instances means there's no \"I need to pick an instance to create an\naccount\" flow, and the portability means that if my host goes down, I can move,\nand my followers are none the wiser.</p>\n<h2>How will others build applications on top of atproto?</h2>\n<p>You can create an atproto app by creating a Lexicon. You'll then want to run\nan App View that does things with data on the network involving your lexicon,\nand your application will want to give people the ability to write data to their\nPDS using your lexicon.</p>\n<p>I myself am considering doing so. We'll see.</p>\n<h2>Concluding thoughts</h2>\n<p>So yeah, on the technical side of things, that's an overview of how atproto and\nBlueSky work. I think this design is very clever. Furthermore, I think the\nseparation of concerns between atproto and BlueSky are very meaningful, as having\na \"killer app\" for the network gives a reason to use it. It also is a form of\ndogfooding, making sure that atproto is good enough to be able to build real\napplications on.</p>\n<p>I'm sure I'll have more to say about all of this in the future.</p>\n<hr />\n<p>Here's my post about this post on BlueSky:</p>\n<p><BlueskyPost uri=\"at://did:plc:3danwc67lo7obz2fmdg6jxcr/app.bsky.feed.post/3km6y2x6cxk2f\" /></p>\n<p>[^1]:\nA commentor points out https://book.peergos.org/, which I had not heard of,\nbut apparently was known to the creators of BlueSky before they made it. Neat.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/using-the-oxide-console/",
"title": "Using the Oxide Console",
"description": null,
"url": "https://steveklabnik.com/writing/using-the-oxide-console/",
"published": "2024-02-17T00:00:00.000Z",
"updated": "2024-02-17T00:00:00.000Z",
"content": "<p>A very, very long time ago, I was introduced to <a href=\"https://en.wikipedia.org/wiki/Gerrit_(software)\">Gerrit</a>. To be honest,\nI hated it. However, lately I have become interested in divesting from\n<code>git</code> and GitHub, and so have decided to re-visit various \"forges\" to see what's\nout there. The \"why\" for this will come later, and I'm not leaving them just yet,\njust, you know, doing some exploring.</p>\n<p>Anyway, in order to play around with Gerrit, I need a server somewhere. This\nwebsite runs entirely on Vercel at the moment, so I don't happen to have a\nmore traditional server lying around to give Gerrit a shot. But as it just so\nhappens, I work at <a href=\"https://oxide.computer/\">Oxide</a>, and so have access to a rack that I can play\naround with. So I thought it would be fun to share what that looks like, given\nthat people are interested in what we do, but don't actually have access to\none of our machines.</p>\n<p>If you'd like to poke at the console yourself, there's a <a href=\"https://oxide-console-preview.vercel.app/\">live demo</a>!\nQuoting <a href=\"https://github.com/oxidecomputer/console?tab=readme-ov-file#live-demo\">the README</a>:</p>\n<blockquote>\n<p>At https://oxide-console-preview.vercel.app the console is deployed as a\nstatic site with a mock API running in a Service Worker. You can create mock\nresources and they will persist across client-side navigations, but they exist\nonly in the browser: nobody else can see them and the mock \"DB\" is reset on\npageload. Request and response bodies in the mock API match the Oxide API's\nOpenAPI spec, but behavior is only mocked in as much detail as is required for\ndevelopment and testing of the console and is not fully representative of the\nreal API.</p>\n</blockquote>\n<p>I happen to think this is extremely cool. If you want to, you could stop reading\nthis post and go play with that instead. It's also, like the rest of Oxide's software\nstack, fully open source!</p>\n<p>Before we begin, I would like to say that I am going to be doing this \"manually\"\nvia the console and a web browser rather than via the CLI. I am doing this for\ntwo reasons:</p>\n<ol>\n<li>I think the console is pretty and I would like to show it off. The folks who\nwork on it do a fantastic job.</li>\n<li>I don't have many images on this blog and so want to write a post with some\nimages and this will look nicer than screenshots of a terminal.</li>\n</ol>\n<p>To be clear, I imagine a lot of real production usage of the Oxide rack will be\ndriven by <a href=\"https://docs.oxide.computer/guides/introduction#_overview\">the API</a>, either via our CLI tools or terraform or\nwhatever. Heck, the console is built on those APIs, so on some level, that's the\nonly way to do it. My point is, don't think of this as being the only workflow\nto interact with Oxide. Maybe if people like this post I'll make some more\ncovering the CLI driven workflow. Let me know.</p>\n<hr />\n<p>After you SSO into the console, you're put on the Projects page:</p>\n<p><img src=\"/img/2024-02-17/projects.png\"></p>\n<p>We're in a \"silo\" named Oxide. A silo is a grouping of users, projects, and\nresources that are isolated from one another. But we don't care about that:\nwe want to make a new project. As you can see, I have censored my co-workers'\nprojects, as I didn't tell anyone I was making this post, and even though\nthe description here is hilarious, it wouldn't be right to share without asking,\nand I'm already like ten layers deep in this yak shave.</p>\n<p>Let's make a new project:</p>\n<p><img src=\"/img/2024-02-17/new-project.png\"></p>\n<p>It's just that easy. After we push the button, we're put on the instances page:</p>\n<p><img src=\"/img/2024-02-17/instances.png\"></p>\n<p>There's a nice little toast in the bottom left that I didn't manage to get a\nscreenshot of because I was re-sizing my window too. Anyway, now that we have\nour project, we can add one or more instances to it. Instances are virtual\nmachines, and so setting one up looks like you might expect. You can give it\na name and description, but for this screenshot I'm only showing off one part\nof this page because this is already a ton of screenshots:</p>\n<p><img src=\"/img/2024-02-17/create-instance.png\"></p>\n<p>We can size instances however we'd like. Even though Gerrit is a Java\napplication these days, I'm going to keep the instance small, because it's just\nfor me and performance isn't a huge deal. Yes, I have made massive instances for\nfun in the past.</p>\n<p>We're using a Debian image that's been provided by this silo, because I am lazy\nand that's fine. Is four gigabytes of disk enough? I don't know, let's find out.</p>\n<p>A few seconds later, our instance is up and running:</p>\n<p><img src=\"/img/2024-02-17/instance.png\"></p>\n<p>I am obscuring the external IP because, well, no reason to give that out.\nAnyway, that's it! We can <code>ssh</code> in:</p>\n<p><img src=\"/img/2024-02-17/login.png\"></p>\n<p>... how, you may ask? Well, I didn't show off the entire \"Create instance\" page:\nI have set up my ssh keys in my account previously, and so by default, because\nthis image uses <code>cloud-init</code> to configure itself, my <code>ssh</code> keys are already\ninjected into the image. Furthermore, I left it on the default <a href=\"https://en.wikipedia.org/wiki/Virtual_private_cloud\">VPC</a>, which\nis currently configured with a firewall rule that allows for incoming <code>ssh</code>\nconnections. All of this is of course absolutely customizable, but given what\nthis rack is being used for, these defaults make sense for us, so I didn't have\nto actually do much setup at all.</p>\n<p>That being said, I do need to open port 8080 so that I can poke at Gerrit in\na web browser. If I go into the VPC configuration, I can add a new firewall rule:</p>\n<p><img src=\"/img/2024-02-17/add-firewall-rule.png\"></p>\n<p>Further down, I can pick port 8080, and that I want to allow TCP. After setting\nup Gerrit, we have success!</p>\n<p><img src=\"/img/2024-02-17/success.png\"></p>\n<hr />\n<p>And that's it! It's just that easy. There's of course a ton of stuff I didn't\ntalk about, but easy things should be easy, and for basic usage, it's just that\neasy.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/memory-safety-is-a-red-herring/",
"title": "Memory Safety is a Red Herring",
"description": null,
"url": "https://steveklabnik.com/writing/memory-safety-is-a-red-herring/",
"published": "2023-12-21T00:00:00.000Z",
"updated": "2023-12-21T00:00:00.000Z",
"content": "<p>TL;DR:</p>\n<p>I think that a focus on memory safe languages (MSLs) versus non memory-safe\nlanguages is a bit of a red herring. The actual distinction is slightly bigger\nthan that: languages which have defined behavior by default, with a superset\nwhere undefined behavior is possible, vs languages which allow for undefined\nbehavior anywhere in your program. Memory safety is an important aspect of this,\nbut it is necessary, not sufficient. Rust's marketing has historically focused\non memory safety, and I don't think that was a bad choice, but I do wonder\nsometimes. Finally, I wonder about the future of the C++ successor languages in\nthe face of coming legislation around MSLs for government procurement.</p>\n<p>Additionally, I would like to thank ixi for discussing some of this stuff with\nme. I do not claim that they agree with me, these opinions are solely my own!\nBut talking about it helped me form those opinions.</p>\n<hr />\n<h2>ThreadId(1)</h2>\n<p>Today I have been programming in Rust for 11 years. I usually write some sort\nof blog post reflecting on the year. This year, I am reflecting on some old\nquestions:</p>\n<ul>\n<li>Should Rust have focused so much on memory safety in its advocacy?</li>\n<li>Does Rust conflate a more general concept of \"safety\" with \"memory safety?\"</li>\n<li>What do people even mean by \"memory safe languages?\"</li>\n</ul>\n<p>But before we get there, we have to talk about some other things.</p>\n<h2>ThreadId(2)</h2>\n<p>The other day, someone <a href=\"https://news.ycombinator.com/item?id=38616605\">asked an interesting question on Hacker\nNews</a>:</p>\n<blockquote>\n<p>Can someone with a security background enlighten me, on why Python is on the\nlist of \"memory safe\" languages? Most of the python code I have worked with is\na thin wrapper on C. Wouldnt that make python vulnerable as well?</p>\n</blockquote>\n<p>This is a very reasonable question! If you click the link, you'd see my answer:</p>\n<blockquote>\n<p>You are correct that if you call C from Python, you can run into problems. But\nthe fault there lies with the C, not the Python. Pure Python itself is memory\nsafe.</p>\n<p>Because memory safety is built on top of memory unsafety, it is generally\nunderstood that when discussing things at this sort of level, that we are\nspeaking about the Python-only subset. (Same as any other memory safe language.)</p>\n</blockquote>\n<p>But this has been gnawing at me for a while. Like it's one of those things that\n\"everybody knows what I mean.\" But isn't that kind of... bullshit? Like something\nis wrong with your definitions. But on the other hand, if we accept FFI as\nbeing not safe, and that since you can FFI from anywhere, that means it's also\nunsafe, then every single program is unsafe. Your operating system does not\noffer memory-safe APIs. To do anything useful, you must call into the operating\nsystem. This means every program would be infected via this conception of safety,\nand therefore, as a definition, it would be useless.</p>\n<p>If we instead make an exception for a language's runtime, which is allowed to\nmake unsafe calls, but users' code is not, that would draw an appropriate\nboundary: only write code in the guest language, and you don't have to worry\nabout safety anymore. And I think that this sort of definition has been the\ndefault for many years. You have \"managed\" languages which are safe, and you have\n\"unmanaged\" languages which are unsafe. Unmanaged languages are used to implement\nmanaged languages. What's that? A Java runtime written in Java? A Ruby runtime\nwritten in Ruby? Yeah those are weird exceptions. Ignore those. And those\n\"native extensions\" that segfault? Well they aren't written in the host language,\nso of course that will happen, and yeah it's kinda annoying when it happens, but\nit's fine. Sometimes you have to risk it for performance.</p>\n<p>I just don't find this satisfying. It's too vague. You just kinda know it when\nyou see it. Ignore FFI, and we're good. And sometimes, you just can't have safety.\nWhat if you want to write a runtime? You're gonna need to call into the system\ndirectly, and now we're back to unsafe, so just use an unsafe language. It is\nwhat it is.</p>\n<h2>ThreadId(3)</h2>\n<p>That comment is on an article titled <a href=\"https://media.defense.gov/2023/Dec/06/2003352724/-1/-1/0/THE-CASE-FOR-MEMORY-SAFE-ROADMAPS-TLP-CLEAR.PDF\">The Case for Memory Safe\nRoadmaps: Why Both C-Suite Executives and Technical Experts\nNeed to Take Memory Safe Coding Seriously\n</a>.\nThis document, published by <a href=\"https://en.wikipedia.org/wiki/Five_Eyes\">the Five Eyes</a>,\nsuggests... that executives and technical experts need to take memory safety\nseriously. Call it boring if you want, but it is what it says on the tin.</p>\n<p>More specifically, it says:</p>\n<blockquote>\n<p>Memory safe programming languages (MSLs) can eliminate memory safety\nvulnerabilities.</p>\n</blockquote>\n<p>As well as an appendix, \"Memory Safe Languages,\" which describes C#, Go, Java,\nPython, Rust, and Swift as memory safe languages.</p>\n<p>Finally, it says:</p>\n<blockquote>\n<p>Programming languages such as C and C++ are examples of memory unsafe programming\nlanguages that can lead to memory unsafe code and are still among the most widely used\nlanguages today.</p>\n</blockquote>\n<p>But it also acknowledges the reality that sometimes you can get around memory\nsafety:</p>\n<blockquote>\n<p>MSLs, whether they use a garbage collection model or not, will almost\ncertainly need to rely on libraries written in languages such as C and C++.\nAlthough there are efforts to re-write widely used libraries in MSLs, no such\neffort will be able to re-write them all anytime soon.</p>\n<p>For the foreseeable future, most developers will need to work in a hybrid model\nof safe and unsafe programming languages. Developers who start writing in an MSL\nwill need to call C and C++ libraries that are not memory safe. Likewise, there\nare going to be situations where a memory unsafe application needs to call into\na memory safe library. When calling a memory unsafe component or application,\nthe calling application needs to be explicitly aware of—and limit any input\npassed to—the defined memory bounds.</p>\n<p>The memory safety guarantees offered by MSLs are going to be qualified when data\nflows across these boundaries. Other potential challenges include differences\nin data marshalling, error handling, concurrency, debugging, and versioning.</p>\n</blockquote>\n<p>Okay, that's enough quoting from the report. And I swear I'm not about to say\nnice things about it because I am cited as #42, though to be honest it's real\nfucking weird that searching \"klabnik\" on a page on defense.gov returns 1/1.\nBut I think this is an excellent formulation of the critical distinction between\nthe languages on one side of the line versus the other, though they make the\nsame conceptual mistake that I believe Rust may have: it's not just about\nmemory unsafety. \"safety\" itself is basically the idea that most of the time,\nyou're in a universe where everything is peachy keen, and occasionally, you'll\nneed to reach across that line into the scary other place, but you gotta do what\nyou gotta do. But there is a line there. But that line is more abstract than\njust memory safety.</p>\n<h2>ThreadId(4)</h2>\n<p>Rust often describes itself as \"data race free.\" But what the heck is a data\nrace anyway? Here's <a href=\"https://blog.regehr.org/archives/490\">John Regehr</a>:</p>\n<blockquote>\n<p>A data race happens when there are two memory accesses in a program where both:</p>\n<ul>\n<li>target the same location</li>\n<li>are performed concurrently by two threads</li>\n<li>are not reads</li>\n<li>are not synchronization operations</li>\n</ul>\n<p>There are other definitions but this one is fine; it’s from Sebastian Burckhardt at Microsoft Research.</p>\n</blockquote>\n<p>Oh. Well, I guess \"Here's Sebastian Burckhardt:\" instead.</p>\n<p>Point is, data races are undefined behavior in Rust, and they're also undefined\nbehavior in C++. When looking for a succinct citation for that fact, I came\nacross <a href=\"https://www.hboehm.info/c++mm/why_undef.html\">this 2006 post by Hans Boehm</a>.\nIts first line:</p>\n<blockquote>\n<p>Our proposed C++ memory model gives completely undefined semantics to programs\nwith data races.</p>\n</blockquote>\n<p>I believe this was the model that was accepted, so that's the best citation you're\ngonna get from me. Okay, as soon as I read that I said \"come on Steve, that's stupid,\nmake sure it's not just trivial to actually find in the standard\" and lo and\nbehold\n<a href=\"https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2020/n4849.pdf\">n4849</a>\nsays:</p>\n<blockquote>\n<p>The execution of a program contains a data race if it contains two potentially\nconcurrent conflicting actions, at least one of which is not atomic, and\nneither happens before the other, except for the special case for\nsignal handlers described below. Any such data race results in undefined\nbehavior.</p>\n</blockquote>\n<p>Okay, the real citation was easier. The article by Bohm is still fascinating.</p>\n<p>But data races aren't undefined behavior in every programming language. For\nexample, let's consider both Java and Go. Now I hate that I even have to\npreemptively say this, but I am <em>not</em> saying Java and Go are bad. I think\nthese are good decisions for both of the two languages. And I think their\ndecisions here are very informative. Language wars are stupid. Stop it.</p>\n<p>Java was the first time I remember hearing, \"yeah you can have data races but\nlike, it's fine\" about. I kinda lodged that fact in the back of my brain, and\ndidn't think about it much. But lately, I was wondering. So I went and read\nthe Java specification, and more specifically, <a href=\"https://docs.oracle.com/javase/specs/jls/se21/html/jls-17.html\">Chapter\n17</a>. It contains\nthis text:</p>\n<blockquote>\n<p>The semantics of operations other than inter-thread actions, such as reads of\narray lengths (§10.7), executions of checked casts (§5.5, §15.16), and\ninvocations of virtual methods (§15.12), are not directly affected by data\nraces.</p>\n<p>Therefore, a data race cannot cause incorrect behavior such as returning the\nwrong length for an array.</p>\n</blockquote>\n<p>This is morally equivalent to \"not undefined behavior\" even though the\nspecification doesn't define \"undefined behavior\" in the way that the C, C++,\nand Rust specifications do. Notably, the specification also defines \"data race\"\nin a slightly different way:</p>\n<blockquote>\n<p>When a program contains two conflicting accesses (§17.4.1) that are not\nordered by a happens-before relationship, it is said to contain a data race.</p>\n</blockquote>\n<p>Not just any ordering or synchronization primitive, a happens-before relation\nspecifically. Fun! We have three technically-slightly different but basically\nmorally equivalent ways of defining a data race. Let's do a fourth. Go also\nallows data races. Sort of. From <a href=\"https://go.dev/ref/mem\">the Go memory model</a>:</p>\n<blockquote>\n<p>A data race is defined as a write to a memory location happening concurrently\nwith another read or write to that same location, unless all the accesses\ninvolved are atomic data accesses as provided by the sync/atomic package.</p>\n</blockquote>\n<p>Given that Rust provides atomic ordering operations in a submodule of its\nstandard library, Rust and Go agree here, though Go's more specific about naming\nit.</p>\n<p>But they also have this to say:</p>\n<blockquote>\n<p>While programmers should write Go programs without data races, there are\nlimitations to what a Go implementation can do in response to a data race. An\nimplementation may always react to a data race by reporting the race and\nterminating the program. Otherwise, each read of a single-word-sized or\nsub-word-sized memory location must observe a value actually written to that\nlocation (perhaps by a concurrent executing goroutine) and not yet overwritten.</p>\n</blockquote>\n<p>So it's not right, but it's also not exactly UB. It goes on:</p>\n<blockquote>\n<p>These implementation constraints make Go more like Java or JavaScript, in that\nmost races have a limited number of outcomes, and less like C and C++, where the\nmeaning of any program with a race is entirely undefined, and the compiler may\ndo anything at all. Go's approach aims to make errant programs more reliable and\neasier to debug, while still insisting that races are errors and that tools can\ndiagnose and report them.</p>\n</blockquote>\n<p>While a Go program may exhibit what a Rust or C++ program would consider\nundefined behavior, and it does also consider it an error, the consequences are\nvery different. You don't get time travel. You get 998 instead of 1,000.</p>\n<p>(After publication, Edoardo Spadolini reached out to me and we had a productive\nconversation about the above paragraph: while my overall point here is still\nvalid, there are slightly more serious consequences to UB in a Go program than\nslightly incorrect sums. For example, torn writes to a slice can lead to serious\nconsequences, and there are other ways to write code that exploits UB to be\nactively malicious, such as a function with the signature\n<code>func Cast[T any](addr uintptr) *T</code> that works. However, it's also worth mentioning\nthat Go ships with a race detector, available by simply passing <code>-race</code>. For some\ninteresting usage of this at scale, check out <a href=\"https://www.uber.com/blog/dynamic-data-race-detection-in-go-code/\">this post by Uber</a>. At the end of the day, this is still\nvery different than the \"your whole program is invalid\" style of issues you\ncan end up with in C or C++, I am slightly underselling the danger above.)</p>\n<p>Go also provides <a href=\"https://go.dev/ref/spec#Package_unsafe\">an unsafe package</a>:</p>\n<blockquote>\n<p>The built-in package unsafe, known to the compiler and accessible through the\nimport path \"unsafe\", provides facilities for low-level programming including\noperations that violate the type system. A package using unsafe must be vetted\nmanually for type safety and may not be portable.</p>\n</blockquote>\n<p>Oh yeah, I forgot, Java also has <code>sun.misc.unsafe</code>, essentially the same thing.\nThey have actually wanted to remove it for a long time, and some progress has\nbeen made, but it's not quite gone yet. <a href=\"https://blogs.oracle.com/javamagazine/post/the-unsafe-class-unsafe-at-any-speed\">The Unsafe Class: Unsafe at Any\nSpeed</a>\nis a good explanation of the topic.</p>\n<h2>ThreadId(5)</h2>\n<p>One weird thing about suggesting that the borrow checker is only about memory\nsafety, and that memory safety means the absence of data races, is that memory\nsafety is more than just the absence of data races. Consider a problem that\nrequires no threads to become an issue: iterator invalidation. And different\nlanguages have different answers to this issue:</p>\n<ul>\n<li>In C++, it is undefined behavior.</li>\n<li>In Java, some data structures detect this and throw a <code>ConcurrentModificationException</code>.</li>\n<li>In JavaScript, Ruby, and other languages, you often get \"weird\" behavior:</li>\n</ul>\n<pre><code>numbers = [1, 2, 3, 4]\nnumbers.each do |number|\n p number\n numbers.shift(1)\nend\np numbers\n</code></pre>\n<p>prints</p>\n<pre><code>1\n3\n[3, 4]\n</code></pre>\n<p>But you don't get full-blown time-travel undefined behavior. Sort of like this\ncode in Rust:</p>\n<pre><code>fn main() {\n let i = i32::MAX + 1;\n println!(\"{i}\");\n}\n</code></pre>\n<p>Okay in this case because it's all at compile time Rust can detect it:</p>\n<pre><code>error: this arithmetic operation will overflow\n --> src/main.rs:2:13\n |\n2 | let i = i32::MAX + 1;\n | ^^^^^^^^^^^^ attempt to compute `i32::MAX + 1_i32`, which would overflow\n |\n = note: `#[deny(arithmetic_overflow)]` on by default\n</code></pre>\n<p>But in the general case, you won't get a compile error. You may get a panic, you\nmay get <code>-2147483648</code>. It's weird. But it's not full-blown time-travel undefined\nbehavior.</p>\n<p>Anyway, back to iterator invalidation:</p>\n<ul>\n<li>In Rust, it is prevented at compile time.</li>\n</ul>\n<p>This is neat! This is cool! This is good! And it's pretty unique to Rust.</p>\n<p>It is nice when our programs do what we expect, instead of what we don't expect.\nIt is even nicer when, if our programs don't do what we expect, we have some sort\nof waypost, some sort of sign where the problem might be. What \"nice\" means may\nbe a bit different in different languages, but the boundary is the important\npart: we're gonna have to potentially violate the rules somewhere, so at least\ngive us some help when that inevitably goes wrong. Over here, logic bugs may\nhappen, some weird behavior may result, but over there? There be dragons.\nNobody can save you out there. Tread carefully.</p>\n<h2>ThreadId(6)</h2>\n<p>If we think about all of these designs, they all are very similar conceptually:\nsafe code is the default, but you can call into some sort of unsafe facility.\nAnd everyone is very clear on the relationship between the two: while the unsafe\nfacility exists to be used, it must uphold the rules that the safe world relies\non. And that means that safety and unsafety have a super/sub-set relationship.\nIn the core is unsafety. But at some point, we draw a line, and on top of that\nline, safety exists.</p>\n<p>In Rust, we often talk about how unsafe is great, because it clearly draws a\nline around code with which serious bugs may happen. More specifically, while\nmost people describe it as \"where memory safety can originate from\" or similar,\nit's actually slightly broader than that: it's where undefined behavior can\noriginate from.</p>\n<p><a href=\"https://doc.rust-lang.org/reference/behavior-considered-undefined.html\">The nomicon</a>:</p>\n<blockquote>\n<p>Rust code is incorrect if it exhibits any of the behaviors in the following\nlist. This includes code within unsafe blocks and unsafe functions. unsafe\nonly means that avoiding undefined behavior is on the programmer; it does not\nchange anything about the fact that Rust programs must never cause undefined\nbehavior.</p>\n<p>It is the programmer's responsibility when writing unsafe code to ensure that\nany safe code interacting with the unsafe code cannot trigger these behaviors.\nunsafe code that satisfies this property for any safe client is called sound;\nif unsafe code can be misused by safe code to exhibit undefined behavior, it\nis unsound.</p>\n</blockquote>\n<p>This is not a new insight, but basically, unsafe is like FFI in a managed\nlanguage. It's not something you do often. But it is something you sometimes\nhave to do. And when you do, you can at least contain the danger: the problem\nhas to lie somewhere behind that veil. You have clear points to begin your\nsearch into where the issues lie, should the abyss claim another soul.</p>\n<p>In some ways I'm also repeating another older slogan Rust had: \"memory safety\nwithout garbage collection.\" But it's more like \"no UB and no garbage collection.\"\nGosh, why didn't we put <em>that</em> on the website? Practically rolls right off the\ntongue. What makes Rust appealing, and I think especially to the \"non-systems\"\ncrowd, is that it shares a property of many managed languages, that of \"no\nundefined behavior by default,\" with a property of many systems languages, that\nof \"no runtime, as little overhead as possible.\" And sure, there's an escape\nhatch, but it's rarely used, and clearly separates the two worlds. This is a\nfundamentally different way of viewing the world than \"unsafe by default.\"</p>\n<p>Another way in which there's an interesting split here in language design\nis between the scripting languages like Ruby, Python, and JavaScript, and\nlanguages like Rust, Go, and Java, is that the boundary in the former is purely\nFFI, while in the latter, there are also unsafe faculties in the host language\nitself, as well as with FFI. In all three cases, they're behind either a specific\npackage or <code>unsafe {}</code>, but they give you some interesting tools that don't\nrequire you to use another language to use. I think this is an under-appreciated\npart of the overall design space.</p>\n<h2>threads.into_iter().for_each(|t| t.join().unwrap());</h2>\n<p>So: did Rust focus too much on memory safety? And what should newer languages\ndo in this area?</p>\n<p>Well, I think that in some sense, the answer is obviously \"no, Rust did not\nfocus too much on memory safety.\" Rust has been wildly successful, reaching\nheights that I only could dream about eleven years ago. Yes, usually I have\ntried to be fairly tempered when talking about Rust's successes, and some\nfolks still try to claim Rust is still super niche. But when I think about the\nways that Rust touched even this blog post, it's staggering: I wrote this post\nin VS: Code, which uses Rust, on Windows, which has Rust in the kernel. I\npublished it to GitHub, which uses Rust for code search. I previewed it in\nChrome, which uses some Rust today and may start having more tomorrow. It's\nhosted on Vercel, which uses Rust in projects like turbopack, though maybe not\nin the critical path for this post. When you read it, the bytes probably passed\nthrough CloudFlare, which uses a ton of Rust, and maybe you're reading this in\nFirefox, which is about 12% Rust. And maybe you're on Asahi Linux, where the\nGPU drivers are written in Rust. Rust has Made It, and is here to stay. And\nthe marketing we used got us here. So in some sense, it would be silly to\ndeclare said messaging a failure.</p>\n<p>However, I also wonder what else could have been. Could Rust have grown faster\nwith a different message? Maybe. That said, what would that message be?\n\"Undefined behavior is scary?\" I'm not sure that's as compelling, even if I think\nit's actually a bigger deal. While memory unsafety is only part of undefined\nbehavior, it is the part that is easy to understand: pointers are the bane of\nevery C newbie. It has consequences that are easy to understand: security\nvulnerabilities are bad. Has Rust <em>really</em> conflated \"memory safety\" and \"safety\"\nthat much? I don't actually think so. People still talk about SQL injection,\nabout Remote Code Execution, about Cross Site Scripting. People don't <em>really</em>\nbelieve that just because it's written in Rust, bugs are impossible. Sure you\ncan find a few comments on the internet as such, but the big organizations that\nare using Rust for meaningful work do not take this position. Given <a href=\"https://en.wikipedia.org/wiki/Amdahl%27s_law\">Amdahl's\nLaw</a>, if 70% of security\nvulnerabilities are related to memory safety, focusing so strongly on it makes\nsense, as fixing that will give you a much larger impact than other things.</p>\n<p>So what should new languages do here? Well, pretty much the only language that\nis gaining developer mindshare that's new and is memory unsafe by default is\nZig. I think Andrew is very sharp, and a tremendous engineer. I am very\ninterested to see how things shake out there. But as a language, it does not\nappeal to me personally, because I think we've sort of moved past \"unsafe by\ndefault.\" That being said, it's also not that Zig is exactly like other\nlanguages in this space either: if we are willing to go beyond \"memory unsafe vs\nmemory safe at compile time\" as a strict binary, and instead look at \"memory\nunsafe at compile time\" itself, we see a pretty big gradient. Zig is doing a\nbunch of things to mitigate issues where possible. Maybe \"safe enough\" truly is\nsafe enough. Time will tell. But beyond that, you also don't have to go full\nRust in the design space. Rust wants to be a language that you can (and at\nOxide, we do) use at the lowest levels of the system. But also memory safe.\nAnd extremely fast. These constraints mean that Rust takes on a lot of\ncomplexity in the type system, where languages that are willing to relax these\nthings wouldn't have to. Here are three languages that I think are sort of\n\"post-Rust\" in a sense, that are learning from Rust but putting their own\ntwist on things:</p>\n<ul>\n<li><a href=\"https://www.hylo-lang.org/\">Hylo</a> (formerly Val) tries to, in their own words,\n\"strengthens [value semantics], while de-emphasizing reference semantics\".\nThis lets them avoid a lot of Rust's complexity. Hylo looks more explicitly\nto Swift than Rust for positive inspiration in many cases, but I'm including\nit here anyway, as I've sort of ended up not talking about Swift in this post.</li>\n<li><a href=\"https://austral-lang.org/\">Austral</a> is \"if affine types are good for systems\nprogramming, linear types are even better.\" Austral also believes that parsimony\nis a virtue in language design, and that linear types can replace a lot of\ndifferent features in other programming languages, while also eschewing other\nfeatures that they deem as not carrying their weight.</li>\n<li><a href=\"https://www.modular.com/max/mojo\">Mojo</a>, \"the usability of Python with the\nperformance of C.\" It also has this sort of \"lower into a more complex language\"\nidea; you can Just Write Python for the most part and it works, but if you want\nto get fancier, you can. <a href=\"https://github.com/modularml/mojo/blob/main/proposals/lifetimes-and-provenance.md\">Lifetimes are a current\nproposal.</a></li>\n</ul>\n<p>I think there are some other languages that are forming the next big cohort of\nprogramming languages, and they're interesting too, but these three I think have\nthe most explicit connection to Rust in the design and goals of the language\nitself. Shout out to Nu, Inko, Gleam, Roc, Oil Shell, Unison, and others. Lots\nof cool stuff going on.</p>\n<h2>int atexit(void (*func)(void))</h2>\n<p>There is one way in which the question \"is the focus on memory unsafety\nmisguided?\" is much more serious than battles in the comment sections of\nvarious fora: the government.</p>\n<p>We talked about this earlier, but various government organizations have been\nslowly putting out recommendations that organizations should be moving towards\nMSLs. And the first reactions to these were always \"well they're not laws so\nthey don't have teeth\" as well as \"laws are inevitably coming.\" To be honest,\nI have mostly been in the former camp, and viewed the later as fearmongering.\nSure, maybe someday, but like, dipping your toes in does not mean that you're\nabout to cannonball moments later.</p>\n<p>But then I got <a href=\"https://news.ycombinator.com/item?id=38700267\">this interesting reply from David Tolnay on Hacker\nNews</a>. In it, he refers me to\n<a href=\"https://adspthepodcast.com/about/\">ADSP: The Podcast</a>, and more specifically,\n<a href=\"https://adspthepodcast.com/2023/12/15/Episode-160.html\">Episode 160: Rust & Safety at Adobe with Sean\nParent</a>. In it, Sean\nmakes reference to two pieces of legislation, <a href=\"https://www.congress.gov/bill/118th-congress/house-bill/2670\">one in the\nUS</a>\nand <a href=\"https://digital-strategy.ec.europa.eu/en/library/cyber-resilience-act\">one in\nthe EU</a>.\nNow, I am not a politican, and these bills are huge, so I wasn't able to figure\nout how these bills do this specifically, but to quote Sean:</p>\n<blockquote>\n<p>The one in the U.S. that's pending basically says that the Department of\nDefense is going to within 270 days of the bill passing (and it's a funding bill\nwhich means it will probably pass late this year - early next year) that the\nDepartment of Defense will establish guidelines around safety and security\nincluding memory safety for software products purchased by Department of\nDefense. The E.U. has a similar wording in a bill that's slowly winding its way\nthrough their channels. I don't have insight into when that will pass. The U.S.\none will almost certainly pass here within a month or two.</p>\n</blockquote>\n<p>This sounds much more realistic, of course. Now, this does not mean that\nC++ is outlawed or something silly like that, but it does mean that using Rust\ncould become a more serious competitive advantage when it comes to selling to\ngovernment: if you don't need an exception for your product, that's an advantage\nover a product which does.</p>\n<p>This all also may go the way of another anecdote Sean talks about in the past:\nwhen a similar bill tried to mandate POSIX compliance. What I'm saying is, go\nlisten to the podcast. And while this still isn't something as wild as \"not\nusing MSLs is illegal,\" it is really shocking to me how quickly things seem to\nbe moving in that vague direction, though of course it will never actually\narrive there. There seems to be at least some sort of thinking that's not\nimmediately lauged out of the room that memory unsafety could be a consumer\nsafety issue in a similar sense that other unsafe materials are a consumer\nsafety issue. Is that right? I'm not sure, but it seems to be where the\nconversation is heading.</p>\n<p>So where does that leave existing widely used memory-unsafe languages? Well,\nthe C++ community has... had a wide variety of discussions on this topic. I\nshould also say I mean \"community\" at large, and the committee feels like it\nhas a big divide as well. Now I should re-iterate that while I pay attention to\nthe C++ standardization process, I am very much an outsider: I have never\nattended a meeting, I have no direct involvement in these things, I just read\na lot. And while historically most of the community wrote Rust off as a novelty,\nI think that at this point, those with their head in the sand about Rust's usage\nare mostly a fringe minority. But the big question that remains is, what to do?</p>\n<p>Two approaches have appeared:</p>\n<ul>\n<li>Improve the safety of C++ in some fashion.</li>\n<li>Create a \"successor language\" that will fix C++'s problems.</li>\n</ul>\n<p>What is interesting to me about these approaches is that they are both good,\nfor different reasons. The former is a sort of harm reduction approach: don't\nlet the perfect be the enemy of the good. <a href=\"https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p3023r1.html\">What can we do today to improve the\nlives of a ton of\npeople?</a>\nBut if you want to move into the realm of an MSL, you have to break backwards\ncompatibility. This is tantamount to just doing #2 in the first place, and so\nsome are cutting straight to that approach. But if we look at it in a slightly\ndifferent way, an interesting thing happens:</p>\n<ul>\n<li>breaking backwards compatibility vs not</li>\n<li>memory safety by default vs not</li>\n</ul>\n<p>This leads to four approaches:</p>\n<ul>\n<li>We do not break backwards compatibility, and we ignore the memory safety\nissue (this is the 'do nothing' option).</li>\n<li>We do not break backwards compatibility, but we try to make things as memory\nsafe as possible given the constraints we have.</li>\n<li>We break backwards compatibility, and we try to make things as memory safe as\npossible but still remain unsafe.</li>\n<li>We break backwards compatibility, and we become memory safe by default.</li>\n</ul>\n<p>The first option is seemingly untenable, but is also where I see C heading.\nThere's seemingly not nearly as much desire for movement in the C space as the\nC++ space, even though I am following what JeanHeyd Meneide is doing and\nappreciate it.</p>\n<p>The second option is how I view at least one of the possible C++ successor\nlanguages, <a href=\"https://www.circle-lang.org/\">Circle</a>. Circle is incredibly\ninteresting and is worth its own post. But in short, it is a C++ compiler that\nalso implements a lot of extensions, both ones that are proposals in the\ncommittee, but also ones that its author is interested in. He has been doing\nlifetime experiments.</p>\n<p>The third option is where the most famous successor languages live:\n<a href=\"https://github.com/carbon-language/carbon-lang\">Carbon</a> and\n<a href=\"https://github.com/hsutter/cppfront\">cpp2</a>. I haven't gotten to spend a ton of\ntime with either of these yet, but they explicitly aim to do to C++ what C++ did\nto C, or what TypeScript did to JavaScript: create a new language for new code,\nwhile allowing you to use the older language for older code. This allows them\nto break backwards compatibility more easily: new code can more easily be\nwritten under the new constraints. This gives them a larger degree of freedom\nto make changes, which may be necessary to move the needle significantly on\nthe safety issue. But to my knowledge, they do not attempt to be memory safe\nby default. They're conceptually similar to Zig in this way.</p>\n<p>But what about a fourth option? What if someone did a TypeScript for C++, but\none that was closer to what Rust does? You might argue that this is basically\njust Rust? (Interestingly, it is also sort of what Zig is with C: you can use\nthe same compiler to compile a mixed codebase, which is close enough in my\nopinion.) What makes me worry about option #3 is that it doesn't pass the\nseeming test that is coming: memory safe by default. I still think moving\nto a #3 style solution is better than staying with #1, but is that enough?\nI don't know, time will tell. I wonder to what degree you can get \"C++ but memory\nsafe\" without just being Rust. As we talked about earlier, most languages that\nare trying to improve on Rust in this space are willing to trade off some core\nconstraints on Rust's design to get them, but C++ and Rust share the same\nconstraints. That said, it would take a tremendous amount of hubris to declare\nthat Rust is the final, ultimate, best language in the \"memory safe without GC\"\nspace. Rust has made a lot of mistakes too. Maybe reflecting on those will make\nfor a good twelfth year post. Someone get started on Rust++, please!</p>\n<p>The reason that today is my Rust-a-versary is that I usually play with a new\nlanguage over Christmas break, and in 2012, that language was Rust. Maybe this\nChristmas, I'll give cpp2 a try.</p>\n<p>In the final minutes before publishing this, I also found another <a href=\"https://www.cisa.gov/sites/default/files/2023-12/CSAC_TAC_Recommendations-Memory-Safety_Final_20231205_508.pdf\">interesting\nreport</a>.\nThis is a report to the CISA director by the Technical Advisory Council on\nhow to engage with the memory safety question. Rust has 17 mentions. I haven't\nread it yet, but I figured I'd include it here.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/updating-buck/",
"title": "Updating Buck",
"description": null,
"url": "https://steveklabnik.com/writing/updating-buck/",
"published": "2023-05-08T00:00:00.000Z",
"updated": "2023-05-08T00:00:00.000Z",
"content": "<p>Hey there! A shorter post today, but I wanted to continue my series on Buck\nby going over some things that have changed since this series started.</p>\n<h2>A series</h2>\n<p>This post is part of a series:</p>\n<ul>\n<li><a href=\"using-buck-to-build-rust-projects\">Using buck to build Rust projects</a></li>\n<li><a href=\"using-cratesio-with-buck\">Using Crates.io with Buck</a></li>\n<li><a href=\"#\">Updating Buck</a> (you are here)</li>\n</ul>\n<h2>Updating Buck</h2>\n<p>Lots of great stuff has been happening since the initial release of buck2, and\nwe'd like to take advantage of that. If we try and update things, though,\nwe'll get (well, you might not, but I did) an error:</p>\n<pre><code>~> cargo +nightly-2023-03-07 install --git https://github.com/facebook/buck2.git buck2\n<snipped>\n Replacing C:\\Users\\steve\\.cargo\\bin\\buck2.exe\nerror: failed to move `C:\\Users\\steve\\.cargo\\bin\\cargo-installYqDCSV\\buck2.exe` to `C:\\Users\\steve\\.cargo\\bin\\buck2.exe`\n\nCaused by:\n Access is denied. (os error 5)\n</code></pre>\n<p>(Why yes, my console prompt has changed...)</p>\n<p>See on Windows, we can't replace a program that's running, while it's running.\nOn other systems, this may work for you. However, it's probably a good idea to\nnot take advantage of this particular feature, because of the underlying\ncause: buck runs a daemon in the background. This daemon is still going. Trying\nto replace it while it's running means you'll still have the old one going\naround in the back, and while that's fine (is it? I don't actually know), best\nto cleanly shut down first. So do this:</p>\n<pre><code>~\\Documents\\GitHub\\buck-rust-hello> buck2 killall\nKilled buck2.exe (4788). C:\\Users\\steve\\.cargo\\bin\\buck2.exe --isolation-dir v2 daemon --dont-daemonize\nKilled buck2.exe (40200). C:\\Users\\steve\\.cargo\\bin\\buck2.exe --isolation-dir v2 daemon --dont-daemonize\n</code></pre>\n<p><code>buck2 killall</code> will kill every instance of the daemon. As you can see, it found\ntwo of mine, and shut them down. As you can see, you want to run this command\nfrom within one of your projects.</p>\n<p>And now an upgrade works.</p>\n<h2>Update Reindeer</h2>\n<p>If you're building Rust code and using Reindeer, like we talked about in the\nlast post, go ahead and grab that too. Lots of good stuff in there:</p>\n<pre><code>~> cargo install --git https://github.com/facebookincubator/reindeer/ reindeer -f\n</code></pre>\n<p>Now that we're all updated, let's fix up our project to make use of the latest\nchanges that are relevant.</p>\n<h2>Update the prelude</h2>\n<p>I no longer need my weird toolchain hacks to get MSVC Rust working. Thank you\ndtolnay! 🙏</p>\n<p>Let's pull in changes to the prelude:</p>\n<pre><code>~\\Documents\\GitHub\\buck-rust-hello> cd prelude\n~\\Documents\\GitHub\\buck-rust-hello\\prelude> git fetch origin\nremote: Enumerating objects: 28, done.\nremote: Counting objects: 100% (28/28), done.\nremote: Compressing objects: 100% (6/6), done.\nremote: Total 17 (delta 12), reused 15 (delta 10), pack-reused 0\nUnpacking objects: 100% (17/17), 3.60 KiB | 102.00 KiB/s, done.\nFrom https://github.com/facebook/buck2-prelude\n 920d3f2..370cd4d main -> origin/main\n~\\Documents\\GitHub\\buck-rust-hello\\prelude> git reset --hard origin/main\nHEAD is now at 370cd4d Http_archive execution platform hack\n~\\Documents\\GitHub\\buck-rust-hello\\prelude> cd ..\n~\\Documents\\GitHub\\buck-rust-hello>\n</code></pre>\n<p>And remove the hack (if you had to do it too) in <code>toolchains\\BUCK</code>:</p>\n<pre><code>system_cxx_toolchain(\n name = \"cxx\",\n visibility = [\"PUBLIC\"],\n)\n</code></pre>\n<p>No more <code>link.exe</code> stuff!</p>\n<h2>Fix some configuration</h2>\n<p>Some good changes have landed in the <code>buck2 init</code> subcommand that we'll want\nto add ourselves.</p>\n<p>First up, we should <a href=\"https://github.com/facebook/buck2/commit/54e1eb5e29eca8a021579b859d0a34c73754eb2a\">add an empty file named <code>.buckroot</code> to the root of our\nrepository</a>.\nBecause buck works with your filesystem hierarchy, it can and will traverse\nupwards looking for things at times. Adding this file ensures that if it does\nso, it will stop before it starts trying to traverse even higher. There's no\nneed for anything in the file, as the contents are ignored.</p>\n<p>I think this kind of change is an interesting way to look at the usability of\nvarious systems. Adding an empty file here is <em>sort of</em> \"more complex\" than\nsay, Cargo. But it's also more explicit. Which means it can be more... I want\nto say \"legible\", if you also read insufferable books like I do sometimes. And\ntherefore, easier. Anyway, more thoughts on all of this in the future.</p>\n<p>Next, we have a change that is demonstrated by this example. Can you guess\nwhat it is?</p>\n<pre><code>~\\Documents\\GitHub\\buck-rust-hello> buck2 run //src/bin:hello_world\nFile changed: root//.git/modules/prelude\nBuild ID: 1a24b418-acdb-4d23-a5e1-6e9b644c01e6\nJobs completed: 83. Time elapsed: 1.9s. Cache hits: 0%. Commands: 3 (cached: 0, remote: 0, local: 3)\n</code></pre>\n<p>There's no reason for buck to be watching our <code>.git</code> directory for changes. And\nthat's why <a href=\"https://github.com/facebook/buck2/commit/2083261102e867bce2b6385b78b598e758b424a8\">it is now put in the default\nconfiguration</a>\nwhen you <code>buck2 init --git</code>. But we've gotta make that change ourselves. Open\nup your <code>.buckconfig</code> and add this at the bottom:</p>\n<pre><code>[project]\nignore = .git\n</code></pre>\n<p>We want it to ignore the <code>.git</code> directory. Seems good.</p>\n<p>... I lied though, there's one more improvement we want to make: we also don't\nwant buck to bother listening to the <code>target</code> directory either, as those files\nare for Cargo's output. So what we <em>actually</em> want is this:</p>\n<pre><code>[project]\nignore = .git, target\n</code></pre>\n<p>After doing that we'll want to <code>buck2 kill</code> to shut the daemon down, so that\nit can pick up our new configuration on the next boot.</p>\n<h2>Regenerating reindeer</h2>\n<p>Since we've got new bugfixes in Reindeer too, let's regenerate our config\nfor our dependencies:</p>\n<pre><code>~\\Documents\\GitHub\\buck-rust-hello> reindeer --third-party-dir third-party buckify\n[WARN reindeer::fixups] semver-1.0.17 has a build script, but I don't know what to do with it: Unresolved build script at ..\\..\\..\\..\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\semver-1.0.17\\build.rs. Dependencies:\n</code></pre>\n<p>We still have to deal with the build script! We didn't talk about the\ncontents of <code>third-party\\BUCK</code> last time, and we won't this time either. If you\nwant to see what's changed, you can take a peek though. One change that we\ndidn't explicitly talk about before, but you may end up noticing, is that it did\nnot generate a target to try and build our build script.</p>\n<p>Let's try it out now:</p>\n<pre><code>~\\Documents\\GitHub\\buck-rust-hello> buck2 run //src/bin:hello_world\nFile changed: root//BUCK\nFile changed: root//third-party/BUCK\nFile changed: root//third-party\nBuild ID: 2eb0c121-d1fb-43d2-b8a4-f923d8dda657\nJobs completed: 20. Time elapsed: 1.1s. Cache hits: 0%. Commands: 2 (cached: 0, remote: 0, local: 2)\n</code></pre>\n<p>Nice. Also note that we aren't seeing any more <code>.git</code> or <code>target</code> changes, not\nthat we've run anything that would inherently change those files, but go ahead,\ninvoke Cargo or git, and then build again. You shouldn't see notifications about\nthose directories anymore.</p>\n<h2>Fixing some breakage</h2>\n<p>Speaking of invoking Cargo, remember how I said this in the last post?</p>\n<blockquote>\n<p>We do have two different Cargo.tomls now. That is a bit of a bummer. But at\nleast it is easy to determine if there’s a problem: dependency failures are\nloud, and if you’re building with both in CI, you’ll notice if stuff goes wrong.\nThere also may be a solution to this I’m just not aware of.</p>\n</blockquote>\n<p>Well...</p>\n<pre><code>~\\Documents\\GitHub\\buck-rust-hello> cargo run\n Compiling hello_world v0.1.0 (C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello)\nerror[E0432]: unresolved import `semver`\n --> src\\bin\\main.rs:1:5\n |\n1 | use semver::{BuildMetadata, Prerelease, Version, VersionReq};\n | ^^^^^^ use of undeclared crate or module `semver`\n\nFor more information about this error, try `rustc --explain E0432`.\nerror: could not compile `hello_world` due to previous error\n</code></pre>\n<p>Going back and re-reading my last post, I did have a <code>cargo add semver</code> in\nthere, so maybe I just forgot to commit that in my last post. Just in case,\nwe'll fix that:</p>\n<pre><code>~\\Documents\\GitHub\\buck-rust-hello> cargo add semver\n Updating crates.io index\n Adding semver v1.0.17 to dependencies.\n Features:\n + std\n - serde\n</code></pre>\n<p>With that, <code>cargo build</code> and <code>cargo run</code> are back in business.</p>\n<p>We also have... well \"breakage\" isn't exactly right, but we have a buck\nconfiguration issue. Let's try to build every target:</p>\n<pre><code>~\\Documents\\GitHub\\buck-rust-hello> buck2 build ...\nError running analysis for `root//:build (prelude//platforms:default#fb50fd37ce946800)`\n\nCaused by:\n 0: Error looking up configured node root//:build (prelude//platforms:default#fb50fd37ce946800)\n 1: `root//src/bin:hello_world` is not visible to `root//:build` (run `buck2 uquery --output-attribute visibility root//src/bin:hello_world` to check the visibility)\nBuild ID: 51657a07-112c-46b4-a2eb-91d60a4b0aed\nJobs completed: 8. Time elapsed: 0.0s.\nBUILD FAILED\n</code></pre>\n<p>We didn't declare that our binary was visible anywhere, and so when we try and\nbuild it, it isn't happy. We do want this to be public, so change <code>src\\bin\\BUCK</code>\nby adding this <code>visibility</code> line near the end. It should look like this:</p>\n<pre><code>rust_binary(\n name = \"hello_world\",\n srcs = [\"main.rs\"],\n crate_root = \"main.rs\",\n deps = [\n \"//third-party:semver\",\n ],\n visibility = [\"PUBLIC\"],\n)\n</code></pre>\n<p>And now that will work:</p>\n<pre><code>~\\Documents\\GitHub\\buck-rust-hello> buck2 build ...\nFile changed: root//src/bin/BUCK\nBuild ID: 9da23da4-3f99-4e78-8c58-ae5e2f1facaa\nJobs completed: 25. Time elapsed: 0.6s. Cache hits: 0%. Commands: 3 (cached: 0, remote: 0, local: 3)\nBUILD SUCCEEDED\n</code></pre>\n<p>Okay, now that we've fixed both Cargo <em>and</em> buck... let's make sure that isn't\ngonna happen again. We aren't testing any of this. So it broke. Just like I said\nit was easy to not let break. Sigh.</p>\n<p>We're going to use GitHub Actions because this is already on GitHub. I'm sure\nyou can adapt it to your setup of choice.</p>\n<p>Put this in <code>.github\\workflows\\ci.yml</code>:</p>\n<pre><code>name: CI\n\non:\n pull_request:\n push:\n branches: [main]\n\njobs:\n test:\n name: ${{matrix.name || format('Rust {0}', matrix.rust)}}\n runs-on: ${{matrix.os || 'ubuntu'}}-latest\n strategy:\n fail-fast: false\n matrix:\n include:\n - rust: nightly\n - rust: beta\n - rust: stable\n - rust: 1.69.0\n - name: Cargo on macOS\n rust: nightly\n os: macos\n - name: Cargo on Windows (msvc)\n rust: nightly-x86_64-pc-windows-msvc\n os: windows\n steps:\n - uses: actions/checkout@v3\n - uses: dtolnay/rust-toolchain@master\n with:\n toolchain: ${{matrix.rust}}\n - run: cargo check\n - run: cargo run\n\n buck:\n name: Buck2 on ${{matrix.os == 'ubuntu' && 'Linux' || matrix.os == 'macos' && 'macOS' || matrix.os == 'windows' && 'Windows' || '???'}}\n runs-on: ${{matrix.os}}-latest\n if: github.event_name != 'pull_request'\n strategy:\n matrix:\n os: [ubuntu, macos, windows]\n steps:\n - uses: actions/checkout@v3\n with:\n submodules: true\n - uses: dtolnay/rust-toolchain@stable\n with:\n components: rust-src\n - uses: dtolnay/install-buck2@latest\n - run: buck2 build ...\n - run: buck2 run //src/bin:hello_world\n\n - uses: actions/cache/restore@v3\n id: cache\n with:\n path: ~/.cargo/bin/reindeer${{matrix.os == 'windows' && '.exe' || ''}}\n key: ${{matrix.os}}-reindeer\n\n - run: cargo install --git https://github.com/facebookincubator/reindeer reindeer\n if: steps.cache.outputs.cache-hit != 'true'\n\n - uses: actions/cache/save@v3\n if: steps.cache.outputs.cache-hit != 'true'\n with:\n path: ~/.cargo/bin/reindeer${{matrix.os == 'windows' && '.exe' || ''}}\n key: ${{steps.cache.outputs.cache-primary-key}}\n\n - run: reindeer buckify\n working-directory: third-party\n if: matrix.os == 'ubuntu'\n - name: Check reindeer-generated BUCK file up to date\n run: git diff --exit-code\n if: matrix.os == 'ubuntu'\n</code></pre>\n<p>This is... you guessed it, based off of <a href=\"https://github.com/dtolnay/cxx/blob/master/.github/workflows/ci.yml\">dtolnay's CI for\ncxx</a>.\nWhat can I say, he writes good code. This version is a bit stripped down, since\nthis is primarily a project for showing off a build system, rather than a\nregular project. This has:</p>\n<ul>\n<li>Builds with Cargo on Linux against nightly, beta, stable, and 1.69.0 as a sort of MSRV check.</li>\n<li>Builds with Cargo on MacOS and Windows against nightly Rust.</li>\n<li>Buck on Linux, MacOS and Windows against stable Rust.</li>\n<li>On Linux, checks that we have comitted any relevant changes to dependencies.</li>\n</ul>\n<p>This does not have:</p>\n<ul>\n<li>A \"fail if there are any warnings\" build</li>\n<li>A clippy build</li>\n<li>A full matrix of every os/language/build system combination</li>\n</ul>\n<p>or other things you may want out of a build. Again, you'll probably want to\ncustomize this heavily, but this is what I'm going to do here.</p>\n<h2>Wrapping up</h2>\n<p>And with that, we are done! Next post we're going to deal with that build\nscript. And use some crates that have more intense dependencies, and get them\nall working.</p>\n<p>As always, you can <a href=\"https://github.com/steveklabnik/buck-rust-hello/commit/914e3aa7cdc269b2536f8118255eae12bf552f8d\">check out the code</a> at this point if you'd like.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/using-cratesio-with-buck/",
"title": "Using Crates.io with Buck",
"description": null,
"url": "https://steveklabnik.com/writing/using-cratesio-with-buck/",
"published": "2023-04-27T00:00:00.000Z",
"updated": "2023-04-27T00:00:00.000Z",
"content": "<p>In <a href=\"using-buck-to-build-rust-projects\">a previous post</a>, I laid out the basics\nof building a Rust project with <a href=\"https://buck2.build/\">buck2</a>. We compared and\ncontrasted it with Cargo. But what about one of the biggest and best features\nthat Cargo has to offer, the ability to use other Rust packages from crates.io?\nThey don't use buck, so how can we integrate them into our build?</p>\n<h2>A series</h2>\n<p>This post is part of a series:</p>\n<ul>\n<li><a href=\"using-buck-to-build-rust-projects\">Using buck to build Rust projects</a></li>\n<li><a href=\"#\">Using Crates.io with Buck</a> (you are here)</li>\n<li><a href=\"updating-buck\">Updating Buck</a></li>\n</ul>\n<p>This post represents how to do this at the time that this was posted; future\nposts may update or change something that happens here. Here's a hopefully\ncomplete but possibly incomplete list of updates and the posts that talk\nabout it:</p>\n<ul>\n<li><code>build-script-build</code> target is no longer generated, see \"Updating Buck\"</li>\n</ul>\n<h2>Depending on semver</h2>\n<p>Let's use the <code>semver</code> package example as our program. I am choosing this for\na few reasons:</p>\n<ul>\n<li>I used to be involved in maintaining it, and I'm sentimental</li>\n<li>It's authored by dtolnay, who authors many great Rust crates.</li>\n<li>More specifically than that, his excellent <a href=\"https://github.com/dtolnay/cxx/\">cxx library</a> maintains build rules for Cargo, Buck, and Bazel, and examining how he uses Buck in cxx helped me write both the last post and this one. I wanted to make sure to shout that out.</li>\n<li>It's pure Rust (this is easier than something that depends on C, since as I mentioned I'm still having an issue or two with getting my own C toolchain working so far).</li>\n<li>It has one optional dependecy on serde, but no others. Again, this is just easier, not a fundamental limitation.</li>\n</ul>\n<p>Here's the example, from <a href=\"https://crates.io/crates/semver\">the page on crates.io</a>:</p>\n<pre><code>use semver::{BuildMetadata, Prerelease, Version, VersionReq};\n\nfn main() {\n let req = VersionReq::parse(\">=1.2.3, <1.8.0\").unwrap();\n\n // Check whether this requirement matches version 1.2.3-alpha.1 (no)\n let version = Version {\n major: 1,\n minor: 2,\n patch: 3,\n pre: Prerelease::new(\"alpha.1\").unwrap(),\n build: BuildMetadata::EMPTY,\n };\n assert!(!req.matches(&version));\n\n // Check whether it matches 1.3.0 (yes it does)\n let version = Version::parse(\"1.3.0\").unwrap();\n assert!(req.matches(&version));\n}\n</code></pre>\n<p>Let's change <code>src/bin/main.rs</code> to contain this code. What do we need to do\nto add this with Cargo?</p>\n<pre><code>buck-rust-hello〉cargo add semver\n Adding semver v1.0.17 to dependencies.\n Features:\n + std\n - serde\nbuck-rust-hello〉cargo run\n Compiling semver v1.0.17\n Compiling hello_world v0.1.0 (C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello)\n Finished dev [unoptimized + debuginfo] target(s) in 0.71s\n Running `target\\debug\\main.exe`\nbuck-rust-hello〉\n</code></pre>\n<p>Easy enough! We expect no output, becuase the asserts should pass. (Note that\nwe didn't ask for the <code>serde</code> feature, so we aren't using it, and therefore\ndon't depend on anything other than <code>semver</code>.)</p>\n<h2>How Cargo handles this</h2>\n<p>Before we move on, let's talk for a moment about what Cargo actually does here.</p>\n<p>First, <code>cargo add</code> has added this to our <code>Cargo.toml</code>:</p>\n<pre><code>[dependencies]\nsemver = \"1.0.17\"\n</code></pre>\n<p>This is the latest release of semver at the time of this writing.</p>\n<p>When we <code>cargo build</code>, Cargo will figure out all of the crates we need. It will\nthen check the cache for the source code for that crate, and if it's not there,\ndownload it. On my system, this lives here:</p>\n<pre><code> ~\\.cargo\\registry\\src\\github.com-1ecc6299db9ec823\\semver-1.0.17\\\n</code></pre>\n<p>This means we have a <em>global</em> cache of source code.</p>\n<p>Cargo will then compile <code>semver</code>, as you can see in the output above. It places\nthat output in the <code>target</code> directory, more specifically</p>\n<pre><code>.\\target\\debug\\deps\n</code></pre>\n<p>This directory will contain <code>libsemver-ded1559592aad8f7.rlib</code>,\n<code>libsemver-ded1559592aad8f7.rmeta</code>, and <code>semver-ded1559592aad8f7.d</code>. These have\nhashes embedded in them so that if we had multiple versions of <code>semver</code> in our\nproject, they can be disambiguated. If you're not familiar with <code>rustc</code> output:</p>\n<ul>\n<li>rlib files are simlar to archive files, like Unix <code>ar</code>. The details are not standard, and may change at any time. Object code lives in here, as well as other things.</li>\n<li>rmeta files contain crate metadata. This can be used for <code>cargo check</code>, for example.</li>\n<li><code>.d</code> files are a dependency file. This format comes from gcc/make, and is sorta standardized. This is in theory useful for use with other build systems, but we won't be using this today.</li>\n</ul>\n<p>Cargo will then build our project, passing in <code>libsemver-*.rlib</code> as a\ndependency.</p>\n<p>If you're curious about the exact commands and flags, <code>cargo build -v</code> will\nshow that. Make sure to <code>cargo clean</code> first, or else you'll get no output, given\nthat we've already built this project.</p>\n<p>For example, here's the <code>rustc</code> invocation that Cargo makes for building this step:</p>\n<pre><code>rustc --crate-name main --edition=2021 src\\bin\\main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=c398fba113f38e08 --out-dir C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello\\target\\debug\\deps -C incremental=C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello\\target\\debug\\incremental -L dependency=C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello\\target\\debug\\deps --extern hello_world=C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello\\target\\debug\\deps\\libhello_world-c5b898f1ca338cca.rlib --extern semver=C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello\\target\\debug\\deps\\libsemver-ded1559592aad8f7.rlib\n</code></pre>\n<p>Since we are not using Cargo, we need to replace all of that stuff, and get\nBuck to generate that <code>rustc</code> line, or at least, some equivalent of it.</p>\n<p>Let's talk about the various things we need to do:</p>\n<h3>The mismatch</h3>\n<p>Let's start with one thing that is basically the same: <code>buck-out</code> and <code>target</code>\nare both directories in our project that cache the output of our build. Yeah\nthe name and details are different, but we're not going to try and somehow\nunify these, as they're both considered implementaiton details of the\nrespective systems, and trying to get them to share is a lot of work for not\nmuch gain.</p>\n<p>Buck does not have a central registry of packages that we can download code\nfrom.</p>\n<p>Buck is interested in reproducable builds, and therefore, a global cache of\nsource code doesn't make as much sense. You want the code stored locally, with\nyour project. The dreaded (or beloved) vendoring.</p>\n<p>Buck does not understand the crate index, Cargo configuration for a given\npackage, and other implementation details. As a more general build system,\nthose are pretty much out of scope.</p>\n<p>Lucikly, other people have done a lot of work here.</p>\n<h3>Reindeer</h3>\n<p>Enter <a href=\"https://github.com/facebookincubator/reindeer/\">Reindeer</a>. Reindeer is a\nproject that will help us bridge this gap. Here's how this will work: Reindeer\nwill create and (mostly) manage a <code>third-party</code> directory for us. It will\ngenerate <code>BUCK</code> files that we can then use to depend on these external crates.\nWe can even choose to vendor our sources or not.</p>\n<p>You can install reindeer through Cargo:</p>\n<pre><code>> cargo install --git https://github.com/facebookincubator/reindeer/ reindeer\n</code></pre>\n<p>Let's set this up. First off, we need to create some files:</p>\n<pre><code>> mkdir third-party\n> code third-party\\Cargo.toml # or vim, or whatever\n</code></pre>\n<p>In that <code>Cargo.toml</code>, we'll need to put this:</p>\n<pre><code>[workspace]\n\n[package]\nname = \"rust-third-party\"\nversion = \"0.0.0\"\npublish = false\nedition = \"2021\"\n\n# Dummy target to keep Cargo happy\n[[bin]]\nname = \"fake\"\npath = \"/dev/null\"\n\n[dependencies]\nsemver = \"1.0.17\"\n</code></pre>\n<p>We're creating a fake package here, so nothing truly matters except the\n<code>[dependencies]</code> section. Here we depend on <code>semver</code> as usual.</p>\n<p>We're also going to configure Reindeer to not vendor our source code, because\nthat's how I prefer to do things. Vendoring was the default behavior, with\nnon-vendoring being added very recently, so if you prefer to vendor, that\nworkflow works very well</p>\n<p>Anyway put this in <code>third-party/reindeer.toml</code>:</p>\n<pre><code>vendor = false\n</code></pre>\n<p>So run this inside of your <code>third-party</code> directory:</p>\n<pre><code>〉cd third-party\n〉reindeer buckify\n[WARN reindeer::fixups] semver-1.0.17 has a build script, but I don't know what to do with it: Unresolved build script at vendor\\semver\\build.rs. Dependencies:\n</code></pre>\n<p>Oh no, build scripts! Yeah I said that I picked <code>semver</code> because it should be easy,\nbut it does have a build script, which is another Cargo-specific feature. Now,\nthe <code>semver</code> crate's build script is used as a feature to support older versions\nof the compiler; all it does is detect old versions and then spit out some\nconfiguration to make sure to not use the newer features of the language. This\nis why this is a warning, not an error; in this case, we don't actually need\nthe build script since we are using a new compiler. So, we are going to skip this\n<em>for now</em>, but we'll come back and fix it.</p>\n<p>At this point, <code>reindeer</code> has generated a <code>BUCK</code> file for <code>semver</code>. Let's see\nwhat targets we have now:</p>\n<pre><code>〉buck2 targets //...\nJobs completed: 4. Time elapsed: 0.0s.\nroot//:build\nroot//src/bin:hello_world\nroot//src/lib:hello_world\nroot//third-party:semver\nroot//third-party:semver-1.0.17\nroot//third-party:semver-1.0.17-build-script-build\n</code></pre>\n<p>We have a few new ones! One for <code>semver</code>, one for the specific version of\n<code>semver</code>, and one for the build script of <code>semver</code>. The general <code>semver</code> is\nan alias for <code>semver-1.0.17</code>.</p>\n<p>Do you know how to modify our build so that buck builds successfully?</p>\n<p>Here's the answer: change <code>src\\bin\\BUCK</code>:</p>\n<pre><code>rust_binary(\n name = \"hello_world\",\n srcs = [\"main.rs\"],\n crate_root = \"main.rs\",\n deps = [\n \"//third-party:semver\",\n ],\n)\n</code></pre>\n<p>And now we can build and run:</p>\n<pre><code>〉buck2 run //src/bin:hello_world\nFile changed: root//src/bin/BUCK\nBuild ID: b18ba58d-8a77-439a-9d95-6051f3cf21d4\nJobs completed: 26. Time elapsed: 1.9s. Cache hits: 0%. Commands: 3 (cached: 0, remote: 0, local: 3)\n</code></pre>\n<p>Success! Our program has no output, if the assertions failed we would have\ngotten something, but this is expected given the example code.</p>\n<p>Now, whenever we need to add or remove a dependency, we can modify\n<code>third-party\\Cargo.toml</code>, re-run the <code>buckify</code> command, and we're good.</p>\n<p>We <em>do</em> have two different <code>Cargo.toml</code>s now. That is a bit of a bummer. But\nat least it is easy to determine if there's a problem: dependency failures are\nloud, and if you're building with both in CI, you'll notice if stuff goes wrong.\nThere also may be a solution to this I'm just not aware of.</p>\n<p>If you'd like to see the whole thing at this point, <a href=\"https://github.com/steveklabnik/buck-rust-hello/commit/2abd1ada7dbbc7f89cd8678eace1e07b3df2ae2f\">this\ncommit</a>\nshould have you covered.</p>\n<p>This should get you going with building more advanced projects in Rust using\nbuck2. In the next post, we'll talk about fixups, which you'll need for\ndependencies that are more complex than <code>semver</code>.</p>\n<p>By the way, if you're interested in this stuff, I've <a href=\"https://discord.gg/ZTEmwypZ6K\">made a Discord for buck\nfans</a>. Come hang out, chat about stuff, ask\nquestions, it's gonna be a good time.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/using-buck-to-build-rust-projects/",
"title": "Using buck to build Rust projects",
"description": null,
"url": "https://steveklabnik.com/writing/using-buck-to-build-rust-projects/",
"published": "2023-04-13T00:00:00.000Z",
"updated": "2023-04-13T00:00:00.000Z",
"content": "<p>A few days ago, Facebook/Meta/idk <a href=\"https://engineering.fb.com/2023/04/06/open-source/buck2-open-source-large-scale-build-system/\">announced that buck2 is now open\nsource</a>.</p>\n<blockquote>\n<p>Buck2 is an extensible and performant build system written in Rust and\ndesigned to make your build experience faster and more efficient.</p>\n</blockquote>\n<p>As it just so happens, I have taken an increasing interest in build systems\nlately. I have mostly ignored the cool kids' build system things, because\nI have never worked at a FAANG, or at a true monorepo shop. I also personally\ntry and avoid the JVM wherever possible, and the first generation of these tools\nwere all built on top of it. (Yes, that bias may be outdated, I'm not saying\n<em>you</em> should avoid the JVM, just stating my own bias up front.)</p>\n<p>So this timing was perfect! Let's explore what using buck looks like.</p>\n<blockquote>\n<p>A brief aside: listen, I have no love for Facebook. I <a href=\"im-deleting-my-facebook-tonight\">deleted my\naccount</a> almost a dozen years ago. That\ndoesn't mean I'm not interested in using good tools they produce. If you\nfeel differently, fine, but that's not what I want to talk about today, so\nI'm not going to.</p>\n</blockquote>\n<p>Oh, one last bit before I begin: I'm not going to do a lot of motivating on\n\"why would you want to use Buck?\" in this post. There's a few reasons for that,\nbut for now, if that's what you're looking for, this post isn't it. We're doing\nthis purely for the novelty of trying out some new tech right now. I will\nprobably end up with a post giving better motivations at some point in the\nfuture, but I think it makes more sense once you see how it works, rather than\nstarting there.</p>\n<h2>A series</h2>\n<p>This post is part of a series:</p>\n<ul>\n<li><a href=\"#\">Using buck to build Rust projects</a> you are here</li>\n<li><a href=\"using-cratesio-with-buck\">Using Crates.io with Buck</a></li>\n<li><a href=\"updating-buck\">Updating Buck</a></li>\n</ul>\n<p>This post represents how to do this at the time that this was posted; future\nposts may update or change something that happens here. Here's a hopefully\ncomplete but possibly incomplete list of updates and the posts that talk\nabout it:</p>\n<ul>\n<li><code>buck2 init</code> also creates a file named <code>.buckroot</code>, see \"Updating Buck\"</li>\n</ul>\n<h2>Getting started with buck2</h2>\n<p><a href=\"https://buck2.build/docs/getting_started/\">The Getting Started page</a> will give\nyou instructions on installing buck. As of this moment, the instructions are:</p>\n<pre><code>$ rustup install nightly-2023-01-24\n$ cargo +nightly-2023-01-24 install --git https://github.com/facebook/buck2.git buck2\n</code></pre>\n<p>This is mega-convenient for me as a Rust user, but probably not if you don't\nhave Rust installed. That said, this is a first release, and so I don't expect\nanything fancier. This is what <code>cargo install</code> is good for!</p>\n<p>Let's make a new directory, <code>hello</code>:</p>\n<pre><code>$ mkdir buck-rust-hello\n$ cd buck-rust-hello\n</code></pre>\n<p>To initialize a project, we use this command:</p>\n<pre><code>$ buck2 init --git\n</code></pre>\n<p>Before we move forward, let's examine what this generated for us.</p>\n<h2>Initial project files</h2>\n<p>We now have this stuff in our directory:</p>\n<pre><code>$ git add .\n$ git status\nOn branch main\n\nNo commits yet\n\nChanges to be committed:\n (use \"git rm --cached <file>...\" to unstage)\n new file: .buckconfig\n new file: .gitignore\n new file: .gitmodules\n new file: BUCK\n new file: prelude\n new file: toolchains/BUCK\n</code></pre>\n<p>Let's talk about each of these in turn.</p>\n<h3><code>.buckconfig</code></h3>\n<p>The <code>.buckconfig</code> file is... a configuration file for Buck, go figure. It looks\nlike this:</p>\n<pre><code>[repositories]\nroot = .\nprelude = prelude\ntoolchains = toolchains\nnone = none\n\n[repository_aliases]\nconfig = prelude\nfbcode = none\nfbsource = none\nbuck = none\n\n[parser]\ntarget_platform_detector_spec = target:root//...->prelude//platforms:default\n</code></pre>\n<p>That <code>none = none</code> is kind of amusing. Regardless of that, this file is\nextremely important: it configures the entire thing. In a sense, it's like\n<code>Cargo.toml</code>: just like a package is defined by the existence of a <code>Cargo.toml</code>,\na <code>.buckconfig</code> defines the existence of a... cell. Which defines a package.\nWe'll get there. Point is, this is the top level configuration. We say that\nthe repository root is in the current directory, we'd like to use the default\nprelude and toolchains.</p>\n<p>I uh... I don't know what the <code>none = none</code> is for. It might just be a bug.\nI haven't seen it in some of the other config files I've poked at. Let's just\nleave that alone for now. I do have a suspicion though... and it involves the\nnext section.</p>\n<p>We also have a table for repository aliases. I couldn't find any documentation\non this, but I would imagine this means we could use the name <code>config</code> instead\nof <code>prelude</code> later. Looks like we don't have any way to refer to <code>fbcode</code> and\n<code>fbsource</code>, which makes sense, and same with <code>buck</code>.</p>\n<p>(I wonder if this is what the <code>none = none</code> is about above, defining a sort of\n\"none repository\" that we can then alias these to.)</p>\n<p>Finally, we have a parser table, with one entry, pointing out where a thing\nexists. I know this configures Buck's parser, but other than that... I'm sure\nI'll figure it out eventually.</p>\n<h3><code>.gitmodules</code> & <code>prelude</code></h3>\n<p>We have a git submodule, pointing to\n<code>https://github.com/facebook/buck2-prelude.git</code>,\nthat lives at the <code>prelude</code> directory. If you poke around in there, you'll\nfind a bunch of <code>.bzl</code> files that implement useful features for us to use.\nWe'll get into those in a moment, but the point is that this is sort of like\na 'standard library' if you will. You could also not use it and define your own.\nIf you're that kind of person.</p>\n<h3><code>.gitignore</code></h3>\n<p>A very simple <code>.gitignore</code> will be created, that contains one line: <code>/buck-out</code>.\nThis is where buck stores artifacts produced by your builds, so we don't want\nthat checked into version control.</p>\n<h3><code>BUCK</code></h3>\n<p>Now we get to the good stuff. Here's the generated <code>BUCK</code> file:</p>\n<pre><code># A list of available rules and their signatures can be found here: https://buck2.build/docs/generated/starlark/prelude/prelude.bzl\n\ngenrule(\n name = \"hello_world\",\n out = \"out.txt\",\n cmd = \"echo BUILT BY BUCK2> $OUT\",\n)\n</code></pre>\n<p><code>genrule</code> is like a function, provided by our prelude. If you're curious, the\nimplementation is in <code>prelude/genrule.bzl</code>. This command, as you may imagine,\nsets up a rule, named <code>hello_world</code>, that produces a file called <code>out.txt</code>.\nIt does this by running the <code>cmd</code>. Nice and straightforward. We'll give that\na try in a moment, but first, one last file:</p>\n<h3><code>toolchains/BUCK</code></h3>\n<p>This file describes a toolchain. Here's the contents:</p>\n<pre><code>load(\"@prelude//toolchains:genrule.bzl\", \"system_genrule_toolchain\")\n\nsystem_genrule_toolchain(\n name = \"genrule\",\n visibility = [\"PUBLIC\"],\n)\n</code></pre>\n<p>This loads a certain rule from the prelude, and then defines this as a public\ntoolchain. We can define as many toolchains as we want here, for example, if\nwe wanted to build both Rust and Python, we could define both toolchains here\nfor later use.</p>\n<p>The \"genrule\" toolchain is used to generate files from a shell command, as we\nsaw before with our rule that produces <code>out.txt</code>. So, in my understanding, here\nwe are defining that we wish to actually use that. And then, in the <code>BUCK</code> file,\nwe're using this toolchain to implement our rule.</p>\n<h2>Invoking our first build</h2>\n<p>Okay, let's actually give this a shot. To instruct Buck to build something, we\ninvoke it with the \"target pattern\" as an argument. Let's ask Buck what targets\nit knows how to build. To do this:</p>\n<pre><code>C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello〉buck2 targets //... 04/10/2023 02:01:50 PM\nBuild ID: cd778a29-2ba4-484b-8956-dc67f6fc0625\nJobs completed: 4. Time elapsed: 0.0s.\nroot//:hello_world\n</code></pre>\n<p>The <code>//...</code> is a \"target pattern.\" The <code>/...</code> means \"all build targets in build\nfiles in subdirectories\", and <code>/</code> means our root directory, so <code>//...</code> means\n\"all targets in all build files in all subdirectories.\" By passing this target\nto <code>buck2 targets</code>, we can see every target in the project. This shows our\none target we've defined, <code>root://:hello_world</code>. This name was defined in our\n<code>BUCK</code> file above. If you change that to</p>\n<pre><code>genrule(\n name = \"lol\",\n</code></pre>\n<p>then <code>buck2 targets //...</code> would show <code>root://:lol</code>.</p>\n<p>Let's actually build our target:</p>\n<pre><code>〉buck2 build //:hello_world\nFile changed: root//BUCK\nBuild ID: 73f4b797-2238-47bc-8e43-7ffcb2b7d9b7\nJobs completed: 36. Time elapsed: 0.0s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)\nBUILD SUCCEEDED\n</code></pre>\n<p>Okay the build succeeded, but where is our <code>out.txt</code>? We can ask buck!</p>\n<pre><code>〉buck2 build //:hello_world --show-output\nBuild ID: 7ce93845-ab1e-4b0a-9274-51fed9f9e295\nJobs completed: 3. Time elapsed: 0.0s.\nBUILD SUCCEEDED\nroot//:hello_world buck-out\\v2\\gen\\root\\fb50fd37ce946800\\__hello_world__\\out\\out.txt\n</code></pre>\n<p>It lives in a deeply nested subdirectory of <code>buck-out</code>, a new top-level directory\nthat was created for us. If you remember from before, this directory is ignored\nin our <code>.gitignore</code>.</p>\n<p>If we look at the file, you can see it contains the text we wanted it to\ncontain.</p>\n<p>Let's build a second time!</p>\n<pre><code>〉buck2 build //:hello_world\nFile changed: root//.git/index.lock\nFile changed: root//.git\nFile changed: root//.git/modules/prelude/index.lock\n31 additional file change events\nBuild ID: c00e4bfa-a1f8-40c7-a61c-2a394dca5da5\nJobs completed: 5. Time elapsed: 0.0s.\nBUILD SUCCEEDED\n</code></pre>\n<p>Buck has noticed that we've changed some files, but since our rule doesn't\ndepend on any of them, we're good to go.</p>\n<h2>Building some Rust code</h2>\n<p>Okay, <code>echo</code> to a file is fun, but let's actually build some Rust. Create a\nfile, <code>hello.rs</code>:</p>\n<pre><code>fn main() {\n println!(\"Hello, world!\");\n}\n</code></pre>\n<p>and then update the <code>BUCK</code> file to this:</p>\n<pre><code>rust_binary(\n name = \"hello_world\",\n srcs = [\"hello.rs\"],\n crate_root = \"hello.rs\",\n)\n</code></pre>\n<p>This says \"hey, we're building a Rust binary, it has this target name, these\nsource files, and the crate root lives here.\" Given we only have one file,\nthere's some reptition. It happens. Let's build:</p>\n<pre><code>〉buck2 build //:hello_world\nFile changed: root//BUCK\nError running analysis for `root//:hello_world (prelude//platforms:default#fb50fd37ce946800)`\n\nCaused by:\n 0: Error looking up configured node root//:hello_world (prelude//platforms:default#fb50fd37ce946800)\n 1: Error looking up configured node toolchains//:cxx (prelude//platforms:default#fb50fd37ce946800) (prelude//platforms:default#fb50fd37ce946800)\n 2: looking up unconfigured target node `toolchains//:cxx`\n 3: Unknown target `cxx` from package `toolchains//`.\n Did you mean one of the 1 targets in toolchains//:BUCK?\nBuild ID: f126ce07-efe8-41d3-8aae-8b7d31a4dafc\nJobs completed: 4. Time elapsed: 0.0s.\nBUILD FAILED\n</code></pre>\n<p>Oops! We didn't set up a rust toolchain! Let's do that now. Edit\n<code>toolchains/BUCK</code>:</p>\n<pre><code>load(\"@prelude//toolchains:rust.bzl\", \"system_rust_toolchain\")\n\nsystem_rust_toolchain(\n name = \"rust\",\n default_edition = \"2021\",\n visibility = [\"PUBLIC\"],\n)\n</code></pre>\n<p>And.... when we build again, the same error. Now. I am not 100% sure what's\ngoing on here, but this is what we need to do:</p>\n<pre><code>load(\"@prelude//toolchains:rust.bzl\", \"system_rust_toolchain\")\nload(\"@prelude//toolchains:genrule.bzl\", \"system_genrule_toolchain\")\nload(\"@prelude//toolchains:cxx.bzl\", \"system_cxx_toolchain\")\nload(\"@prelude//toolchains:python.bzl\", \"system_python_bootstrap_toolchain\")\n\nsystem_genrule_toolchain(\n name = \"genrule\",\n visibility = [\"PUBLIC\"],\n)\n\nsystem_rust_toolchain(\n name = \"rust\",\n default_edition = \"2021\",\n visibility = [\"PUBLIC\"],\n)\n\nsystem_cxx_toolchain(\n name = \"cxx\",\n visibility = [\"PUBLIC\"],\n)\n\nsystem_python_bootstrap_toolchain(\n name = \"python_bootstrap\",\n visibility = [\"PUBLIC\"],\n)\n</code></pre>\n<p>I <em>believe</em> that this is because, to compile the Rust compiler, we need\nPython and a C++ compiler. Well, I <em>did</em> believe that, but after digging into\nthings some more, it's that the Rust toolchain from the prelude depends on\nthe CXX toolchain in the prelude, because the Rust toolchain invokes the C\ncompiler to invoke the linker. I'm still not 100% sure why Python needs to be\nin there. Anyway.</p>\n<p>Now, when I run, I got this:</p>\n<pre><code>〉buck2 build //:hello_world -v 3\nAction failed: prelude//python_bootstrap/tools:win_python_wrapper (symlinked_dir resources)\nInternal error: symlink(original=../../../../../../../../../prelude/python_bootstrap/tools/win_python_wrapper.bat, link=C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello\\buck-out\\v2\\gen\\prelude\\fb50fd37ce946800\\python_bootstrap\\tools\\__win_python_wrapper__\\resources\\win_python_wrapper.bat): A required privilege is not held by the client. (os error 1314)\nBuild ID: 57a66885-f7e7-474b-a782-b49fc4425be9\nJobs completed: 14. Time elapsed: 0.0s.\nBUILD FAILED\nFailed to build 'prelude//python_bootstrap/tools:win_python_wrapper (prelude//platforms:default#fb50fd37ce946800)'\n</code></pre>\n<p>I got this becuase I'm on Windows, and Windows restricts the ability to create\nsymlinks by default. Turning on \"Developer Mode\" (which I'm surprised that I\nhaven't had to turn on so far yet), I get further:</p>\n<pre><code><whole bunch of output>\n = note: 'clang++' is not recognized as an internal or external command,\n operable program or batch file.\n</code></pre>\n<p>Here's that \"invokes the compiler to get the linker\" thing I was referring to\nabove.</p>\n<p>Now... by default, the Rust support is for the GNU version of the Windows\ntoolchain. I never use that. Upstream has said that they want everything\nto be supported, so that change will come at some point, maybe by the time\nyou read this! But in the meantime, I could get my (pure Rust) projects\nbuilding with two small patches:</p>\n<pre><code>diff --git a/prelude/toolchains/cxx.bzl b/prelude/toolchains/cxx.bzl\nindex c57b7b8..dc14ca7 100644\n--- a/prelude/toolchains/cxx.bzl\n+++ b/prelude/toolchains/cxx.bzl\n@@ -39,7 +39,7 @@ def _system_cxx_toolchain_impl(ctx):\n CxxToolchainInfo(\n mk_comp_db = ctx.attrs.make_comp_db,\n linker_info = LinkerInfo(\n- linker = RunInfo(args = [\"clang++\"]),\n+ linker = RunInfo(args = [\"link\"]),\n linker_flags = [\"-fuse-ld=lld\"] + ctx.attrs.link_flags,\n archiver = RunInfo(args = [\"ar\", \"rcs\"]),\n archiver_type = archiver_type,\ndiff --git a/prelude/toolchains/rust.bzl b/prelude/toolchains/rust.bzl\nindex 8172090..4545d55 100644\n--- a/prelude/toolchains/rust.bzl\n+++ b/prelude/toolchains/rust.bzl\n@@ -23,7 +23,7 @@ _DEFAULT_TRIPLE = select({\n # default when we're able; but for now buck2 doesn't work with the MSVC\n # toolchain yet.\n \"config//cpu:arm64\": \"aarch64-pc-windows-gnu\",\n- \"config//cpu:x86_64\": \"x86_64-pc-windows-gnu\",\n+ \"config//cpu:x86_64\": \"x86_64-pc-windows-msvc\",\n }),\n })\n</code></pre>\n<p>Now a build works!</p>\n<pre><code>〉buck2 build //:hello_world\nFile changed: root//BUCK\nFile changed: root//.git/index.lock\nFile changed: root//.git\n6 additional file change events\nBuild ID: 65fc80aa-7bfa-433a-bfa7-57919147b550\nJobs completed: 65. Time elapsed: 1.0s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)\nBUILD SUCCEEDED\n</code></pre>\n<p>We can run it to see the output:</p>\n<pre><code>〉buck2 run //:hello_world\nBuild ID: 78b0ca23-2c7c-4c02-a161-bba15e3b38bd\nJobs completed: 3. Time elapsed: 0.0s.\nhello world\n</code></pre>\n<p>Same idea as <code>cargo run</code>.</p>\n<p>Speaking of <code>cargo run</code>, what might this look like with Cargo? Well, we can create\na <code>Cargo.toml</code>:</p>\n<pre><code>[package]\nname = \"hello_world\"\nversion = \"0.1.0\"\nedition = \"2021\"\n\n[[bin]]\nname = \"hello_world\"\npath = \"hello.rs\"\n</code></pre>\n<p>and try it out. Oh, and you'll probably want to put <code>target</code> into your\n<code>.gitignore</code>.</p>\n<p>Let's build. The \"benchmark\" command in nushell is sort of like <code>time</code> on a\nUNIX system:</p>\n<pre><code>〉benchmark { cargo build }\n Compiling hello_world v0.1.0 (C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello)\n Finished dev [unoptimized + debuginfo] target(s) in 0.34s\n416ms 490us 100ns\n〉benchmark { cargo build }\n Finished dev [unoptimized + debuginfo] target(s) in 0.00s\n77ms 317us 200ns\n</code></pre>\n<p>Not too bad, a bit under half a second for the initial build, and near immediate\non a subsequent build. What about buck?</p>\n<pre><code>〉benchmark { buck2 build //:hello_world -v 3 }\nRunning action: <snip>\nBuild ID: 47ebd9f1-3394-4f72-a0fb-02c936035d2b\nJobs completed: 58. Time elapsed: 0.8s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)\nBUILD SUCCEEDED\n945ms 557us\n〉benchmark { buck2 build //:hello_world -v 3 }\nBuild ID: 5eed24e8-a540-454f-861a-855464aba3c9\nJobs completed: 3. Time elapsed: 0.0s.\nBUILD SUCCEEDED\n78ms 274us 100ns\n</code></pre>\n<p>Not too shabby; Buck is a <em>teeny</em> bit slower on the initial build, but when stuff\nis cached, both are the same speed. All of this is fast enough to qualify as\n\"basically instant.\"</p>\n<h2>Adding a library</h2>\n<p>Let's up the complexity a bit, by adding a library that we want to depend on.\nHere's a <code>lib.rs</code>:</p>\n<pre><code>pub fn print_hello() {\n println!(\"Hello, world!\");\n}\n</code></pre>\n<p>We want to change our top-level <code>BUCK</code> to add this:</p>\n<pre><code>+rust_library(\n+ name = \"print_hello\",\n+ srcs = [\"lib.rs\"],\n+ edition = \"2021\",\n+ visibility = [\"PUBLIC\"],\n+)\n+\n rust_binary(\n name = \"hello_world\",\n srcs = [\"hello.rs\"],\n crate_root = \"hello.rs\",\n+ deps = [\n+ \":print_hello\",\n+ ],\n )\n</code></pre>\n<p>Here, we make a new library, <code>print_hello</code>, and then make our binary depend\non it.</p>\n<p>Let's change the code in <code>main.rs</code> to use the library:</p>\n<pre><code>fn main() {\n println!(\"hello world\");\n print_hello::print_hello();\n}\n</code></pre>\n<p>And that's it! Let's examine our targets:</p>\n<pre><code>〉buck2 targets //...\nBuild ID: 4646f2e7-0ea3-4d59-8590-3da0708ce96e\nJobs completed: 4. Time elapsed: 0.0s.\nroot//:hello_world\nroot//:print_hello\n</code></pre>\n<p>They're both there! We can now build one, the other, or everything:</p>\n<pre><code># build everything\n〉buck2 build //...\n# build just the library\n〉buck2 build //:print_hello\n# build 'just' the binary, this will of course end up building the library too\n〉buck2 build //:print_hello\n</code></pre>\n<p>Let's make sure it still prints our output:</p>\n<pre><code>〉buck2 run //:hello_world\nBuild ID: d76c80fb-dd77-463a-86a1-b6443cea10f6\nJobs completed: 3. Time elapsed: 0.0s.\nHello, world!\n</code></pre>\n<p>Nice.</p>\n<p>Let's compare that to Cargo. Modify <code>Cargo.toml</code>:</p>\n<pre><code>[lib]\nname = \"print_hello\"\npath = \"lib.rs\"\n</code></pre>\n<p>and build:</p>\n<pre><code>〉cargo run\n Compiling hello_world v0.1.0 (C:\\Users\\steve\\Documents\\GitHub\\buck-rust-hello)\n Finished dev [unoptimized + debuginfo] target(s) in 0.42s\n Running `target\\debug\\hello_world.exe`\nHello, world!\n</code></pre>\n<p>Nice.</p>\n<h3>more cargo-ish</h3>\n<p>Before we move forward, this isn't <em>exactly</em> an apples to apples comparison; we've been doing a\nlot of configuration for Cargo that we normally wouldn't have to do, and also,\nwhat if you already have a Cargo project, but you want to try out Buck with it?</p>\n<p>Do this:</p>\n<pre><code>> mkdir src\n> mv hello.rs src/main.rs\n> mv lib.rs src/main.rs\n</code></pre>\n<p>And delete the configuration from <code>Cargo.toml</code>, leaving just the <code>package</code> table.\nFinally, we need to change <code>src/main.rs</code>, given that we're using the default\ncrate name for the library crate, which is <code>hello_world</code> and not <code>print_hello</code>:</p>\n<pre><code>fn main() {\n hello_world::print_hello();\n}\n</code></pre>\n<p>After this, <code>cargo build</code> works just fine. But what about Buck?</p>\n<p>So. We have a bit of weirdness here, and I'm not sure if it's actually\nwork-around-able in Buck or not, since I'm still learning this myself. But\nif we do the basic translation, we'll get an error. Let's try it. This is\nhow you modify the <code>BUCK</code> file:</p>\n<pre><code>rust_library(\n name = \"hello_world\",\n srcs = glob([\"src/**/*.rs\"]),\n edition = \"2021\",\n visibility = [\"PUBLIC\"],\n)\n\nrust_binary(\n name = \"hello_world\",\n srcs = [\"src/main.rs\"],\n crate_root = \"src/main.rs\",\n deps = [\n \":print_hello\",\n ],\n)\n</code></pre>\n<p>Cargo produces a binary and a library, both called <code>hello_world</code>, but buck\ndoesn't like that:</p>\n<pre><code>〉buck2 run //:hello_world\nError evaluating build file: `root//:BUCK`\n\nCaused by:\n Traceback (most recent call last):\n * BUCK:8, in <module>\n rust_binary(\n error: Attempted to register target root//:hello_world twice, re-run the command with `--target-call-stacks` to obtain a call stack of the first registration\n --> BUCK:8:1\n |\n 8 | / rust_binary(\n 9 | | name = \"hello_world\",\n 10 | | srcs = [\"src/main.rs\"],\n 11 | | crate_root = \"src/main.rs\",\n 12 | | deps = [\n 13 | | \":print_hello\",\n 14 | | ],\n 15 | | )\n | |_^\n |\n\nBuild ID: d6a8925d-0180-4308-bcb9-fbc888e7eca1\nJobs completed: 4. Time elapsed: 0.0s.\nBUILD FAILED\n</code></pre>\n<p>It's like hey! You have two targets named <code>hello_world</code>! That's confusing!\nIt also reveals a difference between Buck and Cargo. With Cargo, if you remember\nour configuration, we had to point it to the crate root. From there, Cargo just\nleans on <code>rustc</code> to load up all of the other files that may be required if you\nhave a bunch of modules. But with Buck, we need to tell it up front which files we use.\nSo as you can see above:</p>\n<pre><code> srcs = glob([\"src/**/*.rs\"]),\n</code></pre>\n<p>We can use the <code>glob</code> command to glob up all of our files, which is nice, but\nit's... it's actually wrong. We want to glob everything <em>except</em> <code>main.rs</code>.\nIf <code>main.rs</code> were to change, this would try and re-build both the binary and\nthe library, in my understanding. So that's annoying.</p>\n<p>It's not just annoying for Buck, though. Having both a <code>src/main.rs</code> and a\n<code>src/lib.rs</code> has led to so much confusion from beginners over the years. At some\npoint, someone puts <code>mod lib;</code> into <code>src/main.rs</code> and everything goes to hell.\nThe original intention of this layout, to make simple things simple, is a good\nidea, but I think that sadly, we missed the mark here. Luckily, Cargo also\nsupports a <code>bin</code> directory:</p>\n<pre><code>> mkdir src/bin\n> mv src/main.rs src/bin\n</code></pre>\n<p>We can still <code>cargo run</code> and <code>cargo build</code> and this all works just fine.\nThis doesn't fix our glob issue, though, because <code>src/bin</code> is still inside\nof <code>src</code>.</p>\n<p>I <em>think</em>, in the abstract, I'd prefer a layout like <code>src/{lib,bin}</code>. You want\nthings to not really be nested. So let's do that. Both Cargo and Buck can\nhandle it! It's just not as nice as being purely default in Cargo, since that\nconvention is so strong.</p>\n<pre><code># if you didn't do this above\n> mkdir src/bin\n> mv src/main.rs src/bin\n\n> mkdir src/lib\n> mv src/lib.rs src/lib/print_hello.rs\n</code></pre>\n<p>We have to change <code>src/bin/main.rs</code> to use <code>hello_world</code> again</p>\n<pre><code>fn main() {\n hello_world::print_hello();\n}\n</code></pre>\n<p>And we have to re-add some configuration into <code>Cargo.toml</code>:</p>\n<pre><code>[lib]\npath = \"src/lib/print_hello.rs\"\n</code></pre>\n<p>Everything should build just fine. But what about Buck?</p>\n<p>So, once you start getting into subdirectories, you can also start using\nmultiple <code>BUCK</code> files. So we can empty out our root <code>BUCK</code> file (I'm leaving it\nexisting but empty, if you want to delete it you can but you'll recreate it in\nthe next part anyway), and create two new ones. Here's <code>src/lib/BUCK</code>:</p>\n<pre><code>rust_library(\n name = \"hello_world\",\n srcs = glob([\"**/*.rs\"]),\n crate_root = \"print_hello.rs\",\n edition = \"2021\",\n visibility = [\"PUBLIC\"],\n)\n</code></pre>\n<p>and <code>src/bin/BUCK</code>:</p>\n<pre><code>rust_binary(\n name = \"hello_world\",\n srcs = [\"main.rs\"],\n crate_root = \"main.rs\",\n deps = [\n \":hello_world\",\n ],\n)\n</code></pre>\n<p>We added in a <code>crate_root</code> to the library as well. Okay, let's try this:</p>\n<pre><code>〉buck2 run //:hello_world\nFile changed: root//src/lib/BUCK\nUnknown target `hello_world` from package `root//`.\nDid you mean one of the 0 targets in root//:BUCK?\nBuild ID: d5059fc9-8001-47c4-ba5a-6ba605a4182c\nJobs completed: 2. Time elapsed: 0.0s.\nBUILD FAILED\n</code></pre>\n<p>Oops! Since we moved files around, the names of our targets have changed. Let's\nexamine them:</p>\n<pre><code>〉buck2 targets //...\nBuild ID: c4165964-cb87-49b4-8afe-4a3fc2c526bc\nJobs completed: 4. Time elapsed: 0.0s.\nroot//src/bin:hello_world\nroot//src/lib:hello_world\n</code></pre>\n<p>We had only seen very basic target patterns, but this is enough to show off:</p>\n<p><code>root/src/bin:hello_world</code></p>\n<p>can be read as</p>\n<blockquote>\n<p>The \"hello_world\" target defined in <code>/src/bin/BUCK</code>.</p>\n</blockquote>\n<p>Our target names changing also means we made one mistake in our new <code>BUCK</code> files.\nLet's give it a try:</p>\n<pre><code>〉buck2 run //src/bin:hello_world\nError running analysis for `root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)`\n\nCaused by:\n 0: Error looking up configured node root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)\n 1: Cyclic computation detected when computing key `root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)`, which forms a cycle in computation chain: `root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800),root//src/bin:hello_world (prelude//platforms:default#fb50fd37ce946800)`\nBuild ID: 930ab541-c2dd-44f5-aef1-f6658a2b7c53\nJobs completed: 2. Time elapsed: 0.0s.\nBUILD FAILED\n</code></pre>\n<p>Right. Our binary depends on <code>:hello_world</code>, which it is itself named <code>hello_world</code>,\nso there's a problem. But that's just it, we don't want to depend on any old\n<code>hello_world</code>, we want to depend on our libary. Can you write out the target\npattern that should go in <code>src/bin/BUCK</code>?</p>\n<p>It looks like this:</p>\n<pre><code> deps = [\n \"//src/lib:hello_world\",\n ],\n</code></pre>\n<p>\"The <code>hello_world</code> target in <code>/src/lib/BUCK</code>. Perfect. And now it works!</p>\n<pre><code>〉buck2 run //src/bin:hello_world\nFile changed: root//src/bin/BUCK\nBuild ID: c6d2fdaa-298a-425a-9091-d3f6b38c4336\nJobs completed: 12. Time elapsed: 0.5s. Cache hits: 0%. Commands: 1 (cached: 0, remote: 0, local: 1)\nHello, world!\n</code></pre>\n<p>It kinda stinks to have to type all of that out. Luckily, Buck supports aliases\nfor target patterns. Take our top-level <code>BUCK</code> file, and add this:</p>\n<pre><code>alias(\n name = \"build\",\n actual = \"//src/bin:hello_world\",\n visibility = [\"PUBLIC\"],\n)\n</code></pre>\n<p>And now we can use it:</p>\n<pre><code>〉buck2 build\nBuild ID: a87ed1e2-cfab-47b0-830e-407217997fd7\nJobs completed: 2. Time elapsed: 0.0s.\nBUILD SUCCEEDED\n</code></pre>\n<p>Fun.</p>\n<p>Okay! This is getting a bit long, so let's end there. We have more to learn\nbefore buck can actually replace Cargo in our normal development workflow, but\nI hope this helped you see how you could get started with Buck if you wanted to.</p>\n<p>If you want to check out this on your own, I've published this on GitHub:\n<a href=\"https://github.com/steveklabnik/buck-rust-hello/tree/024ef54ba45627e87a65aaf2f69c6661198c336c\">https://github.com/steveklabnik/buck-rust-hello/tree/024ef54ba45627e87a65aaf2f69c6661198c336c</a></p>\n<p>Next up, we'll be tackling other features, like \"using crates from crates.io.\"\nNo promises on when that'll get published though!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rusts-golden-rule/",
"title": "Rust's Golden Rule",
"description": null,
"url": "https://steveklabnik.com/writing/rusts-golden-rule/",
"published": "2023-03-27T00:00:00.000Z",
"updated": "2023-03-27T00:00:00.000Z",
"content": "<p>I find myself thinking about a particular design principle of Rust today. I'm\nnot sure I've ever seen it named specifically before, but it gets referred to\nfrom time to time, and I think it's an under-rated but very important aspect\nof why Rust works so well. I was going to refer to it as \"the signature is the\ncontract\" today, but then I ended up changing it. Regardless of that, if someone\nelse has already written this out somewhere, and used a different name, please\nlet me know!</p>\n<p>Magic: the Gathering is a really interesting project. I say \"project\" rather\nthan \"card game\" because while it is a card game, it also pioneered a whole\nbunch of incedental other things that had big effects on related hobbies.</p>\n<p>I learned MtG in the late 90s. The rules were a bit different then, but many\nof them are the same. The very first rule I was taught was sort of the \"Magic\nGolden Rule,\" though in today's Comprehensive Rulebook, there are four of them.\nThis one is still the first, though:</p>\n<blockquote>\n<ol>\n<li>The Magic Golden Rules</li>\n</ol>\n<p>101.1. Whenever a card’s text directly contradicts these rules, the card takes\nprecedence. The card overrides only the rule that applies to that specific\nsituation. The only exception is that a player can concede the game at any\ntime (see rule 104.3a).</p>\n</blockquote>\n<p>This rule is the most important rule because it kind of creates the spaces of\npossibilities for cards: many cards exist to tweak, modify, or break some sort\nof fundamental rule.</p>\n<p>That being said, all these years later, this idea is so pervasive in games like\nthis that it's barely even considered an actual rule. It's just part of the\nphysics of the genre, it's how these games work. Yet it's critical to the\nentire enterprise.</p>\n<p>Rust also has a rule. It's kinda funny, because in some senses, this rule\nis almost the opposite of Magic's, if you can even stretch the comparison this\nfar. Here it is:</p>\n<blockquote>\n<p>Whenever the body of a function contradicts the function's signature, the\nsignature takes precedence; the signature is right and the body is wrong.</p>\n</blockquote>\n<p>This rule is also so pervasive in Rust that we take it for granted, but it\nis really, truly important. I think it is also important for Rust users to\ninternalize the implications of this rule, so that they know why certain things\nwork the way that they do.</p>\n<p>Here is the most famous implication of this rule: Rust does not infer function\nsignatures. If it did, changing the body of the function would change its\nsignature. While this is convenient in the small, it has massive ramifications.</p>\n<p>Consider this example program:</p>\n<pre><code>fn foo(x: i32) -> i32 {\n dbg!(x);\n \n x\n}\n</code></pre>\n<p>This function prints out <code>x</code>, and then returns it. Nothing fancy going on here,\nthis is just random stuff to make an example. This compiles just fine. But let's\nimagine that we have a version of Rust that infers our signatures. So we could\ntype this instead:</p>\n<pre><code>fn foo(x) {\n</code></pre>\n<p>This is what a Ruby-ish Rust might look like; we declare the name of our\nargument but not its type, and we don't declare a return type. Now, let's\ndo a small refactoring, we're gonna comment out the final value there:</p>\n<pre><code>fn foo(x) {\n dbg!(x);\n \n //x\n}\n</code></pre>\n<p>the final expression has changed; it's no longer <code>x</code>, but instead is <code>()</code>,\nwhich is what <code>dbg!(x);</code> evaluates to. Because of type inference, the inferrred\ntype of <code>foo</code> is now <code>fn(i32) -> ()</code>. Our function typechecks! It's all good,\nright?</p>\n<p>Well, no:</p>\n<pre><code>error[E0369]: cannot add `{integer}` to `()`\n --> src/lib.rs:5:11\n |\n5 | y + 1\n | - ^ - {integer}\n | |\n | ()\n</code></pre>\n<p>Wait, what? We don't have a <code>y + 1</code> anywhere in our code?! Where's that\nerror coming from... <code>src/lib.rs:5</code>. What's at the top of <code>lib.rs</code>?</p>\n<pre><code>mod bar {\n fn baz() -> i32 {\n let y = crate::foo(5);\n \n y + 1\n }\n}\n</code></pre>\n<p>Oh. Some other code was using <code>foo</code>. When its signature changed, we broke\nthis invocation. It's nice that alt-Rust caught this for us, but this error is\n(would be, anyway!) really bad: no longer is it telling us that our code\nis wrong, but instead points to some other code somewhere else we weren't even\nmessing with. Sure, we can go \"hey why is <code>y</code> <code>()</code>?\" and then figure it out,\nbut compare that to today's error:</p>\n<pre><code>error[E0308]: mismatched types\n --> src/lib.rs:9:19\n |\n9 | fn foo(x: i32) -> i32 {\n | --- ^^^ expected `i32`, found `()`\n | |\n | implicitly returns `()` as its body has no tail or `return` expression\n10 | dbg!(x);\n | - help: remove this semicolon to return this value\n\n</code></pre>\n<p>This error is far better: it points out that the body of our function\ncontradicts the signature. It points out what in our body is generating the\nvalue that contradicts the signature. And it doesn't complain about callers.</p>\n<p>So sure, this gets us nicer error messages, but is that really a big deal?\nI think it is, but it's not the only implication here. I'd like to talk about\ntwo more: one that's clearly an advantage, and one that has led to some pain\nthat people would like to resolve. Balance :)</p>\n<p>First, the one that's an advantage. The advantage is modularity. That makes\nsome forms of analysis much more reasonable, or sometimes even possible, as\nopposed to super difficult. Because everything you need for memory safety is\ndescribed in the signature of the function, Rust doesn't need to examine your\nentire program to determine if there's some shenanigans going on elsewhere.\nThis is far, far less work than just checking the signatures of the functions\nyou call. Each function can be checked in isolation, and then assembled\ntogether. This is a very nice property.</p>\n<p>Second, where this leads to some pain. Users have nicknamed this one \"borrow\nsplitting,\" or \"partial borrowing.\" It looks like this:</p>\n<pre><code>struct Point {\n x: i32,\n y: i32, \n}\n\nimpl Point {\n pub fn x_mut(&mut self) -> &mut i32 {\n &mut self.x\n }\n\n pub fn y_mut(&mut self) -> &mut i32 {\n &mut self.y\n }\n}\n</code></pre>\n<p>I find accessors, and especially mutators, to be where this sort of thing pops\nup most often. This is the classic example. The above code is fine, but if we\ntry to do this:</p>\n<pre><code>// this doesn't work\nimpl Point {\n pub fn calculate(&mut self) -> i32 {\n let x = self.x_mut();\n let y = self.y_mut();\n \n // yes I picked multiplication because this looks kinda funny\n *x * *y\n }\n}\n\n// we would call it like this:\nlet answer = p.calculate();\n</code></pre>\n<p>We get this:</p>\n<pre><code>error[E0499]: cannot borrow `*self` as mutable more than once at a time\n --> src/lib.rs:19:17\n |\n18 | let x = self.x_mut();\n | ------------ first mutable borrow occurs here\n19 | let y = self.y_mut();\n | ^^^^^^^^^^^^ second mutable borrow occurs here\n20 | \n21 | *x * *y\n | -- first borrow later used here\n</code></pre>\n<p>However, if we didn't have these accessors, <code>x</code> and <code>y</code> were instead just\npublic, this very similar free function:</p>\n<pre><code>fn calculate(x: &mut i32, y: &mut i32) -> i32 {\n *x * *y\n}\n\n// called like this:\nlet answer = calculate(&mut point.x, &mut point.y);\n</code></pre>\n<p>works just fine. Why? Because of the signatures. This signature:</p>\n<pre><code>pub fn calculate(&mut self) -> i32 {\n</code></pre>\n<p>and these signatures:</p>\n<pre><code>pub fn x_mut(&mut self) -> &mut i32 {\npub fn y_mut(&mut self) -> &mut i32 {\n</code></pre>\n<p>says \"hey, I am going to borrow all of <code>self</code> mutably,\" which implies an\nexclusive reference to <code>self</code>. That the body of <code>calculate</code> borrows two\ndifferent parts of <code>self</code> independently is 100% irrelevant, that's what the\nsignature says! And so rustc looks at this and says \"hey wait a minute,\ncalling <code>x_mut</code> borrows <code>self</code> mutably, and calling <code>y_mut</code> borrows <code>self</code>\nmutably. That's aliasing! Bad programmer!</p>\n<p>Whereas in the second example, this signature:</p>\n<pre><code>fn calculate(x: &mut i32, y: &mut i32) -> i32 {\n</code></pre>\n<p>says \"hey, I have two mutable references, to two different integers.\" And\nat the call site, Rust sees that we're creating two different borrows to two\ndifferent integers, even though they're both part of our <code>point</code>, and so it\nokays things.</p>\n<p>This is kind of a pain in the butt! But what it saves us from is that scary\naction at a distance in our typecheck example. Imagine that Rust somehow\ninferred that the first version was okay, due to the body only being disjoint.\nWhat happens in the future, when we refactor our function, and the borrows need\nto change? That would have the same problem as before; we'd get weird errors\nelsewhere in our code. There have been some proposals over the years to possibly\nfix this pain point, but it's a tough one. Putting \"here's the fields I'm\nborrowing in the body\" into the signature, in order to conform with the Golden\nRule, looks very odd. Maybe something will happen here, maybe not. I have\ncomplicated feelings.</p>\n<p>So beyond error messages and making accessors/mutators awkward, how does this\naffect you day to day? Well, one way is that I find I end up paying more\nattention to signatures and less attention to bodies, when I'm trying to sort\nsomething out. In many cases, what you're doing in the body is irrelevant, I\ncare only about what your signature affords me. This isn't <em>quite</em> as true\nas in some pure functional languages, but it has that vague feel to it. Another\nway, and a slightly bigger one, has to do with the relationships between types\nand TDD, but I think I'm going to save that for its own post.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/you-re-probably-learning-a-technology-in-its-seventh-season-not-its-pilot/",
"title": "You're probably learning a technology in its seventh season, not its pilot",
"description": null,
"url": "https://steveklabnik.com/writing/you-re-probably-learning-a-technology-in-its-seventh-season-not-its-pilot/",
"published": "2023-01-27T00:00:00.000Z",
"updated": "2023-01-27T00:00:00.000Z",
"content": "<p>I first heard this adage in the Rails space. I don't know who to attribute this\nto, but I also never wrote it down. So, time to write it down, and if you know\nof the origin of this aphorism, let me know and I'd love to put it here!</p>\n<p>I <em>think</em> that the first formulation of this sentiment goes something like this:</p>\n<blockquote>\n<p>Rails is like a soap opera. You're getting started with it in a later season.\nThe cast of characters has already been established, there's a lot of history.\nYou kinda just gotta dive in and go with the flow and catch up on older stuff\nat your leasure.</p>\n</blockquote>\n<p>That's obviously a bit long, so I think I'll be using something like this\nin the future:</p>\n<blockquote>\n<p>You're probably learning a technology in its seventh season, not its pilot.</p>\n</blockquote>\n<p>I was thinking about this today because I happened to come across <a href=\"https://www.reddit.com/r/rust/comments/xj2a23/comment/ipd6tcv/\">this reddit post from a few months ago</a>.</p>\n<blockquote>\n<blockquote>\n<p>In fact, there is hardly any analysis of the downsides of Rust’s error model in the first place, which is quite disheartening.</p>\n</blockquote>\n<p>Not really planning to add much to the convo, but back in mid 2019 when I first\nstarted learning and writing Rust, about half the posts I saw here in the\nsubreddit were related to errors, how to do them right, how to minimize the\nboilerplate, and so on and so forth...</p>\n<p>Over the last couple years, the convo has died down because we've all more or\nless agreed that the major pain points around error handling have been solved\nwith a simple rule: thiserror for library code, anyhow for application code.</p>\n<p>There's obvs still more things that can be done around error handling. try is\none such example, along with standardizing thiserror and anyhow so its part of\nthe stdlib.</p>\n<p>But to say there is no discussion about Rust and it's error handling story is\nvery dishonest imo as that discussion was raged for years before I saw it and\nonly recently came to a proper conclusion for the majority of cases, and thus\ntheres now much less talk about it. I even learned 3 different \"defacto standard\nerror libs\" (that are all now defunct and no one cares about) out of a much\nlonger list of them that came and went before them due to better and better\npatterns being being found over the years!</p>\n<p>There was plenty of healthy and constructive discussion on this topic for an\ninsanely long time and just because you didn't see it doesn't mean everyone was\nsatisfied with the error handling story as it is in std from the 1.0 of Rust.\nRust users actually engage with the system constantly, so ofc they'd discuss the\nproblems with it and work towards fixing it! Hell, there's still people that\nprefer things other than thiserror and anyhow and thats part of why those havent\nbeen standardized yet too, and why there still might seem to be \"no progress\" on\nfixing things.</p>\n<p>TL;DR: Rust developers use Rust's error handling literally all the time. Your\nassertion that they do not understand its shortcomings and pain points and\nrefuse to even discuss them is weird given that there's been a multi-years long\nbattle between all kinds of error handling crates that only recently resulted in\na proper and clear winner.</p>\n</blockquote>\n<p>Now, I don't want to take a position on the original thread, or what they have\nto say, or if they actually <em>are</em> falling afoul of this aphorism. But I think\nthat this particular text shows the meaning of what I mean by \"you're in the\nseventh season\" beautifully. Sometimes, when we learn a new technology, we go\n\"wtf, why is this like this?\" And sometimes, that outside perspective is good!\nBut sometimes, it just means that you weren't there for the discussion, you either\ndon't know or aren't in the same contexts, or a variety of other reasons why\nsomething is the way that it is, even if that way is surprising to you at first.</p>\n<p>In that sense, I think this idea is pretty closely related to Chesterton's Fence.\nIt may even just be the same thing, formulated in a slightly different way.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/too-many-words-about-rusts-function-syntax/",
"title": "Too many words about Rust's function syntax",
"description": null,
"url": "https://steveklabnik.com/writing/too-many-words-about-rusts-function-syntax/",
"published": "2023-01-03T00:00:00.000Z",
"updated": "2023-01-03T00:00:00.000Z",
"content": "<p>There's a famous quote by Eleanor Roosevelt:</p>\n<blockquote>\n<p>great minds discuss semantics, average minds discuss syntax and small minds discuss syntax of comments</p>\n</blockquote>\n<p>(Apologies, Phil Wadler.)</p>\n<p>I'd like to preface this post that I do <em>not</em> consider this to be an actual\nproposal to change Rust, even a pre-RFC. I don't think that making changes like\nthis has enough benefits once a language is as established as Rust is. Consider\nthis post more of some musings about some things that there wasn't really time\nto truly consider when Rust was younger, so that maybe someone who's making a\nnew language can learn from those things early enough in its life to do so.</p>\n<p>Also, I am not the first person to think about this, but I have lost the\nprevious post to time. There also may be a newfangled language that has this or\nsimilar syntax already I'm not aware of. If you know of similar previous\nproposals or languages, please let me know so that I can link to them here.</p>\n<p>Anyway, consider this post a \"Steve decided to write up a thing instead of\njust tweeting, it is not particularly well thought out but at least it's down\non paper\" kind of thing. A serious proposal here, even if I thought the idea was\ngood in general, which to repeat again, I do not, would require examining a lot\nmore corner cases than I do in this post.</p>\n<p>With all of that out of the way, let's begin.</p>\n<hr />\n<h2>The fn syntax</h2>\n<p>Rust does a really good job of being orthogonal in many ways. While it isn't a\ntiny language, that things fit together nicely is one of the things that can\nmake it feel a lot smaller, in my opinion. There's one thing, though, that\nis really a bit inconsistent today, and that's the syntax and semantics of\nfunctions.</p>\n<p>Let me explain: you declare a function item (read: define a function) like this:</p>\n<pre><code>fn foo(x: i32) -> i32 {\n todo!()\n}\n</code></pre>\n<p>Simple enough. However, there's a lot more going on for more complex\ndeclarations. <a href=\"https://doc.rust-lang.org/1.66.0/reference/items/functions.html\">Here</a> is the beginning of the current grammar\nof functions:</p>\n<pre><code>Function :\n FunctionQualifiers fn IDENTIFIER GenericParams?\n ( FunctionParameters? )\n FunctionReturnType? WhereClause?\n ( BlockExpression | ; )\n</code></pre>\n<p>I read this like this: \"A function consists of some function qualifiers, fn,\nan identifier, and an optional list of generic parameters. We then have some\noptional function parameters inside of parenthesis, an optional return type,\nan optional where clause, and then a block expression or a semicolon.\"</p>\n<p>This is in sharp contrast to a C-style function definition, which would\nlook like this:</p>\n<pre><code>int foo(int x) {\n // todo\n}\n</code></pre>\n<p>In general, most new languages have tended to settle closer to Rust's syntax\nhere than C's. There are a few reasons, and it's not like Rust invented this\nstyle in the first place. One of the core nice things about Rust style syntax\nis that <code>fn IDENTIFIER</code>, that is, if you want to find out where <code>foo()</code> is\ndefined, grep for <code>fn foo</code> and you can find it pretty quickly. You can't\ndo this as well with the <code>int</code> syntax. Some code styles have you write</p>\n<pre><code>int\nfoo(int x) {\n // todo\n}\n</code></pre>\n<p>Notably, the GNU project uses this style:</p>\n<blockquote>\n<p>It is also important for function definitions to start the name of the\nfunction in column one. This helps people to search for function definitions,\nand may also help certain tools recognize them.</p>\n</blockquote>\n<p>You instead grep for <code>^foo</code>, and you can get a similar benefit.</p>\n<p>So, what's bad about this syntax? Well, there are useful 'modifiers' to\nfunctions, and Rust has added some and may add more in the future. For example,\nif we want <code>foo</code> to be callable in a const context, we'd make it a <code>const fn</code>:</p>\n<pre><code>const fn foo(x: i32) -> i32 {\n todo!()\n}\n</code></pre>\n<p>That's all well and good, we still even retain the \"grep for <code>fn</code>\" idea here,\nbut these are <em>also</em> useful on arbitrary blocks of code:</p>\n<pre><code>let x = const {\n 1 + 2\n};\n</code></pre>\n<p>This is of course a trivial example that's not very useful, but it shows off\nthe syntax here. My point is... this starts to feel a bit... nonuniform. In\nthe function delcaration, we put const at the front, but when it's any other\nblock expression, we put it right before the block. Rust's function bodies\nare similar to blocks in other ways, such as the whole \"evaluates to the final\nexpression\" thing. But they're different when they're being used to describe\nthe body of a function. That's a bit less than ideal.</p>\n<p>We could try the naive thing:</p>\n<pre><code>fn foo(x: i32) -> i32 const {\n todo!()\n}\n</code></pre>\n<p>This is... weird, but it does sort of unify some syntaxes. We now can turn\nany block of code, including a function body, <code>const</code> by adding the <code>const</code>\nright before the opening brace. However... I kinda shudder at <code>i32 const</code>.\nThere's not enough separation between the return type and the modifier to make\nquick sense of things, in my opinion. However, there's another small tweak we\ncould make here. But first, a small divergence into another corner of Rust's\nsyntax: variables.</p>\n<h2>Variables</h2>\n<p>Rust also has a different syntax for variable declaration. Like C, it's similar\nto its function declaration syntax:</p>\n<pre><code>// Rust\nlet x: i32 = 5;\n</code></pre>\n<pre><code>// C\nint x = 5;\n</code></pre>\n<p>We have <code>let</code> instead of <code>fn</code>, we have the <code>name: type</code> instead of <code>type name</code>.\nWe can also declare a variable but not initialize it:</p>\n<pre><code>let x: i32;\n</code></pre>\n<p>Rust will make us initialize it before we use it, but we can declare the\nvariable on its own, and then give it an initial value later.</p>\n<p>But what's this has to do with functions?</p>\n<h2>A marginally better syntax</h2>\n<p>Functions also happen to have a \"declare the signature but not the body\" syntax\nin Rust too, though it's almost exclusively used in an FFI context. Remember\nthe very end of our grammar?</p>\n<blockquote>\n<p>and then a block expression or a semicolon.</p>\n</blockquote>\n<p>If we have a function <code>foo</code> that we're calling through FFI, we can define it\nlike this:</p>\n<pre><code>#[link(name = \"something\")]\nextern {\n fn foo(x: i32) -> i32;\n}\n</code></pre>\n<p>We don't provide a body, but we do provide a semicolon. This is strikingly\nsimilar to the variable syntax. So why not have the regular function definition\nalso follow the variable syntax?</p>\n<pre><code>fn foo(x: i32) -> i32 = {\n todo!()\n}\n</code></pre>\n<p>We've now added one little extra bit to the grammar: a <code>=</code> after the return\ntype, if any. This one little change allows us to unify the rest of the syntax\naround blocks more easily:</p>\n<pre><code>// full signature\nfn foo(x: i32) -> i32 = {\n todo!()\n};\n\n// empty return type\nfn foo(x: i32) = {\n todo!()\n};\n\n// const\nfn foo(x: i32) -> i32 = const {\n todo!()\n};\n</code></pre>\n<p>I happen to <em>really</em> like this. It's a pretty small tweak but I think it\ncleans stuff up nicely.</p>\n<pre><code>let name = value;\nfn name() = value;\n</code></pre>\n<p>Where both could be values or blocks, with <code>let</code> it's most often a value but\nwith <code>fn</code> it's most often a block. The symmetry pleases me.</p>\n<p>(Sharp readers will also notice the introduction of a semicolon. Leaving it\noff would be closer to the old syntax, requring it would be similar to the\nvariable syntax, but then that would diverge from the other item declaration\nsyntax... I don't feel strongly about it but just in case anyone noticed I\nfigured I'd mention that as well, yet another rabbit hole to chase down.)</p>\n<h2>Going too far</h2>\n<p>This of course raises another divergence. But I'm not sure that fixing this one\nis worth it; it starts to get <em>too</em> weird, I think. But maybe I'm wrong. This\none is about the types.</p>\n<p>One of the things that makes <code>name: type</code> syntax nicer than <code>type name</code> syntax\nis that it degrades nicely in the presence of type inference:</p>\n<pre><code>// Rust\nlet x: i32 = 5;\nlet x = 5;\n</code></pre>\n<pre><code>// C23 (https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3007.htm)\nint x = 5;\nauto x = 5;\n</code></pre>\n<p>Rust's function syntax isn't like this though, partially because there's no\ntype inference on function signatures, so there's not a lot of pressure for\nit to degrade nicely. The choice not to offer inference is the right one for\nRust, but for a future language where that's not the case, we could unify\nthem with something like this:</p>\n<pre><code>// explicit type\nfn foo(x): fn(i32) -> i32 = {\n todo!()\n}\n\n// inferred\nfn foo(x) = {\n todo!()\n}\n</code></pre>\n<p>This one... this may be too far afield. And, while Rust doesn't have named\nparameters, and possibly never will, I have no idea how this style of syntax\nwould work with that. There's also some repetition of <code>fn</code> here that might be\nworth trying to remove, but I'm trying to make the smallest possible deltas\nfrom existing syntax here, and <code>fn(i32) -> i32</code> <em>is</em> the type of that function\nin today's syntax.</p>\n<p>What I mostly take away from this part of the exercise is that consistency is\na good goal, but it's not the <em>only</em> goal. Even if Rust didn't allow for\ninference, making its function declaration syntax be like this may have simply\nbeen a bridge too far for wide adoption. Rust already takes a lot of flak for\nbeing weird, and spending some <a href=\"the-language-strangeness-budget\">Strangeness Budget</a> to unify these\nsyntaxes probably wouldn't be the right choice. But maybe I'm too conservative!\nGiven that Rust already takes that flak, maybe a bigger jump would have been\nokay here. Who knows.</p>\n<p>... actually, we have one way to find out, I guess. This is <em>very</em> similar\nto the syntax that Herb Sutter came up with for <a href=\"https://github.com/hsutter/cppfront\">cppfront</a>:</p>\n<pre><code>foo: (x: i32) -> i32 = {\n // todo\n}\n</code></pre>\n<p>There's no <code>fn</code> preceding. The type syntax works a bit differently. But it's\nclose. And Herb is targetting the people who I'd be most worried about hating\nthis syntax: C++ folks. Maybe this idea isn't so farfetched at all. He's\nalso got an example of how making it any expression after the <code>=</code> can be\nnice for one-liner functions. If we applied this to our fake Rust syntax:</p>\n<pre><code>fn main() = println!(\"Hello, world!\");\nfn hello(name): fn(&str) = println!(\"Hello, {name}!\");\n</code></pre>\n<p>That's kinda nice too.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/ten-years-of-ru---ewriting-my-website/",
"title": "Ten Years of Ru...ewriting my website",
"description": null,
"url": "https://steveklabnik.com/writing/ten-years-of-ru---ewriting-my-website/",
"published": "2022-12-21T00:00:00.000Z",
"updated": "2022-12-21T00:00:00.000Z",
"content": "<p>December 21 is the anniversary of when I first heard about Rust way back in\n2012. I used to write yearly posts about it; the last time I did was <a href=\"../six-years-with-rust\">in\n2018</a>. That makes today ten years. I thought I'd have\nsomething big to say here, but... I just don't.</p>\n<p>Part of that is because I haven't written a blog post since late 2020.\nPart of <em>that</em> is well, <em>gestures wildly at everything</em>, but another part of it\nis that the way I had set up my website was cool, but fragile, and I didn't have\nthe time or energy to keep up with it.</p>\n<p>You see, last time I had basically taken a starter project for a new framework\nI had never used before, and just kinda wrote code until it works. That's one\nof the joys of having your own website; you can do whatever you want with it,\nincluding things you wouldn't do in a professional context. The downside is that\nyou can do whatever you want with it, including things you wouldn't do in a\nprofessional context. My RSS feed has been broken for who knows how long, and\nI just couldn't muster up the effort to fix it.</p>\n<p>So last night, I did something new: I did the tutorial for a new framework I\nhad never used before, and wrote code until it works.</p>\n<p>Yeah yeah that's funny and all, but I actually feel much better about things\nthis time. We'll see how I feel in six months. The framework this time is\n<a href=\"https://astro.build/\">Astro</a>, and I think it's really neat. At some point after\nI do some more with it and digest some more of the way that it works, I'll give\na more thorough writeup, but I find the <a href=\"https://docs.astro.build/en/concepts/why-astro/\">Why Astro?</a> page on their\nsite does a good job of explaining.</p>\n<p>Why not a Rust based framework? I want to do more in that space too, but the goal\nhere is to stretch my legs, and using a language (TypeScript, in this case) that\nI don't use often is more interesting than the thing I use daily.</p>\n<p>However, one little thing did come up during the development of this project\nthat I thought was kind of amusing, in a 'compare and contrast' kind of way.</p>\n<p>I like for my blog to have entries grouped by year. There's no database in this\nproject, so I have to sort things myself. Here's the relevant code as it stands\ntoday, with some small omissions for clarity:</p>\n<pre><code>import type { MarkdownInstance } from \"astro\";\nconst allPosts = await Astro.glob('../pages/writing/*.md');\n\nlet allPostsByDate = (() => {\n let posts: { [key: string]: MarkdownInstance<Record<string, any>>[] } = {};\n allPosts.map((post) => {\n let year = new Date(post.frontmatter.pubDate).getFullYear();\n let array = posts[year.toString()];\n\n // if this is the first post of this year, we have to create the inner array\n if (!array || !array.length) {\n posts[year.toString()] = [];\n }\n\n posts[year.toString()].push(post);\n })\n\n return posts;\n})();\n\nconst years = Object.keys(allPostsByDate).reverse();\n---\n<BaseLayout>\n <p>This is my blog.</p>\n {years.map((year) => (\n <h3>{year}</h3>\n <ul>\n {allPostsByDate[year].map((post) => <BlogPost url={post.url} title={post.frontmatter.title} />)}\n </ul>\n ))}\n</BaseLayout>\n</code></pre>\n<p>I'm sure there's like, ten thousand things wrong with this code; like I said,\nTypeScript beginner over here. I figured this stuff out myself, there's\nprobably a better way to do these things, but this works. (Okay, actually while\nwriting this I realized I didn't sort <em>within</em> the years, so the real code now\nhandles that too, but I left it out because it's not super relevant to the issue\nat hand here.)</p>\n<p>This is a pretty classic \"I'm doing something in a scripting language\" style\nsetup: you make some sort of data structure out of maps (or objects in this\ncase, of course) and arrays, and then spit it out in some form. But TypeScript\nprefers types, it's right there in the name! And I had configured TypeScript\nto be in its strictest mode, becuase I like types too. Here's the one big type\nannotation I had to use:</p>\n<pre><code>let posts: { [key: string]: MarkdownInstance<Record<string, any>>[] } = {};\n</code></pre>\n<p>If I didn't put a type on <code>posts</code>, it complains later in the program:</p>\n<pre><code>`Element implicitly has an 'any' type because expression of type 'string' can't be used to index type '{}'.\n No index signature with a parameter of type 'string' was found on type '{}'.\n</code></pre>\n<p>It needs a bit of help to figure out the type here. Figuring out how to say\n\"hey this is on object with string keys and arrays of <code>MarkdownInstance</code> as\nvalues\" took me a second, but it's not too bad. But I did struggle with it a bit,\nfirst to figure out why I was getting the error in the first place, and second\nso that I could fix it. I kinda laughed at struggling with types in this dymanic\nfeeling context.</p>\n<p>So anyway, I also had to do a bit of translating markdown files; I had exported\nall of my old posts from Notion, but I couldn't just plop them in and expect\nthem to work; Notion doesn't give you the metadata in a frontmatter block, for\nstarters. I have something like 200 posts, so doing this by hand is <em>just</em>\nannoying enough that I said \"I bet I can code this up faster than if I did it\nall by hand, even with a vim macro.\"</p>\n<p>But by now, I was in \"get stuff done\" mode, so I decided to switch back to Rust.\nThe program I wrote needed to:</p>\n<ol>\n<li>Open each markdown file in the directory.</li>\n<li>Try to parse out the metadata.</li>\n<li>use that metadata to write out a new file, with a new filename, and the\nmetadata in a more friendly format.</li>\n<li>Delete the old file.</li>\n</ol>\n<p>This is a classic scripting task! But at this point, I know how to do it in Rust\neasier than in a scripting language. I started like this:</p>\n<pre><code>> cargo new migrate\n> cd migrate\n> cargo add anyhow\n> cargo add walkdir\n> code .\n</code></pre>\n<p>And ended up writing this:</p>\n<pre><code>use anyhow::Result;\nuse std::io::Write;\nuse std::{\n fs::{self, File},\n path::PathBuf,\n};\nuse walkdir::WalkDir;\n\nfn main() -> Result<()> {\n let from = std::env::args().skip(1).next().unwrap();\n println!(\"from: {}\", from);\n for entry in WalkDir::new(from) {\n let entry = entry.unwrap();\n println!(\"Processing: '{}'\", entry.path().display());\n\n if entry.file_type().is_dir() {\n continue;\n }\n\n let contents = fs::read_to_string(entry.path())?;\n\n if &contents[0..3] == \"---\" {\n println!(\"already migrated, skipping\");\n continue;\n }\n\n let header: Vec<_> = contents.lines().take(7).collect();\n if &header[0][0..2] != \"# \" {\n eprintln!(\"couldn't read title, skipping\");\n continue;\n }\n let title = &header[0][2..];\n\n if &header[3][0..6] != \"Blog: \" {\n eprintln!(\"couldn't read blog, skipping\");\n continue;\n }\n let blog = &header[3][6..].to_ascii_lowercase().replace(' ', \"-\");\n\n if &header[4][0..6] != \"Date: \" {\n eprintln!(\"couldn't read date, skipping\");\n continue;\n }\n let date = &header[4][6..];\n\n if &header[6][0..6] != \"Slug: \" {\n eprintln!(\"couldn't read slug, skipping\");\n continue;\n }\n let slug = &header[6][6..];\n dbg!(slug);\n let mut slug = PathBuf::from(slug);\n slug.set_extension(\"md\");\n\n let output = entry.path().parent().unwrap();\n let output_path = output.join(slug);\n println!(\"writing to: {}\", output_path.display());\n\n let mut output = File::create(&output_path)?;\n write!(\n output,\n \"---\nlayout: ../../layouts/MarkdownPostLayout.astro\ntitle: \\\"{}\\\"\npubDate: {}\nblog: {}\n---\\n\\n\",\n title, date, blog\n )?;\n for line in contents.lines().skip(7) {\n writeln!(output, \"{}\", line)?;\n }\n fs::remove_file(entry.path())?;\n }\n\n Ok(())\n}\n</code></pre>\n<p>76 lines. Not too bad. This is <em>quick and dirty</em>, it is not perfect. For\nexample, like ten of my posts had no 'blog' metadata (that tells me which of\nmy old blogs the post was originally written for), and so it didn't work on\nthem. I still needed to do some small cleanup, but I wanted to double check\nevery post anyway, so that was fine.</p>\n<p>But there's a few other things that are kind of amusing to me about this code:</p>\n<ol>\n<li>I struggled with types in the scripty scenario above, but I did not here. I\nam more experienced with Rust so that's probably it, but I needed to annotate\none type, just like in the TypeScript.</li>\n<li>I didn't deal with lifetimes at all. Owned values, baby! And passing references\nto some functions.</li>\n<li>I didn't deal with memory at all. No borrow checker complaints, none of that.\nIt didn't take me any extra time to \"think about ownership and borrowing.\"</li>\n<li>It didn't take me that long to write this. You can actually see a vestigial\nthing of how it was implemented, can you spot the <code>dbg!</code> statement? I wrote\neach part in sequence, printing out the information to manually validate that\nit looked correct, and then eventually adding the \"write new file\" and\n\"delete old file\" stuff when I was sure the process would complete correctly.\nLots of copy/paste there too, as you can see. I don't care. I'm never using\nthis code again now that the migration is done. That's also why I didn't try\nto do some sort of struct and \"deserialize\" the contents, instead just\nparsing it out like I'd never heard of the term \"parsing\" before. It's fine.</li>\n</ol>\n<p>So yeah, I had too many type problems in my 'real' JavaScript program, and no\ntype problems in my script-y Rust. Kinda funny.</p>\n<p>I guess I have ended up making this a post about Rust anyway, even though I\ndidn't think I had much to say when I started. Now that Rust has grown so much,\npeople who love it are using it for all kinds of things, and sometimes that\nseems to piss some people off. \"You can't use a systems language for scripting\ntasks, nooo!!\" they cry. If you had asked me ten years ago if I should write web\nbackends in Rust or replace shell scripts with Rust, I'd have said the same\nthing. But now that I've used Rust enough, a lot of those sort of generic\ncomplaints about Rust don't speak to my actual experience: I had a problem, I\nwrote ~70 lines of code in not very much time, problem solved. Seems like Rust\nworked just as well as anything else would have. Yes, it's true that maybe if\nyou've just picked up the language, you'd have written that code much more\nslowly than in another language you're more familiar with, or maybe you'd have\nhit some sort of error that was confusing and took up a bunch of time, just like\nI did with my TypeScript. But just because you may have those problems, doesn't\nmean that everyone does. And, it also means that in the future, if you stick with\nit, you won't have those problems either! Or at least, it's possible not to. Not\nevery tool clicks with every person.</p>\n<p>Here's to another ten years.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/are-out-parameters-idiomatic-in-rust/",
"title": "Are out parameters idiomatic in Rust?",
"description": null,
"url": "https://steveklabnik.com/writing/are-out-parameters-idiomatic-in-rust/",
"published": "2020-10-16T00:00:00.000Z",
"updated": "2020-10-16T00:00:00.000Z",
"content": "<p>There is a pretty straightforward design pattern called \"out parameters\" that you may have heard of before. In short, the question is, does your function wear pants... err, I mean, does your function modify data like this:</p>\n<pre><code>fn foo() -> i32 {\n // body elided\n}\n\nlet x = foo();\n\n</code></pre>\n<p>or like this:</p>\n<pre><code>fn foo(out: &mut i32) {\n // body elided\n}\n\nlet mut x = 0;\n\nfoo(&mut x);\n\n</code></pre>\n<p>That is, does your function return a value, or modify a parameter?</p>\n<p>There are a number of different reasons you may or may not use one or the other of these techniques in different languages, and they often depend on the semantics of the language itself.</p>\n<p>In Rust, you almost always want to return a value from a function, that is, prefer option one over option two. There are a few reasons for this.</p>\n<p>The first is that it's generally semantically more accurate. Your function is producing a value of some kind, so it should, well, produce a value. Additionally, with Rust's relatively rich data type support, it's easy to do things like return a tuple if you need to return more than one thing.</p>\n<p>The second is that, in some languages and with some compilers, the out parameter is a performance optimization that you as a programmer do by hand. Imagine if I wasn't returning an <code>i32</code>, which is just a few bytes. The most straightforward way of implementing the function in assembly is to copy the return value from the function to its parent's stack. This also may be required by your ABI. Now, I am not an expert here, so I am not 100% sure of the exact current state of things, but at least in theory, optimizations can be done by the compiler. This is a pretty well-known optimization, see <a href=\"https://en.wikipedia.org/wiki/Copy_elision\">Wikipedia for more</a>. But, I think that we do already do this today in some cases, if I compile both of these functions:</p>\n<pre><code>fn returns_huge(out: &mut [u64; 1024]) {\n *out = [0; 1024];\n}\n\nfn returns_huge() -> [u64; 1024] {\n [0; 1024]\n}\n\n</code></pre>\n<p>I get identical assembly:</p>\n<pre><code>playground::returns_huge: # @playground::returns_huge\n# %bb.0:\n movl $8192, %edx # imm = 0x2000\n xorl %esi, %esi\n jmpq *memset@GOTPCREL(%rip) # TAILCALL\n # -- End function\n\n</code></pre>\n<p>However, it is possible that this breaks for other examples, and so you may, may, may want to move to an out parameter if you see extraneous copies appear in your performance profiling.</p>\n<p>C++ guarantees this optimization in certain circumstances; it would be really nice for Rust to do so as well. My understanding is that's the intention.</p>\n<h2>When you should use out parameters</h2>\n<p>There is one case where it is idiomatic to use an out parameter in Rust though. That's when you are writing into some kind of buffer, and you want to be able to allow your caller to provide the buffer to you, rather than making a new one. This would allow the caller to re-use the same buffer for multiple operations.</p>\n<p>(I say \"buffer\" here but really, this generalizes to any sort of grow-able container, to be clear. I am using \"buffer\" in a loose sense here.)</p>\n<p>You can see this play out in the Rust standard library. <code>std::io::Read</code> looks like this:</p>\n<pre><code>pub trait Read {\n fn read(&mut self, buf: &mut [u8]) -> Result<usize>;\n}\n\n</code></pre>\n<p>This accepts a <code>&mut [u8]</code> rather than returning a new buffer. This allows for significant savings by re-using a buffer. For example, in <a href=\"https://doc.rust-lang.org/stable/book/ch02-00-guessing-game-tutorial.html\">Chapter 2 of the book</a>, we have this example:</p>\n<pre><code>use rand::Rng;\nuse std::cmp::Ordering;\nuse std::io;\n\nfn main() {\n println!(\"Guess the number!\");\n\n let secret_number = rand::thread_rng().gen_range(1, 101);\n\n loop {\n println!(\"Please input your guess.\");\n\n let mut guess = String::new();\n\n io::stdin()\n .read_line(&mut guess)\n .expect(\"Failed to read line\");\n\n let guess: u32 = match guess.trim().parse() {\n Ok(num) => num,\n Err(_) => continue,\n };\n\n println!(\"You guessed: {}\", guess);\n\n match guess.cmp(&secret_number) {\n Ordering::Less => println!(\"Too small!\"),\n Ordering::Greater => println!(\"Too big!\"),\n Ordering::Equal => {\n println!(\"You win!\");\n break;\n }\n }\n }\n}\n\n</code></pre>\n<p>We aren't <em>quite</em> properly re-using the buffer here, but we could modify this to look like this:</p>\n<pre><code>use rand::Rng;\nuse std::cmp::Ordering;\nuse std::io;\n\nfn main() {\n println!(\"Guess the number!\");\n\n let secret_number = rand::thread_rng().gen_range(1, 101);\n\n let mut guess = String::new();\n\n loop {\n println!(\"Please input your guess.\");\n\n guess.clear();\n\n io::stdin()\n .read_line(&mut guess)\n .expect(\"Failed to read line\");\n\n let guess: u32 = match guess.trim().parse() {\n Ok(num) => num,\n Err(_) => continue,\n };\n\n println!(\"You guessed: {}\", guess);\n\n match guess.cmp(&secret_number) {\n Ordering::Less => println!(\"Too small!\"),\n Ordering::Greater => println!(\"Too big!\"),\n Ordering::Equal => {\n println!(\"You win!\");\n break;\n }\n }\n }\n}\n\n</code></pre>\n<p>Here, we create our string outside of the loop, and then on each iteration of the loop, clear it. Clearing it does not deallocate the backing memory; it only changes the length to zero. When the next line is read in, if it's smaller than the existing size of the string, it won't allocate any new memory. This would not be possible if <code>stdin()</code> returned a new <code>String</code> every time, so even though the out parameter is a bit less nice to use, it can be much faster.</p>\n<p>It took 26 releases of Rust for <a href=\"https://doc.rust-lang.org/stable/std/fs/fn.read.html\"><code>std::fs::read</code></a> and <a href=\"https://doc.rust-lang.org/stable/std/fs/fn.read_to_string.html\"><code>std::fs::read_to_string</code></a> to be added for this reason; we eventually did because in many cases, you don't care about re-using the buffer, and these functions are nicer to use, but they weren't included originally because they're pretty simple, and are built on top of the re-usable solutions:</p>\n<pre><code>pub fn read_to_string<P: AsRef<Path>>(path: P) -> io::Result<String> {\n fn inner(path: &Path) -> io::Result<String> {\n let mut file = File::open(path)?;\n let mut string = String::with_capacity(initial_buffer_size(&file));\n file.read_to_string(&mut string)?;\n Ok(string)\n }\n inner(path.as_ref())\n}\n\n</code></pre>\n<p>That said, I am exceedingly glad we did add them. It is a much nicer user experience.</p>\n<p>(You may be wondering \"what's up with that inner function there?\"... that maybe deserves another post! It's not neccesary, strictly speaking, but is another case of \"the compiler does not do an optimization yet so write some code in a strange way by hand.\")</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-cxx-debate/",
"title": "The CXX Debate",
"description": null,
"url": "https://steveklabnik.com/writing/the-cxx-debate/",
"published": "2020-08-22T00:00:00.000Z",
"updated": "2020-08-22T00:00:00.000Z",
"content": "<p>Let's talk about safe and unsafe in Rust.</p>\n<p>To review, Rust has a concept of \"unsafe code.\" Unsafe Rust is a superset of safe Rust, letting you do some extra things that are dangerous. It's your job as a programmer to verify that your code is playing by the rules, because the compiler cannot check them for you.</p>\n<p>Imagine we have a C program, <code>add.c</code> that looks like this:</p>\n<pre><code>#include <stdint.h>\n\nuint32_t add_one(uint32_t x) {\n return x + 1;\n}\n</code></pre>\n<p>We can call this function from Rust like this:</p>\n<pre><code>extern {\n fn add_one(x: u32) -> u32;\n}\n\nfn main() {\n let x = 5;\n\n let y = unsafe { add_one(x) };\n\n println!(\"{}\", y);\n}\n</code></pre>\n<p>We need to invoke <code>unsafe</code> here because any <code>extern</code> function doesn't come from Rust; it comes from another language, and it's possible that that function does something that would cause undefined behavior. We look at <code>add_one</code>, see that it's not violating any rules, and we're good.</p>\n<p>However, we know that we can pass any <code>u32</code> to this function, and it's safe. We don't need to example the value of the argument every single time, they're all fine. So we do this:</p>\n<pre><code>mod internal {\n extern {\n pub fn add_one(x: u32) -> u32;\n }\n}\n\nfn add_one(x: u32) -> u32 {\n unsafe {\n internal::add_one(x)\n }\n}\n\nfn main() {\n let x = 5;\n\n let y = add_one(x);\n\n println!(\"{}\", y);\n}\n</code></pre>\n<p>This is a classic way that unsafe code gets used in Rust; we have our binding, <code>internal::add_one</code>, and our wrapper, <code>add_one</code>. The wrapper is safe, but uses unsafe internally. This is totally fine! The idea is that this gives us a sort of \"seam\" in our code. Imagine that <code>add_one</code> would do bad things if we passed it a number less than five. We could use our wrapper to check this:</p>\n<pre><code>// we would use a real error type in real code but we're using () here because we don't\n// actually care about the error handling here.\nfn add_one(x: u32) -> Result<u32, ()> {\n if x < 5 {\n return Err(());\n }\n\n unsafe {\n Ok(internal::add_one(x))\n }\n}\n</code></pre>\n<p>But, <code>add_one</code> doesn't need to do this. Every <code>u32</code> we pass is totally fine.</p>\n<p>Nothing so far is controversial here, this is just background and setup. The purpose of these examples is for you to read and ask yourself, \"Is this okay? How do I feel about this?\"</p>\n<p>Somewhere after this point, opinions start to diverge.</p>\n<hr />\n<p>Our C library is growing new functionality! Here it is:</p>\n<pre><code>#include <stdint.h>\n\nuint32_t add_one(uint32_t x) {\n return x + 1;\n}\n\nuint32_t add_two(uint32_t x) {\n return x + 2;\n}\n</code></pre>\n<p>We can now add two, not only one. Amazing, I know. Let's write another wrapper:</p>\n<pre><code>mod internal {\n extern {\n pub fn add_one(x: u32) -> u32;\n pub fn add_two(x: u32) -> u32;\n }\n}\n\nfn add_one(x: u32) -> u32 {\n unsafe {\n internal::add_one(x)\n }\n}\n\nfn add_two(x: u32) -> u32 {\n unsafe {\n internal::add_two(x)\n }\n}\n\nfn main() {\n let x = 5;\n\n let y = add_one(x);\n let z = add_two(x);\n\n println!(\"{}, {}\", y, z);\n}\n</code></pre>\n<p>Two functions, both have an unsafe binding, and a safe wrapper.</p>\n<p>Is this okay? How do you feel about this?</p>\n<p>Now, imagine we had 100 of these functions. Writing this all out ends up being a massive amount of error-prone boilerplate. Heck, I almost forgot to switch to <code>internal::add_two</code> after I copy/pasted the new function there. Rust has a tool to abstract over code like this: macros! So let's refactor our code with a macro:</p>\n<pre><code>macro_rules! call_c {\n // implementation elided\n}\n\ncall_c! {\n fn add_one(x: u32) -> u32;\n fn add_two(x: u32) -> u32;\n}\n\nfn main() {\n let x = 5;\n\n let y = add_one(x);\n let z = add_two(x);\n\n println!(\"{}, {}\", y, z);\n}\n</code></pre>\n<p>I am not showing you the macro because the exact contents are gnarly and immaterial to the point. I also think that maybe this needs to be a procedural macro not a <code>macro_rules</code> macro but again, those details don't really matter. The point is, we have a macro, it generates the exact same structure that we had before.</p>\n<p>Is this okay? How do you feel about this?</p>\n<hr />\n<p>This wrapper is also known as <a href=\"https://github.com/dtolnay/cxx\">CXX</a>. The syntax is slightly different, but conceptually, this is what it is doing. The reason it is controversial is that some people do not in fact feel that the above macro is okay. The reason that they feel it is not okay is that you do not write <code>unsafe</code> by hand, yet, if you're not careful, things can go wrong. I think this is a totally understandable and reasonable reaction, but I also don't think that it makes a ton of sense once you actually dig into the details.</p>\n<p>We actually don't need C to think about this problem, we can do this with unsafe Rust purely. And I think that using unsafe Rust makes it a bit more clear, so let's talk about this in some more depth, but purely with Rust.</p>\n<p>The <code>unsafe</code> keyword has two, complimentary uses. It can be used to require your caller to check invariants, and it can be used to tell the compiler \"I have checked the invariants.\" In other words:</p>\n<pre><code>// I am leaving off parameters and return types because it doesn't matter, but real code\n// would of course have them.\n\n// When calling foo, please check some invariants.\nunsafe fn foo() {\n // ...\n}\n\n// I have checked the invaraints, and this is okay to do.\nunsafe {\n foo();\n}\n</code></pre>\n<p>The compiler forces these combinations of things; using a feature that generates a requirement to check something will require you to also declare that you've checked it. These requirements bubble up, in other words, we can also pass that obligation to our callers:</p>\n<pre><code>unsafe fn foo() {\n // ...\n}\n\nfn bar1() {\n // I have checked the invariants, and this is okay to do\n unsafe {\n foo();\n }\n}\n\n// I have not checked the invariants, so since I'm calling foo, you must check\n// the invariants\nunsafe fn bar2() {\n foo();\n}\n</code></pre>\n<p>Rust's built-in FFI works like <code>bar2</code>; if the contents of <code>foo</code> are written in another language, it has to be unsafe. This means that, even if it is perfectly safe to call <code>foo</code> with any possible arguments, or in any circumstances, we still need to write a wrapper.</p>\n<p>CXX's take on FFI is that it works like <code>bar1</code>; by using the library we are saying \"hey, I know that <code>foo</code> is safe in all circumstances, so please generate the wrapper for me.\" In other words, in some sense, it pushes the unsafe block across the other side of the boundary, because that's where it actually belongs in this case.</p>\n<hr />\n<p>This has generated a lot of feelings on both sides. The people who don't like it are upset that the letters <code>['u', 'n', 's', 'a', 'f', 'e']</code> are not present, and feel that this makes it quite easy to make a mistake and accidentally use the macro on some code that is not actually safe to call in all circumstances. The people who do like it argue that when you have a large number of calls that are actually safe, using a macro to abstract them away is fine, like you'd abstract away any other boilerplate. Sometimes, repeating the same thing over and over again makes mistakes more likely, not less.</p>\n<p>In the meantime, <a href=\"https://www.reddit.com/r/rust/comments/ielvxu/the_cxx_debate/g2i1r7a/?context=3\">dtolnay has decided</a> to change the macro and re-add the <code>unsafe</code> token, which helps assuage the fears of the people who are upset. But it also centralizes these sorts of uses of <code>unsafe</code>, which helps achieve the goals of the library in the first place. I am continually impressed by dtolnay's ability to design libraries, and hope he keeps writing Rust code forever.</p>\n<p>Anyway, if you were wondering what this all was about, that's my take on it. I personally feel that CXX is fine the way that it is, but I'm also happy with this compromise, and hope that others are too.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/today-is-my-first-day-at-oxide-computer-company/",
"title": "Today is my first day at Oxide Computer Company",
"description": null,
"url": "https://steveklabnik.com/writing/today-is-my-first-day-at-oxide-computer-company/",
"published": "2020-06-15T00:00:00.000Z",
"updated": "2020-06-15T00:00:00.000Z",
"content": "<p><img src=\"/img/2020-06-15/oxide.png\" alt=\"Oxide Logo\" /></p>\n<p>Today is my first day at <a href=\"https://oxide.computer/\">Oxide Computer Company</a>, where I will be doing engineering work.</p>\n<hr />\n<p>When Oxide introduced themselves to the world, I saw a lot of folks who were pretty confused. At its core, what Oxide is doing is very straightforward: you give Oxide money, and Oxide gives you a computer. This company sells goods, for money. That raises the issue of what kind of computer, and why would you want one?</p>\n<p>In the late 90s and early 2000s, if you were running a web service of some kind, you'd have a server sitting in a rack in a data center somewhere. Got a lot of traffic? You'd buy a bigger, beefier server. This was the era of \"vertical scaling.\" Servers were big, expensive, and hard to manage, so you didn't buy more servers, you bought bigger servers.</p>\n<p>That started changing. In 2006, you saw a bunch of articles like <a href=\"https://www.networkworld.com/article/2304459/google-builds-own-servers-for-efficiency.html\">this one</a>:</p>\n<blockquote>\n<p>Google, typically tight-lipped about the technology behind its data centers, builds its own servers to save costs and because standard products don't exactly meet its needs, Google's senior vice president of operations said on Thursday.</p>\n</blockquote>\n<p>This era saw companies like Google start to reject the notion of \"Server grade\" hardware, and rely purely on cheaper, consumer-grade hardware. As the quote said, this required building a bunch of software that would make the cluster of servers more reliable.</p>\n<p>But eventually, there were downsides to this approach, as well. As these companies grew, so did a new generation of companies building \"the cloud\" as we know it today. And once you start managing that many computers, possibilities open up to gain certain efficiencies and cut costs. Years later, we see stuff <a href=\"https://www.geekwire.com/2017/amazon-web-services-secret-weapon-custom-made-hardware-network/\">like this</a>:</p>\n<blockquote>\n<p>It’s not unusual for internet and software giants to design and even make their own hardware, to increase efficiency and build a competitive advantage. Google custom-designs its own servers, filling them with millions of chips from Intel, and announced in May that it has designed its own application-specific integrated circuit (ASIC) for use on neural networks. Facebook uses its own switches in its data centers. But market-leading public-cloud company Amazon Web Services may have gone farthest down this path — designing not only its own routers, chips, storage servers and compute servers but also its own high-speed network.</p>\n</blockquote>\n<p>The cloud was printing money, and folks were running into scaling issues. The solution was pretty straightforward: hire some people to build amazing infrastructure. Why not just buy it? Well, it simply didn't exist for purchase in the first place. The current crop of computer companies really grew up in the vertical scaling era. But beyond that, you also have to remember that this stuff was pretty <em>new</em> at that time. The Cloud was still cutting-edge, and so the requirements weren't really known. There was a lot of R&D work to figure out what things would actually work in these environments. It turns out that a lot of it was not just hardware, but the software too. There is a ton of junk and waste in off-the-shelf server systems today, in all parts of the system.</p>\n<p>Because these developments were proprietary, it also meant that if you weren't willing to build your own hardware, the only way you could get access to this new tech was by renting it. The public cloud's usage-based billing model is pretty standard, and pretty lucrative. Custom hardware helped dig an even deeper moat around cloud providers. If you want to actually own computers, your options are to use previous-generation tech, or hire a team and try to catch up to the big players. So the world kind of split in two: the hyperscalers and everyone else.</p>\n<p>There is a <strong>ton</strong> more to say here, but this is the short of it: what Oxide is doing is building computers that are suitable for hyperscalers, but selling them, rather than building another public cloud. These computers will have hardware and software designed together to create excellent systems. Or I should say \"will be selling them,\" it is still early days.</p>\n<hr />\n<p>I do not come from a hardware background. But there's a lot more to a system than hardware. And as you might guess from the name of the company, Oxide will be building a lot of software in Rust. I also have known two of the three founders of the company for a while, and they are folks I respect and whose work I admire.</p>\n<p>I am also looking forward to doing engineering work again. I still love doing product work, but I think I prefer it in a small company setting. In a new company, engineers can still have a lot of ownership over product, so I won't be giving that up completely. But I'm excited to finally have a job where I get to write Rust code as a direct responsibility. It's not like I've not written any Rust over the past few years, but I'm looking forward to writing more of it, regardless.</p>\n<p>Finally, I'm excited to continue to help others as well; Oxide cannot only hire folks who are already Rust experts. There just aren't enough of them yet, and there's folks who do fantastic work who don't yet write Rust. I'll be helping to bridge that gap.</p>\n<p>There is so much work to do. It's almost 9am. Time to get started.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-next-iteration-of-my-blogs/",
"title": "The next iteration of my blogs",
"description": null,
"url": "https://steveklabnik.com/writing/the-next-iteration-of-my-blogs/",
"published": "2020-06-08T00:00:00.000Z",
"updated": "2020-06-08T00:00:00.000Z",
"content": "<p>I've had a few different blogs over the years. Every so often, I change things. One of the nice things about having your own site is that you can do whatever you want! So, I'm folding my old blogs' content into my main website. I'm sick of adding new subdomains, and it all feels better in one place these days. The old URLs will redirect to posts on steveklabnik.com, but I wanted to record what they looked like at the time.</p>\n<hr />\n<p>My oldest blog that's survived is <a href=\"https://web.archive.org/web/20200519230707/http://blog.steveklabnik.com/\">Literate Programming</a>. It looked like this:</p>\n<p><img src=\"/img/2020-06-08/Untitled.png\" alt=\"untitled\" />\n<img src=\"/img/2020-06-08/Untitled%201.png\" alt=\"untitled 01\" />\n<img src=\"/img/2020-06-08/Untitled%202.png\" alt=\"untitled 02\" /></p>\n<p>This was my own static site generator! I loved making that table of contents stuff over on the left.</p>\n<hr />\n<p>Then, there's <a href=\"https://web.archive.org/web/20200604152340/https://words.steveklabnik.com/\">words</a>. This used <a href=\"https://svbtle.com/\">Svbtle</a>, which is a great platform. I liked the Meeple, and the red, and the Kudos button that made random internet people <em>so mad</em> for some reason. Anyway, it looked like this:</p>\n<p><img src=\"/img/2020-06-08/Untitled%203.png\" alt=\"untitled 03\" />\n<img src=\"/img/2020-06-08/Untitled%204.png\" alt=\"untitled 04\" />\n<img src=\"/img/2020-06-08/Untitled%205.png\" alt=\"untitled 05\" /></p>\n<p>The kudos count does nothing, other than record that you did it. The style was really nice, it was easy to write Markdown. Then Medium got popular. Oh well.</p>\n<hr />\n<p>Now, I'm writing this in <a href=\"https://www.notion.so/\">Notion</a>. I really, really like writing in Notion. Someone put together <a href=\"https://github.com/ijjk/notion-blog\">some code</a> to use Notion like a headless CMS. Exactly what I wanted. I've modified it a bunch, I'm using I'm using <a href=\"https://concrete.style/\">concrete.css</a> as the base of my look, for example. They're using <a href=\"https://nextjs.org/\">Next.js</a> and <a href=\"https://vercel.com/\">Vercel</a>. I've been a fan of those folks (and pretty much everyone in the JAMStack space, to be honest) for a while, so I'm giving that a go.</p>\n<p>I think it's going pretty good so far.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/how-often-does-rust-change/",
"title": "How often does Rust change?",
"description": null,
"url": "https://steveklabnik.com/writing/how-often-does-rust-change/",
"published": "2020-04-11T00:00:00.000Z",
"updated": "2020-04-11T00:00:00.000Z",
"content": "<p>I’ve been thinking a lot lately about how often Rust changes. There are some people that assert that Rust stays fairly static these days, and there are some people who say Rust is still changing far too much. In this blog post, I want to make a <em>data driven</em> analysis of this question. First, I will present my hypothesis. Next, my methodology. Then, we’ll talk about the results. Afterward, we’ll discuss possible methodological issues, and possible directions for further research.</p>\n<p>As <a href=\"https://twitter.com/adam_chal/status/1248705340163620866\">said on twitter</a>:</p>\n<p><img src=\"https://svbtleusercontent.com/66pLpg8EDEH5dvt5BGQ7LH0xspap_small.jpg\" alt=\"https://svbtleusercontent.com/66pLpg8EDEH5dvt5BGQ7LH0xspap_small.jpg\" /></p>\n<h2>The hypothesis</h2>\n<p>My personal take on Rust changing: to me, it feels like we had more change in the past than we have today. The changes are also getting more minor. This will be my hypothesis.</p>\n<h2>The methodology</h2>\n<p>There are tons of different ways that we could measure the question “how often does Rust change?” So before I started collecting data, I had to decide some details.</p>\n<p>I realized that my premise here is about how people <em>feel</em> about Rust. So, how do they tend to understand how Rust changes? Well, the primary way that we communicate change to the Rust community is through release blog posts. So I decided to start there.</p>\n<p>What I did was, I opened up every release blog post for Rust, from 1.0.0 to 1.42.0, in a new tab. For fun, I also opened the release notes for 1.43, which will come out soon.</p>\n<p>I then looked at every single change announced, and categorized them as follows:</p>\n<ul>\n<li>adds syntax?</li>\n<li>language</li>\n<li>standard library</li>\n<li>toolchain</li>\n<li>major</li>\n<li>medium</li>\n<li>minor</li>\n</ul>\n<p>I initially included “deprecation” and “soundness” as categories too, but I ended up not including them. More on that later.</p>\n<p>From here, I had to decide what these categories <em>meant</em>. What standard would I be using here? Here’s the easier ones:</p>\n<p><em>language</em> changes mean some sort of modification to the language definition. These are additive changes, thanks to our stability policy, but they’re new additions.</p>\n<p><em>standard library</em> means a change to the standard library, primarily, new functions, new types, new methods, stuff like that. This one is pretty simple as a criteria but there’s some small interesting methodological issues we’ll get into later.</p>\n<p><em>toolchain</em> changes related to <code>cargo</code>, <code>rustup</code>, new compiler target support, stuff like that. Not part of the language proper, but part of the Rust distribution and things Rust programmers would use.</p>\n<p>Now, for the harder ones:</p>\n<p><em>major</em>/<em>medium</em>/<em>minor</em> changes are how big of a change I think that the change is. There’s a few interesting parts to this. First of all, this is, of course, very subjective. I also decided to try and evaluate these as of today, that is, there are some changes we maybe thought were major that aren’t used all that often in practice today, so I categorized those as <em>medium</em> rather than <em>major</em>. This felt more consistent to me than trying to remember how I felt at the time.</p>\n<p><em>adds syntax</em> changes <em>sound</em> easy, but are actually <em>really</em> tricky. For example, consider a feature like “<a href=\"https://github.com/rust-lang/rfcs/blob/master/text/1492-dotdot-in-patterns.md\">dotdot in patterns</a>”. This does change the grammar, for example, so you could argue that it adds syntax. But as a programmer, I don’t <em>really</em> care about the grammar. The summary of this feature is “Permit the .. pattern fragment in more contexts.” It goes on to say:</p>\n<blockquote>\n<p>This RFC is intended to “complete” the feature and make it work in all possible list contexts, making the language a bit more convenient and consistent.</p>\n</blockquote>\n<p>I think these kinds of changes are actually very core to this idea, and so I decided to categorize them according to my opinion: this is <em>not</em> a new syntax change. You already had to know about <code>..</code>.</p>\n<p>I believe this may be the most controversial part of my analysis. More on this later, of course.</p>\n<p>Okay, so that’s the stuff I covered. But there’s also stuff I didn’t cover. I was doing enough work and had to draw the line somewhere. I left this stuff out:</p>\n<ul>\n<li>compiler speedups. This is interesting with numbers, but that means actually compiling stuff, and I don’t have time for that. This is a whole study on its own.</li>\n<li>Documentation work. This tends to be not tracked as new features, but sometimes it appears in release notes, for bigger things. Better to just leave it out. I also don’t think that it affects my hypothesis at all.</li>\n<li>Library types implementing new traits. These can be hard to number, especially with generics involved. I decided that it would be better to leave them off.</li>\n<li>compiler internal news. We sometimes talk about things like “MIR exists now!” or “we ported the build system from make to cargo!” but similar to documentation, we talk about it very little. It’s also not a change that affects end users, or rather, it affects them more directly by stuff that is counted. MIR enabled NLL, and NLL was tracked as a language feature.</li>\n</ul>\n<h2>The results & analysis</h2>\n<p>I am not as good with Google Sheets as I thought, so I asked for help. Big thanks to <a href=\"https://twitter.com/ManishEarth\">Manish</a> for helping me get this together. I should note that he set me up, but then I tweaked a ton of stuff, so any mistakes are mine, not his. For example, I got a little lazy and didn’t realize that the colors aren’t the same across the various charts. That’s my bad.</p>\n<p>Here’s some quick totals of how many changes and of each kind I found, with some sparklines. You can click on any of the images in this post to enlarge them:</p>\n<p><img src=\"https://svbtleusercontent.com/i84Jyi8Ku2vJ2VRGfYmyBV0xspap_small.png\" alt=\"https://svbtleusercontent.com/i84Jyi8Ku2vJ2VRGfYmyBV0xspap_small.png\" /></p>\n<p>And here’s some charts, per release:</p>\n<p><img src=\"https://svbtleusercontent.com/oZTrBYjy26f5AKURN7z3db0xspap_small.png\" alt=\"https://svbtleusercontent.com/oZTrBYjy26f5AKURN7z3db0xspap_small.png\" /></p>\n<p>First of all, it is fairly clear that, at least numerically speaking, the standard library is the part of Rust that changes most often. It’s a clear outlier by volume. I find this result kind of hilarious, because Rust is well-known for having a small standard library. We’ll talk more about why I think this is in the “Issues and further research” section below.</p>\n<p>Let’s look at the chart without the standard library:</p>\n<p><img src=\"https://svbtleusercontent.com/qXu9v6Li5qzg8Ag6HV1j8p0xspap_small.png\" alt=\"https://svbtleusercontent.com/qXu9v6Li5qzg8Ag6HV1j8p0xspap_small.png\" /></p>\n<p>You can see there’s a <em>ton</em> of toolchain changes early on. We had a lot of work to do early on, and so made a ton of changes! It’s calmed down in recent times, but toolchain changes happen almost twice as often as language changes. I should also note that this reduction in toolchain changes may be due to methodology; this is talked about later in the post.</p>\n<p>So, the core question here: it looks like the language is changing <em>more</em>, recently, rather than less. I think it’s pretty clear that this graph is <em>not</em> exactly what I would imagine, that is, I expected a nice gentle curve down, but that’s not what has happened. I want to dig into this a bit more, but first, let’s look at some other graphs:</p>\n<p><img src=\"https://svbtleusercontent.com/3Xa9rr1SM3atm69gAohMkM0xspap_small.png\" alt=\"https://svbtleusercontent.com/3Xa9rr1SM3atm69gAohMkM0xspap_small.png\" /></p>\n<p>Changes that add syntax are a subset of language changes. There hasn’t been a whole ton of this, overall. Rust 2018 was the large bump. Half of our releases do not add syntax, though 10 out of 43 did introduce language changes. Our first 29 releases, skipping 1.26, had around one or two changes on average, but every since, it’s been between three and four. I believe this has to do with my methodology, but at the same time, this pretty resoundingly refutes my hypothesis. Very interesting!</p>\n<p>Here’s major/medium/minor:</p>\n<p><img src=\"https://svbtleusercontent.com/cm4iWVjB9XZw4qp3c5b94o0xspap_small.png\" alt=\"https://svbtleusercontent.com/cm4iWVjB9XZw4qp3c5b94o0xspap_small.png\" /></p>\n<p>There’s a peak around Rust 2018, and one from 1.12 to 1.19, but otherwise, it’s been pretty steady lately, in terms of overall changes. If we look at just major changes:</p>\n<p><img src=\"https://svbtleusercontent.com/N3Dud8Du3fUEC8LL8a9Fn0xspap_small.png\" alt=\"https://svbtleusercontent.com/N3Dud8Du3fUEC8LL8a9Fn0xspap_small.png\" /></p>\n<p>Rust 2018 had a huge bump. After 1.0, and after Rust 2018, it calmed. I think this chart is <em>very</em> interesting, and demonstrates the 3 year edition cycle well. We ship an edition, stuff calms down, things slowly build back up, we ship an edition, they calm down again.</p>\n<p>So, why was my hypothesis wrong? I think this conclusion is <em>fascinating</em>. I think part of it has to do with my methodology, and that maybe, that difference explains some of the ways people feel about Rust. You see, even if we release extremely often, and a lot of releases don’t have a <em>ton</em> of changes in them (as you can see, it’s vaguely around 8 or 10, when you exclude the standard library, on average), the very fact that we do release often and have a decent, but fairly small amount of change means that things get into the release notes that may not if we had the exact same amount of change, but did yearly releases.</p>\n<p>So for example, in 2019, we released Rust 1.32 to 1.40. That involved 35 language changes. Would I have included, say, “you can now use <code>#[repr(align(N))]</code> on enums” if I had been writing a post about all of these changes for the whole year? Probably not. But because there were only eight total changes in that release, with half of those being language changes, it made sense to include in the 1.37 release post.</p>\n<p>Does this mean that folks who read the release posts and think Rust changes a lot are wrong? No, it does not. For those of us in the thick of it, a lot of the smaller changes kind of blur into the background. I actually initially typed “<code>#[repr(N)]</code> on enums” when writing the above sentence, because this change isn’t relevant to me or my work, and is pretty small, and makes things more orthogonal, and so it is just kind of vague background noise to me. But to someone who isn’t as in-tune with the language, it’s harder to know what’s big and what’s small. The endless release posts with tons of stuff in them makes it <em>feel</em> like a lot is happening. But the same amount of stuff may be happening in other languages too, you just don’t see them in release posts, because they only put out one once a year. You both see them less often, as well as see less in them.</p>\n<p>I don’t think this means that the Rust project should change its release schedule, and I’m not entirely sure if this means we should change how we write release posts. However, maybe there’s a better way to surface “these are big changes” vs “this is a small thing,” or something like that. I’m not sure. But I do think this helps me understand the disconnect a lot better.</p>\n<h2>Issues and further research</h2>\n<p>There are some big issues with this methodology. No analysis is perfect, but I want to be fair, so I’m calling out all of the stuff I can think of, at least.</p>\n<p>The first of which is, this is inherently subjective. There’s different degrees of subjectivity; for example, “total changes in the notes” is fairly objective. But not completely; folks have to decide what goes into the release notes themselves. So there’s already a layer of filtering going on, before I even started looking at the data. Doing this analysis took me two days of real time, but only a few hours of actual work. Something that analyses the git repo in an automated fashion may see other results. I argue that this level of subjectivity is okay, because we’re also testing something subjective. And I think I designed my methodology in a way that captures this appropriately. On some level, it doesn’t matter if the “real” answer is that Rust rarely changes: folks still get this info by reading the posts, and so I don’t think saying that “oh, yeah you feel that way, but this graph says your feelings are wrong” is really going to help this debate.</p>\n<p>The second problem expands on this. We sort of had three eras of release note writing: the first few posts were written by Aaron and Niko. As of Rust 1.4, I took over, and wrote <em>almost</em> every post until 1.33. I then stepped back for a while, though I did help write the 1.42 post. The release team wrote 1.34 onward, and it was more of a collaborative effort than previous posts. This means that the person doing the filter from release notes -> blog post changed over time. This can mess with the numbers. For example, there are some blog posts where I would have maybe omitted a small family of features, but the folks writing that post included them all, individually. This may explain the uptick in language changes lately, and why many of them were not rated as “major.” Additionally, the people who did the filter from “list of PRs” to “release notes” <em>also</em> changed over time, so there’s some degree of filtering change there, as well.</p>\n<p>To complicate things further, starting with Rust 1.34, Cargo lost its section in the release blog posts. Big changes in Cargo still made it into the text, but the previous posts had more robust Cargo sections, and so I feel like post-1.34, Cargo was under-counted a bit. Cargo changes are toolchain changes, and so the recent calm-ness of the toolchain numbers may be due to this. This still is okay, because again, this analysis is about folks reading the release notes, but it’s worth calling out. Likewise, I think the standards for writing the library stabilizations changed a bit over time too; it’s gone from more to less comprehensive and back again a few times.</p>\n<p>This was mentioned above a bit, but for completeness, the structure of Rust releases means there’s a variable criteria for what makes it into a release post. If it’s a smaller release, smaller changes may get featured, whereas if those same feature were in a bigger release, they may not have been. I think this aspect is a significant factor in the rejection of my hypothesis, as I mentioned above.</p>\n<p>I think there’s interesting future research to be done here. I initially tried to track deprecations and soundness fixes, but soundness fixes happened so little it wasn’t worth talking about. There were 7 in those 42 releases. This is another weakness / area for future research, because I did not include point releases, only 1.x.0 releases. This would have added a few more soundness fixes, but it also would have added a lot of random noise where not much happened in the graphs, so that’s why I left those out. They also don’t happen on the same regular schedule, so time would get a bit funny… anyway. Deprecations was also just hard to track because there weren’t a lot of them, and sometimes it was hard to say if something was deprecated because of a soundness issue vs other reasons.</p>\n<p>Is my critera for adding syntax correct? I think a reasonable person could say “no.” If you have something that’s special cased, and you’re making it more general, there’s a good argument that this is <em>removing</em> a restriction, rather than adding a feature. But reasonable folks could also say you’ve added something. I don’t think that this part is core to my analysis, and so I think it’s fine that I’ve made this decision, but you may not care for this section of results if you disagree.</p>\n<p>This analysis doesn’t get into something that I think is a huge deal: the ecosystem vs the language itself. I only track the Rust distribution itself here, but most Rust programmers use many, many ecosystem libraries. When people talk about churn in Rust, do they really mean the ecosystem, not the language? In some sense, doing so is correct: Rust having a small standard library means that you have to use external packages a lot. If those churn a lot, is it any better than churn in the language itself?</p>\n<p><em>Does</em> Rust have a small standard library? There were 962 changes in 42 releases, that’s almost 23 changes per release. A lot of them are small, but still. Maybe Rust has a small, but deep standard library? Due to coherence, I would maybe think this is true. Just how big are standard libraries anyway? I’ve only seen discussion of this concept in terms of design philosophy. I’ve never seen a numerical analysis of such. Is it out there? Do you know of one? I’d love to hear about it!</p>\n<p>I think there’s a number of reasons that the standard library changes dominate the total change rate. Primarily, this section of the blog post <em>tends</em> to be the most complete, that is, out of the total amount of changes, more standard library changes make it into blog posts than other kinds of changes. Why? Well, it’s really easy to measure: count up stuff stabilized and put it in a big list in the post. It’s also about how these changes are measured; when a method is added to every number type, that’s <code>u8</code> + <code>u16</code> + <code>u32</code> + <code>u64</code> + <code>u128</code> = <code>5</code> changes, even if conceptually it’s one change, in a different sense.</p>\n<p>This leads to another question about those standard library changes: what is up with that weird peak around Rust 1.33? Well, we added the <code>const fn</code> feature in Rust 1.31. After that landed, we could <code>const</code>-ify a number of functions in the standard library. The initial huge bulk of changes here landed in 1.33. It had 138 changes, but only 13 were not “this function is now <code>const</code>.” And the feature continues to drive change outside of 1.33; the initial landing of <code>const fn</code> in 1.31 was very limited, and as it expands, more functions that already existed can be made <code>const</code>.</p>\n<p>In conclusion, even though I still believe that Rust has slowed down its rate of change a lot, I think that it makes total sense that not everyone agrees with me. By some metrics, I am just flat-out wrong.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/c-is-how-the-computer-works-is-a-dangerous-mindset-for-c-programmers/",
"title": "\"C is how the computer works\" is a dangerous mindset for C programmers",
"description": null,
"url": "https://steveklabnik.com/writing/c-is-how-the-computer-works-is-a-dangerous-mindset-for-c-programmers/",
"published": "2020-03-31T00:00:00.000Z",
"updated": "2020-03-31T00:00:00.000Z",
"content": "<p>A long time ago, I wrote <a href=\"https://words.steveklabnik.com/should-you-learn-c-to-learn-how-the-computer-works\">“Should you learn C to “learn how the computer works”?\"</a>. This was part 1 in a three part series. A year later, I got around to part 2, <a href=\"https://words.steveklabnik.com/c-is-not-how-the-computer-works-can-lead-to-inefficient-code\">““C is not how the computer works” can lead to inefficient code\"</a>.</p>\n<p>It’s been five more months. I don’t think I really want to write out the third part; I expected this to be a quick series of blog posts, not something that was on my plate for a year and a half. Here’s the thesis of part 3:</p>\n<blockquote>\n<p>Part three is going to show what happens if you make a mistake with the ideas from part two. If you incorrectly assume that C’s abstract model maps directly to hardware, you can make mistakes. This is where UB gets dangerous. While you can take advantages of some properties of the machine you’re relying on, you have to know which ones fit within C’s model and which ones don’t.</p>\n</blockquote>\n<p>I honestly don’t feel like writing more than that, so I’m just gonna leave it there. Basically, the overall thrust of this series has been this: C is not the hardware, it’s an abstract machine. But that machine runs on real hardware, and abstractions are leaky. If you go too far into “purely only the abstract machine,” you may not be able to accomplish your tasks. If you go too far into “C runs directly on the hardware,” you may be surprised when a compiler does something that you didn’t expect.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/a-sad-day-for-rust/",
"title": "A sad day for Rust",
"description": null,
"url": "https://steveklabnik.com/writing/a-sad-day-for-rust/",
"published": "2020-01-17T00:00:00.000Z",
"updated": "2020-01-17T00:00:00.000Z",
"content": "<p><code>actix-web</code> is dead.</p>\n<p>This situation is bad, from all sides. When Rust was a tiny, tiny community, I thought to myself, “wow, I wonder how long this will last? Probably 1.0 will kill it.” Sort of playing off of <a href=\"https://en.wikipedia.org/wiki/Eternal_September\">Eternal September</a>, I assumed that over time, the community would grow, and we’d encounter problems.</p>\n<p>Today is the first day where I say to myself, okay, has that happened?</p>\n<p>This story is not super clear-cut. I’m not going to link to a dozen citations, or try to prove that I’m some sort of neutral party here. I’m going to give you account of this story as I remember it and as I felt it. Because this isn’t really about playing judge. This is about thinking about the future.</p>\n<p>It’s been very clear from the beginning that the Rust project saw Rust as more than just the language. The community and the people mattered. From the earliest days, leadership explicitly took the position that it wasn’t just the code, but the people around the project were important. Of course, people are also people, and so this wasn’t perfect; we’ve made several fairly large mis-steps here over the years. But Rust has been an experiment in community building as much as an experiment in language building. Can we reject the idea of a BDFL? Can we include as many people as possible? Can we be welcoming to folks who historically have not had great representation in open source? Can we reject <a href=\"https://blog.aurynn.com/2015/12/16-contempt-culture\">contempt culture</a>? Can we be inclusive of beginners?</p>\n<p>Rust has a necessary feature, <code>unsafe</code>, that allows you to escape Rust’s guarantees. This is a really important part of Rust, but also a very dangerous one, hence the name. But one of the things that’s important about it is that, since <code>unsafe</code> means “I have checked this code, and it upholds all of the guarantees the compiler cannot check,” you can build a safe interface with unsafe guts.</p>\n<p>Interacting with anything outside of the language, for example, using the operating system to print to the screen, or write a file, has to be <code>unsafe</code>, and so without this property, all programs would be <code>unsafe</code>. This premise is treated skeptically by many, but has generally been demonstrated to have worked out. Several folks are in the process of providing tooling to prove that your <code>unsafe</code> code is correct, and provided proofs for a bunch of the standard library’s <code>unsafe</code> code. It not only found a few places where there were bugs, but also a few places where the restrictions were too tight!</p>\n<p>But “generally” is doing a lot of work in that previous sentence. This setup creates a gnawing fear in the back of people’s minds: what if some foundational package uses <code>unsafe</code>, but uses it incorrectly? What happens when this causes problems for every package that uses that package? This relationship between safe and unsafe is also a bit hard to understand, and so, when bugs are found around unsafe, people <em>outside</em> of Rust often use it to suggest that all of Rust is a house of cards. While Rust has “made it” in many senses, it has not quite yet in many others, and so I think this produces a fundamental anxiety in the community.</p>\n<p>Speaking of making it, actix-web is a good web framework. It came onto the scene, and people generally liked it. It also performed extremely well. Its score on <a href=\"https://www.techempower.com/benchmarks/\">Techempower</a> in particular was impressive, and got more impressive over time. Its author also works at Microsoft, and suggested that Microsoft is using actix-web in production. This was really exciting to people. This was the second project using Rust at Microsoft, and so this seemed really fantastic.</p>\n<p>But then, the anxiety.</p>\n<p>Before we get into that, though, there’s also the matter of Reddit. The team has basically rejected Reddit for said social reasons. We provide alternate forums for folks, but Reddit is a huge place. The Rust subreddit has ~87,000 subscribers, partially thanks to this. And, for a while, the Rust reddit was a decent place. I still think it’s better than most reddits, but it’s degraded much more quickly than the community at large. “Why Reddit is like this” is a whole other essay; I think it’s built into reddit’s structure itself. But the point is, while Reddit is not official, and so not linked to by any official resources, it’s still a very large group of people, and so to suggest it’s “not the Rust community” in some way is both true and very not true. For the purposes of this story, I think it has to be included in the broader community, and I think that this situation has brought new questions about the relationship of the project and Reddit, though this is already far too long.</p>\n<p>So, someone takes a peek under the covers, and it turns out actix-web is using a <em>lot</em> of unsafe code. That’s not inherently a problem. The problem is, a lot of that unsafe code is not actually needed. This is where things start to go wrong.</p>\n<p>The author of actix-web basically says “nah it’s fine.” This is basically the perfect storm of unsafe anxiety: a big, important, visible library, a bunch of <code>unsafe</code>, and an unresponsive author.</p>\n<p>Now, it’s important to mention that people were not just complaining: they were writing code. It is easy to frame this as a case of open source entitlement, and it still very may well be, but this isn’t just empty complaints. Patches were included, they just weren’t welcome. And that’s also not inherently wrong either; a project maintainer shouldn’t be required to accept a patch just because it exists…</p>\n<p>Reddit whips itself into a frenzy. Lots of ugly things are said. More issues are opened, and closed, and locked. Eventually, the author accepts some code that drops the total unsafe count significantly and has more sound usage of what’s left.</p>\n<p>People were not happy about how all this played out, for a variety of reasons and in a variety of ways. I haven’t spoken to actix-web’s author, but I can’t imagine that he was very happy about it either.</p>\n<p>And then it happens all over again. More unsafe, more patches rejected, more Reddit, more bad blood. I thought that at this point, the author said he was quitting, but that may be faulty memory, or maybe he changed his mind. Regardless, how responsive would you be, if you were the maintainer, after the shit-show of last time?</p>\n<p>People were not happy about how all this played out, for a variety of reasons and in a variety of ways.</p>\n<p>All was quiet for a while, and actix-web kept climbing the Techempower benchmarks, reaching the top of most of them by a <em>lot</em>. It is wicked fast.</p>\n<p>And then, yesterday, a post titled <a href=\"https://medium.com/@shnatsel/smoke-testing-rust-http-clients-b8f2ee5db4e6\">Smoke-testing Rust HTTP clients</a> gets published. It focuses on a, well, smoke test of Rust HTTP clients, but it also comments a lot on the complexity of each project, and the amount of <code>unsafe</code>. The author found and filed a lot of bugs.</p>\n<p>But:</p>\n<blockquote>\n<p>A quick glance at the dependencies reveals that it relies on actix-service, which underpins all of Actix and has a bespoke and unsound Cell implementation. For example, this method violates memory safety by handing out multiple mutable references to the same data, which can lead to e.g. a use-after-free vulnerability. I have reported the issue to the maintainers, but they have refused to investigate it.</p>\n</blockquote>\n<p>This causes the now-usual Reddit uproar. It’s extra nasty this time. Some people go far, far, far over the line.</p>\n<p>And now the maintainer has quit.</p>\n<p>This means, on some level, this situation is over: there will not be a fourth huge actix-web drama. But it’s really left a bad taste in my mouth. It’s unfortunate that a well-known project had soundness issues. It’s unfortunate that the maintainer wasn’t receptive to fixing them. At the same time, I’ve been there: being a maintainer is tough. It’s also unfortunate that there’s this <em>style</em> of response, and kind, and volume. It <em>is</em> true that the situation was dangerous. But did it have to be handled this way? You have to understand a lot of nuance here to even know the main points of the story. One version of this story that will certainly be told is “The Rust community says they’re nice but they will harass you if you use unsafe wrong.” Is that what we want? I, for one, do not. If you’re a part of Rust, you gotta ask yourself: are you happy with this outcome? How did your actions (or lack thereof) contribute to it? Could you have done something better?</p>\n<p>I’m not sure where we go from here, and I’m not sure what we could have done to prevent this from happening. But I feel like this is a failure, and it’s set Rust back a bit, and I’m just plain sad.</p>\n<p>EPILOGUE</p>\n<p>actix-web has new maintainers, and will now live on: https://github.com/actix/actix-web/issues/1289</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/c-is-not-how-the-computer-works-can-lead-to-inefficient-code/",
"title": "\"C is not how the computer works\" can lead to inefficient code",
"description": null,
"url": "https://steveklabnik.com/writing/c-is-not-how-the-computer-works-can-lead-to-inefficient-code/",
"published": "2019-10-25T00:00:00.000Z",
"updated": "2019-10-25T00:00:00.000Z",
"content": "<p>A little over a year ago, I wrote <a href=\"https://words.steveklabnik.com/should-you-learn-c-to-learn-how-the-computer-works\">“Should you learn C to ‘learn how the computer works’”</a>. It was a bit controversial. I had promised two follow-up posts. It’s taken me a year, but here’s the first one.</p>\n<p>In that post, I argued that C doesn’t “work like the computer”, but rather, by a concept called the “C abstract machine.” It happens to be close to how computers operate in some sense, but that when you write C, you’re writing for the abstract machine, not the computer itself. Here’s the trick, though: the C abstract machine doesn’t define every single aspect of how computation happens. And so sometimes, you can write code in two different ways, that the C abstract machine says are equivalent, yet are <em>not</em> equivalent in performance. This means that, even if you buy into my original point, you still need to know the boundaries of this definition, and can exploit those boundaries in important ways.</p>\n<p>Let’s start with a made-up example, and then we’ll show how this happens in real code. Imagine that we have a very simple programming language, built on an abstract machine. It defines only the semantics for the multiplication of two 32-bit integers. It also ignores things like overflow. You could write a program like “3 * 4” and this machine would say “hey the result is 12.” Let’s also imagine we have a very simple computer: it has a single instruction, “add two numbers together.” This takes one clock cycle to execute. If we wrote a compiler for this machine, and didn’t apply any optimizations, it would execute “4 + 4”, and then take that result and add four to it. We have two add operations, and so our program takes two clock cycles.</p>\n<p>Let’s imagine a new version of our computer comes out. It adds a new instruction: multiplication. This also takes one cycle, but can multiply two numbers directly. Our compiler adds a new back-end, and when compiling our “3 * 4” program for this fancy new machine, it emits a single multiplication instruction. Our program now takes one clock cycle.</p>\n<p>Now imagine a new version of our compiler. It is able to do “constant folding”, an optimization. Our program does no computation at all, only produces a “12” directly. This now means our program takes zero cycles on both bits of hardware.</p>\n<p>Is all of this okay?</p>\n<p>Yes! The machine only defines how the <code>*</code> operator works on numbers in a “what number comes out the other side” sense. It doesn’t specify how to actually accomplish this task. Compilers for our little language are free to implement the details in any way they choose, as long as they respect the rules of the abstract machine. Other details are fair game. As long as the program “3 * 4” produces “12”, the rest of the computational process doesn’t actually matter.</p>\n<p>This is how you can write “portable” code that is portable in one sense, but not at all in another sense. As long as you respect the rules of the abstract machine, you’ll be guaranteed (we live in a world with no compiler bugs, of course) to get the correct result. If it takes too long, or takes no time at all, that’s not the language’s concern, strictly speaking.</p>\n<p>If this idea seems silly to you, well, I can assure you it’s real. For example, let’s consider the actual C language, and some actual hardware, specifically, x86.</p>\n<p>Imagine we have an array of arrays, four by four:</p>\n<pre><code>0\t0\t0\t0\n0\t0\t0\t0\n0\t0\t0\t0\n0\t0\t0\t0\n</code></pre>\n<p>There are two naive ways to process it: row by row:</p>\n<pre><code>1\t2\t3\t4\n5\t6\t7\t8\n9\t10\t11\t12\n13\t14\t15\t16\n</code></pre>\n<p>or column by column:</p>\n<pre><code>1\t5\t9\t13\t\n2\t6\t10\t14\n3\t7\t11\t15\n4\t8\t12\t16\n</code></pre>\n<p>Here’s a fun question: is one of these faster than the other?</p>\n<p>At this small size, probably not. But there’s a way to do this that’s faster than either of these ways, and that’s called “blocking” or “tiling”. It means that you process a sub-set, or “block”, of the array each time. Something like this:</p>\n<pre><code>1\t2\t5\t6\n3\t4\t7\t8\n9\t10\t13\t14\n11\t12\t15\t16\n</code></pre>\n<p>On x86, this is <em>meaningfully</em> faster in many circumstances. This has to do with the way that the CPU caches information. For more, see <a href=\"https://software.intel.com/en-us/articles/how-to-use-loop-blocking-to-optimize-memory-use-on-32-bit-intel-architecture\">this page from Intel</a>, including a visualization that’s probably better than mine.</p>\n<p>This raises an interesting point: is this code portable? Imagine a machine that didn’t have the cache behavior of x86 chips here. This code would run on them, but it may be 14 times slower. Strictly speaking, this program is “portable” in the sense that you’ll get the right computation in the end, but maybe this speed is part of the user requirements for this software to be successful, and so being super slow means that it “doesn’t work” according to those requirements, even though from a programming language point of view, it’s all good.</p>\n<p>Because C’s abstract machine is so thin, these kinds of details can really, really matter, as we’ve seen. And this is where the good part of the “C teaches you how the computer works” meme comes in. Because the machine is thinner, you can learn more about the details, and as long as they’re not details that the C abstract machine cares about, exploit them. And there’s a good reason I mentioned an optimizing compiler above; it’s also the job of compiler authors to realize the difference between a given computer and the C abstract machine, and then exploit the difference to make your code go as fast as possible. But this really blurs the line between abstract machine and physical machine. And that’s why we all argue online about this all the time.</p>\n<hr />\n<p>I have a third post to wrap up this series. I hope it won’t take me another year, but no guarantees…</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/porting-steveklabnik-com-to-workers-sites-and-zola/",
"title": "Porting steveklabnik.com to Workers Sites and Zola",
"description": null,
"url": "https://steveklabnik.com/writing/porting-steveklabnik-com-to-workers-sites-and-zola/",
"published": "2019-09-26T00:00:00.000Z",
"updated": "2019-09-26T00:00:00.000Z",
"content": "<p>One fun thing about having a personal domain is that you can keep the contents the same forever, but take a few days every so often to port it to some new stack you want to try out.</p>\n<p>A while back, I had my site on <a href=\"https://pages.github.com/\">GitHub Pages</a> using <a href=\"https://jekyllrb.com/\">Jekyll</a>. This has long been the sort of default, go-to stack for me. At some point, I decided to simplify even further, and wrote a small script to take the few Markdown files I have and just make HTML out of them. I then used <a href=\"https://www.netlify.com/\">Netlify</a> to do the hosting. I wanted to try out their legendary ease of use, and it was indeed super simple.</p>\n<p>Earlier this year, I <a href=\"https://words.steveklabnik.com/i-m-joining-cloudflare\">took a job at Cloudflare</a> to work on part of <a href=\"https://workers.cloudflare.com/\">Cloudflare Workers</a>. To get more familiar with the project I’d be working on, <a href=\"https://www.cloudflare.com/products/workers-kv/\">Workers KV</a>, I came up with a plan: I’d put the Markdown files into KV, and then write a worker in Rust and WebAssembly that would generate the HTML on the fly, and run that in a Worker. This worked really well, even though it was <em>incredibly</em> overkill for a simple static site.</p>\n<p>Today we’re launching a new product, <a href=\"https://blog.cloudflare.com/workers-sites/\">Workers Sites</a>. You see, I wasn’t the only one who was interested in using Workers for static site hosting; I’d see tweets every so often where others were doing the same. But doing so was a completely manual process, you had to write the code on your own, you had to do all the setup yourself, and there was no assistance in doing so. Workers Sites changes this, and makes it easy to host your own site, if you’d like.</p>\n<p>The rest of this post is going to walk you through my porting process for <a href=\"https://www.steveklabnik.com/\">steveklabnik.com</a>.</p>\n<hr />\n<p>I decided to go with <a href=\"https://www.getzola.org/\">Zola</a> for my static site generator, as it’s written in Rust. I’d been wanting to give Zola a try for a while, and this was a good excuse to do so.</p>\n<p>I needed these pre-requisites:</p>\n<ul>\n<li>Zola, instructions <a href=\"https://www.getzola.org/documentation/getting-started/installation/\">here</a>.</li>\n<li>Wrangler, instructions <a href=\"https://github.com/cloudflare/wrangler\">here</a></li>\n<li>A Cloudflare Workers account, <a href=\"https://workers.cloudflare.com/\">here</a> if you don’t have a Cloudflare account already, and if you do, you can click the Workers tab in your dashboard to set it up.</li>\n</ul>\n<p>Additionally, you need to have a $5/month subscription to Workers, to get access to Workers KV. I want us to offer a free tier of KV in the future, but we’re not there yet. That $5/month gets you a lot more than one site, to be clear, but this isn’t a completely free solution today.</p>\n<p>From there, it was time to make a new site with Zola:</p>\n<pre><code>> zola init steveklabnik.com\n> cd steveklabnik.com\n</code></pre>\n<p>Zola places the contents of your site in the <code>contents</code> directory. I only have one level for my site, so I dumped my two pages in there, as well as re-named <code>index.md</code> to <code>_index.md</code> to follow Zola convention.</p>\n<p>I had a few YAML headers on pages, I had to convert those to Zola’s TOML format:</p>\n<pre><code>+++\ntitle = \"Deleuzional\"\n+++\nI really like the writing of [Gilles Deleuze]. I need\n</code></pre>\n<p>The <code>+++</code>s are like <code>---</code> in Jekyll, and the metadata is in TOML instead of YAML. I only needed <code>title</code> because I am keeping things very simple.</p>\n<p>I then needed to set up templates. Like most static site generators, the template forms the outer contents of the HTML on a page, and renders the Markdown into the middle somewhere. I have a slightly different template for my root than the other two pages, and Zola handles this pretty well.</p>\n<pre><code>> ls .\\templates\\\n\n Directory: C:\\Users\\Steve Klabnik\\src\\steveklabnik.com\\templates\n\nMode LastWriteTime Length Name\n---- ------------- ------ ----\n-a---- 9/25/2019 5:28 PM 1288 index.html\n-a---- 9/25/2019 5:28 PM 1234 page.html\n</code></pre>\n<p><code>index.html</code> is for the root page, and <code>page.html</code> is for the other pages. This is a Zola convention.</p>\n<p>Zola templates are written in <a href=\"https://tera.netlify.com/docs/templates/\">Tera</a>, which is sorta like <a href=\"https://shopify.github.io/liquid/\">Liquid</a> or <a href=\"https://jinja.palletsprojects.com/en/2.10.x/\">Jinja2</a>. Here’s a snippit of my template:</p>\n<pre><code> <body>\n <h2>{{ page.title }}</h2>\n {{page.content | safe}}\n </body>\n</code></pre>\n<p>The <code>| safe</code> bit says that it shouldn’t escape the HTML produced by <code>page.content</code>, which is the rendered contents of the Markdown pages.</p>\n<p>Finally, I have some static assets for the site: a CSS file, some JS, and the favicon, etc. These get dumped in the <code>static</code> directory, and Zola will copy them over when I build the site.</p>\n<p>Previewing during this process is very useful:</p>\n<pre><code>> zola serve\n</code></pre>\n<p>It took me a bit to figure out the conventions and get everything in place, and being able to change the contents and watch my browser refresh was invaluable.</p>\n<p>Once I had my site ported to Zola, it’s time to deploy it to Workers Sites!</p>\n<p>Wrangler is the command-line tool for Cloudflare Workers. If you haven’t used it before, you’ll have to log in:</p>\n<pre><code>> wrangler config\n</code></pre>\n<p>To check that your auth is set up correctly, you should run this command:</p>\n<pre><code>> wrangler whoami\n You are logged in with the email '[email protected]'.\n</code></pre>\n<p>Of course, this should show your own email, not mine.</p>\n<p>Let’s add Workers Sites to our project:</p>\n<pre><code>> wrangler init --site website\n</code></pre>\n<p>This creates two main things: <code>wrangler.toml</code>, used to configure Wrangler, and <code>workers-sites</code>, a directory with your worker in it. I used the extra <code>website</code> parameter to wrangler becuase it defaults to the name of the subdirectory, and the period in <code>steveklabnik.com</code> is invalid in a worker name, so I chose <code>website</code>. You can put whatever you’d like.</p>\n<p>We need to set up <code>wrangler.toml</code>:</p>\n<pre><code>> code wrangler.toml\n</code></pre>\n<p>Mine looks like this:</p>\n<pre><code>account_id = \"f5518bfbf88c230551a64797c221c7da\"\nname = \"website\"\ntype = \"webpack\"\nroute = \"www.steveklabnik.com/*\"\nzone_id = \"9359b254c46ff9f5c546203686794862\"\nworkers_dev = false\n\n[site]\nbucket = \"./public\"\nentry-point = \"workers-site\"\n</code></pre>\n<p>All of these IDs are fine to be public; <code>wrangler</code> stores your sensitive stuff in <code>~/.wrangler</code>.</p>\n<p>Important bits:</p>\n<p>The <code>route</code> needs to be set up properly; this one means that the worker will serve everything on <code>www.steveklabnik.com</code>. My account and zone IDs are needed; you can find these on the Cloudflare dashboard on the bottom right of the main page for it. <code>workers_dev</code> is false; I’m deploying to my site, not to a subdomain of <code>workers.dev</code>. If we did, we wouldn’t need the <code>zone_id</code>, or <code>route</code>. Finally, the <code>bucket</code> setting needs to be set to whatever your site generator’s output directory is, which for Zola is <code>public</code>.</p>\n<p>And that’s it! You can test out what this looks like:</p>\n<pre><code>> wrangler preview\n</code></pre>\n<p>If that all looks good, we can deploy:</p>\n<pre><code>> zola build\n> wrangler publish\n</code></pre>\n<p>And that’s it! And wow is it fast:</p>\n<p><img src=\"https://svbtleusercontent.com/hS1xpZgYuQ9sRhUxP1iqq20xspap_small.png\" alt=\"https://svbtleusercontent.com/hS1xpZgYuQ9sRhUxP1iqq20xspap_small.png\" /></p>\n<hr />\n<p>… well, we’re <em>almost</em> done. Right now, this means I can only publish from my laptop. I’d prefer to have GitHub deploy this. We don’t have that as a built-in feature today, so I’d have to write an Action to do so. I haven’t yet, so I can’t blog about that. Maybe next time.</p>\n<p>Additionally, there’s one more twist. Zola generates different URLs than my previous ones. I had previously had <code>.html</code> on the ends of my pages, but Zola does not do this.</p>\n<ul>\n<li>old: https://www.steveklabnik.com/deleuzional.html</li>\n<li>new: https://www.steveklabnik.com/deleuzional/</li>\n</ul>\n<p>I think I’d prefer the new style anyway, so rather than figure out how to get Zola to do this, let’s set up some redirects. This is one cool thing about using Workers for this kind of task; while it’s slightly more manual, we can also make customizations very easily. The contents of the Worker that serves up the site lives in <code>workers-sites/index.js</code>. It’s very short.</p>\n<pre><code>import { getAssetFromKV } from \"@cloudflare/kv-asset-handler\";\n\naddEventListener(\"fetch\", event => {\n event.respondWith(handleEvent(event));\n});\n\nasync function handleEvent(event) {\n try {\n return await getAssetFromKV(event);\n } catch (e) {\n let pathname = new URL(event.request.url).pathname;\n return new Response(`\"${pathname}\" not found`, {\n status: 404,\n statusText: \"not found\"\n });\n }\n}\n</code></pre>\n<p>Given that I only have two URLs, rather than doing something big and fancy, I did something dumb and straightforward:</p>\n<pre><code>import { getAssetFromKV } from \"@cloudflare/kv-asset-handler\";\n\naddEventListener(\"fetch\", event => {\n event.respondWith(handleEvent(event));\n});\n\nasync function handleEvent(event) {\n\n // redirect some old URLs\n let pathname = new URL(event.request.url).pathname;\n\n if (pathname === \"/security.html\") {\n return Response.redirect(\"https://www.steveklabnik.com/security/\", 301);\n } else if (pathname === \"/deleuzional.html\") {\n return Response.redirect(\"https://www.steveklabnik.com/deleuzional/\", 301);\n }\n\n try {\n return await getAssetFromKV(event);\n } catch (e) {\n let pathname = new URL(event.request.url).pathname;\n return new Response(`\"${pathname}\" not found`, {\n status: 404,\n statusText: \"not found\"\n });\n }\n}\n</code></pre>\n<p>I’m constructing <code>pathname</code> twice here, but I just don’t care enough to bother to refactor this out. Additionally, it’s nice to purely add, rather than customize what was generated too much.</p>\n<p>But with that, and another <code>zola build && wrangler publish</code>, I have the redirects set up for my site.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-culture-war-at-the-heart-of-open-source/",
"title": "The culture war at the heart of open source",
"description": null,
"url": "https://steveklabnik.com/writing/the-culture-war-at-the-heart-of-open-source/",
"published": "2019-05-26T00:00:00.000Z",
"updated": "2019-05-26T00:00:00.000Z",
"content": "<p>There’s a war going on. When isn’t there a war going on? But I’m not talking about a physical war here: I’m talking about a war over meaning. This particular war is a fight over what “open source” means.</p>\n<p>Let’s take a few steps back.</p>\n<h2>The Free Software Foundation</h2>\n<p>People organize into groups for many reasons. This story starts with the story of an organization called the “GNU Project.” It was started in 1983, and <a href=\"https://groups.google.com/forum/#!msg/net.unix-wizards/8twfRPM79u0/1xlglzrWrU0J\">here’s the initial announcement on USENET</a>. I’ve pulled out four important paragraphs:</p>\n<blockquote>\n<p>Starting this Thanksgiving I am going to write a complete Unix-compatible software system called GNU (for Gnu’s Not Unix), and give it away free to everyone who can use it. Contributions of time, money, programs and equipment are greatly needed.I consider that the golden rule requires that if I like a program I must share it with other people who like it. I cannot in good conscience sign a nondisclosure agreement or a software license agreement.So that I can continue to use computers without violating my principles, I have decided to put together a sufficient body of free software so that I will be able to get along without any software that is not free.If I get donations of money, I may be able to hire a few people full or part time. The salary won’t be high, but I’m looking for people for whom knowing they are helping humanity is as important as money. I view this as a way of enabling dedicated people to devote their full energies to working on GNU by sparing them the need to make a living in another way.</p>\n</blockquote>\n<p>There’s a lot of different ways to talk about this post: it contains all sorts of interesting things. But for right now, I want to talk about the motivations here. There were two main ones: first, to produce a software artifact. The second one is the big one: the artifact needs to be produced because existing ones were not compatible with a particular set of values.</p>\n<p>The word “ideology” is a tricky one, but let’s go with this definition, which is <a href=\"https://en.wikipedia.org/wiki/Ideology\">from Wikipedia</a>:</p>\n<blockquote>\n<p>An ideology is a collection of normative beliefs and values that an individual or group holds for other than purely epistemic reasons.</p>\n</blockquote>\n<p>The GNU project was formed to produce software according to a particular ideology: one of sharing software. I’m not here to argue if the project has accomplished this goal, or if this goal is good or not. My point is that the origins of the GNU project were specifically motivated by a collection of normative beliefs and values.</p>\n<p>Two years later, the Free Software Foundation would be formed to support the GNU project, and promote the concept of Free Software. Free Software was software that aligns with the ideology of the GNU project, and a definition of Free Software was created and published in Feburary of 1986. Here is that definition:</p>\n<blockquote>\n<p>The word “free” in our name does not refer to price; it refers to freedom. First, the freedom to copy a program and redistribute it to your neighbors, so that they can use it as well as you. Second, the freedom to change a program, so that you can control it instead of it controlling you; for this, the source code must be made available to you.</p>\n</blockquote>\n<p>Since then, the Free Software Definition has expanded to four points. You can read the current definition <a href=\"https://www.gnu.org/philosophy/free-sw.en.html\">here</a>.</p>\n<h2>Open Source appears</h2>\n<p>A decade passes, and trouble is brewing. I’ll <a href=\"https://en.wikipedia.org/wiki/Open_source#Origins\">quote wikipedia again</a>:</p>\n<blockquote>\n<p>[T]he modern meaning of the term “open source” was first proposed by a group of people in the free software movement who were critical of the political agenda and moral philosophy implied in the term “free software” and sought to reframe the discourse to reflect a more commercially minded position. In addition, the ambiguity of the term “free software” was seen as discouraging business adoption. The group included Christine Peterson, Todd Anderson, Larry Augustin, Jon Hall, Sam Ockman, Michael Tiemann and Eric S. Raymond. Peterson suggested “open source” at a meeting held at Palo Alto, California, in reaction to Netscape’s announcement in January 1998 of a source code release for Navigator. Linus Torvalds gave his support the following day, and Phil Hughes backed the term in Linux Journal.</p>\n</blockquote>\n<p>Here we see the creation of the open source movement. If you’d prefer a primary source, here’s <a href=\"http://www.catb.org/~esr/open-source.html\">Eric S. Raymond</a>:</p>\n<blockquote>\n<p>Specifically, we have a problem with the term “free software”, itself, not the concept. I’ve become convinced that the term has to go.The problem with it is twofold. First, it’s confusing; the term “free” is very ambiguous (something the Free Software Foundation’s propaganda has to wrestle with constantly). Does “free” mean “no money charged?” or does it mean “free to be modified by anyone”, or something else?Second, the term makes a lot of corporate types nervous. While this does not intrinsically bother me in the least, we now have a pragmatic interest in converting these people rather than thumbing our noses at them. There’s now a chance we can make serious gains in the mainstream business world without compromising our ideals and commitment to technical excellence – so it’s time to reposition. We need a new and better label.</p>\n</blockquote>\n<p>Shortly after, the <a href=\"http://opensource.org/\">Open Source Initiative</a> was founded. A sort of mirror of the FSF, the OSI would support the promotion of the term “open source,” and the ideology behind it. Like the Free Software Definition, the <a href=\"https://opensource.org/osd\">Open Source Definition</a> was created, derived from Debian’s guidelines for producing Free Software.</p>\n<p>Again, we have an organization that’s created along ideological lines. But in this case, slightly different ones. <a href=\"https://web.archive.org/web/20090413035630/https://opensource.org/history\">An older version of the OSI website says</a>:</p>\n<blockquote>\n<p>The conferees decided it was time to dump the moralizing and confrontational attitude that had been associated with “free software” in the past and sell the idea strictly on the same pragmatic, business-case grounds that had motivated Netscape.</p>\n</blockquote>\n<p><a href=\"https://opensource.org/history\">Today’s version says</a>:</p>\n<blockquote>\n<p>The conferees believed the pragmatic, business-case grounds that had motivated Netscape to release their code illustrated a valuable way to engage with potential software users and developers, and convince them to create and improve source code by participating in an engaged community. The conferees also believed that it would be useful to have a single label that identified this approach and distinguished it from the philosophically- and politically-focused label “free software.”</p>\n</blockquote>\n<p>The idea here is plain: Free Software, but for business.</p>\n<p>For more on this topic, I recommend <a href=\"https://thebaffler.com/salvos/the-meme-hustler\">The Meme Hustler</a>.</p>\n<h2>Moving forward</h2>\n<p>Twenty years have passed, and a lot has changed.</p>\n<p>For ideological movements to persist, they need to re-produce their values in new membership. And for a while, the FSF and OSI did a good job of that. These two movements, Free Software and Open Source, produced a lot of software, and gained a lot of new converts. But then… something happened.</p>\n<p>I’m not sure exactly how it happened. I think the lazy answer is “GitHub!!!!”. I do think GitHub played a role, but I think the answer is more complex than that. I personally think that gender plays a huge role. But that’s a different essay. Regardless of <em>why</em> it happened, something did happen.</p>\n<p>Somewhere along the way, Open Source ran into a problem that many movements face: the members of the movement no longer understood the ideology that created the movement in the first place.</p>\n<p>If you ask a random developer what “open source” means to them, you won’t often hear “software that follows the Open Source Definition.” If you ask them “what’s the difference between free software and open source software,” you’ll often hear “aren’t those the same thing?” or “you can charge money for open source software, it’s not always free.” You may even hear “it’s on GitHub.”</p>\n<p>In talking to developers about open source, you’ll also hear something else. Something that’s been itching in the back of my brain for a while, and the thing that led me to write this. You’ll often hear developers talk about how the relationship between businesses and open source developers is messy. Complicated. Businesses don’t “give back enough,” won’t pay people to work on open source. That it’s all take and no give. I’ve said a lot of that stuff myself.</p>\n<p>But here’s the thing: that’s why the concept of open source was created in the first place. It’s there, plain as day, if you ask the folks who actually created it. Open source was a way to make free software better for businesses. This doesn’t mean that it’s beyond critique, of course. The relationship can improve. I don’t think these developers are stupid, or hypocrites.</p>\n<p>This is where we’re at today. The Free Software movement was created, and then Open Source was created as a reaction to that. Today’s developers have never learned about this history, or don’t care about it, or actively think it’s irrelevant. And so we have a war. A war for the meaning of open source. A war fought with tweets, and blog posts, and discussions. A war between the old guard, those who created the idea originally, and the new generation, the developers who have taken the base idea and run with it.</p>\n<p>I think history will repeat itself. In the same way that the open source movement said “we’re like free software, but with these changes,” I think we’ll end up with a new movement. For the same reasons that “open source” came up with a new name, I think the movement that will arise from today’s developers will also need a new name. I’m not sure what that movement will look like, and I’ll explore why in another post. To give you a teaser: the problem is in the way that both Free Software and Open Source are formulated. The rot is in the roots, and I’m not yet sure what will replace it.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/i-m-joining-cloudflare/",
"title": "I'm joining Cloudflare",
"description": null,
"url": "https://steveklabnik.com/writing/i-m-joining-cloudflare/",
"published": "2019-04-05T00:00:00.000Z",
"updated": "2019-04-05T00:00:00.000Z",
"content": "<p><img src=\"https://svbtleusercontent.com/jpH1brqXUvSAz1tNQ8uVKb0xspap_small.png\" alt=\"https://svbtleusercontent.com/jpH1brqXUvSAz1tNQ8uVKb0xspap_small.png\" /></p>\n<p>Yesterday, I signed an offer letter from Cloudflare. I’ll be the product manager of Storage, which is the part of the company that’s working on data storage products for the <a href=\"https://www.cloudflare.com/products/cloudflare-workers/\">Cloudflare Workers</a> platform. I’ll be starting at the end of the month.</p>\n<p>I think a lot of you will say “that makes perfect sense,” but many more will say “wait, what?”. I hope this post will make things a bit more clear.</p>\n<p>Before we start, I also want to say that I’ll still be continuing my work on Rust, though obviously in a more limited capacity. I’ll still be working on revisions to <a href=\"https://doc.rust-lang.org/stable/book/\">The Rust Programming Language</a>, and doing core team work, for example.</p>\n<hr />\n<p>Before I worked on Rust, I worked on and with Ruby on Rails. When I got out of the web application game, the hot topic was this new “platform as a service” stuff. I’ve been watching all of the latest changes from afar, and they’ve been quite interesting. I’ve always loved the web, and the people building it. In the last 18 months or so, I’ve also been extremely interested in WebAssembly. Working at Cloudflare is the intersection of four things for me:</p>\n<ul>\n<li>Edge compute</li>\n<li>WebAssembly</li>\n<li>Rust</li>\n<li>Personal and career growth</li>\n</ul>\n<h2>Edge Compute</h2>\n<p>There’s sort of been four eras of “how do I web server”:</p>\n<ul>\n<li>Physical servers (“on premise”)</li>\n<li>Infrastructure as a service</li>\n<li>Platform as a service</li>\n<li>Function as a service</li>\n</ul>\n<p>A lot of people see the relationship between these things as “what you control” vs “what does the provider control”, but I don’t think that’s quite right. But I’m getting ahead of myself.</p>\n<p>Back in the day, if you wanted a web site, you bought a physical server, and you put it in a data center somewhere. I wasn’t around for those days. The point is, you managed everything: the hardware, the software. All of it.</p>\n<p>In <a href=\"https://www.shapeways.com/blog/archives/305-CloudFab-Distributed-Fabrication-3D-printing,-Nick-Pinkston-interview.html\">my days as a startup CTO</a>, my services were deployed to a Virtual Private Server. Instead of owning servers ourselves, we rented a virtual machine, running on someone else’s servers. This is “infrastructure as a service”; I didn’t have to worry about machines, I could just ask for one, and it’d be given to me. I still had to manage the operating system and everything above it.</p>\n<blockquote>\n<p>Fun side note: CloudFab was a Rails application with a big C extension to do data processing that was too much for Ruby. This later informed a lot of my Rust work.</p>\n</blockquote>\n<p>Next up came “platform as a service.” To me, Heroku was the platonic ideal of this era. Now, you didn’t even need to manage servers. You got to worry about your application, and that’s it. Maybe you’d need to set a slider to scale during times of demand.</p>\n<p>I’ve been reading a lot about “functions as a service” over the past few years. Instead of deploying an entire application, each endpoint becomes an application, basically. This is where I think the previous thinking about the relationships between these categories of service is a little wrong. The actual underlying trend here is something else: what affordances do I have to scale?</p>\n<p>In the on-premise days, if I wanted to add capacity, I’d need to buy new servers, install them, set everything up. There’s a <em>lot</em> of latency there. And if I need to remove capacity, well, I have to sell those servers? The move to infrastructure as a service was significant, because it more clearly separated concerns. Each company didn’t need a data center; a data center provides homogeneous compute. As a user, if I need to scale, I can spin up some more virtual servers much more quickly than I can physical ones. Managing that required a lot of tooling, however.</p>\n<p>This was the insight that led to platform as a service: by managing this tooling, you could make it even easier for users to scale. There’s a reason why the slider became the ubiquitous synonym with PaaS. There’s nothing that you could do with PaaS that was impossible with IaaS, but it was much, much easier.</p>\n<p>This brings us to Functions as a Service, also known as “serverless.” The reason that this architecture matters is one very similar to previous insights. By breaking your application into multiple functions, they can scale independently. This is one of the dreams of microservice architecture, but there’s a bit of a difference. Microservice architecture focuses on how you build your system, and how you split up components internally. FaaS focuses on how people interact with your system, and splits up the components according to that. This framing is a bit more significant to your ability to scale, because it means that during times of high load, you can scale only the parts under that load. In the same way that IaaS said “let’s not force people to buy servers to scale” and PaaS said “let’s not force people to build out infrastructure to scale”, FaaS says “let’s not force people to scale their help page to scale their shopping cart.”</p>\n<p>This brings us to edge compute. Edge compute is a particular kind of FaaS, but an interesting take on it. Here’s the concept: your users are all over the world. Why do their packets need to go to <code>us-east-1</code> and back? To which you might say, “sure, that’s why we’ve developed our application to work in multiple regions.” That’s the same relationship between IaaS and PaaS. That is, why should you have to worry about making your application be available across multiple regions? You may not be worrying about individual physical servers anymore, but you’re still worrying about data centers. And some FaaS products, like AWS Lambda, can already run easily across multiple regions, and with some setup, go between them. However, Amazon has about 20 total regions.</p>\n<p>So how is edge compute different? Well, at some point, various CDN companies realized “wait a minute. We have severs around the world. Why can’t we run compute on them as well?”</p>\n<blockquote>\n<p>Side note: Cloudflare is not really a CDN company, though that’s how I always thought of them. They do a lot more stuff than CDNs.</p>\n</blockquote>\n<p>This is interesting because, well, <a href=\"https://aws.amazon.com/about-aws/global-infrastructure/\">here’s those 20 Amazon regions</a>:</p>\n<p><img src=\"https://svbtleusercontent.com/qxGa9h6TshtRrXZmh6tsRU0xspap_small.png\" alt=\"https://svbtleusercontent.com/qxGa9h6TshtRrXZmh6tsRU0xspap_small.png\" /></p>\n<p>Compare this with <a href=\"https://www.fastly.com/network-map\">Fastly’s network</a>:</p>\n<p><img src=\"https://svbtleusercontent.com/ePzMAv7Q4NVLARUKambWH80xspap_small.jpg\" alt=\"https://svbtleusercontent.com/ePzMAv7Q4NVLARUKambWH80xspap_small.jpg\" /></p>\n<p>And finally, <a href=\"https://www.cloudflare.com/network/\">Cloudflare’s network</a>:</p>\n<p><img src=\"https://svbtleusercontent.com/bQzn9XqBY98mw269Q1tsdH0xspap_small.png\" alt=\"https://svbtleusercontent.com/bQzn9XqBY98mw269Q1tsdH0xspap_small.png\" /></p>\n<p>(the page is animated, so those highlighted ones don’t mean anything)</p>\n<p>You’re much likely to be physically closer to a Fastly or Cloudflare server than you are an AWS one. And that’s what we mean by “edge compute”, that is, you’re running your code on the edge, rather than in the data center.</p>\n<p>Now, I should also mention that there is a form of Lambda called Lambda@Edge that runs off of CloudFront locations, rather than the above map. So that looks like this:</p>\n<p><img src=\"https://svbtleusercontent.com/vwJvtVvGK7DpGZYEPcrjzW0xspap_small.png\" alt=\"https://svbtleusercontent.com/vwJvtVvGK7DpGZYEPcrjzW0xspap_small.png\" /></p>\n<p>Much better! My point about regions is more applicable to “edge compute” vs more general compute, than saying something about Lambda as a particular product.</p>\n<p>CloudFlare Workers don’t require you to think about where your service is deployed: when you upload it, it’s automatically running on all 175 locations, within minutes.</p>\n<p>I think this space is interesting and significant. Cloudflare has <a href=\"https://www.cloudflare.com/products/cloudflare-workers/\">Workers</a>, and Fastly has <a href=\"https://wasm.fastlylabs.com/\">Terrarium</a>. Amazon has <a href=\"https://docs.aws.amazon.com/lambda/latest/dg/lambda-edge.html\">Lambda@Edge</a>. These are all very interesting platforms for building high performance applications.</p>\n<p>My role as part of Storage will be to consider “what does data access and storage look like in this world?” If your code is moved to the edge, but your data is still in a central server, you don’t gain the full benefit of having the code close to the client. There’s a lot of interesting stuff in this space!</p>\n<h2>WebAssembly</h2>\n<p>Both ClouldFlare Workers and Fastly’s Terrarium use <a href=\"https://webassembly.org/\">WebAssembly</a> as a core part of the platform. And that’s significant:</p>\n<blockquote>\n<p>If WASM+WASI existed in 2008, we wouldn't have needed to created Docker. That's how important it is. Webassembly on the server is the future of computing. A standardized system interface was the missing link. Let's hope WASI is up to the task! https://t.co/wnXQg4kwa4\n— Solomon Hykes (@solomonstre) March 27, 2019</p>\n</blockquote>\n<p>In order for this stuff to work, you need a runtime for WebAssembly. One interesting thing we’re seeing in this space is sort of two parallel paths emerge: do you want to support JavaScript, or not? Recently, Fastly open sourced <a href=\"https://www.fastly.com/blog/announcing-lucet-fastly-native-webassembly-compiler-runtime\">Lucet</a>, the WebAssembly runtime that powers Terrarium. It’s an impressive bit of tech. There are others too, like <a href=\"https://wasmer.io/\">Wasmer</a> and <a href=\"https://github.com/CraneStation/wasmtime\">Wasmtime</a>. By focusing purely on WebAssembly, you can build specialized tech, and it can be really, really fast.</p>\n<p>However, you can’t compile JavaScript to WebAssembly. You can sort of do it with AssemblyScript, a TypeScript subset the compiles to wasm. But JavaScript is arguably the most popular programming language in the world. I personally believe in JavaScript, even with the rise of WebAssembly. And so Cloudflare is taking a different approach than those projects. Instead of building a wasm-only runtime, they’re building on top of Google’s V8. This means that they can support both WebAssembly and JavaScript in workers. Additionally, by leaning on V8, you can take advantage of all of the excellent engineering resources that Google pours into it, similar to how the Rust compiler gets to take advantage of people working on improving LLVM.</p>\n<p>Anyway, the point is, WebAssembly is core to this new edge compute world. And so I’m excited to be in the space. I know people that work at Fastly, wasmer, and Mozilla, and they’re all doing great work. I think there’s space for both approaches, but I’m mostly excited to be there and see how it all turns out.</p>\n<p>Oh, one other thing I wanted to say: right now, everything around Workers is closed source. Kudos to Fastly for open-sourcing Lucet. I asked about this in my interview, and Cloudflare is very interested in doing more open source work, and so I’m hoping it doesn’t take us long to catch up in this regard. We’ll see, of course, I haven’t actually started yet. But I think that this stuff needs to be open source, personally.</p>\n<h2>Rust</h2>\n<p>You may remember a little story called <a href=\"https://en.wikipedia.org/wiki/Cloudbleed\">CloudBleed</a>. In short, Cloudflare had a pretty bad security bug in 2017. This bug happened because of a parser, written in Ragel. Here, <a href=\"https://blog.cloudflare.com/incident-report-on-memory-leak-caused-by-cloudflare-parser-bug/\">I’ll let Cloudflare explain it</a>:</p>\n<blockquote>\n<p>The Ragel code is converted into generated C code which is then compiled. The C code uses, in the classic C manner, pointers to the HTML document being parsed, and Ragel itself gives the user a lot of control of the movement of those pointers. The underlying bug occurs because of a pointer error.</p>\n</blockquote>\n<p>CloudBleed happened due to memory unsafety. When talking to Cloudflare today, it’s pretty clear that this bug was taken extremely seriously, to the point where it created a culture change inside the company itself. These days, it seems to me (I haven’t started yet, mind you) that you have to use a memory safe language by default, and only use a memory unsafe one if you have an extremely good justification. But Cloudflare needs to be able to have a lot of performance, and control latency, in much of their stack.</p>\n<p>Enter Rust. Cloudflare started exploring Rust after CloudBleed, and now uses a significant amount of Rust in production. They host the local Rust meetup here in Austin, and sometimes the one in San Francisco. I’m excited to help Cloudflare be successful with Rust, and to use that experience to help improve Rust too.</p>\n<p>My understanding is that I’ll be dealing with a bunch of TypeScript and possibly Go as well, which is also pretty cool.</p>\n<h2>Career growth</h2>\n<p>Finally, the move to product management. I really enjoy product work, and have had several jobs that were sorta-kinda product jobs. I don’t think that management jobs are inherently better than engineer jobs, of course, but it’s a way for me to grow. My role at Balanced was sorta-kinda-PM-y, and I loved that job.</p>\n<p>I don’t have too much more to say about this, just that it’s a bit different, but something I’m really excited to do.</p>\n<h2>I’m excited for the future</h2>\n<p>So yeah, that’s my extremely long-winded explanation of what’s going on. I’m not starting until the end of the month, so I’ve got a few weeks to purely enjoy. I’ll still be playing a lot of <a href=\"http://www.celestegame.com/\">Celeste</a> and Fortnite, writing blog posts, and working on some Rust open source. But then I get to get back to work. I’m pumped about all of this! New role, building some new tech with tech I already love.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/what-comes-after-open-source/",
"title": "What comes after open source",
"description": null,
"url": "https://steveklabnik.com/writing/what-comes-after-open-source/",
"published": "2019-04-02T00:00:00.000Z",
"updated": "2019-04-02T00:00:00.000Z",
"content": "<p>In <a href=\"https://words.steveklabnik.com/the-culture-war-at-the-heart-of-open-source\">a previous post</a>, I discussed the history of open source, and ended with this claim:</p>\n<blockquote>\n<p>Today’s developers have never learned about this history, or don’t care about it, or actively think it’s irrelevant. … For the same reasons that “open source” came up with a new name, I think the movement that will arise from today’s developers will also need a new name.</p>\n</blockquote>\n<p>We talked about the ideological history of open source, but that’s not what developers object to, really. I don’t think developers are moving back towards a world of making source code private. Instead, it’s something related to a very old discussion in free software. To quote <a href=\"https://www.gnu.org/philosophy/free-sw.en.html\">the FSF</a>:</p>\n<blockquote>\n<p>“Free software” means software that respects users’ freedom and community. Roughly, it means that the users have the freedom to run, copy, distribute, study, change and improve the software. Thus, “free software” is a matter of liberty, not price. To understand the concept, you should think of “free” as in “free speech,” not as in “free beer”. We sometimes call it “libre software,” borrowing the French or Spanish word for “free” as in freedom, to show we do not mean the software is gratis.</p>\n</blockquote>\n<p>In a similar fashion, I don’t think that developers are turning against the concept of “free as in free speech”. I think that they don’t believe that the current definitions of free software and open source actually produce software that is “free as in speech.”</p>\n<h2>What does “freedom” mean anyway?</h2>\n<p>What “free as in speech” means is itself, yet another schism between different camps in the Free Software/Open Source camp. At the root of this schism is a difference of <em>strategy</em>. The specific tactic is straightforward: there are two kinds of licenses, and which do you choose? The two kinds are:</p>\n<ul>\n<li><em>permissive</em> licenses</li>\n<li><em>viral</em> licenses if you don’t like them, <em>copyleft</em> licenses if you do</li>\n</ul>\n<p>Open Source advocates prefer permissive licenses, and Free Software advocates prefer viral/copyleft licenses. What’s the difference? Well, that’s the topic for a different essay. Because here’s the thing: the jig is up. We’ve already hit the root problem. And you probably didn’t even notice. It took me a long, long time to notice.</p>\n<p>Before I expose the trick, a personal interlude.</p>\n<h2>Losing my religion</h2>\n<p>I used to be a card-carrying, passionate advocate of Free Software. I used to see Stallman lecture. I only installed Free Software on my personal computer to the fullest extent possible. I was considering buying a coreboot-compatible laptop to get rid of the rest.</p>\n<p>But then, slowly, I stopped caring. It felt really similar to when I decided to leave Catholicism. I started noticing that these beliefs weren’t really actually helpful, and were frankly, mostly harmful. Now, I’m not saying Free Software is harmful. What I am saying is that refusing to use a wifi connection on your laptop because there aren’t free drivers helps basically nobody, and only harms yourself. Even if you are the kind of person who thinks boycotts work, there just aren’t enough free software diehards to form a large enough boycott. And even then, you’d want to do the boycott <em>together</em>, <em>simultaneously</em>, to send the message.</p>\n<p>I realized that a lot of software I used was under a permissive license, and even though I’d prefer if it was GPL’d, the only way to actually realize the dream of improving the software thanks to the source I’d been given was to contribute back, under those terms. So I started to.</p>\n<p>And at that point, the permissive licenses were becoming more and more popular. So I found myself in a strange position: even though I was a passionate Free Software advocate, I found myself near-exclusively building Open Source software. All around me, I saw the assault on copyleft, from all sides. And copyleft was losing.</p>\n<p>At some point, I just stopped caring.</p>\n<h2>Free Software and Open Source are about licenses</h2>\n<p>With that out of the way, here’s the problem: note that I seamlessly switched above from talking about what Free Software and Open Source <em>are</em>, to immediately talking about <em>licenses.</em> This is because these two things are effectively synonymous. Quoting <a href=\"https://www.gnu.org/licenses/licenses.html\">the FSF again</a>:</p>\n<blockquote>\n<p>Published software should be free software. To make it free software, you need to release it under a free software license.</p>\n</blockquote>\n<p>The <a href=\"https://opensource.org/osd\">OSI</a>:</p>\n<blockquote>\n<p>Open source doesn’t just mean access to the source code. The distribution terms of open-source software must comply with the following criteria:</p>\n</blockquote>\n<p>Both Free Software and Open Source are, almost by definition, specific design constraints on software licenses. While this started with the FSF, the OSI inherited it.</p>\n<p>I think a quote from <a href=\"https://www.gnu.org/licenses/licenses.html#WhatIsCopyleft\">further down the FSF’s page</a> really drives this home:</p>\n<blockquote>\n<p>To copyleft a program, we first state that it is copyrighted; then we add distribution terms, which are a legal instrument that gives everyone the rights to use, modify, and redistribute the program’s code or any program derived from it but only if the distribution terms are unchanged. Thus, the code and the freedoms become legally inseparable.</p>\n</blockquote>\n<p>The FSF sees copyleft, and therefore copyright, as a legal tool to enforce software freedom. This sometimes puts Free Software advocates in a strange position. For example, they will <a href=\"https://www.fsf.org/campaigns/drm.html\">campaign passionately against Digital Rights Management</a>, a software tool to protect copyrighted works, yet also <a href=\"https://fsfe.org/news/2019/news-20190326-01.en.html\">support expanding copyright in some cases</a> when it can further the aims of the GPL. (Due to controversy, they’ve stepped back a bit since, as you can see.)</p>\n<h2>Licenses are not sufficient</h2>\n<p>So why is it a problem that the concepts of free software and open source are intrinsically tied to licenses? It’s that the aims and goals of both of these movements are about <em>distribution</em> and therefore <em>consumption</em>, but what people care about most today is about the <em>production</em> of software. Software licences regulate <em>distribution</em>, but cannot regulate <em>production</em>. (technically they can, but practically, they can’t. I get into this below.) This is also the main challenge of whatever comes after open source; they cannot rely on the legal tactics of the last generation. I don’t have solutions here.</p>\n<p>Let’s talk about what developers want first, and then we’ll get into why licenses can’t accomplish this.</p>\n<h2>Developers care about production</h2>\n<p>In <a href=\"https://words.steveklabnik.com/the-culture-war-at-the-heart-of-open-source\">my previous post</a>, I claimed that developers are mostly confused about open source. Here’s two concrete examples of what I mean.</p>\n<ul>\n<li>Imagine a company wants to take some open source software using a permissive license and use it in their product. In the course of using it in their product, they improve it. Are they required to give anything back to the parent project, at all?</li>\n<li>Imagine a company wants to take some free software using a copyleft license and use it in their product. In the course of using it in their product, they improve it. In order to comply with the license, they include a line in their 250 page “software license agreement” that says “Certain components of the software, and third party open source programs included with the software, have been or may be made available by $COMPANY on its Open Source web site (http://www.opensource.mycompany.com/”. That web site contains zip files with the contents of the (heavily modified) source. Are they required to do anything more than that?</li>\n</ul>\n<p>When presenting these questions to most developers today, I suspect you’d get these answers to these two questions, in order:</p>\n<ul>\n<li>Yes, and that’s why open source is such a mess; companies take and don’t give back.</li>\n<li>… that sounds extremely shady.</li>\n</ul>\n<p>Those of you who have been around the block a few times may recognize situation number two: it’s what Apple did with WebKit. There was a project called KHTML, which was licensed under the GPL. Apple forked it, and in order to comply with the GPL, did exactly the above. They were completely within their rights to do so. Yet, many recognized, even at this time, that this “wasn’t in the spirit of open source.” These tactics are sometimes called “source dumps” or “code bombs.”</p>\n<p>But that attitude, that this may be following the letter, <em>but not the spirit</em>, is the crux of it here. Most developers don’t understand open source to be a particular license that certain software artifacts are in compliance with, but an attitude, an ideology. And that ideology isn’t just about the consumption of the software, but also its production. An open source project should have a public bug tracker. There should be a mailing list, for discussion. You should be able to observe, and ideally participate in, the development of the software. Focusing on the code being open is putting the cart before the horse. In fact, this is one of the reasons why there’s such derision for “source available” licenses; it’s not about the source being open. It’s that the source being open is a necessary, but not sufficient, component of being open source. Now, this is normally framed as a problem of distribution, but I think that many also understand it as a problem of production.</p>\n<p>I believe that this focus on process is why the GNU project has fallen out of favor as well. The tools that the GNU project uses to develop its Free Software are arcane, ugly, and unique to them. It’s the same with many of the old-school Open Source projects as well. If I never have to look at a Bugzilla instance again, I will die happy. This is why GitHub took off; it provided a significantly nicer developer experience for building software. You may not personally agree, but the numbers speak for themselves. The FSF didn’t move to GitHub because GitHub is proprietary, and they see that as inconsistent with their values. Most developers see that you can use it for no money, and that the software produced with it is open source. They see this as consistent with their values.</p>\n<p>When developers talk about problems they see in open source, it’s often that there are production problems. Companies don’t “give back” money or developer hours. Programmers today don’t seem to be upset that, if they’ve developed any proprietary extensions to their open source software, that those extensions are not shared back with the community. They care that the production process is impeded by additional pressure, without providing resources. If a company were to add a proprietary feature to an open source project, yet pays five employees to develop the open source part further, the FSF sees this as a tragedy. The commons has not been enriched. The new generation of open source developers sees this as a responsible company that thankfully is contributing to the development of something they use and care about.</p>\n<p>Software licenses can only restrict what people can do when they distribute the source code, and that’s it. It cannot force someone to have a bug tracker, or a code of conduct, or accept your patch. Copyleft can force an absolute minimal “contribution” back to your project, but it can’t force a good-faith one. This makes it an inadequate tool towards building something with the kinds of values that many developers care about.</p>\n<h2>The challenges</h2>\n<p>How would one go about building a software movement around the open <em>production</em> of software? There are a number of challenges.</p>\n<p>First of all, do you even want the legal system to enforce such a thing? There are pros and cons. Without legal teeth, companies are unlikely to comply. That’s part of why we’re in this mess to begin with!</p>\n<p>Okay, so you want the legal system to somehow enforce this kind of control over the production of software. But how? If we look closer at the strategy used by Free Software and Open Source, they use licenses, which are a form of intellectual property law, which is modeled after property law. Earlier, I said that you can’t use licenses to regulate production, and that’s <em>technically</em> not true. For example, say that I own a brand, like McDonalds. I own the intellectual property surrounding that brand. I can licence that intellectual property to others, contingent on them producing hamburgers (and whatever else) in a certain way, according to my specification.</p>\n<p>This doesn’t really work with the way that open source is set up. The entities are in reverse here; it’s the software developers that want to be able to dictate things, but it’s the project that sets the license terms.</p>\n<p>That got me thinking about a different pattern for doing this kind of thing. Have you ever seen one of these?</p>\n<p><img src=\"https://svbtleusercontent.com/wWhQCoC3mGGKwQ1nS6CC9H0xspap_small.gif\" alt=\"https://svbtleusercontent.com/wWhQCoC3mGGKwQ1nS6CC9H0xspap_small.gif\" /></p>\n<p>This image on a product is part of a process called “certification.” The image itself is referred to as a “certification mark.” In order to use this image on your product, you apply to a “certification body”, in this case, the <a href=\"https://www.usda.gov/\">USDA</a>. This body has set up some kind of tests, and if your product passes them, you gain the ability to say that you’ve passed the certification. I chose organic food on purpose here; most aspects of this certification are about the process by which the food is produced.</p>\n<p>Technology is no stranger to these kinds of processes:</p>\n<p><img src=\"https://svbtleusercontent.com/fMstP8GhZwpMhHxNdZ3aTn0xspap_small.png\" alt=\"https://svbtleusercontent.com/fMstP8GhZwpMhHxNdZ3aTn0xspap_small.png\" /></p>\n<p>So in theory, one could imagine an organization that produces a different kind of document. Instead of a license for the source code, they would provide a way to say uh, let’s go with “Open Development Certified.” Projects could then submit for certification, they’d get accepted or rejected.</p>\n<p>I’m not confident this solution would work, however.</p>\n<p>For one, even though some parts of our industry have accepted certifications, I feel like software developers have a pretty big bias against them. Beyond that, there’s two other major questions: who would do this certification, and how would they determine the criteria? It’s not clear who has the moral authority to suggest that they are the arbiter of what is correct here, and that a majority of people would agree. And then they would have a big job of actually determining what those sets of rules would be. It does have a nice property of a built-in business model; you can charge for application for the certification. But that also has a host of issues. And even if you sort all of that out, it runs afoul of the same “boycott” problems that I talked about above with Free Software. This certification only makes sense if people demand that the software they use is Open Development Certified. I’m not sure that this would be the case.</p>\n<p>Another option is some sort of “Developer Union,” which would put pressure on the companies that those developers work at to contribute back to open source projects. Many developers seem <em>rabidly</em> anti-union, and tech companies are as well. I’m not sure this is a viable path, today.</p>\n<h2>So where do we go from here?</h2>\n<p>I’m still, ultimately, left with more questions than answers. But I do think I’ve properly identified the problem: many developers conceive of software freedom as something larger than purely a license that kinds in on redistribution. This is the new frontier for those who are thinking about furthering the goals of the free software and open source movements. Our old tools are inadequate, and I’m not sure that the needed replacements work, or even exist.</p>\n<p>Something to think about, though.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rust-has-finally-outgrown-me/",
"title": "Rust has finally outgrown me",
"description": null,
"url": "https://steveklabnik.com/writing/rust-has-finally-outgrown-me/",
"published": "2019-03-29T00:00:00.000Z",
"updated": "2019-03-29T00:00:00.000Z",
"content": "<p>Today is a very interesting day. It’s giving me a lot of feels. Today is <a href=\"https://rustlatam.org/\">Rust Latam</a>, a Rust conference in Montevideo.</p>\n<p>And I’m not there.</p>\n<p>This is significant because, as far as we know, I’m the only person who has been to every Rust conference in existence so far. I went to RustCamp, all three RustConfs, all five RustFests so far, all three Rust Belt Rusts. One RustRush. Am I forgetting any? Thirteen Rust conferences in the past four years.</p>\n<p>I really wanted to attend Rust Latam. The main organizer, Santiago, has been a friend for a very long time. It’s amazing that we’re getting a new conference in this part of the world. There’s been a lot of energy with Rust there for quite a while. It would have been nice to meet all those people whose tweets I’ve seen.</p>\n<p>But in the end, I’m still between jobs (news on that soon, hopefully!), and spending that money would be irresponsible. I’m just getting settled in Austin, and it’s nice to stay in one place for a little while. I didn’t only attend Rust conferences last year, I attended many more. I missed Thanksgiving with my friends, because I was at a conference. I made five trips in October last year; Eastern Europe, then San Francisco, then Michigan, then New York (for a conf, not to stay at home), then Berlin. I did three more in November. I did about twenty conferences, and a few personal trips, last year. I am very tired.</p>\n<p>Don’t get me wrong, I love travel, and plan to do more of it in the future. But I’ve been taking a bit of a breather; I’ve only done four trips so far this year, twice to Europe, twice in the US. But it’s been nice to take a break. For now.</p>\n<p>More importantly, it’s time to let go of the pressure I felt to keep that streak up. Some of those trips really took a toll on me, mentally and physically. Nobody should ever fly from New York to Belarus for 30 hours, then immediately fly to San Francisco for three days, then back to New York. I’m not old, but I’m not as young as I once was, either.</p>\n<p>It’s also time to acknowledge that I cannot do it all on my own. This conference streak is one thing, but Rust has outgrown me in other ways. I used to be on <code>#rust</code> every day, answering almost every question. Then it was both <code>#rust-beginners</code> and <code>#rust</code>. Not to mention all the other channels. I’m not even on IRC anymore. I was on the core team when the core team approved every RFC; now my team very rarely has RFCs. I used to read every single GitHub email on <code>rust-lang/rust</code>, and then <code>rust-lang/cargo</code>, and then all their other libraries. Now I skim <code>rust-lang/rust</code>, and have unsubscribed from a lot of the other repos. I used to know every single production user of Rust. Those times are long gone; I’m starting to get used to hearing “oh yeah our team has been using Rust in production for a while now, it’s great.” The first time that happened, it felt <em>very</em> strange. Exciting, but strange.</p>\n<p>Back when I first decided I wanted to devote my time to Rust, I used to submit every single blog post about Rust to Hacker News and Reddit, and then answer any question anyone had about things. I diligently did this for <em>years</em>. I’m slowly letting go of this one, too. We’ll see.</p>\n<p>All of these things are excellent. This is the future I wanted. This is why I spent so much time over the years doing all of this stuff. I did it, mission accomplished. It feels good. But it also feels a little weird. That Rust is bigger than <em>me</em> is an amazing, wonderful thing.</p>\n<p>I wonder what the next stage of Rust’s growth will feel like.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/learning-ada/",
"title": "Learning Ada",
"description": null,
"url": "https://steveklabnik.com/writing/learning-ada/",
"published": "2019-03-26T00:00:00.000Z",
"updated": "2019-03-26T00:00:00.000Z",
"content": "<p>I decided to try and learn Ada. I love programming languages. This post documents my attempt, and explains what I learned. This is pretty stream of consciousness and basically is a bunch of random notes, so if you hate that, better close the tab. I typed my thoughts up pretty raw, so I may be incorrect in places too. Let me know if you know the answer to something I don’t!</p>\n<hr />\n<p>The first step, of course, is to google:</p>\n<p><img src=\"https://svbtleusercontent.com/72NhELV1orufu8jasf5iiT0xspap_small.png\" alt=\"https://svbtleusercontent.com/72NhELV1orufu8jasf5iiT0xspap_small.png\" /></p>\n<p>Great! I know that Adacore are sort of the stewards of Ada? Vaguely, that might be wrong. Let’s look at all three of those.</p>\n<blockquote>\n<p>Learn.adacore.com is an interactive learning platform designed to teach the Ada and SPARK programming languages.</p>\n</blockquote>\n<p>Ah, that’s awesome! No installation needed.</p>\n<p>The second one looks <strong>extremely</strong> like spam. It does have a page of learning materials that look legit enough. I wonder if this is a low-effort attempt to make money through ads. Or maybe it’s actually good and real and feels outdated, it does have a high google ranking. Upon doing some more research, it seems like the latter. Oops…</p>\n<p>The third one is a parked domain, it apparently expired. Oops.</p>\n<p>Anyway, I’m going to pursue learn.adacore.com</p>\n<h2>learn.adacore.com</h2>\n<p>Here’s the initial code presented:</p>\n<pre><code>with Ada.Text_IO; use Ada.Text_IO;\n\nprocedure Learn is\n\n subtype Alphabet is Character range 'A' .. 'Z';\n\nbegin\n\n Put_Line (\"Learning Ada from \" & Alphabet'First & \" to \" & Alphabet'Last);\n\nend Learn;\n</code></pre>\n<p>Okay! This clearly has some style to it. Seems like <code>Pascal_Snake_Case</code>. I’m not super hyped about this, but it’s fine. Syntax discussions are like the tabloids of programming language discussion. They’re fun, but they don’t really have any educational comment. One thing I will point out: are apostrophes used for method calls? Otherwise, this seems to be fairly Pascal like, which I think makes sense given what I know about Ada.</p>\n<p>I can click run and see it execute right there! Awesome.</p>\n<p>There seems to be three sections here: About, Courses, and Books. About is a single page of information that reinforces what I already know:</p>\n<blockquote>\n<p>The Ada programming language was designed from its inception to be used in applications where safety and security are of the utmost importance.The SPARK programming language is a formally verifiable subset of the Ada language which allows developers to mathematically prove program correctness through static means.Founded in 1994, AdaCore is the leading provider of commercial and open-source software solutions for Ada</p>\n</blockquote>\n<p>Cool.</p>\n<p>There are two courses, introduction to Ada, and introduction to SPARK. I think SPARK is cool but I’ll learn that after I learn Ada itself. I also do like books, but let’s do that first.</p>\n<p>I’m not going to write everything about the book as I read it, but I will say that I like the basic structure here: first some history, then imperative programming stuff, then talking about types. After that it gets into packages, more type stuff, privacy, generics, exceptions, tasks, interfacing with C, the standard library, and some appendices. This feels pretty logical. Time to do some reading, I’ll jump back here at random points with notes.</p>\n<blockquote>\n<p>The first Ada standard was issued in 1983; it was subsequently revised and enhanced in 1995, 2005 and 2012, with each revision bringing useful new features.</p>\n</blockquote>\n<p>I heard that Ada 2020 is a thing; seems like they’re on a roughly 7-8ish year timeframe generally. Cool.</p>\n<blockquote>\n<p>In terms of modern languages, the closest in terms of targets and level of abstraction are probably C++ and Rust.</p>\n</blockquote>\n<p>We got a shout-out! That’s nice.</p>\n<blockquote>\n<p>Readability is more important than conciseness. Syntactically this shows through the fact that keywords are preferred to symbols, that no keyword is an abbreviation, etc.</p>\n</blockquote>\n<p>This is interesting; it seems to suggest that readability and verbosity are synonymous. I often find concise things easier to read, personally. This is, of course, subjective, and I wonder how I’ll feel after I learn more of the language.</p>\n<blockquote>\n<p>Ada is a multi-paradigm language with support for object orientation and some elements of functional programming, but its core is a simple, coherent procedural/imperative language akin to C or Pascal.</p>\n</blockquote>\n<p>This makes a lot of sense, especially given the time it was created.</p>\n<p>I’m finding it a <em>little</em> hard to understand what <code>main</code> is. The docs say</p>\n<blockquote>\n<p>Greet is a procedure, and the main entry point for our first program. Unlike in C or C++, it can be named anything you prefer. The builder will determine the entry point. In our simple example, gprbuild, GNAT’s builder, will use the file you passed as parameter.</p>\n</blockquote>\n<p>It will use it as … parameter? Like, is the fact that we called the file <code>greet.adb</code> the reason it calls <code>Greet</code>? It’s not clear to me at the moment.</p>\n<blockquote>\n<p>Integer’Image is a function that takes an Integer and converts it to a String. It is an example of a language construct known as an attribute, indicated by the “’” syntax, which will be covered in more detail later.</p>\n</blockquote>\n<p>Ah, it’s not a method call, it’s an attribute.</p>\n<blockquote>\n<p>In Ada, an integer type is not specified in terms of its machine representation, but rather by its range. The compiler will then choose the most appropriate representation.</p>\n</blockquote>\n<p>Very interesting. I wonder how well-defined these mappings are, for example, if you need to call a C function, you’re gonna need to give it the correct sized type… I’m sure I’ll get to that.</p>\n<blockquote>\n<p>Mainly for efficiency reasons, while machine level overflow always results in an exception, type level overflows will only be checked at specific boundaries, like assignment:</p>\n</blockquote>\n<p>Okay, <em>this</em> is the good stuff. Ranged integers are something people always say Ada is better at than Rust; I wondered if this was a static or dynamic check. It <em>is</em> a dynamic check. Cool. I’ve always wanted to know!</p>\n<blockquote>\n<p>As mentioned previously, every “built-in” type in Ada is defined with facilities generally available to the user.</p>\n</blockquote>\n<p>Pretty cool!</p>\n<blockquote>\n<p>While unconstrained arrays in Ada might seem similar to variable length arrays in C, they are in reality much more powerful, because they’re truly first-class values in the language. You can pass them as parameters to subprograms or return them from functions, and they implicitly contain their bounds as part of their value. This means that it is useless to pass the bounds or length of an array explictly along with the array, because they are accessible via the ’First, ’Last, ’Range and ’Length attributes explained earlier.</p>\n</blockquote>\n<p>Awesome. This is the way to do it, for sure.</p>\n<blockquote>\n<p>Here is how the string type is defined in Ada:type String is array (Positive range <>) of Character;</p>\n</blockquote>\n<p>Hmm, what’s a <code>Character</code> though? Let’s try to mess with the example:</p>\n<pre><code>with Ada.Text_IO; use Ada.Text_IO;\n\nprocedure Greet is\n Message : String (1 .. 2) := \"😊\";\n -- ^ Pre-defined array type.\n -- Component type is Character\nbegin\n for I in reverse Message'Range loop\n -- ^ Iterate in reverse order\n Put (Message (I));\n end loop;\n New_Line;\nend Greet;\n</code></pre>\n<p>Here’s the output:</p>\n<pre><code>Run...\ngreet.adb:4:04: warning: \"Message\" is not modified, could be declared constant\ngreet.adb:4:33: warning: wrong length for array of subtype of \"Standard.String\" defined at line 4\ngreet.adb:4:33: warning: \"Constraint_Error\" will be raised at run time\nraised CONSTRAINT_ERROR : greet.adb:4 length check failed\n</code></pre>\n<p>Hm, wrong length. What is the right length? This particular emoji is four bytes, so let’s try to set it to 5, since we start at 1. Nope that doesn’t work either. What about 4?</p>\n<pre><code>Run...\nthe machine running the examples is not responding, please try again later\n</code></pre>\n<p>Very suspicious. Anyway, I’m guessing it’s ASCII, maybe it’ll tell me later.</p>\n<blockquote>\n<p>One last feature of Ada arrays that we’re going to cover is array slices. It is possible to take and use a slice of an array (a contiguous sequence of elements) as a name or a value.</p>\n</blockquote>\n<p>Hell yeah, I love slices.</p>\n<blockquote>\n<p>Ada has multidimensional arrays, which are not covered in this course. Slices will only work on one dimensional arrays.</p>\n</blockquote>\n<p>Ah, bummer. I guess I’ll have to read up on those separately.</p>\n<blockquote>\n<p>Returning variable size objects in languages lacking a garbage collector is a bit complicated implementation-wise, which is why C and C++ don’t allow it, prefering to depend on explicit dynamic allocation / free from the user.The problem is that explicit storage management is unsafe as soon as you want to collect unused memory. Ada’s ability to return variable size objects will remove one use case for dynamic allocation, and hence, remove one potential source of bugs from your programs.Rust follows the C/C++ model, but with safe pointer semantics. However, dynamic allocation is still used. Ada can benefit from an eventual performance edge because it can use any model.</p>\n</blockquote>\n<p>Hmmm, I wonder <em>how</em> it is doing this. Doesn’t that still dynamically allocate in Ada?</p>\n<blockquote>\n<p>Ada doesn’t have a tuple construct and does not allow returning multiple values from a subprogram (except by declaring a full-fledged record type). Hence, a way to return multiple values from a subprogram is to use out parameters.</p>\n</blockquote>\n<p>I love tuples, oh well.</p>\n<blockquote>\n<p>While reading an out variable before writing to it should, ideally, trigger an error, imposing that as a rule would cause either inefficient run-time checks or complex compile-time rules. So from the user’s perspective an out parameter acts like an uninitialized variable when the subprogram is invoked.</p>\n</blockquote>\n<p>Hmmmmmmmmmmmmmmmmmm. This seems <em>extremely</em> against Ada’s safety focus.</p>\n<pre><code>with Ada.Text_IO; use Ada.Text_IO;\n\nprocedure Outp is\n procedure Foo (A : out Integer) is\n begin\n Put_Line (Integer'Image (A));\n end Foo;\n A : Integer := 12;\nbegin\n Foo(A);\nend Outp;\n</code></pre>\n<p>gives</p>\n<pre><code>Run...\noutp.adb:4:19: warning: formal parameter \"A\" is read but never assigned\noutp.adb:10:08: warning: \"A\" modified by call, but value might not be referenced\n32514\n2 errors.\n</code></pre>\n<p>Each time I run it, I get a different number. Seems bad?</p>\n<blockquote>\n<p>There are two ways in which Ada helps shield programmers from the dangers of pointers:One approach, which we have already seen, is to provide alternative features so that the programmer does not need to use pointers. Parameter modes, arrays, and varying size types are all constructs that can replace typical pointer usages in C.Second, Ada has made pointers as safe and restricted as possible, but allows “escape hatches” when the programmer explicitly requests them and presumably will be exercising such features with appropriate care.</p>\n</blockquote>\n<p>Seems great! I’m very here for this.</p>\n<p>… but they have <code>null</code>. Of course they do, given the timeframe Ada was developed. Oh well.</p>\n<p>Okay, so <code>new</code> is a keyword, and is in the language. But freeing is an unsafe standard library API? I guess this makes sense, similarly to how leaking memory isn’t unsafe in Rust. Feels weird, though.</p>\n<p>I guess, reading more, the idea is that you should basically never need to do this yourself. I think this is where I was getting caught up earlier with the “return an unknown type” bit too. It <em>is</em> allocating, but since you’re not typing <code>new</code>, you’re not responsible, and so that’s safer. That makes sense. I hope my understanding is correct here, but I’m not totally sure.</p>\n<p>This does mean that it feels like allocations exist, but are hidden from you, the user. That’s okay, but it’s very different than Rust.</p>\n<blockquote>\n<p>Ada’s variant records are very similar to Sum types in functional languages such as OCaml or Haskell. A major difference is that the discriminant is a separate field in Ada, whereas the ‘tag’ of a Sum type is kind of built in, and only accessible with pattern matching.There are other differences (you can have several discriminants in a variant record in Ada). Nevertheless, they allow the same kind of type modeling as sum types in functional languages.</p>\n</blockquote>\n<p>This is pretty cool.</p>\n<p>The “Character Types” section still does not explain to me what valid characters are. Oh well. Googling it gives me <a href=\"https://www.adaic.org/resources/add_content/standards/05rm/html/RM-3-5-2.html\">this</a> which says</p>\n<blockquote>\n<p>The predefined type Character is a character type whose values correspond to the 256 code positions of Row 00 (also known as Latin-1) of the ISO/IEC 10646:2003 Basic Multilingual Plane (BMP). Each of the graphic characters of Row 00 of the BMP has a corresponding character_literal in Character. Each of the nongraphic positions of Row 00 (0000-001F and 007F-009F) has a corresponding language-defined name, which is not usable as an enumeration literal, but which is usable with the attributes Image, Wide_Image, Wide_Wide_Image, Value, Wide_Value, and Wide_Wide_Value; these names are given in the definition of type Character in A.1, “The Package Standard”, but are set in italics.</p>\n</blockquote>\n<p>Ah. Yeah so that’s why the emoji didn’t work. Did that crash the compiler earlier, is that what was happening?</p>\n<blockquote>\n<p>Generic subprograms or packages can’t be used directly. Instead, they need to be instantiated, which we do using the new keyword, as shown in the following example:</p>\n</blockquote>\n<p>Wait, is this the same <code>new</code> keyword as before? Do generics require heap allocation? That is a reasonable design, I’m just a bit surprised.</p>\n<blockquote>\n<p>Ada exceptions are not types, but instead objects, which may be peculiar to you if you’re used to the way Java or Python support exceptions.</p>\n</blockquote>\n<p>I found this sentence confusing; aren’t the objects in Java or Python as well? When I was reading this sentence, halfway through, I thought to myself “oh, like the stuff I know, cool” and then the second half was “this is unlike the stuff you know”. Hrm.</p>\n<blockquote>\n<p>A task can be thought as an application that runs concurrently with the main application. In other programming languages, a task can be called a thread, and tasking can be called multithreading.</p>\n</blockquote>\n<p>Cool, this is a big topic I’ve been wondering about.</p>\n<blockquote>\n<p>As we’ve just seen, as soon as the main task starts, its subtasks also start automatically. The main task continues its processing until it has nothing more to do. At that point, however, it will not terminate. Instead, the task waits until its subtasks have finished before it allows itself to terminate.</p>\n</blockquote>\n<p>Ah interesting, this is different than Rust. Well, Rust’s standard library, anyway.</p>\n<p>I wish it explained if tasks <em>are</em> system threads, or if they’re green threads.</p>\n<p>I did some googling, and apparently this is implementation-defined. GNAT uses system threads for some of the same reason that Rust decided to expose system threads. Seems good.</p>\n<blockquote>\n<p>The pragma Assertion_Policy statement is used to force the compiler to generate code to check the precondition.</p>\n</blockquote>\n<p>Design by contract works at runtime. This is something I always thought was true, but was never actually sure of. Cool.</p>\n<blockquote>\n<p>To interface with C’s built-in types, we use the Interfaces.C package, which contains most of the type definitions we need. For example:</p>\n</blockquote>\n<p>Ah ha, this answers my question from earlier. Cool.</p>\n<blockquote>\n<p>Object-oriented programming (OOP) is a large and ill-defined concept in programming languages and one that tends to encompass many different meanings because different languages often implement their own vision of it, with similarities and differences from the implementations in other languages.However, one model mostly “won” the battle of what object-oriented means, if only by sheer popularity. It’s the model used in the Java programming language, which is very similar to the one used by C++. Here are some defining characteristics:</p>\n</blockquote>\n<p>Spicy! But not incorrect.</p>\n<p>Dot notation exists! But only if you’re actually using objects, er, “tagged types”.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/what-s-next-for-semver/",
"title": "What's next for SemVer",
"description": null,
"url": "https://steveklabnik.com/writing/what-s-next-for-semver/",
"published": "2019-02-11T00:00:00.000Z",
"updated": "2019-02-11T00:00:00.000Z",
"content": "<p>On December 18, 2009<a>1</a>, the Semantic Versioning specification was announced. More commonly known as SemVer, it provided guidance for developers regarding software versions:</p>\n<blockquote>\n<p>I propose a simple set of rules and requirements that dictate how version numbers are assigned and incremented. For this system to work, you first need to declare a public API. This may consist of documentation or be enforced by the code itself. Regardless, it is important that this API be clear and precise. Once you identify your public API, you communicate changes to it with specific increments to your version number. Consider a version format of X.Y.Z (Major.Minor.Patch). Bug fixes not affecting the API increment the patch version, backwards compatible API additions/changes increment the minor version, and backwards incompatible API changes increment the major version.I call this system “Semantic Versioning.” Under this scheme, version numbers and the way they change convey meaning about the underlying code and what has been modified from one version to the next.</p>\n</blockquote>\n<p>The author came from the Ruby world, and around the same time, some big things were brewing. <a href=\"https://bundler.io/\">Bundler</a> was being brought into the world. Bundler brought a few new things, but the most important features involved specifying dependencies for your projects. Normally, you’d specify which version of a gem your project needed. Instead, Bundler allowed you to specify a range of versions you’d accept, and it would help figure out which version you should use, according to the SemVer rules.</p>\n<p>In January of 2010, npm, a similar program for the shiny new Node.js, also shipped, and also used SemVer.</p>\n<p>Thanks to these tools, Ruby and Node enjoyed (and still do, to this day) a vibrant open source ecosystem. SemVer is the underlying concept that made this possible.</p>\n<p>However, SemVer is not perfect. In some ways, I think of SemVer as the <code>gofmt</code> of versioning:</p>\n<blockquote>\n<p>Gofmt’s style is no one’s favorite, yet gofmt is everyone’s favorite.Go Proverbs</p>\n</blockquote>\n<p>SemVer has its flaws, but it empirically works better than previous attempts at such a thing. That such a thing exists is far more important than how perfect it is. Fundamentally, version numbers are a communication tool, and SemVer provides a framework for said communication.</p>\n<p>That said, SemVer <em>does</em> have flaws. Bundler, <code>npm</code>, Rust’s Cargo, NuGet, and a variety of tools have implemented the specification. But there are edge cases, as well as gaping holes, in the spec, and so in some ways, these tools are incompatible.</p>\n<p>Over the last few months, several of us<a>2</a> have been talking, and today, we have an announcement to make. We’ve formed a semver team, and we intend to start work on producing a new version of the spec.</p>\n<p>Who is “we”? The team roster will be <a href=\"https://github.com/orgs/semver/teams/maintainers/members\">in a GitHub team</a><a>3</a>, but for posterity’s sake, here’s the initial list of members, in the same order GitHub lists them:</p>\n<ul>\n<li>Anand Gaurav, NuGet</li>\n<li>Dave Herman, Notion</li>\n<li>André Arko, Bundler</li>\n<li>isaacs, npm</li>\n<li>Samuel Giddins, CocoaPods</li>\n<li>Steve Klabnik, Cargo</li>\n</ul>\n<p>(I’m putting myself as “Cargo,” because Cargo is the main client of my SemVer implementation. In most of these projects, the person who maintains the SemVer implementation is the same as the project, but in Rust, they’re separate. Cargo is the higher-order bit here, though.)</p>\n<p>We have not yet started the work on the next iteration of the specification, but we have agreed on a governance model for SemVer. Described in the new <code>[CONTRIBUTING.md](https://github.com/semver/semver/blob/master/CONTRIBUTING.md)</code>, we’re going with an RFC process, similar to the Rust programming language and many other projects. In short, “substantial” changes to the SemVer spec need to go through a process by which a written description of the changes are proposed, and the team reaches consensus on acceptance.</p>\n<p>It also contains some principles we have for the future of SemVer. I’ve copied the important ones for the specification itself here:</p>\n<ul>\n<li>No RFC will be approved if it is deemed to cause significant breakage to any of the SemVer-using communities represented by the SemVer Maintainers group.</li>\n<li>RFCs will be considered formally adopted only when they are approved by the SemVer Maintainers group, and implemented in a simple majority of represented implementations.</li>\n<li>Implementations may add functionality in advance of an approved RFC but all such functionality must be flagged as “experimental”, so that users understand it may change in the future.</li>\n</ul>\n<p>In essence, we don’t intend to re-write the spec, but instead, fill in holes in the spec, find out where our implementations differ, and unify all of our implementations as much as is feasible. We believe that this will not only help all of us, but also help new tools as well. Being compatible with each other is an important part of the value of these tools.</p>\n<p>I look forward to letting you know when our work is done! If you’d like to get involved, please watch that governance repository, and submit RFCs! While we’re all excited about the future of SemVer, it’s nobody’s job to do this work. As such, we’ll be moving fairly slowly. Please be patient. Thanks :)</p>\n<hr />\n<p>You may also want to read <a href=\"https://haacked.com/archive/2019/02/11/semver-collective/\">Phil’s post</a>, reflecting on this transition from the other side. Thanks for being so much of a good steward of SemVer so far, Phil!</p>\n<hr />\n<ol>\n<li>At least <a href=\"https://web.archive.org/web/20091218170740/http://semver.org/\">according to the WayBack Machine</a><a>↩</a></li>\n<li><a href=\"https://crates.io/crates/semver\">I maintain Cargo’s <code>semver</code> implementation</a>.<a>↩</a></li>\n<li>Ugh, I’m told apparently this can’t be made public? Even better that I put the list here, then.<a>↩</a></li>\n</ol>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/thank-u-next/",
"title": "thank u, next",
"description": null,
"url": "https://steveklabnik.com/writing/thank-u-next/",
"published": "2019-01-06T00:00:00.000Z",
"updated": "2019-01-06T00:00:00.000Z",
"content": "<p>I started working at 15, when I took a job as a pizza cook. Over the next seven years, I moved up the ranks, to a driver, shift manager, and then as part of the “new store opening team.” The franchise was growing, and we needed to help new franchisees open their new stores. I’d travel to where the new store was a week before they would open, help train the new staff, and then work their opening weekend. It was really fulfilling work; if pizza paid as well as tech, I’d seriously consider doing it forever.</p>\n<p>One time, somewhere in Maryland, I got a bit ill. It wasn’t a good idea for me to work the rest of the week, but the company’s top brass would usually visit to cut the ribbon on the store. It was about a four hour drive back to home, and our COO graciously agreed to let me come back to the ’burgh with him, so I could recover at home. On the drive, I asked him what he did before working here, and the answer really surprised me. “Oh, I was the CEO of American Eagle. I grew them from 20 locations to 200, and then decided to move on.” To a kid from the suburbs, being in charge of one of the most popular clothing brands among my peers seemed like a great gig. Why’d he quit? The answer is something that’s really stuck with me; this happened about fifteen years ago. What he told me was this: at each stage of a company’s growth, they have different needs. Those needs generally require different skills. What he enjoyed, and what he had the skills to do, was to take a tiny company and make it medium sized. Once a company was at that stage of growth, he was less interested and less good at taking them from there. And that was what he was trying to do for our pizza chain.</p>\n<p>I’ve mostly worked at small companies. I did a startup, then consulted, then worked for a series of companies with five to 15 people. I quit one of those jobs when I found Rust. What I saw in Rust was something that the world really <em>needed</em>. And I wanted to help it get there. Beyond that, the only real way to get a job working on Rust was to work at Mozilla. And independently of Rust, that was something I’m really excited about.</p>\n<p>Here’s a poster on my wall:</p>\n<p><img src=\"https://svbtleusercontent.com/x35Th9VqQFuuZ4SFBEsZkU0xspap_small.jpeg\" alt=\"https://svbtleusercontent.com/x35Th9VqQFuuZ4SFBEsZkU0xspap_small.jpeg\" /></p>\n<p>I’ve been using Firefox before it was called Firefox. This poster is of a two-page ad that Mozilla took out in the New York Times to announce Firefox 1.0. I’ve long believed in Mozilla’s mission. This was an exciting opportunity!</p>\n<p>Fast forward a few years. Rust has been growing, we’re doing great things. I love my team. But every time that this means interacting with Mozilla more broadly, I end up being frustrated. I could say a lot of things here, but I won’t get into the details. What’s really important is this: I’m not proud to be a Mozillian anymore. A variety of incidents contributed to this, but at the end of the day, it’s what’s true. Furthermore, I don’t have any personal opportunity at Mozilla; I recently discovered I’m the lowest-paid person on my team, and Mozilla doesn’t pay particularly well in the first place. In order to have any kind of career growth, I would have to not do the work that aligns with my skills, and what I’m doing now is really how I can have the most impact on Rust moving forward.</p>\n<p>I don’t believe that Mozilla’s direction has any chance of changing; when I’ve tried to express my frustrations, I’ve only gotten disciplined. Mozilla is not interested in hearing what I have to say. And that’s fine, but when I take a step back and think about things, that means it’s time to go, for both my sake and Mozilla’s. So I’ve just put in my two weeks’ notice.</p>\n<p>So what’s next? Well, the first thing is that I don’t plan to stop working on Rust. How much I’ll be able to depends on what’s next, but that’s the great part about open source; I can continue to help the thing I love, even if it might not be full-time anymore.</p>\n<p>I’ve also been enamored with another technology recently: WebAssembly. 2019 is going to be a <em>huge</em> year for WebAssembly, even if many people don’t know it yet, and may not see the effects until 2020. Once again, I find myself seeing something that the world needs, and I think I have the skills to help make that happen. Unlike with Rust, I don’t think that WebAssembly needs my help to actually work on it itself; I think it’s not yet found its killer application. I’d like to help find out what that is, and help both wasm and that application grow.</p>\n<p>There’s something similar in Google’s Fuchsia. I’ve worked with the web for a long time, but doing Rust for the last few years, I’ve also really enjoyed systems-level work. Fuchsia is very interesting and a bit of an enigma, but I can’t wait to see how it turns out. They use a lot of Rust, and plan to use more. I’m not really a kernel hacker, and I’m not sure that Fuchsia is far enough along to really need my skills, but it also really piques my interest.</p>\n<p>Additionally, I’m always open to other things. Before Rust, I worked at Balanced Payments, which was one of my favorite jobs I’ve ever had. I would have never thought I’d want to work at a payments company, but by being open to it, I found an awesome role, with fantastic co-workers. Maybe what I do next will be something completely different!</p>\n<p>In terms of the actual <em>work</em> I would like to do, I don’t think a traditional engineering role really suits me. Don’t get me wrong, I love to write some code, but I don’t think that those kinds of roles really play to my unique strengths. What I really love to do is teaching, evangelizing, and growing something. I’ve done it with Rust, and I can do it again. I could see several roles or hybrid ones here, with titles like “developer evangelist,” “product manager”, or possibly even “VP of Growth.” I want something that challenges me, and where I can grow my career.</p>\n<p>So if you’re:</p>\n<ul>\n<li>A company that wants to invest in Rust, or at Rust in your company</li>\n<li>A company that thinks they can take webassembly (and their product built on it) to the next level</li>\n<li>The Fuchsia team at Google</li>\n</ul>\n<p>or something completely different that you think I’d be psyched about, please <a href=\"mailto:[email protected]\">email me</a> and let’s see where things go!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/thoughts-on-rust-in-2019/",
"title": "Thoughts on Rust in 2019",
"description": null,
"url": "https://steveklabnik.com/writing/thoughts-on-rust-in-2019/",
"published": "2018-12-27T00:00:00.000Z",
"updated": "2018-12-27T00:00:00.000Z",
"content": "<p>Each year, <a href=\"https://blog.rust-lang.org/2018/12/06/call-for-rust-2019-roadmap-blogposts.html\">we ask the community to write blog posts about what they want to see on Rust’s roadmap for the next year</a>. This is my post for Rust in 2019.</p>\n<h2>Rust 2021: Maturity</h2>\n<p>This year is also a bit special; in 2018, we introduced “editions” to Rust, on a roughly three-year schedule. So now is not just a good time to think about 2019, but about 2020 and 2021 as well. Rust 2015 was about “stability”. Rust 2018 was about “productivity.” I’d like Rust 2021 to be about “maturity.”</p>\n<p>In order to get there, here’s what we need in 2019.</p>\n<h2>No <em>new</em> features</h2>\n<p>Emphasis on “new” here. What do I mean by this? Well, there are a few features that are in the pipeline that I <em>do</em> think should land:</p>\n<ul>\n<li><code>async</code>/<code>await</code></li>\n<li>GATs</li>\n<li><code>const</code> generics</li>\n</ul>\n<p>And <em>possibly</em></p>\n<ul>\n<li>Specialization</li>\n</ul>\n<p>None of these features are new; we’ve got their basic designs in place. These features are also <em>significant</em> and <em>foundational</em>; we need <code>async</code>/<code>await</code> (and to some degree, GATs) for a great networking story, we need <code>const</code> generics for an excellent numerics story.</p>\n<p>But after that? I’d like to see a moratorium on major features until 2020, and I’d prefer if we limited 2020 to two or three, if any.</p>\n<p>We’ve hit a sweet spot. We’ve always said that Rust 1.0 was about stability, not completeness. I think we’re fast approaching completeness.</p>\n<p>That said, I don’t think that the language team should be disbanded or anything; I think their work should transition to working on <em>specifying</em> what we already have, in more detail. I’m not sure if we can have the reference be done in 2019 (more on that later), but I’d like it to be much further along. That can only happen with the help of the language team, and they can only do that work if they have the time.</p>\n<h2>Refine the RFC process</h2>\n<p>The RFC process needs to be significantly re-vamped. Niko wrote a great post on this <a href=\"http://smallcultfollowing.com/babysteps/blog/2018/06/20/proposal-for-a-staged-rfc-process/\">back in June</a>, and I think it’s really, really, really important. I’d like to work on an RFC to propose this, so if you’re interested, we should talk.</p>\n<p>Niko already makes the case and lays out some foundations, so I won’t say more there.</p>\n<h2>Pay down organizational debt</h2>\n<p>Consider this section an endorsement of <a href=\"https://boats.gitlab.io/blog/post/rust-2019/\">everything boats said</a>. I cannot say it better, so I will just leave it at that.</p>\n<h2>Figure out documentation sustainability</h2>\n<p>This year was a bad year for the docs team. The book got shipped, and that was great. We had some people pitch in on the reference, and their work was amazing. Some of the other documentation writers moved on to work on <code>rustdoc</code>, which was important.</p>\n<p>But a lot of the goals we wanted to do about writing more docs themselves never came to fruition. For example, contributing to major ecosystem crates? Never happened. The cookbook isn’t done. Rust by Example still languishes. The standard library could use a lot of love.</p>\n<p>We just don’t have the people to make this happen. We’ve tried, but nothing has worked. It’s possible that this is simply unfixable, after all, most programmers don’t like writing docs. But I also don’t want to give up on it either. I’m not sure what to do here, but I do know that it is a major problem.</p>\n<h2>Conclusion</h2>\n<p>There’s a lot of work to do, and I’m very excited to do it. I think Rust is in an <em>excellent</em> place, and with some work, we can make Rust even more amazing in a year’s time.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/six-years-with-rust/",
"title": "Six years with Rust",
"description": null,
"url": "https://steveklabnik.com/writing/six-years-with-rust/",
"published": "2018-12-21T00:00:00.000Z",
"updated": "2018-12-21T00:00:00.000Z",
"content": "<p>Today is six years since I first heard of Rust. <a href=\"https://words.steveklabnik.com/five-years-with-rust\">I wrote a post last year</a> about it.</p>\n<p>This past year was… intense. Rust 1.31 was basically Rust 2.0, at least in the marketing sense. I burned myself out getting the first edition of the book together for Rust 1.0, and I burned myself out getting the edition shipped.</p>\n<p>Let’s talk about the bad and the good. Bad first so we end on the high notes.</p>\n<h2>The Bad</h2>\n<p>This year, the docs team sort of… died. Not entirely, but it’s… different now. We used to have a single docs team that handled most everything docs related. The core of that for the past few years has been me and two others. But early this year, we split the sub-teams into fewer individual teams. The old docs team became the docs team, the <code>rustdoc</code> team, the Rust By Example team, and the Reference team. But we didn’t really attract many more people. We did to some degree! The reference team did a <em>ton</em> of amazing reference work. RBE has still struggled. I stayed on the docs team, and the other two went to the <code>rustdoc</code> team. That’s fine, and they’re doing great work, but it’s a little lonely over here now. I can’t decide how much of this issue is “it’s really hard to get people to volunteer to write docs” and how much of this issue is “I mis-managed the team(s) somehow and so it’s my fault.”</p>\n<p>Shipping the edition was a monumental effort that caused a lot of debt. We did a good job at avoiding <em>technical</em> debt, but we have a lot of social and organizational debt. <a href=\"https://boats.gitlab.io/blog/post/rust-2019/\">See here for more</a>. This has to be paid off now…</p>\n<p>There’s a few other things, but let’s stop there.</p>\n<h2>The Good</h2>\n<p>We shipped the edition! In fact, we invented the entire edition system and then shipped our first one. Whew! This is a <em>massive</em>, <em>massive</em> accomplishment. Figuring out how we are going to manage Rust’s evolution decades into the future is, frankly a pretty amazing thing to do in twelve months. Non-lexical lifetimes finally landed. Procedural macros finally landed. Rocket is almost on stable. <code>async</code>/<code>await</code> is almost here, and <code>Futures</code> are finally where they should be after two years of work. We did <em>so much</em> good work this year, technically speaking. I’m very proud.</p>\n<p>We shipped the new website! This was met with some controversy, but every single website redesign in the history of redesigns has been met with controversy. Doing so in less than a year is also an incredible accomplishment. It almost always takes way longer. We now have the foundations for Rust web properties going forward. This will also pay off a ton later.</p>\n<p>We made wasm a thing! I really really wanted wasm to be a thing. A lot of people put in a ton of work, and Rust’s wasm support is bar-none best-in-class. I think this is still a <em>massive</em> growth area for Rust, and we’re almost a full year ahead of almost everyone else on this.</p>\n<p>Speaking of growth, Rust’s commercial uptake skyrocketed this year. I met someone for the first time who told me “I learned Rust because it’s what we use at work.” A ton of companies, big and small, have started deploying Rust in critical places, and report good things. Out of the FAANGs, we’ve got Facebook, Amazon, and Google. Three out of five ain’t bad. This tweet is wild:</p>\n<blockquote>\n<p>Introducing #HyperledgerSawtooth version 1.1, an enterprise-grade #blockchain platform with support for #WebAssembly smart contracts and great performance with @rustlang. Empower your business. https://t.co/wl9OYVnUsR #HyperledgerForum pic.twitter.com/5vurpbVMP1\n— Intel Security (@IntelSecurity) December 12, 2018</p>\n</blockquote>\n<p>So is this one:</p>\n<blockquote>\n<p>Discover Rust, a C-like language that supports imperative, functional, and object-oriented programming. https://t.co/fCgY4x6zqc\n— IBM Developer (@IBMDeveloper) June 10, 2018</p>\n</blockquote>\n<p>There’s still lots more to do, but things are headed in the right direction.</p>\n<p>Speaking of heading in the right direction, tomorrow will be the compliment to this post; my entry for <code>#rust2019</code>. Catch you then!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/why-is-clojure-so-stable/",
"title": "Why is Clojure so stable?",
"description": null,
"url": "https://steveklabnik.com/writing/why-is-clojure-so-stable/",
"published": "2018-12-18T00:00:00.000Z",
"updated": "2018-12-18T00:00:00.000Z",
"content": "<p>There’s been some Programming Language Discourse lately, and I have some thoughts. It’s mostly centered around <a href=\"https://news.ycombinator.com/item?id=18702802\">a comment on the orange website</a>:</p>\n<blockquote>\n<p>I would put it simply: Clojure was designed by a thinker, who creates when he is away from the keyboard, not in front of it. When one releases and breaks the code in his head first, very few breaking changes are left for the public releases.</p>\n</blockquote>\n<p>I think, regardless of this person being right or wrong, there’s an interesting question here. Let’s look at <a href=\"https://news.ycombinator.com/item?id=18701834\">the comment this is ultimately responding to</a>:</p>\n<blockquote>\n<p>From the perspective of a (fairly large-scale at this point) app developer: I find it great that Clojure places such emphasis on backwards compatibility. In general, migration to newer Clojure versions is completely painless.The language has been designed by experienced and mature people and doesn’t go through “let’s throw everything out and start again” phases like so many other languages do.</p>\n</blockquote>\n<p>Clojure has a well-deserved reputation for stability. The argument is not about that, but about <em>why</em> Clojure is so stable.</p>\n<h2>Individuals and their greatness</h2>\n<p>A lot of people, like the original commentator, believe that Rich, the BDFL of Clojure, is responsible. I think Rich deserves a lot of credit, but at the same time, this explanation is a little bit too close to the “great man theory of history.” The Great Man theory was originally proposed in the mid 1800s by Thomas Carlyle, and goes something like this: \"“the history of the world is but the biography of great men.”</p>\n<blockquote>\n<p>Don’t get me started about Carlyle, who was a pretty terrible guy even in his own day. A lot of the alt-right, “Dark Englightenment”, and Neo-reactionary folks consider him a really important thinker, and that’s all I’ll say about that.</p>\n</blockquote>\n<p>What this <em>means</em> is that Important Men make big decisions, and that the lives of the rest of us are irrelevant in the scheme of things. He cited people like Shakespeare, Martin Luther, and Napoleon. In Carlyle’s mind, history was changed by these men and them alone. The French Army didn’t win wars, Napoleon won wars.</p>\n<p>I tend to subscribe to the opposite, that is, “history from below,” or “a people’s history,” which emphasizes the army over Napoleon, the worker over the CEO. A critic of Carlyle’s, Herbert Spencer, said this:</p>\n<blockquote>\n<p>You must admit that the genesis of a great man depends on the long series of complex influences which has produced the race in which he appears, and the social state into which that race has slowly grown. … Before he can remake his society, his society must make him.</p>\n</blockquote>\n<p>That is, we have to look at historical context to see what created these changes, not just a few famous individuals.</p>\n<blockquote>\n<p>Two sidebars here: first, Spencer is the person who coined the phrase “survival of the fittest.” Second, there is an irony in framing this particular philosophical debate about if individuals or history cause historical change by citing well-known opinions of two individual men. This idea was so popular, and still is, that it’s something many of us latch onto, sometimes subconsciously, even those who don’t believe it!</p>\n</blockquote>\n<p>So, beyond Rich, who as I said before, does deserve a lot of credit… why might Clojure be so stable? I can think of three reasons. There may be more, but these are what comes to my mind.</p>\n<h2>Clojure is a Lisp</h2>\n<p>First of all, Clojure is a Lisp. This has a wide array of reasons why stability is a bit easier than in other languages.</p>\n<p>The first is Lisps’ lack of syntax. Most languages have updates that introduces new syntax. As this grows larger and larger, the design space gets more and more constrained, making updates harder and harder. Lisps, on the other hand, rarely grow syntactically. Clojure has more syntax than most Lisps, but it still has far less syntax than non-Lisp languages.</p>\n<p>Another perspective of this is homoiconicity. This property means that “new syntax” for Lisps can be introduced through macros, and not through the base language. In most other languages, users cannot extend the language to provide new syntax, but in Clojure, they can.</p>\n<p>Lesson: the smaller your core language, the more stable you can be.</p>\n<h2>Clojure is dynamically typed</h2>\n<p>There are many, many, many ways to debate the virtues of static typing against those of dynamic typing. However, one area in which dynamically typed languages clearly win out, in my mind, is that they have a wider set of tools at their disposal to make changes without breaking code. This cuts both ways; they also make it much, much harder to know if you’ve broken code as well. Those are able to be mitigated through a variety of techniques, and we can argue about the tradeoffs here, but I think the possibility of advantage is plainly clear.</p>\n<p>Lesson: it can be easier to change a dynamic system than a static one. This is both good and bad, but for now we’re focused on the good.</p>\n<h2>Clojure’s development is pretty slow</h2>\n<p>Clojure famously optimizes contributions around Rich and his process, at the expense of others. Debating this has been a real sore spot lately, and I don’t mean to pile onto this part of the discussion. I’m not interested in suggesting, in this blog post, if this is good or bad. I’m interested in the effects it has.</p>\n<p>If we ask <a href=\"https://github.com/clojure/clojure/compare/1.0...master\">GitHub how many commits were between 1.0 and <code>master</code></a>, the answer is 1,957. If we look at <a href=\"https://github.com/clojure/clojure/compare/clojure-1.9.0...clojure-1.10.0\">commits between 1.9.0 and 1.10.0</a>, we get 138 commits. Clojure 1.9 was released December 8, 2017. Clojure 1.10 was released December 17, 2018. That’s 138 commits for that release, for a year.</p>\n<p>In contrast, <a href=\"https://github.com/rust-lang/rust/compare/1.30.0...1.31.0\">between 1.30 and 1.31</a>, six weeks of development, Rust had 1,694 commits. that’s almost as many as Clojure has had in the last nine and a half years.</p>\n<p>Lesson: It’s much easier to keep things stable when you don’t change as often.</p>\n<hr />\n<p>All of these factors tie into each other. Clojure doesn’t need to change as much, since the core language is a Lisp. When it does need to change, it has good tools at its disposal to change. Also, note that Rich was the one who made a lot of these decisions! He absolutely deserves credit for a job well done, even if I think that there are many other factors at play.</p>\n<p>Furthermore, ascribing this entirely to Rich makes it harder for other languages to learn good lessons from Clojure’s successes; they can’t just get a Rich. By digging in a bit deeper, we can apply these lessons a bit more broadly. Good language design requires taking a sober look at various tradeoffs, and making the best decision you can make, regardless if you’re a BDFL or part of a language design team.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/borrow-checking-escape-analysis-and-the-generational-hypothesis/",
"title": "Borrow checking, escape analysis, and the generational hypothesis",
"description": null,
"url": "https://steveklabnik.com/writing/borrow-checking-escape-analysis-and-the-generational-hypothesis/",
"published": "2018-10-10T00:00:00.000Z",
"updated": "2018-10-10T00:00:00.000Z",
"content": "<p>There’s a fairly common argument that pops up on programming forums, and it’s about the nature of what “garbage collection” means. In the research world, <a href=\"http://www.memorymanagement.org/glossary/g.html#term-garbage-collection\">this</a> is what GC means:</p>\n<blockquote>\n<p>Garbage collection (GC), also known as automatic memory management, is the automatic recycling of dynamically allocated memory(2). Garbage collection is performed by a garbage collector which recycles memory that it can prove will never be used again. Systems and languages which use garbage collection can be described as garbage-collected.</p>\n</blockquote>\n<p>Historically, there has been two major forms of GC: reference counting, and tracing. The argument happens because, with the rise of tracing garbage collectors in many popular programming languages, for many programmers, “garbage collection” is synonymous with “tracing garbage collection.” For this reason, I’ve been coming around to the term “automatic memory management”, as it doesn’t carry this baggage.</p>\n<hr />\n<p>I’ve often seen people make statements like this one, from the Rust subreddit this morning:</p>\n<blockquote>\n<p>Manual memory management requires more work than garbage collected. Its a trade off of course to be more performant or use lower resources. When and where should Rust be used or not used according to you?</p>\n</blockquote>\n<p>While I don’t <em>completely</em> disagree with this sentiment, it’s also never quite sat right with me. Yes, Rust is a bit harder <em>at the start</em>, but once you get over a hump, I don’t generally find writing Rust to be significantly harder than using a GC’d language. I’ve been trying to figure out why that is.</p>\n<blockquote>\n<p>The idea that it’s easy for me because I’m an amazing programmer is not an answer I’m willing to accept here, both because I don’t think that’s particularly true, but also because I’ve seen Rust have a pretty wide appeal to a fairly large group of people, and they can’t all be amazing programmers. It’s just math.</p>\n</blockquote>\n<hr />\n<p>My first clue comes from my own programming history. Some of my first programming languages were C, C++, and Java. These languages are <em>statically</em> typed, that is, they require you to declare (or have inferred) the type of variables up front, and check that the type holds. That is, they’re known at compile time. I can remember the first time I came across Perl, and the concept of dynamic typing. Wow, this is <em>so much easier</em>! I could do <em>anything</em>. From then on, I pretty much focused on dynamically typed languages.</p>\n<p>While doing my work in Ruby, one thing I worked hard on was testing. Rubyists are big on tests; with a language that’s <em>so</em> dynamic, you really need them to make sure your stuff is working properly.</p>\n<p>When I decided to start writing a bunch of Rust, I had to get used to using static types again. But I also wrote less tests. Many of the things that I was testing for were things that were taken care of by the type system. In other words, my tests were sort of like “dynamic type checking.” Instead of being checked by a compiler at compile-time, these behaviors of my system were checked at run time, by my tests. This isn’t a novel insight; many others have stated this relationship between tests and types.</p>\n<p>We often acknowledge that writing tests is a <em>skill</em>. When I first started with Ruby, I was bad at testing. I got better with practice. But I don’t think we collectively acknowledge that working with types is <em>also</em> a skill. At first, it’s tough. But as you develop those skills, it becomes easier and easier.</p>\n<hr />\n<p>Let’s switch gears a little, to garbage collection.</p>\n<p>In garbage collection research, there’s a term, the <a href=\"http://www.memorymanagement.org/glossary/g.html#term-generational-hypothesis\">“generational hypothesis”</a>. It states</p>\n<blockquote>\n<p>In most cases, young objects are much more likely to die than old objects.</p>\n</blockquote>\n<p>If that succinct description doesn’t make sense, here’s a slightly longer one: there’s a kind of GC called a “generational garbage collector.” There are a few options when implementing one, but I’m going to describe one form in particular. What it does is this: new objects are allocated in a section of memory called the “nursery,” an analogy to where human children go shortly after they’re born. When the GC runs, it looks through the nursery. Any objects that survive are promoted to a new section of memory. Some collectors may have multiple of these “generations.” The generational hypothesis states that most objects die in the nursery. The GC can then take advantage of this fact by scanning the nursery more often than the older generations, which allows you to not scan every object every collection.</p>\n<p>Empirically, the generational hypothesis is true. But I haven’t seen much investigation into <em>why</em> it is true. That said, I’m also not a GC expert. If you have references here, please get in touch.</p>\n<hr />\n<p>There’s another concept related to GCs called “escape analysis.” This is an optimization which several languages do. The idea is fairly straightforward: if something is declared in a function, and never leaves (“escapes”) that function, you don’t need to put the object on the heap, you can stack allocate it instead.</p>\n<blockquote>\n<p>“It isn't that the generational hypothesis isn't true for Go, it's just that the young objects live and die young on the stack.The result is that generational collection is much less effective than you might find in other managed runtime languages” #golang https://t.co/aXEleY2yQ6\n— Dave Cheney (@davecheney) July 18, 2018</p>\n</blockquote>\n<p>If you squint a little at this tweet, you can almost think of escape analysis as a <em>stronger</em> version of the generational hypothesis. The claim is that escape analysis applies so often to Go code that any object that reaches the heap is effectively in the older generation.</p>\n<p>Why is this? I’m not totally sure, but I have a feeling that is has to do with the core concept of functions and a call stack. Additionally, the value/reference type distinction. The more you program with values in functions, rather than references, the less likely objects are to escape past the call stack. in generational hypothesis terms, values allocated lower in the call stack tend to live longer, and ones higher in the call stack tend to live shorter.</p>\n<hr />\n<p>There’s an interesting area of type research called “gradual typing.” The idea is that typing is not an either/or, static vs dynamic. It’s a spectrum. Some things are statically typed, some things are dynamically typed, and it all works together. The idea is that you can start off writing things that are dynamically typed, and then once you’re more sure of the behaviors you want, you can start to statically type them.</p>\n<p>TypeScript is a great example:</p>\n<pre><code>// person is dynamically typed\nfunction greeter(person) {\n return \"Hello, \" + person;\n}\n\n// person is statically typed\nfunction greeter(person: string) {\n return \"Hello, \" + person;\n}\n</code></pre>\n<p>TypeScript is really, really growing in popularity. I think one of those factors is the gradual typing; you can slowly work on your type-using skills, and you don’t have to use them everywhere.</p>\n<hr />\n<p>Interestingly enough, many JIT compilers rely on the fact that in practice, programmers tend to pass the same types of things to functions, even if the language is dynamically typed. This is called <a href=\"https://hacks.mozilla.org/2017/02/a-crash-course-in-just-in-time-jit-compilers/\">“type specialization”</a>. This is similar to the generational hypothesis to me, but in a different domain. And as I said about it, empirically, this is true. But I haven’t seen much investigation into <em>why</em> it is true. That said, I’m also not a JIT expert. If you have references here, please get in touch.</p>\n<hr />\n<p>So what’s all this have to do with Rust?</p>\n<p>Similar to the dichotomy between static and dynamic typing, I think that Rust is proving out a new niche in the concept of garbage collection. That is, historically, we think of GC as something <em>dynamic</em>, that is, it involves a run-time component that does stuff at runtime. However, the idea of automatic memory management doesn’t <em>inherently</em> mean that it has to execute at runtime.</p>\n<blockquote>\n<p>I should mention that C++ paved a lot of the path here too, so maybe the above is worded too strongly. I do think that Rust is significantly different here, but you may not. If you don’t, please replace the above with “Rust is continuing to prove out a niche pioneered by C++,” and that’s fine by me.</p>\n</blockquote>\n<p>This also opens up two more things about why I don’t feel that “Rust is too hard and so you should use a GC’d language when you can afford it.” First of all, Rust’s strong, static checking of memory provides me a similar peace of mind to a runtime GC. To make an analogy to types and tests, when I program in a language like C, I have to think a lot about the details of memory management, because I don’t really have anything at all checking my work. Its memory management is completely unchecked, similar to (compile time) types in a Ruby program. But with automatic memory management, I don’t have to worry about this, as the memory management system has my back. I won’t get a use after free, I won’t get dangling pointers.</p>\n<p>I further think this line of thought is on the right track because lifetimes are generic types, and the ownership system is affine types.</p>\n<hr />\n<p>This also relates to the “difficulty” of Rust. In the same way that learning to use static types takes time, learning to use static memory management takes time. But once you’ve developed that skill, you can rely on those static checks to free up your brain to instead focus on your program itself, and not the house-keeping tasks that these systems are checking for.</p>\n<p>And, just like with static types, different people will make different choices of tradeoff for effort. Some people refuse to use dynamically typed languages, because they’ve invested the time into effectively using static types and prefer to have these things checked. Others don’t want to waste time on learning these kinds of things, and are more focused on shipping things. They have their own set of skills to ensure reliability. I think both approaches make sense depending on what you’re doing.</p>\n<hr />\n<p>In some ways, you can think of the borrow checker as an even stronger form of escape analysis; that is, instead of saying “oh, this escapes, so allocate it on the heap”, the borrow checker says “oh, this escapes, so error at compile time.” For example, this code:</p>\n<pre><code>fn bar() -> i32 {\n 5\n}\n\nfn foo() -> &'static i32 {\n &bar()\n}\n</code></pre>\n<p>gives this error:</p>\n<pre><code>error[E0716]: temporary value dropped while borrowed\n --> src/lib.rs:6:6\n |\n6 | &bar()\n | ^^^^^ creates a temporary which is freed while still in use\n7 | }\n | - temporary value is freed at the end of this statement\n |\n = note: borrowed value must be valid for the static lifetime...\n</code></pre>\n<p><code>bar()</code> produces a value, and so <code>&bar()</code> would produce a reference to a value on <code>foo()</code>’s stack. Returning it would be a dangling pointer. In a system without automatic memory management, this would cause a dangling pointer. In a system with dynamic automatic memory management, this would heap allocate the return value of <code>bar()</code>, so that everything works out. But Rust’s automatic memory management is fully static, and so it has to error in this situation.</p>\n<hr />\n<p>Similar to gradual typing, this means that you can also have gradual GC. For example, Rust does have <code>Rc<T></code> and <code>Arc<T></code>, which are reference-counted types, clearly GC according to the literature. This moves the memory management from static to dynamic.</p>\n<blockquote>\n<p>I don’t think that the transition here is as seamless as in TypeScript, though that’s a whole other post… additionally, it’s not like gradual typing is perfect.</p>\n</blockquote>\n<hr />\n<p>One last thing before I go: the lifetime elision rules are also an empiric statement about how programmers write code, but I’m not sure what the underlying reasons are. From <a href=\"https://github.com/rust-lang/rfcs/blob/master/text/0141-lifetime-elision.md\">the RFC</a>:</p>\n<blockquote>\n<p>We found that of the 169 lifetimes that currently require annotation for libstd, 147 would be elidable under the new rules, or 87%.</p>\n</blockquote>\n<p>If it were 100%, we could never require you to write annotations. But with 87%, we can let you not write annotations, and most code that follows most normal coding practices never needs to write explicit lifetimes.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/should-you-learn-c-to-learn-how-the-computer-works/",
"title": "Should you learn C to \"learn how the computer works\"?",
"description": null,
"url": "https://steveklabnik.com/writing/should-you-learn-c-to-learn-how-the-computer-works/",
"published": "2018-10-02T00:00:00.000Z",
"updated": "2018-10-02T00:00:00.000Z",
"content": "<p>I’ve often seen people suggest that you should learn C in order to learn how computers work. Is this a good idea? Is this accurate? I’m going to start with my conclusion right upfront, just to be crystal clear about what I’m saying here:</p>\n<ul>\n<li>C is not “how the computer works.”</li>\n<li>I don’t think most people mean this phrase literally, so that is sort of irrelevant.</li>\n<li>Understanding the context means that learning C for this reason may still be a good idea for you, depending on your objectives.</li>\n</ul>\n<p>I plan on making two follow-up posts as well, exploring more implications of this idea, but this is already quite a lot. I’ll update this post with links to them when I make them.</p>\n<hr />\n<p>I’ve often seen people suggest this:</p>\n<blockquote>\n<p>By learning C, you can learn how computers work.</p>\n</blockquote>\n<p>I think that this idea is not inherently wrong, but does come with some caveats. As long as you keep those caveats in mind, I think this can be a viable strategy for learning new and important things. However, I rarely see people discuss this slogan in detail, so I’m writing this post so that I can provide that context that I believe is sorely needed. If you’re thinking about learning C for this reason, this post is written for you. I hope it’ll assist you as you learn.</p>\n<blockquote>\n<p>Before we truly begin, I’d like to say one more thing: if you want to learn C, then learn C! Learning things is great. Learning C was quite important to my understanding of computing and my career. Learning C and about its place in programming language history will make you a better programmer. You don’t need a justification. Learn stuff for the sake of learning it. This post is intended to be a guide to your quest for understanding, not a suggestion that you should or should not learn C.</p>\n</blockquote>\n<p>First of all, let’s talk about who is generally recommended this idea. If you’re trying to “learn how computers work,” then it stands to reason that you don’t currently understand that. Which programmers don’t understand how computers work? I’ve mostly seen this sentiment come from people who primarily program in dynamically typed, “scripting” languages, like Ruby, Python, or JavaScript. They “don’t know how computers work” because these languages operate inside of a virtual machine, and therefore, the semantics of the virtual machine are what actually matters. After all, the whole idea of a virtual machine is to provide portability. The goal is to only require knowledge of what’s portable, rather than rely on the details of the hardware that the VM is running on.</p>\n<p>There’s just one problem with this: C <em>also</em> operates inside of a virtual machine.</p>\n<h2>The C abstract machine</h2>\n<p>From the <a href=\"http://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf\">C99 spec</a>, Section 5.1.2.3, “Program Execution”:</p>\n<blockquote>\n<p>The semantic descriptions in this International Standard describe the behavior of an abstract machine in which issues of optimization are irrelevant.</p>\n</blockquote>\n<p>In my opinion, this is the most important concept to understand when learning C. C does not “describe how the computer works,” it describes how the “C abstract machine” works. Everything else of importance flows from this concept and its implications.</p>\n<blockquote>\n<p>A further note: I’ve chosen C99 here, which is not the latest C standard. Why? Well, MSVC has an …interesting amount of C support, and I’m a Windows user these days. Yes, you can run clang and gcc on Windows. There is not a ton of difference between C89, C99, and C11 with regards to what we’re talking about in these posts. At some point you have to pick. The version I’ve linked here includes some amendments to the initial spec.</p>\n</blockquote>\n<p>You may have heard another slogan when talking about C: “C is portable assembler.” If you think about this slogan for a minute, you’ll also find that if it’s true, C cannot be how the computer works: there are many kinds of different computers, with different architectures. If C is like a version of assembly language that works on multiple computers with multiple architectures, then it cannot function exactly how each of those computers work simultaneously. It <em>must</em> hide details, or else it wouldn’t be portable!</p>\n<p>That said, I think this fact is sort of irrelevant, because I don’t think people mean the phrase “C is how the computer works” literally. Before we can talk about that, let’s talk about the C abstract machine, and why many people don’t seem to understand this aspect of the C language.</p>\n<h2>Aside: why do people get this wrong?</h2>\n<p>I can only relate my own experience, though I think it generalizes to many others.</p>\n<p>I learned GW-BASIC, then C, then C++, then Java. I had heard about Java before I started writing it in roughly 1999, four years after it was introduced. Java’s marketing at the time was very anti-C++, and it focused on the JVM as a platform, and how the machine model was what differentiated it from C++, and therefore C. Sun Microsystems no longer exists, but a <a href=\"https://tech-insider.org/java/research/1996/0123.html\">mirror of the press release</a> has this to say:</p>\n<blockquote>\n<p>Java-based applications are platform-independent; only the Java Virtual Machine needs to be ported to each platform. It acts as an interpreter between an end user’s computer and the Java-based application. An application written in the Java environment can run anywhere, ending the need for porting applications to multiple platforms.</p>\n</blockquote>\n<p>“Write once, run everywhere” was the big slogan. These two sentences were how I (and many others) came to understand Java, and how it differed from C++. Java has an interpreter, the Java Virtual Machine. C++ does not have a Virtual Machine.</p>\n<p>With this huge marketing push, “virtual machine” became synonymous with “large runtime and/or interpreter” in the minds of many. Languages without this feature were too tied to the particular computer, and required porting between platforms, because they are not truly platform independent. Java’s very reason to exist was to change this aspect of writing applications in C++.</p>\n<blockquote>\n<p>“Runtime”, “virtual machine”, and “abstract machine” are different words for the same fundamental thing. But they’ve since gained different connotations, due to non-essential variance in different implementations of these ideas.</p>\n</blockquote>\n<p>I personally believe that this marketing, in 1995, is why programmers still misunderstand C’s nature today.</p>\n<p>So is this claim false? Why would Sun Microsystems spend millions and millions of dollars promoting a lie? If C is also based on an abstract machine that offers portability between platforms, why did Java need to exist? I think this is key to understanding what people <em>really</em> mean when they say “C is how the computer works.”</p>\n<h2>What do people actually mean?</h2>\n<p>Even though C operates in the context of a virtual machine, it still is significantly different than something like Java. Sun was not lying. In order to understand why, you have to understand C’s history.</p>\n<p>An operating system was written in assembly language for a computer called the “PDP-7” in 1969 at Bell Labs. In 1970, it was christened UNIX. As time went on, Bell Labs bought more and newer computers; including the PDP-11.</p>\n<p>When it came time to port Unix to the PDP-11, they considered using a higher-level language, which was a pretty radical idea at the time. Imagine that I said to you “I’m going to write an OS in Java” today, you’d probably laugh, even though <a href=\"https://en.wikipedia.org/wiki/JavaOS\">it is possible</a>. The situation (in my understanding, I wasn’t alive then) was pretty analogous. They considered a language called B, but it didn’t support some of the features that the PDP-11 had, and so they created a successor, naming it “C”, since C was the next letter in the alphabet.</p>\n<blockquote>\n<p>There was no “A”; B was a successor to BCPL, the “Basic Combined Programming Language.”</p>\n</blockquote>\n<p>In 1972, on a PDP-11, they wrote the first C compiler, and simultaneously re-wrote UNIX in C. Initially, portability wasn’t the actual goal, but C did resonate with a lot of people, and C compilers were ported to other systems.</p>\n<p>In 1978, the first edition of “The C Programming Language,” a book on C, was published. Affectionately known as “K&R,” after its authors, the book was very much not a specification, but provided enough of a description of the language that others attempted to write C compilers. This would later be called “K&R C.”</p>\n<p>As UNIX spread, so did C, and both were ported to many computers. In the 70s and 80s, the ports grew and grew. In the same way that C was created because B couldn’t support all of the features of the PDP-11, many compilers had extensions to the language. Since there was only K&R, and not a spec, as long as they were close enough, that was considered acceptable. By 1983, the lack of any sort of standardization was causing issues, and so a group was created at ANSI to produce a spec. C89 was published in 1989, and is sometimes called “ANSI C.”</p>\n<p>As such, the C specification was attempting to unify these diverse implementations, on diverse hardware. And so the C abstract machine is sort of the minimal possible specification that would allow the same code to run the same on all platforms. Implementations of C were compiled, not interpreted, and so there was no interpreter, so there was no “VM” in that 1995 sense. However, C programs are written against this abstract, non-existent machine, which is then translated into assembly that is specific to the actual computer that the program is running on. You could not rely on some specific details in order to write portable C code. This makes writing portable C very tricky, as you may have made a platform-specific assumption when writing the initial version of your code.</p>\n<p>This is best illustrated with an example. One of the fundamental data types in C is a <code>char</code>, named after “character.” However, the C abstract machine does not define how many bits a <code>char</code> is. Well, it defines it, but not with a number; it defines it as <code>CHAR_BIT</code> big, which is a constant. Section 5.2.4.2.1 of the spec:</p>\n<blockquote>\n<p>The values given below shall be replaced by constant expressions suitable or use in #if preprocessing directives. … Their implementation-defined values shall be equal or greater in magnitude (absolute value) to those shown, with the same sign.CHAR_BIT: 8</p>\n</blockquote>\n<p>So in other words, you know that a <code>char</code> must be at least 8 bits in size, but implementations are allowed to be larger. In order to properly code against the “C abstract machine”, you must use <code>CHAR_BIT</code> instead of <code>8</code> as the size when doing <code>char</code>-by-<code>char</code> processing. But this isn’t some sort of interpreter feature, in the way that we think of virtual machines; it’s a property of how the compiler turns your source code into machine code.</p>\n<blockquote>\n<p>Yes, there are systems where CHAR_BIT isn’t 8.</p>\n</blockquote>\n<p>So this “abstract machine”, while still technically being the same idea as the Java VM, is more of a compile-time construct to guide compilers in the task of emitting assembly code than some sort of runtime check or property. The equivalent type in Java is <code>byte</code>, which is always 8 bits, and it’s up to the implementation of the JVM to decide how to make that happen on platforms where a byte is larger. (I am not sure if the JVM runs on any of those platforms, but that’s how it would work.) The C abstract machine was created to be a minimal papering over of various hardware, not to be some sort of platform created from whole cloth that’s written in software that your code targets.</p>\n<p>So while Sun is technically incorrect, in practice, they mean something different than what they’re literally saying, and what they <em>mean</em> is accurate. It’s the same with “learn C to learn how computers work.”</p>\n<h2>Learn C to learn <em>more</em> about how computers work</h2>\n<p>What <em>do</em> people actually mean? In the context of “should a Rubyist learn C to learn about how computers work”, this desire to drop “down to the metal,” as it were, is an interest in understanding not only their program, and how it works inside the VM, but to understand how the combo of their program + their VM operates in the context of the machine itself.</p>\n<p>Learning C <em>will</em> expose you to more of these kinds of details, because the abstract machine maps so much more closely to hardware, as well as abstractions provided by operating systems. C is very different than these sorts of languages, and so learning C can teach you a lot.</p>\n<p>But it’s also important to remember that C is fundamentally an <em>abstraction</em> of hardware, and abstractions are leaky. Don’t conflate what C does or how it operates with the machine itself. If you do, you’re bound to run afoul of these differences, which can cause problems. Most learning resources for C, especially today, as hardware becomes more and more homogeneous, will promote the idea that this is how the computer works. So it can be hard, as a learner, to know what’s going on under the hood, and what’s an abstraction provided by C.</p>\n<p>We haven’t even touched on other factors in this discussion, like that due to C’s tremendous popularity, hardware has become more homogeneous because it tends towards being similar to the semantics of C’s abstract machine. If your architecture deviates too much from C’s semantics, C programs may run much more slowly than others, and C programs are often how the speed of hardware is tested. This post is long enough…</p>\n<p>For this reason, I think a more accurate version of this statement would be “By learning C, you can learn <em>more</em> about how computers work.” I do think that a rough familiarity with C can be useful to many programmers, and will serve them well even if they don’t write any C themselves. Learning more about C can also give you insight into the history of how our field has developed.</p>\n<p>There are other ways to learn this stuff as well; C is not <em>inherently</em> the way to learn these topics, but it is a good option. My next two posts will explore some of the implications of this intersection between portability, the C abstract machine, and how people think about C programs.</p>\n<p>There’s so many things to learn in computing. I wish you well on your journey.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/you-can-t-turn-off-the-borrow-checker-in-rust/",
"title": "You can't \"turn off the borrow checker\" in Rust",
"description": null,
"url": "https://steveklabnik.com/writing/you-can-t-turn-off-the-borrow-checker-in-rust/",
"published": "2018-09-14T00:00:00.000Z",
"updated": "2018-09-14T00:00:00.000Z",
"content": "<p>Every once in a while, someone will talk about <code>unsafe</code> in Rust, and how it “turns off the borrow checker.” I think this framing leads to misconceptions about <code>unsafe</code> and how it interacts with safe code.</p>\n<p>Here’s some code that causes a borrow checker error:</p>\n<pre><code>fn main() {\n let mut x = 5;\n \n let y = &x;\n let z = &mut x;\n\n println!(\"{}\", y);\n}\n</code></pre>\n<p>And the error:</p>\n<pre><code>error[E0502]: cannot borrow `x` as mutable because it is also borrowed as immutable\n --> src/main.rs:5:18\n |\n4 | let y = &x;\n | - immutable borrow occurs here\n5 | let z = &mut x;\n | ^ mutable borrow occurs here\n6 | }\n | - immutable borrow ends here\n</code></pre>\n<p>Rust won’t let us have a <code>&T</code> and a <code>&mut T</code> to the same <code>T</code> at the same time.</p>\n<p>If <code>unsafe</code> \"turned off the borrow checker, we’d expect this code to work:</p>\n<pre><code>fn main() {\n let mut x = 5;\n \n unsafe {\n let y = &x;\n let z = &mut x;\n\n println!(\"{}\", y);\n }\n}\n</code></pre>\n<p>However, we get the same error, plus a new warning:</p>\n<pre><code>error[E0502]: cannot borrow `x` as mutable because it is also borrowed as immutable\n --> src/main.rs:6:22\n |\n5 | let y = &x;\n | - immutable borrow occurs here\n6 | let z = &mut x;\n | ^ mutable borrow occurs here\n7 | }\n | - immutable borrow ends here\n\nwarning: unnecessary `unsafe` block\n --> src/main.rs:4:5\n |\n4 | unsafe {\n | ^^^^^^ unnecessary `unsafe` block\n |\n = note: #[warn(unused_unsafe)] on by default\n</code></pre>\n<p>So what does <code>unsafe</code> actually do? Unsafe Rust is a <em>superset</em> of Safe Rust. <code>Unsafe</code> never changes the semantics of Rust code. It instead adds new features that you can only use inside an <code>unsafe</code> block.</p>\n<p>One of those features is the ability to de-reference a “raw pointer”. This code <em>does</em> compile:</p>\n<pre><code>fn main() {\n let mut x = 5;\n \n let y = &x as *const i32;\n let z = &mut x as *mut i32;\n \n unsafe {\n println!(\"y: {} z: {}\", *y, *z);\n }\n}\n</code></pre>\n<p><code>y</code> and <code>z</code> here have the types <code>*const i32</code> and <code>*mut i32</code>, both kinds of raw pointers. In order to print <code>y</code> and <code>z</code>, we must de-reference them, which means that we need an unsafe block.</p>\n<blockquote>\n<p>Incidentally, this code is safe. Raw pointers are allowed to alias. And we have no &T or &mut Ts here. So we’re good.</p>\n</blockquote>\n<p>Why does this matter? Well, a lot of people think that as soon as you drop into <code>unsafe</code>, you’re missing the point of Rust, and that you lose all of its guarantees. It’s true that you have to do a lot more work when writing unsafe, since you don’t have the compiler helping you in certain situations, but that’s only for the unsafe constructs.</p>\n<p>For example, let’s look at Rust’s standard library, and the <code>[LinkedList<T>](https://doc.rust-lang.org/stable/std/collections/linked_list/struct.LinkedList.html)</code> that it contains. It looks like this:</p>\n<pre><code>pub struct LinkedList<T> {\n head: Option<NonNull<Node<T>>>,\n tail: Option<NonNull<Node<T>>>,\n len: usize,\n marker: PhantomData<Box<Node<T>>>,\n}\n</code></pre>\n<p>I’m not gonna go into <em>too</em> much detail here, but it has a head pointer, a tail pointer, a length. <code>[NonNull<T>](https://doc.rust-lang.org/stable/std/ptr/struct.NonNull.html)</code> is like <code>*mut T</code>, but asserts that it will never be null. This means that we can combine it with <code>Option<T></code>, and the option will use the <code>null</code> case for <code>None</code>:</p>\n<pre><code>fn main() {\n println!(\"{}\", std::mem::size_of::<*mut i32>());\n println!(\"{}\", std::mem::size_of::<Option<std::ptr::NonNull<i32>>>());\n}\n</code></pre>\n<p>prints this on my system:</p>\n<pre><code>8\n8\n</code></pre>\n<p>This optimization is guaranteed due to guarantees on both <code>Option<T></code> and <code>NonNull<T></code>.</p>\n<p>So now, we have a sort of hybrid construct: <code>Option<T></code> is safe, and so we can do some operations entirely in safe code. Rust is now forcing us to handle the null checks, even though we have zero runtime overhead in representation.</p>\n<p>I think a good example of how this plays out in practice is in the implementation of <code>[append](https://doc.rust-lang.org/stable/std/collections/linked_list/struct.LinkedList.html#method.append)</code>, which takes two <code>LinkedList<T></code>s and appends the contents of one to the end of the other:</p>\n<pre><code> pub fn append(&mut self, other: &mut Self) {\n match self.tail {\n None => mem::swap(self, other),\n Some(mut tail) => {\n if let Some(mut other_head) = other.head.take() {\n unsafe {\n tail.as_mut().next = Some(other_head);\n other_head.as_mut().prev = Some(tail);\n }\n\n self.tail = other.tail.take();\n self.len += mem::replace(&mut other.len, 0);\n }\n }\n }\n }\n</code></pre>\n<p>Here’s some comments to help understand what’s going on:</p>\n<pre><code> pub fn append(&mut self, other: &mut Self) {\n // do we have a tail?\n match self.tail {\n\n // If we don't, then we have no elements, and so appending a\n // list to this one means that that list is now this list.\n None => mem::swap(self, other),\n\n // If we do have a tail...\n Some(mut tail) => {\n\n // the first thing we do is, check if the other list has\n // a head. if it does...\n if let Some(mut other_head) = other.head.take() {\n\n // we set our tail to point at their head, and we \n // set their head to point at our tail.\n unsafe {\n tail.as_mut().next = Some(other_head);\n other_head.as_mut().prev = Some(tail);\n }\n\n // We set our tail to their tail\n self.tail = other.tail.take();\n\n // finally, we add their length to our length, and \n // replace theirs with zero, because we took all\n // their nodes!\n self.len += mem::replace(&mut other.len, 0);\n\n } // there's no else case for that if let; if they don't\n // have a head, that list is empty and so there's\n // nothing to do.\n }\n }\n }\n</code></pre>\n<p>We only need <code>unsafe</code> when mutating the <code>next</code> of the <code>tail</code> of the first list, and the <code>prev</code> of the <code>head</code> of the second list. And that’s even only in the case where the <code>tail</code> exists in the first place. We can do things like “set our tail to their tail” in safe code, because we’re never de-referencing pointers. Safe Rust is helping us manage many, but not all, of our guarantees. We <em>could</em> move the <code>unsafe</code> block to encompass the entire body of this function, and Rust would still make us check those <code>Option<T></code>s. It would still check that <code>self</code> and <code>other</code> are not aliased by <code>&T</code> or <code>&mut T</code>s in the body of the function.</p>\n<p><code>unsafe</code> is a powerful tool, and should be used with care. But that doesn’t mean that you throw out many of the things that make Rust what it is.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/is-webassembly-the-return-of-java-applets-flash/",
"title": "Is WebAssembly the return of Java Applets & Flash?",
"description": null,
"url": "https://steveklabnik.com/writing/is-webassembly-the-return-of-java-applets-flash/",
"published": "2018-07-25T00:00:00.000Z",
"updated": "2018-07-25T00:00:00.000Z",
"content": "<p>In my <a href=\"http://words.steveklabnik.com/webassembly-is-more-than-just-the-web\">last post on WebAssembly</a>, I made the following claim:</p>\n<blockquote>\n<p>Some have compared WebAssembly to Java applets; in some ways, they’re very right, but in some ways, they’re very wrong. Eventually I’ll write a post about the wrong, but for now, the right: in some sense, WebAssembly is a different way of accomplishing what the JVM set out to do: it’s a common virtual machine that can be used to build very cross-platform software.</p>\n</blockquote>\n<p>A lot of people expressed interest in me elaborating, so let’s get to it! For this post, I’m going to make three comparisons: to Flash, to Java Applets, and <em>occasionally</em> to PNaCL. Secondly, this post is going to focus on the <em>web</em> use-case for WebAssembly, even though the previous post was about non-web uses. We’ll make <em>that</em> comparison in the next post. Finally, this post is kind of like eating tapas, there’s a bunch of little sections. I feel like it’s a bit short, but at the same time, I’m trying to get back into the habit of blogging, and if I try to keep expanding it, it’ll take forever, so it is what it is.</p>\n<hr />\n<p>For what it’s worth, I think this comparison is very <em>natural</em>; that is, when you hear this pitch for WebAssembly:</p>\n<blockquote>\n<p>WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable target for compilation of high-level languages like C/C++/Rust, enabling deployment on the web for client and server applications.</p>\n</blockquote>\n<p>It sounds quite a bit like previous technologies. Asking how they relate, or assuming that they’re a slightly different take on the problem makes a lot of sense. But WebAssembly is significantly different, for a few reasons.</p>\n<h2>Wasm won</h2>\n<p>The first reason that WebAssembly is different is that it’s succeeded, and those technologies did not. When I say this, I mean it in a very specific way; if you look at the number of Flash applications out there, and compare it to the number of WebAssembly apps out there, Wasm loses, for sure. Those of us that are excited about WebAssembly still have a lot of work to do before it’s widely used.</p>\n<p>When I say “WebAssembly won”, I mean that it’s become a part of the Web Platform. Flash, Java Applets, and PNaCL used the browser’s plugin functionality to work. They were, in a sense, <em>outside</em> of the web ecosystem. They were never standardized, and that matters a lot.</p>\n<p>In some sense, this is the final point, really. But just saying that it’s true doesn’t explain <em>why</em>. The rest of this post digs into <em>why</em> this occurred, and those points sort of sum up to this one: other technologies didn’t make it, but WebAssembly did. There’s a <em>lot</em> of overlap here, but I’m trying to reasonably separate it out into individual points.</p>\n<h2>Other technologies didn’t fit nicely into the platform</h2>\n<p>Remember this?</p>\n<p><img src=\"https://svbtleusercontent.com/gY9tj6D62jnC9Fc6RncFhD0xspap_small.gif\" alt=\"https://svbtleusercontent.com/gY9tj6D62jnC9Fc6RncFhD0xspap_small.gif\" /></p>\n<p>Or this?</p>\n<p><img src=\"https://svbtleusercontent.com/qaipxL2e5a4gLH5cAhRHDn0xspap_small.jpg\" alt=\"https://svbtleusercontent.com/qaipxL2e5a4gLH5cAhRHDn0xspap_small.jpg\" /></p>\n<p>What about this?</p>\n<p><img src=\"https://svbtleusercontent.com/9mGgBDzvwYYLGHFg4Pafzt0xspap_small.png\" alt=\"https://svbtleusercontent.com/9mGgBDzvwYYLGHFg4Pafzt0xspap_small.png\" /></p>\n<p>If you built an applet in one of these technologies, you didn’t really build a <em>web application</em>. You had a web page with a chunk cut out of it, and your applet worked within that frame. You lost all of the benefits of other web technologies; you lost HTML, you lost CSS, you lost the accessibility built into the web. These platforms never gained the ability to interact with the rest of the platform provided by your browser. Well, <a href=\"https://docs.oracle.com/javase/tutorial/deployment/applet/manipulatingDOMFromApplet.html\">technically they might have</a>, but that’s not how these technologies were used in practice.</p>\n<p>How could the web sustain an ecosystem that doesn’t integrate with it? It was never going to happen. And users ended up solidly rejecting it. Outside of games, users <em>hated</em> Flash. Java Applets were heavy and slow. Enshrining these technologies into the web platform wasn’t going to work.</p>\n<p>WebAssembly, on the other hand, is much closer to JavaScript. It doesn’t inherently require taking over a part of your screen. It doesn’t expect to be its own little closed off world. Via JavaScript today, and <a href=\"https://github.com/WebAssembly/host-bindings/blob/master/proposals/host-bindings/Overview.md\">on its own in the future</a>, it’s able to interact with the surrounding environment. It just… fits.</p>\n<h2>Other technologies were owned by companies</h2>\n<p>Java was owned by Sun Microsystems, Flash was owned by Adobe. But why does this matter?</p>\n<p>Corporate influence on the web is a complicated topic. But <em>in general</em>, the web runs on a psuedo-consensus model. Java and Flash, on the other hand, were controlled by their respective companies. These companies have significant motive to make profit, not to make the web better. And that’s partially what led to the above situation; these companies didn’t care that they didn’t integrate properly with the rest of the platform. Why would they? It’d be much better for business if you got locked into their platform, and abandoned the rest of the web entirely. The incentives are fundamentally misaligned.</p>\n<p>WebAssembly is a joint venture between Mozilla, Google, Apple, Microsoft, and others. It’s not trying to push anyone’s particular platform, but instead represents the shared interests of a wide number of stakeholders, both corporate and individual.</p>\n<h2>WebAssembly followed the web process</h2>\n<p>This <em>also</em> meant that these technologies never really followed the process that we use to standardize the web. That being said, the web standards process is healthier than its ever been, and at the time these technologies were big, things worked a bit differently. Regardless, WebAssembly followed the way that we do things on the web.</p>\n<p>First, <code>asm.js</code> was created, proving that we could do more impressive things with the web. Its performance was good enough to demonstrate what could be done, though it wasn’t fantastic. The secret sauce of <code>asm.js</code> was that it was just JavaScript; it was fully backwards compatible with the existing ecosystem. “Don’t break the web” is a very, very important rule for browser vendors.</p>\n<p>WebAssembly was sort of <code>asm.js</code> done better. After building some degree of consensus around <code>asm.js</code>, a variety of people came together to make WebAssembly a reality. Given that it wasn’t just JavaScript, it had to go through the usual process for browsers to implement it, and it did so. Now a W3C spec, it went <em>with</em> the grain of web standards, not against them.</p>\n<h3>Corollary: other technologies were too big and unstable</h3>\n<p>There’s another reason Wasm succeeded: it’s tiny. Wasm takes the approach of other web technologies: start small, and build on those foundations. Stay backwards compatible. These technologies also didn’t follow that particular social convention. For Flash and Java Applets, you’d be downloading the particular runtime you needed, and so there was no need for compatibility. PNaCl built on top of LLVM IR, which is completely unstable. They were going to make it a stable subset, but then when LLVM introduces changes, it would have ended up diverging, which isn’t a great situation to be in.</p>\n<p>These technologies were also far too large to ever make it through the stabilization process. Can you imagine the big four browser vendors specifying the JVM in full and then agreeing to those semantics for all time? For all of ActionScript, itself a similar-to-but-not-quite version of ECMAScript? For all of LLVM-IR?</p>\n<p>Large technologies introduce an unacceptable amount of risk in this situation. It’s an all-or-nothing kind of deal, and the safe bet is to go with “nothing.” WebAssembly, on the other hand, does almost nothing. It’s pretty much math and calling into JavaScript. This means it’s <em>much</em> easier to develop consensus around.</p>\n<h3>Collary: other technologies required an entire separate virtual machine</h3>\n<p>I haven’t mentioned Dart in this post, but it sort of fits here too. One big issue with “let’s just put the JVM in every browser” is that browsers <em>already</em> contain a language runtime: JavaScript’s. It’s hard enough to maintain one runtime, let alone two. And then how do you integrate them?</p>\n<p>WebAssembly, on the other hand, was designed to be a small extension to existing JavaScript virtual machines. While you <em>can</em> implement your own WebAssembly VM, and non-web use cases often do, for browsers, the maintenance cost is much, much lower.</p>\n<h2>Other technologies were too specific</h2>\n<p>WebAssembly is fundamentally language-agnostic. Flash and Java Applets were built first and foremost to run ActionScript and Java. They’re deeply tied to their relative semantics. Even PNaCl suffers from this to some degree; LLVM is really designed for C-like languages, though not quite the same amount.</p>\n<p>Do we really want to christen one language as the next language of the web? We already have JavaScript. Are we someday going to introduce a third language? A fourth? An agnostic approach is significantly better for longevity’s sake.</p>\n<h2>Wasm has a strong approach for safety</h2>\n<p>Java Applets and Flash were security <em>nightmares</em>. Even if there were attempts to secure them, in reality, they constantly had problems.</p>\n<p>WebAssembly, on the other hand, piggybacks on that JavaScript VM again; all of the effort going into creating its sandbox also applies to Wasm. Wasm is missing some features of regular assembly language that can cause security vulnerabilities, like stack smashing. Wasm is memory-safe, which is huge!</p>\n<p>Additionally, WebAssembly was designed for <em>validation</em> in mind; its fully typed, and can be checked without running any of its code. It spec includes instructions on how to do validation. This stuff is quite useful!</p>\n<h2>Conclusion</h2>\n<p>One thing I’ll say about this post is that it’s quite <em>implementer</em> focused. But implementers are important; they control the web. Providing a solution that works with their goals is just as important as something that works for users. In a future post, I’d like to try to go over user concerns some as well. I have lots of posts to write!</p>\n<p>At the end of the day:</p>\n<ul>\n<li>Other technologies were not integrated into the platform, and commercial interests didn’t point towards that happening</li>\n<li>Other technologies required too much; too much to agree on, too much to implement</li>\n<li>Wasm has a really strong sandboxing and verification story that others simply did not.</li>\n</ul>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/webassembly-is-more-than-just-the-web/",
"title": "WebAssembly is more than just the web",
"description": null,
"url": "https://steveklabnik.com/writing/webassembly-is-more-than-just-the-web/",
"published": "2018-07-13T00:00:00.000Z",
"updated": "2018-07-13T00:00:00.000Z",
"content": "<p>While WebAssembly was given its name for good reason, I also think it’s quite the misnomer. When you hear about WebAssembly, don’t just think of the web: think bigger.</p>\n<p>Many programmers know that WebAssembly is a technology that lets you run non-JavaScript code in the browser. That is true, but there’s a lot more to it. You see, the spec is written in a very interesting way: there’s the core, what WebAssembly actually is. Here’s <a href=\"https://webassembly.github.io/spec/\">the opening section</a>:</p>\n<blockquote>\n<p>To support the embedding of WebAssembly into different environments, its specification is split into layers that are specified in separate documents.</p>\n</blockquote>\n<blockquote>\n<p>Core specification - Defines the semantics of WebAssembly modules and its instruction set, independent from a concrete embedding.API specifications - Define application programming interfaces enabling the use of WebAssembly modules in concrete embedding environments.</p>\n</blockquote>\n<p>It goes on to say:</p>\n<blockquote>\n<p>Currently, two APIs are specified: JavaScript API … Web API.</p>\n</blockquote>\n<p>So, WebAssembly itself, the core, is completely independent of the web. The web is a particular environment that Wasm is embedded <em>in</em>, but not actually required to use wasm. Other embedding APIs can be created, with their own interfaces, to do various things.</p>\n<p>Let’s make this concrete: on the web, we can write a program that draws graphics using WebGL. But we could also write a desktop program that embeds a WebAssembly interpreter, and provides an OpenGL (or whatever else) API to render to the screen. And now our WebAssembly program is running on the desktop.</p>\n<p>Why would you want to do this? Well:</p>\n<blockquote>\n<p>I wonder if wasm, in being sufficiently minimal, well defined, practical and widely supported, will wind up as an archival format of choice for emulation-based software preservation. https://t.co/33OUOrqiWg\n— Graydon Hoare (@graydon_pub) July 11, 2018</p>\n</blockquote>\n<p>These emulators, with a teeny bit of work, could be made to run on the desktop as well. “Cross-platform” doesn’t just mean “Windows, Linux, and MacOS”, but can also mean running to the web as well.</p>\n<p>There’s so many more things that can be done, too. All it requires is a bit of re-framing of WebAssembly in your mind:</p>\n<p>WebAssembly is <em>not</em> a way to run code other than JavaScript on the web; WebAssembly is a tiny, embeddable language that you can use almost anywhere.</p>\n<p>Some have compared WebAssembly to Java applets; in some ways, they’re very right, but in some ways, they’re very wrong. Eventually I’ll write a post about the wrong, but for now, the right: in some sense, WebAssembly is a different way of accomplishing what the JVM set out to do: it’s a common virtual machine that can be used to build very cross-platform software.</p>\n<p>There’s a couple of other interesting applications of WebAssembly outside of the browser; one of my personal favorites is <a href=\"https://github.com/nebulet/nebulet\">Nebulet</a>:</p>\n<blockquote>\n<p>A microkernel that implements a WebAssembly “usermode” that runs in Ring 0.</p>\n</blockquote>\n<p>I’d describe it like this:</p>\n<blockquote>\n<p>Nebulet is an operating system that runs WebAssembly programs.</p>\n</blockquote>\n<p>None of this has anything to do with a web browser; just like ELF is the standard binary format on many Linuxes, WebAssembly is the standard binary format on Nebulet. It’s a new implementation of a quite old idea, but I’m really excited to see where it goes.</p>\n<p>A second interesting example is Parity. I’m a cryptocurrency skeptic, but from a tech perspective, this is quite cool. Parity is a company who is building a VM for the Ethereum cryptocurrency. Etherium’s VM has a programming language, Solidity, that lets you script their blockchain. Writing a programming language is a tough task on its own, and Solidity has had some issues, leading to pretty serious bugs. And those bugs are affecting people’s money.</p>\n<p>So, rather than keep their custom language, Parity is experimenting with <a href=\"https://wiki.parity.io/WebAssembly-Design\">embedding WebAssembly into the Ethereum VM instead</a>. This gets them a well-tested language that has pretty decent toolchain support, and lets them use a variety of other languages to do said scripting. This is very cool!</p>\n<p>These are a few of the possibilities for WebAssembly, and I’m really excited about where it could go in the future.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust-revisited/",
"title": "\"The Expressive C++17 Coding Challenge (in Rust)\" revisited",
"description": null,
"url": "https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust-revisited/",
"published": "2018-02-14T00:00:00.000Z",
"updated": "2018-02-14T00:00:00.000Z",
"content": "<p>In October of last year, I wrote a post, <a href=\"http://words.steveklabnik.com/the-expressive-c-17-coding-challenge-in-rust\">“The Expressive C++17 Coding Challenge (in Rust)”</a>. For various reasons, it got brought up again in the D world, and <a href=\"https://seb.wilzba.ch/b/2018/02/the-expressive-c17-coding-challenge-in-d/\">seb has written a new post</a>. It’s good, you should check it out!</p>\n<p>However, it links to my gist, not my blog post. As I said back then:</p>\n<blockquote>\n<p>I held myself to the same constraints as the original contest; no external packages is a bit painful in Rust, but it’s not too bad. Mostly it would let me eliminate boilerplate while also improving correctness, and making the code a bit shorter.</p>\n</blockquote>\n<p>So, that got me thinking: What <em>would</em> this look like if I could use external packages? I took about an hour, and knocked it out. I have two versions to show you today, one where I pay no attention to allocations, and one where it’s zero-allocation.</p>\n<p>First, the “whatever just allocate” version:</p>\n<pre><code>extern crate csv;\n#[macro_use]\nextern crate serde_derive;\nextern crate serde;\n#[macro_use]\nextern crate structopt;\n\nuse std::error::Error;\nuse std::fs::File;\nuse structopt::StructOpt;\n\n#[derive(Debug,Serialize,Deserialize)]\nstruct Record {\n name: String,\n surname: String,\n city: String,\n country: String,\n}\n\n#[derive(StructOpt, Debug)]\n#[structopt(name = \"basic\")]\nstruct Opt {\n filename: String,\n column_name: String,\n replacement: String,\n output_filename: String,\n}\n\nfn run(args: &Opt) -> Result<(), Box<Error>> {\n let input = File::open(&args.filename)?;\n let output = File::create(&args.output_filename)?;\n\n let mut rdr = csv::Reader::from_reader(input);\n let mut wtr = csv::Writer::from_writer(output);\n\n for result in rdr.deserialize() {\n let mut record: Record = result?;\n\n let replacement = args.replacement.clone();\n\n match &*args.column_name {\n \"name\" => record.name = replacement,\n \"surname\" => record.surname = replacement,\n \"city\" => record.city = replacement,\n \"country\" => record.country = replacement,\n _ => panic!(\"incorrect column name\"),\n }\n\n wtr.serialize(record)?;\n }\n\n wtr.flush()?;\n\n Ok(())\n}\n\nfn main() {\n let opt = Opt::from_args();\n\n if let Err(err) = run(&opt) {\n eprintln!(\"error: {}\", err);\n std::process::exit(1);\n }\n}\n</code></pre>\n<p>This uses the <code>[csv</code> crate](https://crates.io/crates/csv/1.0.0-beta.5) to do the heavy lifting with regards to the CSV stuff, and the <code>[strucopt</code> crate](https://crates.io/crates/structopt) for argument parsing. Not only is this code much shorter than my “no crates” version, but it’s also got more features!</p>\n<p>There’s a few things that aren’t the best here: first, the <code>match</code>. Rust doesn’t have reflection, exactly, so saying “hey I have this string and I want to index it in this struct” is not as nice. I’m just panic-ing here, but it’d be nicer to validate it up front, and possibly even do some custom derive shenanigans so that I don’t have to write it myself, but I wanted to present a fairly <em>realistic</em> version of what I’d write today. This is good enough.</p>\n<p>Second, there’s that one <code>clone</code>. We need it because we’ve said that our <code>Record</code> type holds <code>String</code>s, and so has ownership over the string. We end up copying <code>London</code> each iteration of the loop. Is copying a city name a problem? Not <em>really</em>, in the context of this example. But what if it <em>was</em>? Fundamentally, there’s no reason to do any allocation here: We’re reading from a file, and we’re writing the same string over and over again.</p>\n<p>Enter version 2:</p>\n<pre><code>extern crate csv;\n#[macro_use]\nextern crate serde_derive;\nextern crate serde;\n#[macro_use]\nextern crate structopt;\n\nuse std::error::Error;\nuse std::fs::File;\nuse structopt::StructOpt;\n\n#[derive(Debug,Serialize,Deserialize)]\nstruct Record<'a> {\n name: &'a str,\n surname: &'a str,\n city: &'a str,\n country: &'a str,\n}\n\n#[derive(StructOpt, Debug)]\n#[structopt(name = \"basic\")]\nstruct Opt {\n filename: String,\n column_name: String,\n replacement: String,\n output_filename: String,\n}\n\nfn run(args: &Opt) -> Result<(), Box<Error>> {\n let input = File::open(&args.filename)?;\n let output = File::create(&args.output_filename)?;\n\n let mut rdr = csv::Reader::from_reader(input);\n let mut wtr = csv::Writer::from_writer(output);\n\n let mut raw_record = csv::StringRecord::new();\n let headers = rdr.headers()?.clone();\n\n while rdr.read_record(&mut raw_record)? {\n let mut record: Record = raw_record.deserialize(Some(&headers))?;\n\n match &*args.column_name {\n \"name\" => record.name = &args.replacement,\n \"surname\" => record.surname = &args.replacement,\n \"city\" => record.city = &args.replacement,\n \"country\" => record.country = &args.replacement,\n _ => panic!(\"incorrect column name\"),\n }\n\n wtr.serialize(record)?;\n }\n\n wtr.flush()?;\n\n Ok(())\n}\n\nfn main() {\n let opt = Opt::from_args();\n\n if let Err(err) = run(&opt) {\n eprintln!(\"error: {}\", err);\n std::process::exit(1);\n }\n}\n</code></pre>\n<p>This is <em>slightly</em> more involved, as you no longer get access to the nice iterator, and have to do things a little more manually. But it took me about five minutes to port this bit over, so it’s not <em>too</em> onerous.</p>\n<p>So, as you can see, once you let people use external packages, the Rust gets <em>significantly</em> nicer. The code pretty much mirrors the task: open the two files, iterate over them, replace the column, write it out. Nice.</p>\n<p>I’m not going to do any performance comparisons here because that’s not really what the contest was about. I am mildly curious… if I really wanted to optimize performance, I’d also skip the string stuff and go for bytes directly. This isn’t a super tough transformation, but moves things to be slightly lower level.</p>\n<p>If you manipulate CSVs often, you should check out <code>[xsv](https://crates.io/crates/xsv)</code>, a command-line tool in Rust that lets you slice and dice these files with speed and ease.</p>\n<hr />\n<p>I should also mention that several people have been working on <code>quicli</code>, a package to make making CLIs super easy. One of its authors <a href=\"https://gist.github.com/killercup/049d759118e3b5029737eb77e157ea42\">posted a gist of what this looks like with quicli</a>, you should check that out too!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/2-sup-5/",
"title": "0b100000",
"description": null,
"url": "https://steveklabnik.com/writing/2-sup-5/",
"published": "2018-01-25T00:00:00.000Z",
"updated": "2018-01-25T00:00:00.000Z",
"content": "<p>As someone who has <a href=\"https://www.inc.com/jeff-bercovici/peter-thiel-young-blood.html\">significantly different beliefs than Peter Thiel</a>, today is likely my second-to-last power of two years alive.</p>\n<p>Two years ago, <a href=\"http://words.steveklabnik.com/a-eulogy-for-my-20s\">I wrote a post about turning 30</a>, and I pretty much feel the same way. Never enough time, tick tock, tick tock. I have today off (thanks Mozilla!), but am still considering working anyway, because I really care about what I’m working on, and am excited about it. We’ll see.</p>\n<p>I feel like I have more to say here, but also don’t. So maybe let’s leave it at that.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/looking-back-at-rust-in-2018/",
"title": "Looking back at Rust in 2018",
"description": null,
"url": "https://steveklabnik.com/writing/looking-back-at-rust-in-2018/",
"published": "2018-01-09T00:00:00.000Z",
"updated": "2018-01-09T00:00:00.000Z",
"content": "<blockquote>\n<p>A careful reader will note the date of this post; it’s fundamentally in response to this call for blog posts, with a rhetorical device of looking backwards as if my personal plans for Rust came true. If you’re reading this in 2019 or beyond, this may be a work of fiction.</p>\n</blockquote>\n<p>It’s been an exciting year for Rust. Well, it depends on what you mean by “exciting”, exactly. In a word, this year has been about <em>polish</em>. I find this kind of thing exciting! It’s a very different kind of exciting than the “new and shiny” exciting. It’s “take what we have and make it shiny.”</p>\n<p>Let’s recap the major achievements of the year.</p>\n<h2>Rust 2018</h2>\n<p>First of all, we shipped our first epoch! If you type <code>cargo new</code> today, you’ll see a new shiny entry in your <code>Cargo.toml</code>:</p>\n<pre><code>[package]\nname = \"new-program\"\nversion = \"0.1.0\"\nauthors = [\"steveklabnik <[email protected]>\"]\n\nepoch = \"2018\"\n\n[dependencies]\n</code></pre>\n<p>What does this mean in practical terms? Well, a few things. First, let’s get the big one out of the way: Try to compile this code with no epoch, or with <code>epoch = \"2015\"</code>:</p>\n<pre><code>fn main() { let catch = 5;}\n</code></pre>\n<p>You’ll see this:</p>\n<pre><code>> cargo build\n Compiling new-program v0.1.0 (file:///C:/Users/steve/tmp/new-program)\nwarning: `catch` will become a keyword in the 2018 epoch\n --> src\\main.rs:2:9\n |\n2 | let catch = 5;\n | ^^^^^\n |\n = note: #[warn(new_keyword)] on by default\n = note: to avoid this warning, consider using `_catch` instead\n</code></pre>\n<p>And if you change it to <code>epoch = \"2018\"</code>, you’ll see this:</p>\n<pre><code>> cargo build\n Compiling new-program v0.1.0 (file:///C:/Users/steve/tmp/new-program)\nerror: expected pattern, found keyword `catch`\n --> src\\main.rs:2:9\n |\n2 | let catch = 5;\n | ^^^^^\n\nerror: aborting due to previous error\n</code></pre>\n<p>Success! However, the specifics here aren’t as interesting as what this change means:</p>\n<p>First, <em>this is one of only two changes that relies on the epoch</em>. When we announced our plans for epochs, many people were skeptical that we would do as we said, and only included small, parser-level changes here. But we did, and will continue to do so. <code>catch</code> and <code>dyn Trait</code> are the only two changes here.</p>\n<p>Second, <em>we’ve demonstrated that this doesn’t split the ecosystem</em>. All of the crates that had no epoch are treated as 2015, and everything still works. Mixing 2015 and 2018 code works just fine.</p>\n<p>Finally, <em>mrustc has not had trouble implementing this split</em>. <code>[mrustc</code> is an alternative implementation of Rust](https://github.com/thepowersgang/mrustc), and some of the worries about epochs would be that they would unnecessarily burden alternative implementations. This fear has not borne fruit yet though.</p>\n<h2>We did ship <em>some</em> new stuff</h2>\n<p>Even though this year was about polish, that doesn’t mean that we had no new features. The key is, these features were all <em>designed</em> in 2017 or before. That is, features like non-lexical lifetimes, <code>impl Trait</code>, and Macros 2.0 didn’t land in stable until 2018, but they’ve been worked on for years. At the end of 2017, we had preliminary implementations already landed, it was just about polishing them up so we could ship.</p>\n<p>Cargo, however, did gain some new abilities. Cargo was sort of neglected in 2017, or at least, wasn’t worked on as much as the rest of the project was. Cargo grew the ability to use alternative registries, and at least one company is offering “crates.io as a service” and “crates.io behind your firewall”. Cargo also became easier to use offline, and the “integrate Cargo into other build systems” work bore some more fruit.</p>\n<h2>WebAssembly and Embedded platforms</h2>\n<p>At the end of 2017, Rust was poised to take advantage of WebAssembly. And take advantage, we did. WebAssembly has not taken over the world yet, but that was also never its intention. It is a common aspect of the Web Platform these days, and Rust’s support for it is so excellent that it’s the primary language with which you write new code for WebAssembly today. Tons of JavaScript programmers use Rust for their various systems-level tasks on the web, and we’re just starting to see completely new kinds of applications form. Adding Rust to your Node or front-end project is trivial, thanks to excellent integration from JavaScript’s build tools, or as we now call them, “compilers”.</p>\n<blockquote>\n<p>An aside: in the 2014-2017 era, even though JavaScript was JIT compiled, many programmers used various “build tools” or “transpilers” to do things like minify and bundle their JavaScript together. This was useful for things like “let me use new ES2017 features today but transpile to ES5.” In 2017, lots of JavaScript programmers, through tools like Rollup, started to appreciate that what they’re doing, on a fundamental level, is compiling. Once these tools realized this, they started taking advantage of decades of existing compiler research, and these tools started doing amazing things. Oddly enough, this makes JavaScript feel more like Java: you compile to a thing that ends up being JIT-ed. And that’s not even half of it; stuff like Ember’s GlimmerVM took this approach to new heights.</p>\n</blockquote>\n<p>Furthermore, it’s considered standard practice to test your crate against the WebAssembly target, just like you would Windows, Linux, and macOS, so the vast majority of the crates.io ecosystem Just Works.</p>\n<p>But this work on wasm has a good side effect: wasm is basically a very minimal embedded platform. By doing the legwork to make wasm awesome, we also made embedded platforms of various kinds awesome. There’s also specific work that had to be done, for example, remember <code>xargo</code>? Cargo just does that now. We’ve elevated ARM to a Tier 1 platform these days, and provide easy instructions for getting a new architecture going.</p>\n<p>This also implies stabilizing some of the lower-level things that are needed to do OSdev work. You can’t write a “real” OS in stable Rust yet, but you can write toy ones, and real OSes should be usable by next year.</p>\n<h2>Stabilization</h2>\n<p>One way in which we did polish was to ship a ton of stuff that was previously in flight. I mentioned non-lexical lifetimes earlier, as well as some other features, but there’s also a sort of “long tail” of stabilizations that we addressed.</p>\n<p>At the start of the year, the teams went through each of their feature gates, and voted: stabilize, put on hold, or remove. Things put on hold this year will get decided next year, but cannot be put on hold for another year. Of course, there were some small course-corrections over the year, but in general, it let us go “oh hey, slice patterns has been unstable since before Rust 1.0, let’s finally ship it.”</p>\n<h2>Bugs & new contributors</h2>\n<p>Rust’s bugtracker was in a much better state in 2017 than in 2016, but got even better in 2018. 2017 saw the infrastructure team actually ensure that all bugs get triaged, that is, they all have tags within two days of being opened. In 2018, however, there was a focus to knock out a lot of the long-standing, papercut style bugs. At the start of 2018, there were ~3100 bugs on the tracker, but today, there are only ~1500. That’s four bugs per day! We couldn’t have gotten there without all of the new contributors and team members we’ve added this year; in 2017, we had about 75 people on our various teams, and now we have 100. Your average Rust release in 2017 had about 120 contributors, it’s 180 for 2018.</p>\n<h2>Community and Job growth</h2>\n<p>It’s not just the teams that have grown, but also the community. For example, in 2016, crates.io had ~7500 crates. In 2017, ~13,000. In 2018, we’re up to 35,000. That’s a lot of packages! One major driver of this growth is jobs; once <strong>REDACTED</strong> announced that they’d been using Rust in a production setting, and that compared to the old system written in <strong>REDACTED</strong>, it saved them millions of dollars in server costs per year, a ton of companies started using Rust. It’s not <em>super</em> easy to get a Rust job, but they’re out there. We’ve also seen our meetups continue to grow, RustConf had two tracks this year, and there’s a new RustFest in South America, with rumors of a conference in China next year.</p>\n<h2>IDEs</h2>\n<p>While I don’t really use IDEs, we put a lot of work into making them awesome, and 2018 was the first time that people who <em>do</em> use IDEs love using one. Specifically, they love using the new IDE that JetBrains has released for Rust. In 2017, JetBrains was sponsoring the plugin for IntelliJ, but in 2018, they’ve released a whole product. Other IDEs are coming along swimmingly; Visual Studio now has really excellent support, for example.</p>\n<h2>Documentation</h2>\n<p>I work on docs, so I’m going to give this a section, even though maybe other things are more “exciting”.</p>\n<p>2018 saw the release of $REDACTED, a sort of “next-generation <code>rustdoc</code>.” It focused on not just API docs, but all of your docs holistically; you can publish guides as well, structure things how you’d like, etc. In some ways, it’s “<code>rustdoc</code> + <code>mdbook</code>” in a nice integrated package. This has led to even more people writing even better docs, since it’s so nice to use.</p>\n<p>There were two other major initiatives this year: focusing on intermediate documentation, and helping major ecosystem crates get excellent docs. In 2017, there was sort of a “what do I do after TRPL” situation, but now we’ve got some great resources to help you grow as a Rust programmer. Additionally, many of the more popular crates have docs that rival the standard library’s, and smaller crates have decent docs.</p>\n<p>Oh, and TRPL saw its initial paper release early in the year. Yay!</p>\n<h2>Epilogue</h2>\n<p>Whew, that’s a lot of stuff! What does 2019 hold for Rust? That post is for another time.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/five-years-with-rust/",
"title": "Five years with Rust",
"description": null,
"url": "https://steveklabnik.com/writing/five-years-with-rust/",
"published": "2017-12-21T00:00:00.000Z",
"updated": "2017-12-21T00:00:00.000Z",
"content": "<p>Today is five years since I first heard of Rust. <a href=\"http://words.steveklabnik.com/four-years-with-rust\">I wrote a post last year</a> about it. This year, I have something slightly different in mind.</p>\n<p>Five years is a long time in tech. It’s also a long time for me to be focused on one specific thing; other than the pizza job I held in high school and college, I’ve never stayed in a single job for five years. (This is true today as well; it’s not like I got a job with Rust the first day I learned of its existence.) I started with Ruby and Rails in 2008/2009, and my final commits to Rails were in August/September of 2013. So at this point, my career has basically been split in half, rather than dominated by Ruby.</p>\n<p>Five years ago tomorrow, I registered <a href=\"http://www.rustforrubyists.com/\">http://www.rustforrubyists.com/</a>.</p>\n<p>Yesterday, I had lunch with someone I’m very honored to call a friend. We talked about a lot of things, but also about Rust’s past, present, and future. One thing I’ve always felt a little awkward about is that there was a crucial point back in 2011/2012 where I wasn’t sure what to do with myself. I was very frustrated by the industry, and considered doing other things instead. But, as they and I talked about, sometimes, it’s not about your interests. If what you want to do is change the world, caring a lot isn’t enough; you have to analyze what you can actually <em>do</em>. While I think our industry is pretty broken, I at least have the ability to try and improve it, thanks to my stature. (God, even just typing that makes me feel gross.)</p>\n<p>I decided to put my time into Rust because I believed, and still believe, that it’s an important thing. And I saw the work that needed to be done, and saw that it’s the kind of work that I could do. And so, it’s maybe not something bigger, but I resolved myself to doing the work.</p>\n<p>Five years in, I think my journey has been successful so far. I’m proud of how far Rust has come. That said, there’s a lot more work to do. I’m still committed as ever to getting that work done, and feel proud to be able to work alongside people as amazing as the Rust community.</p>\n<p>Here’s to five more years. ?</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/user-agent-moz-a/",
"title": "User-Agent: moz://a",
"description": null,
"url": "https://steveklabnik.com/writing/user-agent-moz-a/",
"published": "2017-12-12T00:00:00.000Z",
"updated": "2017-12-12T00:00:00.000Z",
"content": "<p>Every six months, Mozilla gets all of its employees from around the world together in one place, for a week. This morning was the plenary session, with Mozilla leadership talking about our successes this year, as well as plans for the future. I noticed a theme that, while not super overt, I find very interesting, and bodes well for Mozilla’s future.</p>\n<p>The words “user agent” kept coming up in various aspects of the plenary. The <code>User-Agent</code> header has a… shall we say <a href=\"https://webaim.org/blog/user-agent-string-history/\">storied history</a>. When most people hear the term “user agent”, they mostly think of the “<a href=\"https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent/Firefox\">user agent string</a>”. This is passed through the <code>User-Agent</code> header to identify which browser you’re using.</p>\n<p>Well, that’s how most people think of it, but that’s not the whole story. If it is supposed to identify which browser you’re using, why is it not the <code>Browser</code> header? What is this “user agent” term anyway?</p>\n<p>The concept is a bit more broad: a “user agent” is a bit of software that acts on behalf of its user. A web browser is a “user agent” for the web. But an email program is also a “user agent” for email, for example. So “user agent” and “web browser” are not synonymous; a web browser is a specific form of user agent.</p>\n<p>Words have power, however. Framing matters. The way you describe a problem affects the way you think about the solutions. So the shift in language I picked up on was, Mozilla is not building a web browser, Mozilla is building a user agent. So what’s the difference here?</p>\n<p>A web browser is a dumb terminal. A user agent advocates and works for you, to better your experience.</p>\n<p>There’s a lot more to say about what all this framing implies: for security, for privacy, for customization, for openness, for advocacy, for social impact. But I have some meetings to go to, so I’ll just leave you with a question:</p>\n<p>What if your browser worked <em>for</em> you, rather than <em>against</em> you?</p>\n<p>In my mind, this “user agent” framing is a very useful and interesting way to think about Mozilla’s mission, and the goals we pursue in the future.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust/",
"title": "\"The Expressive C++17 Coding Challenge\" in Rust",
"description": null,
"url": "https://steveklabnik.com/writing/the-expressive-c-17-coding-challenge-in-rust/",
"published": "2017-10-25T00:00:00.000Z",
"updated": "2017-10-25T00:00:00.000Z",
"content": "<p>A few weeks ago, two C++ bloggers initiated a challenge (<a href=\"https://www.fluentcpp.com/2017/10/23/results-expressive-cpp17-coding-challenge/\">here</a> and <a href=\"http://www.bfilipek.com/2017/09/the-expressive-cpp17-challenge.html\">here</a>): use the shiny new features of C++17 to write a small program that manipulates CSV files. The link is to the results; you can see the examples people came up with.</p>\n<hr />\n<p>I’ve written <a href=\"http://words.steveklabnik.com/the-expressive-c-17-coding-challenge-in-rust-revisited\">a follow-up post</a> too!</p>\n<hr />\n<p>I decided to implement it in Rust for fun. You should give it a try, in whatever language you use, it’s a nice sample problem. I held myself to the same constraints as the original contest; no external packages is a bit painful in Rust, but it’s not too bad. Mostly it would let me eliminate boilerplate while also improving correctness, and making the code a bit shorter.</p>\n<p>Anyway, here it is:</p>\n<p>https://gist.github.com/steveklabnik/ad0a33acc82e21ca3f763e4278ad31a5</p>\n<p>I implemented <code>From</code> for everything so I could use <code>?</code>, but then <code>main</code> just has a ton of <code>unwrap</code>s. I normally would remove those too, but the challenge does not include good error handling, so I decided to just stick to the text of the challenge.</p>\n<p>I’m not going to say that this is the <em>best</em> Rust code in the world, nor compare it directly to the C++ examples. But it was a fun hour or two to knock it out, and figured I’d at least share my results.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/want-to-join-the-rust-docs-team/",
"title": "Want to join the Rust docs team?",
"description": null,
"url": "https://steveklabnik.com/writing/want-to-join-the-rust-docs-team/",
"published": "2017-03-24T00:00:00.000Z",
"updated": "2017-03-24T00:00:00.000Z",
"content": "<p>One of Rust’s major goals this year is to <a href=\"https://github.com/rust-lang/rust-roadmap/issues/13\">provide mentoring at all levels of the project</a>. This is a goal I’m very on-board with. There’s lots of ways to get involved with Rust, but I’d like to talk about the bit I’m most involved in: documentation.</p>\n<p>There’s many ways to contribute to Rust’s docs. Some people contribute occasional PRs, some people end up contributing a lot. No matter what your interest, I’d like to help. But before I get into those details, <em>why</em> should someone get involved with documentation? I have my own reasons, but I also have a pitch for you:</p>\n<blockquote>\n<p>Working on documentation is a fantastic way to learn something in a deeper way.</p>\n</blockquote>\n<p>In order to document something, you have to understand it, even if only a little bit! It’s a good excuse to dig into the details. And I’ve found that many people are eager to help explain something to you in return for you writing that down and synthesizing it into documentation.</p>\n<p>If that sounds intriguing to you, let’s talk! There’s a number of different ways to do this, and I’m interested generally in helping anyone work on Rust’s documentation, in any form. However, if you don’t have something in mind, I do have a suggestion.</p>\n<p>Back in October of 2015, I made a <a href=\"https://github.com/rust-lang/rust/issues/29329\">big GitHub issue</a> to track giving the whole standard library documentation a once-over. I then made sub-issues for each main module. Slowly, myself and others have been knocking them out, but there’s still a number of issues open.</p>\n<p>Today, I went through all of the remaining ones, and made fine-grained bullet points of what needs to be done. I’ll admit that some of them may not be as explicit as they could be, but it at least gives a starting point to talk about. Here’s all the sections that are currently open:</p>\n<ul>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29349\">convert</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29354\">ffi</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29381\">macros</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29378\">thread</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29377\">sync</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29375\">str</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29371\">ptr</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29370\">process</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29367\">os</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29365\">ops</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29363\">net</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29357\">hash</a></li>\n<li><a href=\"https://github.com/rust-lang/rust/issues/29355\">fmt</a></li>\n</ul>\n<p>Some of the work that needs to be done is very small: add a link here, re-word a sentence there. Other bits are large; “this whole thing needs re-written.”</p>\n<p>So here’s my pitch to you: have you ever wanted to learn a bit more about Rust? Do any of those names sound interesting to you? Maybe you’ve wanted to learn a bit more about threads in Rust. Check out that ‘thread’ issue above, poke around, learn some Rust, and then let’s make the docs better! We can work together on the issues, through email, or on IRC. Whatever works. Maybe you’ll find it as addicting as I do. And I find that people who are non-experts write way better docs than experts. So don’t worry about that!</p>\n<p>Worst case, you’ll have learned a bit of the standard library a bit better, and that’s always a good thing, too.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/an-overview-of-macros-in-rust/",
"title": "An overview of macros in Rust",
"description": null,
"url": "https://steveklabnik.com/writing/an-overview-of-macros-in-rust/",
"published": "2017-01-24T00:00:00.000Z",
"updated": "2017-01-24T00:00:00.000Z",
"content": "<p>Rust has had a lot of stuff going on with “macros”. There are a few, interconnected aspects of these features, many different names for the same thing, and a lot of development work going on. This post will give an overview of the past, present, and future of macros in Rust.</p>\n<p>Ultimately, macros allow you to write things like this:</p>\n<pre><code>let map = hashmap!{\n \"a\" => 1,\n \"b\" => 2,\n};\n</code></pre>\n<p>or</p>\n<pre><code>#[derive(Serialize, Deserialize)]\nstruct S {\n #[serde(rename = \"a\")]\n f: i32,\n}\n</code></pre>\n<p>That is, some macros look like functions: <code>foo!</code>, but allow you to write nearly arbitrary syntax inside of the <code>()</code>s. Some let you control the ability to write custom attributes. Both generate some other kind of valid Rust code for the compiler to interpret. In the first case, the <code>hashmap!</code> macro will expand to a call to <code>HashMap::new</code> and a bunch of <code>insert</code> calls, and the second case, the <code>Serialize</code> and <code>Deserialize</code> traits will be automatically implemented for this struct, along with a bit of custom logic where the field <code>f</code> maps to the name <code>a</code> in the data that’s being {de,}serialized.</p>\n<p>These features are extremely powerful, but also must be done right: metaprogramming is very useful, but can also make code inscrutable, or give poor error messages.</p>\n<p>Oh, and I’d like to give a shout-out to <a href=\"https://danielkeep.github.io/tlborm/book/index.html\">The Little Book of Rust Macros</a>, which is basically <em>the</em> resource for doing this kind of work. If you are interested in writing macros, you should check it out.</p>\n<h2>Pre-1.0</h2>\n<p>As Rust developed, two major forms of macros evolved. One was called “macros” and the other was called “compiler plugins”. Both of these systems were pretty good, but both of them also had a lot of problems. We’ll get into the details of each of these kinds of macros below, but there’s one thing we should talk about first. On the way to 1.0, it was decided to stabilize one kind of macros, but not the other. From <a href=\"https://blog.rust-lang.org/2014/10/30/Stability.html\">Stability as a Deliverable</a>: (also, this uses some terms that we will cover shortly)</p>\n<blockquote>\n<p>After extensive discussion, we plan to release globs and macros as stable at 1.0. For globs, we believe we can address problems in a backwards-compatible way. For macros, we will likely provide an alternative way to define macros (with better hygiene) at some later date, and will incrementally improve the “macro rules” feature until then. The 1.0 release will stabilize all current macro support, including import/export.On the other hand, we cannot stabilize syntax extensions, which are plugins with complete access to compiler internals. Stabilizing it would effectively forever freeze the internals of the compiler; we need to design a more deliberate interface between extensions and the compiler. So syntax extensions will remain behind a feature gate for 1.0.Many major uses of syntax extensions could be replaced with traditional code generation, and the Cargo tool will soon be growing specific support for this use case. We plan to work with library authors to help them migrate away from syntax extensions prior to 1.0. Because many syntax extensions don’t fit this model, we also see stabilizing syntax extensions as an immediate priority after the 1.0 release.</p>\n</blockquote>\n<p>We’ll discuss in each section how this ended up working out, but it’s important to remember that only <em>one</em> form of these macros was available on stable Rust for 1.0, and in fact, as of the time of this writing. That’s going to change soon…</p>\n<p>It’s also important to note that we weren’t entirely happy with macros. They were fine, but they have some flaws. We wanted to fix them, but we also didn’t want to ship 1.0 without macros. Language design is hard. I’ll talk about this more in the appropriate section.</p>\n<h2>Declarative Macros</h2>\n<p>The first form of Macros in Rust, and the one that’s most widely used, are called “declarative macros”, sometimes “macros by example”, sometimes “macro_rules macros”, or sometimes just “macros”. At their core, declarative macros allow you to write something similar to a Rust <code>match</code> statement:</p>\n<pre><code>match x {\n 4 => println!(\"four!\"),\n 5 => println!(\"five!\"),\n _ => println!(\"something else\"),\n}\n</code></pre>\n<p>With <code>match</code>, <code>x</code>’s structure and value will be evaluated, and the right arm will execute based on what matches. So if <code>x</code> is five, the second arm happens. Etc.</p>\n<p>These kinds of macros work in the same way: you set up some sort of pattern, and then, if that pattern matches, some code is generated. One important difference here is that in <code>match</code>, <code>x</code> gets evaluated. With macros, <code>x</code> does not get evaluated.</p>\n<p>There are two kinds of declarative macros: ones defined with the <code>macro_rules!</code> keyword, and ones defined with the <code>macro</code> keyword.</p>\n<h3><code>macro_rules!</code></h3>\n<p>These are most commonly known as “macros” in Rust world, and are the only thing that’s stable in Rust at the time of this writing. Here’s a macro you’ll see when using Rust:</p>\n<pre><code>let x: Vec<u32> = vec![1, 2, 3];\n</code></pre>\n<p>This macro creates a new vector, with three elements inside. Here’s what the macro could look like, written with <code>macro_rules!</code>:</p>\n<pre><code>macro_rules! vec {\n ( $( $x:expr ),* ) => {\n {\n let mut temp_vec = Vec::new();\n $(\n temp_vec.push($x);\n )*\n temp_vec\n }\n };\n}\n</code></pre>\n<p>Whew! That’s a bunch of stuff. The most important line is here:</p>\n<pre><code> ( $( $x:expr ),* ) => {\n</code></pre>\n<p>This <code>pattern => block</code> looks similar to the <code>match</code> statement above. If this pattern matches, then the block of code will be emitted. Given that this is the only pattern in this macro, there’s only one valid way to match; any other will be an error. More complex macros will have more than one rule.</p>\n<p>The <code>$x:expr</code> part matches an expression, and gives it the name <code>$x</code>. The <code>$(),*</code> part matches zero or more of these expressions. Then, in the body of the macro, the <code>$()*</code> part is generated for each part that matches, and the <code>$x</code> within is replaced with each expression that was matched.</p>\n<p>These macros are fine, but there’s a number of bugs and rough edges. For example, there’s no namespacing: if a macro exists, it’s everywhere. In order to prevent name clashes, this means that you have to explicitly import the macros when using a crate:</p>\n<pre><code>#[macro_use]\nextern crate serde;\n</code></pre>\n<p>Otherwise, you couldn’t import two crates that had the same macro name. In practice this conflict doesn’t come up much, but as we grow, it becomes more likely. The hygiene of <code>macro_rules</code> is there, but not perfect. (Only local variables and labels are hygienic…) Et cetera. The plan is that with the new macro systems, you’ll be able to use the regular module system, and use macros just like you use functions today.</p>\n<p>Even with these problems, macros were useful enough to include in Rust 1.0. But plans for a better macro system were already being worked on, and so, they were given the name <code>macro_rules!</code> so that we could eventually deprecate all of this. This is one of the few features that we decided to do this with Rust 1.0; a vestigial thing that was always planned for obsolescence. Doing it hurts, but not having any metaprogramming at all would hurt much more.</p>\n<h3><code>macro</code></h3>\n<p>As such, the <code>macro</code> keyword was reserved for a new macro system. We’re still not 100% sure if we’re calling these “macros by example” (which you’ll sometimes see with a “2.0” at the end) or “declarative macros”. I prefer the latter, because it pairs nicely with “procedural macros.” It seems like most agree with me. We’ll see.</p>\n<p>These macros are an incremental improvement over the existing <code>macro_rules!</code> macros: they’re fundamentally the same thing, but with some nicer features.</p>\n<p>At the time of this writing, <a href=\"https://github.com/rust-lang/rfcs/pull/1584\">an initial RFC is in FCP</a>, laying out some high-level aspects of this feature. An initial implementation is underway, but it hasn’t landed yet, so you can’t use <code>macro</code> in real code yet.</p>\n<h2>Procedural Macros</h2>\n<p>In opposition to the pattern-based declarative macros, the second form are called “procedural macros” because they’re functions: they accept some Rust code as an input, and produce some Rust code as an output. I say “code” but I don’t mean that literally; we’ll get into the exact details in each section.</p>\n<h3>Syntax Extensions</h3>\n<p>Syntax extensions, also sometimes just called “procedural macros” or “compiler plugins”, were not stabilized for Rust 1.0. Here’s (part of) a syntax extension that implements roman numeral literals:</p>\n<pre><code>#![feature(plugin_registrar, rustc_private)]\n\nextern crate syntax;\nextern crate rustc;\nextern crate rustc_plugin;\n\nuse syntax::parse::token;\nuse syntax::tokenstream::TokenTree;\nuse syntax::ext::base::ExtCtxt;\nuse rustc_plugin::Registry;\n\nfn expand_rn(cx: &mut ExtCtxt, sp: Span, args: &[TokenTree])\n -> Box<MacResult + 'static> {\n // implementation goes here\n}\n\n#[plugin_registrar]\npub fn plugin_registrar(reg: &mut Registry) {\n reg.register_macro(\"rn\", expand_rn);\n}\n</code></pre>\n<p>You’d use it like this:</p>\n<pre><code>#![feature(plugin)]\n#![plugin(roman_numerals)]\n\nfn main() {\n assert_eq!(rn!(MMXV), 2015);\n}\n</code></pre>\n<p>That is… a lot of code. And frankly, probably doesn’t even work on today’s Rust right now. Why not? Well, that’s the big problem with syntax extensions:</p>\n<pre><code>extern crate syntax;\nextern crate rustc;\nextern crate rustc_plugin;\n</code></pre>\n<p>This literally loads up bits of the Rust compiler and uses it as a library. Then, it uses Rust’s internal AST to do the processing. This is extremely powerful, but also extremely unstable. We don’t and can’t freeze Rust’s internal AST, or any of its other internal APIs, at least not at this time. Not to mention that this means any alternative Rust compiler would need to either use our AST or map ours to theirs, which puts a damper on alternative Rust implementations. This means that syntax extensions haven’t had a good path towards stability. It’s also why they couldn’t be made stable as-of Rust 1.0. But “run arbitrary Rust code to produce more Rust code” is a useful feature, and so, some projects needed them. In addition, while declarative macros can produce function-style <code>foo!</code> calls, they cannot do the custom attribute stuff. And those are very useful. We <em>could</em> make this work, but there hasn’t been a real reason to, nor much demand.</p>\n<p>A side note: there’s also the term “compiler plugin.” Syntax extensions are a particular kind of compiler plugin; there are others, like lint passes, LLVM pass plugins, MIR plugins… we’re only talking about syntax extensions in this post, however.</p>\n<p>The current plan is to never stabilize syntax extensions, and instead, stabilize “procedural macros.” Again, this isn’t for all compiler plugins, only syntax extensions.</p>\n<h3>Procedural Macros</h3>\n<p>Accepted in <a href=\"https://github.com/rust-lang/rfcs/blob/master/text/1566-proc-macros.md\">RFC 1566</a>, procedural macros will give you the same power as compiler plugins, without needing to reach into the guts of <code>rustc</code>. How? Well, they look like this:</p>\n<pre><code>#[proc_macro]\npub fn foo(TokenStream) -> TokenStream {\n // do stuff to implement the foo macro\n}\n</code></pre>\n<p>This creates a <code>foo!</code> procedural macro. You’ll note that it accepts a <code>TokenStream</code> and produces a <code>TokenStream</code>. To quote the RFC:</p>\n<blockquote>\n<p>The first argument is the tokens between the delimiters in the macro use. For example in foo!(a, b, c), the first argument would be [Ident(a), Comma, Ident(b), Comma, Ident(c)].The value returned replaces the macro use.</p>\n</blockquote>\n<p>Why tokens? Well, while an AST can change, the actual tokens will not. This approach will allow us the flexibility to change <code>rustc</code>’s internal AST, while giving us the full power of compiler plugins.</p>\n<p>So, how do you work with one of these kinds of macros? You can of course work with the token stream directly, but we also expect to see libraries that will provide easier-to-use interfaces on top of this. You could imagine a package that converts a <code>TokenStream</code> to its own AST, and then provides an interface to that AST. This is nicer to use than working with tokens directly. There are already some libraries that do similar kinds of things, like <a href=\"https://github.com/dtolnay/syn\">syn</a>. More on that in a moment.</p>\n<h3>Function-like vs attribute like</h3>\n<p>The above is for a macro that gives you a <code>foo!</code> macro, that is, one that looks like a function. What about one that gives you an attribute? That’d look like this:</p>\n<pre><code>#[proc_macro_attribute]\npub fn foo(Option<TokenStream>, TokenStream) -> TokenStream {\n // implementation goes here\n}\n</code></pre>\n<p>To quote the RFC:</p>\n<blockquote>\n<p>The first argument is a list of the tokens between the delimiters in the macro use. Examples:#[foo] => None#[foo()] => Some([])#[foo(a, b, c)] => Some([Ident(a), Comma, Ident(b), Comma, Ident(c)])The second argument is the tokens for the AST node the attribute is placed on.</p>\n</blockquote>\n<p>Internally, it’s the same thing: use a <code>TokenStream</code>, produce a <code>TokenStream</code>.</p>\n<h3>“Macros 1.1”</h3>\n<p>You may have also heard a lot of commotion about “Macros 1.1.” In fact, they will be stable as of the next release of Rust, 1.15. Macros 1.1 are the <a href=\"https://en.wikipedia.org/wiki/Pareto_principle\">Pareto principle</a> applied to procedural macros: as it turns out, the vast majority of people who use procedural macros today use it for “custom derive”, that is, the ability to say <code>#[derive(MyTrait)]</code> and implement a trait automatically. As such, a plan was hatched: we could stabilize <em>just the very very very basics</em> of procedural macros, enough to get the ability to write custom derives, and stabilize that, without needing to implement everything. What does that look like? Here’s one that implements <code>#[derive(Double)]</code></p>\n<pre><code>extern crate proc_macro;\n\nuse proc_macro::TokenStream;\n\n#[proc_macro_derive(Double)]\npub fn double(input: TokenStream) -> TokenStream {\n let source = input.to_string();\n\n // Parse `source` for struct/enum declaration, and then build up some new\n // source code representing a number of items in the implementation of\n // the `Double` trait for the struct/enum in question.\n let source = derive_double(&source);\n\n // Parse this back to a token stream and return it\n source.parse().unwrap()\n}\n</code></pre>\n<p>You’ll notice that this still looks like <code>TokenStream</code> -> <code>TokenStream</code>, but there’s something more subtle here: <code>input.to_string()</code>. That is, for Macros 1.1, <code>TokenStream</code> has two methods; all it has is the ability to be converted to a string, with the original Rust code inside of it, and a <code>parse</code> method, which can take a string of Rust code and produce another <code>TokenStream</code>. What you do with that <code>String</code> is your own business.</p>\n<p>What most people will do is use <a href=\"https://github.com/dtolnay/syn\">syn</a> to parse the Rust code, and <a href=\"https://github.com/dtolnay/quote\">quote</a> to generate Rust code from that. These libraries will make it feasible to build real-world custom derive code, without us needing to stabilize anything we don’t want to. Of course, once full procedural macros are implemented, it’s not likely that people will want to turn <code>TokenStream</code>s into and parse <code>TokenStream</code>s from strings, but it’s not <em>totally</em> unheard of, and will be the tiniest wart. It’s worth it to move a significant chunk of the ecosystem onto the stable compiler.</p>\n<p>For more details, see <a href=\"https://github.com/rust-lang/rfcs/blob/master/text/1681-macros-1.1.md\">RFC 1681</a>.</p>\n<h3>“Macros 1.2” ?</h3>\n<p>There’s also been some small talk about possibly doing a “macros 1.2”, that is, another step towards full procedural macros without needing the whole thing. This would be “procedural macros without hygiene”, though it’s still early days for this idea; there isn’t even an RFC yet. We’ll see if this happens. There’s still some controversy around this idea.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/announcing-rust-contributors/",
"title": "Announcing Rust Contributors",
"description": null,
"url": "https://steveklabnik.com/writing/announcing-rust-contributors/",
"published": "2017-01-12T00:00:00.000Z",
"updated": "2017-01-12T00:00:00.000Z",
"content": "<p>When I was working with Ruby on Rails, someone once sent me this link:</p>\n<p>http://contributors.rubyonrails.org/</p>\n<p>In some ways, this link changed my life. You see, I respond <em>really</em> strongly to integers going upwards. I am a sucker for points, hits, whatever. That web page, however, puts that integer towards something useful: open source contributions. It’s fairly well known that open source contribution is a long tail, but that’s also <em>perfect</em> for this kind of gamification. One contribution, as of today, will put you in 2461th place. Your second? 1473! That’s almost a thousand places! Just <em>one</em> more contribution puts you at number 1114. Still three hundred spots. Your forth commit will let you get in the top 900, #893. You get the idea. (I ended up at #47.)</p>\n<p>Today, I’m working on Rust. And one of the things that I do is write the release announcements, or at least, their first draft. And one of my favorite things to do is the last bit: thanking all of the people who have put in work toward this release. It’s one of my favorite things because I really do appreciate all of the patches, but also because I remember how hooked I was when people gave me a little bit of credit for my contribution. Thanking people is powerful.</p>\n<p>So to write those blog posts, I have to run some <code>git</code> commands. And I always forget <em>exactly</em> what I have to write, given our branching strategy. So I wrote a little <code>Rakefile</code> almost a year ago that will generate a new post with the appropriate thank yous and statistics. But I never made it general enough; I have to tweak the numbers every release. It’s annoying. I’ve always wanted to fix it up. There’s one more way in which I don’t like this setup: we only thank people who contribute to <code>rust-lang/rust</code>. Even those with a commit to Cargo don’t make it in there. It’s a historical artifact of the release process, but I don’t like it. We should be thanking those people too.</p>\n<p>While all of this has been rolling around in the back of my brain, I’ve been thinking about contributors.rubyonrails.org. I’ve said for a while I wanted it in Rust. So, over the last few days, I hacked one together.</p>\n<p>https://rust-contributors.herokuapp.com/</p>\n<p>Right now, it only replicates what we say in the blog posts. It doesn’t show commit counts, or order by it, or anything. But the foundation is there. Also, since this is a fun hack, I didn’t take too much care when writing it: there’s a number of low-hanging fruit kinds of things to do. I didn’t use a fancy web framework. I didn’t focus on refactoring relentlessly. I didn’t write any tests. The error handling is basically non-existent. I just knocked it out as fast as I can. But it’s a web app, written in Rust, and that’s fun.</p>\n<p>If you’d like to help me hack on this, head over to https://github.com/steveklabnik/contributors and take a look at the issues. I’ve already identified a number of things, and if you do web work but have been meaning to mess with Rust, this might be a good chance to give it a shot.</p>\n<p>Thank you to everyone who submits patches to Rust; you’re all great, and I’m lucky to be able to work with you all.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/fire-mario-not-fire-flowers/",
"title": "Fire Mario not fire flowers",
"description": null,
"url": "https://steveklabnik.com/writing/fire-mario-not-fire-flowers/",
"published": "2016-12-29T00:00:00.000Z",
"updated": "2016-12-29T00:00:00.000Z",
"content": "<p><a href=\"http://words.steveklabnik.com/rust-is-more-than-safety\">My post yesterday</a> got a lot of great responses. Two of them are from people I greatly respect, <a href=\"https://thefeedbackloop.xyz/safety-is-rusts-fireflower/\">Dave Herman</a> and <a href=\"http://graydon2.dreamwidth.org/247406.html\">Graydon Hoare</a>. You should go read those two posts; this one is in response to them. I’m going to respond to both at the same time by mixing them together.</p>\n<p>Graydon says:</p>\n<blockquote>\n<p>Safety in the systems space is Rust’s raison d’être. Especially safe concurrency (or as Aaron put it, fearless concurrency). I do not know how else to put it.</p>\n</blockquote>\n<p>“Fearless concurrency” is a different way to put it! That is, to use Dave’s metaphor, “safety” is the fire flower. But “fearless concurrency” is Fire Mario. This is where I think Dave misses the mark slightly as well. I agree with his metaphor, but disagree with his conclusion.</p>\n<p>I’m not saying that we get rid of fire flowers. I’m saying that we focus on “Rust makes you Fire Mario,” not “Rust is made of fire flowers”, when talking to people about Rust.</p>\n<p>Dave also misses my mark slightly, but in a different way:</p>\n<blockquote>\n<p>I don’t think the Rust community should worry about shying away from talking about safety, but I hope we all remember that it’s in service of something much more exciting: making more badass systems programmers!</p>\n</blockquote>\n<p>We are so close here that it feels like nitpicking, but I think the difference is important. I think one of Rust’s greatest potentials, like Dave does, is bringing a ton of new people into systems programming. But my point is this: “memory safety without garbage collection” doesn’t speak to that <em>at all</em>. That is, memory safety without garbage collection is the mechanism of how that’s accomplished, but it’s not the accomplishment itself. It’s the Fire Flower, not Fire Mario.</p>\n<p>I’d like to make sure we’re talking about being Fire Mario, rather than being Fire Flower enthusiasts.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rust-is-more-than-safety/",
"title": "Rust is more than safety",
"description": null,
"url": "https://steveklabnik.com/writing/rust-is-more-than-safety/",
"published": "2016-12-28T00:00:00.000Z",
"updated": "2016-12-28T00:00:00.000Z",
"content": "<p>Rust is most well-known for its features around writing low-level code in a safer way than its predecessors. But it’s also a lot more than that. The Rust community is slowly evolving the way that we talk about Rust to people who haven’t yet given Rust a try, but it can take time. But how did we get here?</p>\n<p>In Rust’s initial release announcement, Graydon used a phrase that is amazing:</p>\n<blockquote>\n<p>[Rust is] Technology from the past come to save the future from itself</p>\n</blockquote>\n<p>He goes on to elaborate a bit:</p>\n<blockquote>\n<p>Many older languages [are] better than new ones. We keep forgetting already-learned lessons.</p>\n</blockquote>\n<p>This is one of the many things that the “Rust” name evokes. Rust was not intended to be a cutting-edge PLT (programming language theory) research language. It was challenging an assumption that I think many of us hold dear: that “progress” means that things always get better. Reality is a bit more complex than that. Yes, we’ve improved on some things, but there are some older things that were actually “better” in some sense, but never quite made it for other reasons. I’ve always phrased this sentiment like this:</p>\n<hr />\n<p>Let’s look at the initial release dates of some of the top programming languages:</p>\n<ul>\n<li>C: 1972</li>\n<li>C++: 1983</li>\n<li>C#: 2000</li>\n<li>Java: 1995</li>\n<li>JavaScript: 1995</li>\n<li>Python: 1991</li>\n<li>PHP: 1995</li>\n<li>Ruby: 1995</li>\n</ul>\n<p>The first thing you’ll notice is: what the hell were they putting in the water in 1995?</p>\n<p>The second is that, other than C# in 2000, 1995 or earlier is where most of our programming languages come from. Have we really not made any advancements significant enough in the last 20 years to have gotten people’s attention?</p>\n<hr />\n<p>Now, there are a multitude of newer languages, and people do use them. And these languages have had more releases since then. But none of these languages has fundamentally changed. They’re all pretty much the same, but better. And specifically, you’ll notice that many of these languages are scripting or higher-level languages: if we restrict ourselves to “what most operating systems are programmed in”, we haven’t had a new generation of programming languages in thirty or forty years!</p>\n<p>PLT research and industry live in two different worlds. Industry is effectively using ancient PLT tech. There’s lots of <em>good</em> reasons for this too, mind you. But if you’re looking to make a new hit programming language, you don’t actually have to be innovative in the PLT sense. You can take PLT tech from the 2005-2010 era and you’ll still be ten years ahead!</p>\n<p>So in some senses, this is what Rust is trying to do: not be a cutting-edge programming language in a PLT sense, but advance the industry by mining “older” research.</p>\n<p>However, plans don’t always work out the way you want to. <a href=\"https://www.youtube.com/watch?v=79PSagCD_AY\">As Rust evolved</a>, many of its original bits were thrown out, and replaced with new ones. Rust <em>has</em> advanced a small bit of the universe in PLT with the ownership/borrowing system. Like most research, it’s built on top of older work. But that’s exciting!</p>\n<p>Another thing about the borrow checker is that it’s “unique” about Rust as well. So if someone asks you, “hey, what’s cool about Rust?”, it’s extremely tempting to answer with the thing that makes it most clearly different than other programming languages. And for some people, this pitch is compelling! But not for everyone…</p>\n<h2>Shifting priorities</h2>\n<p>Marketing is hard. But there’s one thing that I’ve learned about it that I think applies to Rust:</p>\n<blockquote>\n<p>Your marketing should be focused on what problems your product solves for the consumer.</p>\n</blockquote>\n<p>Rust’s marketing, historically, hasn’t been as good at this as it could have been. That is, we’ve been too focused on the details, and not enough on the big picture. We’ve been worried about what’s interesting in a PLT aspect, and not what problems Rust solves for users. I can think of two big examples where this has happened:</p>\n<ul>\n<li>Rust is safer than C or C++. (“I never have problems with safety in the C++ code I write” is a common reply.)</li>\n<li>Rust is as low-level of a programming language as C! (“I never write C. Rust is not for me.”)</li>\n</ul>\n<p>Both of these pitches for Rust focus on the <em>how</em>, and not on the problem itself. And that means that sometimes people draw the opposite inference as to what you were intending with the pitch in the first place.</p>\n<p>I have some ideas on a better pitch for Rust, and I even had a draft of it here, but I’ve decided to delete it for now. Instead, I’d like to ask all of you who love Rust to give this some thought, and write your own blog posts about what problems Rust solves for you. I’ll edit this post and link to yours if you ask me to. I’ve seen several people do a good job at articulating this kind of thing before. And I’ll work on my own post as well, as a follow-up.</p>\n<p>Let’s figure out the best way to share our love of Rust with the rest of the programming world.</p>\n<hr />\n<p>Here’s some of the responses generated from this:</p>\n<p>I’ve written <a href=\"http://words.steveklabnik.com/fire-mario-not-fire-flowers\">Fire Mario, not Fire Flowers</a>, as a response to two responses to this post.</p>\n<p>The <a href=\"https://www.reddit.com/r/rust/comments/5krhr0/rust_is_more_than_safety/\">/r/rust</a>, <a href=\"https://www.reddit.com/r/programming/comments/5krw7r/rust_is_more_than_safety/\">proggit</a>, and <a href=\"https://news.ycombinator.com/item?id=13272474\">orange website</a> threads are full of good replies.</p>\n<ul>\n<li><a href=\"https://medium.com/@Hisako1337/rust-in-2017-8f2b57a67d9b\">Rust in 2017</a></li>\n<li><a href=\"https://insanitybit.github.io/2016/12/28/why-rust-introduction\">Why Rust?</a> and <a href=\"https://insanitybit.github.io/2016/12/28/why-rust-sum-types\">Why Rust? Sum Types</a></li>\n</ul>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/four-years-with-rust/",
"title": "Four years with Rust",
"description": null,
"url": "https://steveklabnik.com/writing/four-years-with-rust/",
"published": "2016-12-21T00:00:00.000Z",
"updated": "2016-12-21T00:00:00.000Z",
"content": "<p>Today is four years since I first learned about the existence of Rust. I know this because Rust 0.5 was the first release I used. Rust has changed a lot in that time. For a brief overview of its history, <a href=\"https://www.youtube.com/watch?v=79PSagCD_AY\">watch this talk of mine</a>. But for today, I thought it would be fun to check out <a href=\"https://mail.mozilla.org/pipermail/rust-dev/2012-December/002787.html\">the release announcement</a> and see what’s the same, and what’s changed.</p>\n<h2>rust-dev</h2>\n<p>First up, the announcement itself. The <a href=\"https://mail.mozilla.org/listinfo/rust-dev\">rust-dev mailing list</a> used to be your one-stop shop for talking about Rust in a persistent form. But in <a href=\"https://mail.mozilla.org/pipermail/rust-dev/2015-January/011558.html\">January of 2015</a>, we decided to close it down. Why? Let’s see what Brian said at the time:</p>\n<blockquote>\n<p>You likely have already noticed, but traffic to rust-dev has decreased dramatically in recent months. This was a result of natural changes in project coordination at first, and then an intentional effort to phase out the list.In the beginning of the project there were only two places to talk about rust: #rust and rust-dev, and they served us well for all purposes for several years. As Rust grew though the coordination of the project moved to different venues, conversations were happening in a number of other places, and the purpose of rust-dev became less clear. At the same time there were also a number of heated and essentially unmoderatable discussions that caused many to lose faith in the effectiveness of the mailing list.</p>\n</blockquote>\n<p>I love mailing lists, but I know I’m fairly unique in that. But they do have one big problem, which is that the tools for moderating them are extremely thin. You can pretty much ban someone, and that’s about it. No removing only certain posts, no cooldown periods, no shadow banning. While Rust has a generally positive reputation for community and moderation, that doesn’t mean things were always perfect. We learned some lessons the hard way.</p>\n<p>More than that, mailman represents something of the “old guard” of open source. People see it as a sign of something backwards. Very few people want even more email. The culture is changing. We recognized this, and decided to move to <a href=\"http://www.discourse.org/\">Discourse</a> instead. rust-dev was survived by two children, <a href=\"http://users.rust-lang.org/\">users</a> and <a href=\"https://internals.rust-lang.org/\">internals</a>. This created a clear split between “discussion about Rust” and “discussions about building Rust.” It’s still free/open source software, it has good moderation tools, and you can even have it send you email!</p>\n<p>We also don’t make release announcements on these forums; we do it on <a href=\"https://blog.rust-lang.org/\">our blog</a>.</p>\n<p>Let’s dig into those notes!</p>\n<h2>900 changes, numerous bugfixes</h2>\n<p>900 changes is a lot! At the time, Rust was on a roughly six-month release schedule, so that’s about five patches merged per day. Rust 1.14 <a href=\"https://github.com/rust-lang/blog.rust-lang.org/pull/141\">will have</a> 1230 patches over its six-week release cycle: that’s just over 29 per day. That’s a much, much higher velocity. And that only counts the main repository; we’ve since added many more, like <a href=\"https://github.com/rust-lang/cargo/\">Cargo</a> and <a href=\"https://github.com/rust-lang/crates.io\">crates.io</a>. This is actually something I’ve been worried about for a while now; we only credit people for commits to <code>rust-lang/rust</code> in release announcements, basically due to this history, but there’s a lot more work that goes on here these days. I want <a href=\"http://contributors.rubyonrails.org/\">Rails Contributors</a>, but for the Rust project. Anyone want to work on it with me sometime?</p>\n<h2>Syntax changes</h2>\n<ul>\n<li>Removed <code><-</code> move operator</li>\n</ul>\n<p>Rust went through several iterations of “when do you move, when do you copy” over its lifetime. Today, that distinction is made between types that implement <code>Copy</code> and ones that don’t. But older Rusts had various schemes. Given that this one was removed in the first version of Rust I used, I don’t <em>exactly</em> remember this, but I think it was</p>\n<pre><code>x <- y; // move\nx = y; // copy\n</code></pre>\n<p>Rust also had this at one point in time, I believe:</p>\n<pre><code>x = move y; // move\nx = y; // copy\n</code></pre>\n<p>We decided to unify the two syntactically for two reasons: one, the annotations felt like busywork, the compiler would tell you what to put if you got it wrong. Secondly, at least in today’s Rust, move and copy are identical operations, with the exception of what you can do with the older variable after the move. Sharing the syntax reinforces that.</p>\n<p>Niko’s blog has a lot of great history in it, and <a href=\"http://smallcultfollowing.com/babysteps/blog/2012/10/01/moves-based-on-type/\">this topic is no exception</a>.</p>\n<ul>\n<li>Completed transition from <code>#fmt</code> extension syntax to <code>fmt!</code></li>\n</ul>\n<p>Long ago, Rust had a “no keywords can have more than five letters” rule, and so many things were abbreviated. This particular extension lives on today, but with the longer name <code>[format!](https://doc.rust-lang.org/stable/std/macro.format.html)</code>.</p>\n<ul>\n<li>Removed old fixed length vector syntax - <code>[T]/N</code></li>\n</ul>\n<p>Rust’s vector and array types, as we know them today, went through a <em>lot</em> of internal iteration, across many axes. The moral equivalent of <code>[T]/N</code> today is <code>[T; N]</code>, but I’m sure there are/were lots of subtle differences between the two. I can’t quite remember.</p>\n<ul>\n<li>New token quasi-quoters, <code>quote_tokens!</code>, <code>quote_expr!</code>, etc.</li>\n</ul>\n<p>These live on in some sense; they’re still not stable, so much so that they’re note even shown in the official docs. <a href=\"https://manishearth.github.io/rust-internals-docs/syntax/ext/quote/index.html\">Manish hosts a copy though</a>. These are tools for “syntax extensions”, Rust’s most powerful form of metaprogramming. A final design for them was <a href=\"https://github.com/rust-lang/rfcs/pull/1566\">just accepted eight days ago</a>, so it’ll be a while still before you see this on stable Rust.</p>\n<ul>\n<li>Macros may now expand to items and statements</li>\n</ul>\n<p>I don’t know why macros couldn’t do this before, but they still can today, and it’s very useful.</p>\n<ul>\n<li><code>a.b()</code> is always parsed as a method call, never as a field</li>\n</ul>\n<p>This is a fun edge case. Here’s the problem:</p>\n<pre><code>struct Env<F: Fn(i32)> { f: F,}let e = Env { f: |i| println!(\"Hello, {}\", i) };e.f(); // what does this do?\n</code></pre>\n<p>According to this entry, this would be a method call. In today’s Rust, this would be an error, you need to write <code>(e.f)();</code> At least the error tells you exactly what to do!</p>\n<pre><code>error: no method named `f` found for type `Env<[closure@<anon>:6:22: 6:50]>` in the current scope\n --> <anon>:8:7\n |\n8 | e.f(); // what does this do?\n | ^\n |\nnote: use `(e.f)(...)` if you meant to call the function stored in the `f` field\n --> <anon>:8:7\n |\n8 | e.f(); // what does this do?\n | ^\n\n</code></pre>\n<ul>\n<li><code>Eq</code> and <code>IterBytes</code> implementations automatically generated with <code>#[deriving_eq]</code> and <code>#[deriving_iter_bytes]</code> respectively</li>\n</ul>\n<p>We now have a more generic way to derive traits: <code>#[derive(Eq)]</code>, for example. <code>Eq</code> is still here today, but <code>IterBytes</code> is not. I can’t recall exactly what it did.</p>\n<p>Today, the list of traits you can derive is still only blessed ones by the compiler, and it’s a huge cause of people using nightly today, for better ergonomics with libraries like <a href=\"http://serde.rs/\">Serde</a> and <a href=\"http://diesel.rs/\">Diesel</a>. But as of Rust 1.15, this restriction will be lifted, and one of the largest blockers of people using stable Rust will be eliminated! ???</p>\n<ul>\n<li>Removed the special crate language for <code>.rc</code> files</li>\n</ul>\n<p>The moral equivalent of <code>.rc</code> files today are <code>.crate</code> files, which Cargo will upload to crates.io. But unlike in the 0.5 days, you almost never need to worry or think about them, since Cargo makes it Just Work (tm).</p>\n<ul>\n<li>Function arguments may consist of any irrefutable pattern</li>\n</ul>\n<p>This is true today, and is a little known fact! It works like this:</p>\n<pre><code>struct Point { x: i32, y: i32,}fn takes_point(Point {x, y}: Point) { println!(\"({}, {})\", x, y);}fn main() { let origin = Point { x: 0, y: 0 }; takes_point(origin);}\n</code></pre>\n<p>That is, arguments to functions are <code>PATTERN: TYPE</code>, not <code>NAME: TYPE</code>. Inside of <code>takes_point</code>, <code>x</code> and <code>y</code> are in scope, rather than a whole point.</p>\n<h2>Semantic changes</h2>\n<ul>\n<li><code>&</code> and <code>~</code> pointers may point to objects</li>\n</ul>\n<p>Rust did have objects, long ago, but I thought they were removed at this point. This one is unclear to me.</p>\n<p>Worth mentioning that <code>~</code> is <code>Box<T></code> now, though. That’s a long story…</p>\n<ul>\n<li>Tuple structs - <code>struct Foo(Bar, Baz)</code>. Replace newtype enums.</li>\n</ul>\n<p>These are <a href=\"https://doc.rust-lang.org/stable/book/structs.html#tuple-structs\">still in Rust today</a>.</p>\n<ul>\n<li>Enum variants may be structs</li>\n</ul>\n<p>This is true today:</p>\n<pre><code>enum Foo {\n Variant { x: i32, y: i32 },\n}\n</code></pre>\n<ul>\n<li>Destructors can be added to all nominal types via Drop trait</li>\n</ul>\n<p>This is <a href=\"https://doc.rust-lang.org/stable/std/ops/trait.Drop.html\">still true</a>!</p>\n<p>I think the “nominal” here is alluding to something older in Rust’s type system; we don’t make these kinds of distinctions today, as IIRC, all types are nominal. Not 100% sure.</p>\n<ul>\n<li>Structs and nullary enum variants may be constants</li>\n</ul>\n<p>This is true today.</p>\n<ul>\n<li>Values that cannot be implicitly copied are automatically moved without writing <code>move</code> explicitly</li>\n</ul>\n<p>I alluded to this above with <code><-</code>.</p>\n<ul>\n<li><code>&T</code> may now be coerced to <code>*T</code></li>\n</ul>\n<p><code>*T</code> is <code>*const T</code> or <code>*mut T</code> today, but this is still true:</p>\n<pre><code>let x = &5;\nlet y: *const i32 = x;\n</code></pre>\n<ul>\n<li>Coercions happen in <code>let</code> statements as well as function calls</li>\n</ul>\n<p>We’ve gone through many different iterations of “what coerces and what doesn’t” over the years; I actually expected the example with <code>*const i32</code> above to require an <code>as</code>.</p>\n<ul>\n<li><code>use</code> statements now take crate-relative paths</li>\n</ul>\n<p>This is true today. I really like it, but a lot of people find it confusing. Basically, “use” always starts from the root of the crate. So</p>\n<pre><code>mod bar { struct Foo; mod baz { use bar::Foo; // from the root use super::Foo; // if you want to start from the parent use self::super::Foo; // self makes it be relative. Not even sure if self + super works, but you get the idea. }}\n</code></pre>\n<ul>\n<li>The module and type namespaces have been merged so that static method names can be resolved under the trait in which they are declared</li>\n</ul>\n<p>Not sure on this one, it’s too in the weeds.</p>\n<h2>Improved support for language features</h2>\n<ul>\n<li>Trait inheritance works in many scenarios</li>\n</ul>\n<p>This has always been a bit of a misnomer, inheritance means:</p>\n<pre><code>trait Foo: Bar {\n}\n</code></pre>\n<p>If you implement <code>Foo</code>, you must also implement <code>Bar</code>. That’s it. This line makes me smile: “it works more often.”</p>\n<ul>\n<li>Support for explicit self arguments in methods - <code>self</code>, <code>&self</code> <code>@self</code>, and <code>~self</code> all generally work as expected</li>\n</ul>\n<p>Today’s Rust doesn’t just support this; it’s required! However:</p>\n<ul>\n<li><code>self</code> is still <code>self</code></li>\n<li><code>&self</code> is still <code>&self</code></li>\n<li><code>@self</code> is <code>Rc<self></code>, sorta kinda.</li>\n<li><code>~self</code> is <code>Box<self></code></li>\n</ul>\n<p><code>@self</code> was supposed to be a garbage-collected type, but was never really more than refcounting.</p>\n<p>In addition, the last two don’t actually work like that: they would be <code>self: Rc<Self></code> and <code>self: Box<self></code>. The <code>Box</code> version will compile today, but the <code>Rc</code> version does not, because <code>Box</code> is magical. Eventually this will be rectified.</p>\n<ul>\n<li>Static methods work in more situations</li>\n</ul>\n<p>Another “yay more features work” line. :)</p>\n<ul>\n<li>Experimental: Traits may declare default methods for impls to use</li>\n</ul>\n<p>This works today, and is very useful. Consider <code>[Iterator](https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html)</code>: all of those methods have default implementations, so you only need to define <code>next()</code>, and you get the rest automatically.</p>\n<h2>Libraries</h2>\n<ul>\n<li>New condition handling system in <code>core::condition</code></li>\n</ul>\n<p>Conditions were a Lisp way of handling errors that Rust supported for a while. They were really cool, but nobody really knew how to effectively use them, so they never got used, so they’re gone now.</p>\n<ul>\n<li>Timsort added to <code>std::sort</code></li>\n</ul>\n<p><code>std::sort</code> doesn’t exist anymore, though we do have <a href=\"https://doc.rust-lang.org/stable/std/primitive.slice.html#method.sort\">a sort function on slices</a>. We don’t declare it to have a particular algorithm, though we do say “This sort is stable and O(n log n) worst-case but allocates approximately 2 * n where n is the length of self.”</p>\n<p><a href=\"https://github.com/rust-lang/rust/pull/38192\">Recently it was improved a ton</a>, by a first-time contributor, no less! Extremely well-done. In the PR, he explains the algorithm:</p>\n<blockquote>\n<p>However, if we take the main ideas from TimSort (intelligent merging strategy of sorted runs) and drop galloping, then we’ll have great performance on random inputs and it won’t be bad on partially sorted inputs either.</p>\n</blockquote>\n<p><code>TimSort</code> is still around, kicking.</p>\n<ul>\n<li>New priority queue, <code>std::priority_queue</code></li>\n</ul>\n<p>Today this is <code>[std::collections::binary_heap](https://doc.rust-lang.org/std/collections/binary_heap/)</code>.</p>\n<ul>\n<li>Pipes for serializable types, `std::flatpipes’</li>\n</ul>\n<p>No idea where this went.</p>\n<ul>\n<li>Serialization overhauled to be trait-based</li>\n</ul>\n<p>This is true today. See the above comments about Serde, though. <code>[rustc-serialize](https://crates.io/crates/rustc-serialize)</code> got to cheat though, and the compiler understands <code>RustcEncodable</code> and <code>RustcDecodable</code>. Can’t wait for Rust 1.15.</p>\n<ul>\n<li>Expanded <code>getopts</code> definitions</li>\n</ul>\n<p>We’ve <a href=\"https://github.com/rust-lang-nursery/getopts\">moved <code>getopts</code> out of tree</a>, but it’s still there.</p>\n<ul>\n<li>Moved futures to <code>std</code></li>\n</ul>\n<p>Hoo boy! I forgot that we used to have futures in the standard library. Futures are one of the hottest topics in Rust today. <a href=\"https://github.com/tokio-rs\">Tokio</a> is one of the most anticipated releases of all time in Rust world. For a taste, <a href=\"https://www.youtube.com/watch?v=bcrzfivXpc4\">watch this talk</a>. A little birdie told me that Tokio should have a <code>0.1</code> release soon…</p>\n<ul>\n<li>More functions are pure now</li>\n</ul>\n<p>Rust no longer has a notion of purity, though <code>[const fn](https://github.com/rust-lang/rfcs/blob/master/text/0911-const-fn.md)</code> (still unstable) sorta feels like it in some senses. For some history on purity in Rust, see <a href=\"http://smallcultfollowing.com/babysteps/blog/2012/10/12/extending-the-definition-of-purity-in-rust/\">Niko’s blog</a>, or <a href=\"https://mail.mozilla.org/pipermail/rust-dev/2013-April/003926.html\">this explanation by Graydon on why it was removed</a>.</p>\n<ul>\n<li><code>core::comm</code> renamed to <code>oldcomm</code>. Still deprecated</li>\n</ul>\n<p>Very, very long gone.</p>\n<ul>\n<li><code>rustdoc</code> and <code>cargo</code> are libraries now</li>\n</ul>\n<p>Rustdoc still exists, though not <em>really</em> as a library. Cargo however, is not the cargo you’re thinking of. <a href=\"https://github.com/rust-lang/rust-wiki-backup/blob/73816d6f888c24fb8115d78078a1601805cbecb5/Doc-using-cargo-to-manage-packages.md\">It’s a totally different one</a>. Rust went through many iterations of package managers before coming up with today’s Cargo; there was also <code>[rustpkg](https://github.com/rust-lang/rust/blob/3e39e3e80dcf726a96ec0fe778f96e2a9dde620b/doc/guide-rustpkg.md)</code>, for example, which was <em>after</em> the Cargo referenced here, but before the Cargo we’re using today.</p>\n<h2>Misc</h2>\n<ul>\n<li>Added a preliminary REPL, <code>rusti</code></li>\n</ul>\n<p>We removed the repl from the tree, as it never <em>really</em> worked, and was a maintenance nightmare. <a href=\"https://github.com/murarth/rusti\">This one</a> is the only one I know of today. Some people ask about one, but nobody has done the work to get a good one together yet. It’s tough!</p>\n<ul>\n<li>License changed from MIT to dual MIT/APL2</li>\n</ul>\n<p>This is still true today, with no plans on changing.</p>\n<h2>Contributors to Rust 0.5:</h2>\n<p>We had 41 people contribute to 0.5. 1.14 will have 144 contributors, and that’s around average for a release these days. Out of those 41 people, at a glance, I recognize the names of about 17 individuals, and roughly six of them are still involved with Rust and Servo today. What about the other eleven? Some were interns that got jobs at other places and can’t work on Rust anymore, some are contributors that left for various reasons, and then of course, there’s Graydon.</p>\n<p>I personally opened <a href=\"https://github.com/rust-lang/rust/pull/4305\">my first pull request</a> six days after discovering it. You’ll note this PR wasn’t merged due to a procedural issue: I sent it into the wrong branch! Today, GitHub lets you change this, but I had to <a href=\"https://github.com/rust-lang/rust/pull/4308\">open a new PR instead</a>.</p>\n<p>As I said in that PR:</p>\n<blockquote>\n<p>I’ve just started using Rust, and I really like it. One of the things that makes Rust hard to use is its lack of documentation, which is obviously fine, given that Rust isn’t production-ready yet.I’d like to help change that.To get started, I’ve just modified this one little description in the rustdoc. If this works out, and you all like my changes, I’ll start working on more docs like this for the rest of core, and maybe the stdlib after that.</p>\n</blockquote>\n<p>You might say it did work out. I’m now <a href=\"https://blog.rust-lang.org/2014/12/12/Core-Team.html\">on the core team</a>, I’ve sent in 866 PRs since then, and that first patch landed in <a href=\"https://mail.mozilla.org/pipermail/rust-dev/2013-April/003427.html\">Rust 0.6</a>. Rust 0.10 is the only release since then that I’ve not gotten at least one patch into, and today, I write the release announcements.</p>\n<p>I hope you’ve enjoyed this little trip down memory lane! I’m still really loving Rust and its community. Here’s to many more years ?♥</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/whats-new-with-the-rust-programming-language/",
"title": "What's new with \"The Rust Programming Language\"",
"description": null,
"url": "https://steveklabnik.com/writing/whats-new-with-the-rust-programming-language/",
"published": "2016-08-15T00:00:00.000Z",
"updated": "2016-08-15T00:00:00.000Z",
"content": "<p>It’s been a while since I’ve mentioned my work on “The Rust Programming Language”, and I wanted to give you all an update. Before we get into the news, though, I’d like to take a moment to reflect on where the book has been, and then we’ll talk about it’s future.</p>\n<p>I <a href=\"http://words.steveklabnik.com/rusts-documentation-is-about-to-drastically-improve\">previously wrote about my history with Rust and docs</a>. As I said back then:</p>\n<blockquote>\n<p>But my first area of focus will be on the area of Rust’s documentation that’s the most weak, and simultaneously the most important: the tutorial.</p>\n</blockquote>\n<p>“The tutorial” eventually turned into “The Rust Programming Language.” But there’s a funny thing about writing a book for a language that isn’t done yet: you don’t really know <em>how</em> to write a good book on something that doesn’t exist, because, well, you haven’t played with it yourself. I could make decent guesses, given that Rust was evolving incrementally, but that’s still very different. I think “technical debt” is a great metaphor here: the evolution of the language meant that I had to keep up with the work of many, many people to try and update the documentation. Furthermore, since nobody had tried to learn Rust 1.0 yet, since it didn’t exist, I couldn’t write a book that would do an amazing job of teaching them, since I hadn’t used it either!</p>\n<p>In some ways, this mirrors the compiler itself, which accrued a <em>massive</em> amount of technical debt until 1.0. And so, the story since 1.0 has largely about been paying off this debt; culminating in <a href=\"https://blog.rust-lang.org/2016/04/19/MIR.html\">MIR</a>. By the same token, since 1.0, a lot of people have read the book, and a lot of people have tried to learn Rust. I now know so much more about how people go about doing so, and so am much, much better situated to write something that’ll be great for them.</p>\n<p>And so, now that time has passed, re-writing the book is what I’ve been spending a lot of my time on. I’ve been mostly quiet about this work, since I’ve wanted to get a chunk finished before starting to talk about it, and so today is finally that day.</p>\n<p>Let’s talk about what’s going on with the book today.</p>\n<h2>A new coauthor.</h2>\n<p>The first bit of news is something that I’ve been wanting to tell you all for a few weeks now, and is the bit that I’m most excited about: I’m bringing <a href=\"https://twitter.com/carols10cents\">Carol (Nichols || Goulding)</a> on as a coauthor. TRPL is an incredibly personal work for me, one that I’ve poured my life into over the last few years. Carol and I have been friends for a long time, and there’s nobody else I’d rather share an authorship with. Not only is Carol a friend, but she’s been doing Rust longer than most. You might know her from <a href=\"https://github.com/carols10cents/rustlings\">Rustlings</a>, or the <a href=\"http://rust-belt-rust.com/\">Rust Belt Rust conference</a>, or as a cofounder of <a href=\"http://integer32.com/\">Integer32</a>, the first Rust-focused consulting company. Carol knows her stuff, and I’m proud to have her name next time mine on the cover. The book will be far better through her influence.</p>\n<h2>A new beginning</h2>\n<p>One of Carols’ first tasks was taking a bunch of the stuff I’d already written and checking it out herself, making improvements. She’s taken the first six chapters of the book and whipped them into shape. Some of the content you might recognize from the first edition of TRPL, but much of it is brand-new.</p>\n<ul>\n<li><a href=\"http://rust-lang.github.io/book/\">Read the first six chapters online</a></li>\n<li><a href=\"https://github.com/rust-lang/book\">Source and issue tracker</a></li>\n</ul>\n<p>As an example of how much better this book is, can, and will be than the first edition, compare the <a href=\"http://rust-lang.github.io/book/ch04-00-understanding-ownership.html\">Ownership chapter</a> to <a href=\"https://doc.rust-lang.org/stable/book/ownership.html\">the current one</a> ( plus <a href=\"https://doc.rust-lang.org/stable/book/references-and-borrowing.html\">borrowing</a> ). Instead of an abstract set of rules, we took one of the biggest things facing a new Rustacean, strings, and used them as a way to make these concepts concrete.</p>\n<p>If you have the time, please check it out, and file issues with any suggestions!</p>\n<h2>More to come</h2>\n<p>If you’d like to see chapters as we work on them, just follow the repo: everything is open source. Initial PRs are already up for the start of the next three chapters, and hopefully we’ll get them merged and move forward soon. I’m really proud of how the book is shaping up, and hope you enjoy it too.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/structure-literals-vs-constructors-in-rust/",
"title": "Structure literals vs. constructors in Rust",
"description": null,
"url": "https://steveklabnik.com/writing/structure-literals-vs-constructors-in-rust/",
"published": "2016-07-26T00:00:00.000Z",
"updated": "2016-07-26T00:00:00.000Z",
"content": "<p>Learning the basics of a language and its syntax is easy. Learning how all those bits fit together is a bit harder. There’s a neat intersection between three of Rust’s features that I’ve seen people use, but never seen written down. I was explaining this technique to someone in <code>#rust-beginners</code> the other day, and thought I’d write it down in case it helps you, too.</p>\n<p>A small review: If you have a <code>struct</code> in Rust, like this:</p>\n<pre><code>struct Point {\n x: i32,\n y: i32,\n}\n</code></pre>\n<p>You can use ‘<code>struct</code> literal syntax’ to create a new instance of the <code>struct</code>:</p>\n<pre><code>let origin = Point { x: 0, y: 0 };\n</code></pre>\n<p>However, this syntax only works if you have the proper access to the <code>struct</code> and its members via Rust’s privacy rules.</p>\n<pre><code>mod foo {\n pub struct Point {\n x: i32,\n y: i32,\n }\n\n pub fn foo(x: i32, y: i32) -> Point {\n Point { x: x, y: y } // this is fine, as we're in the same module\n }\n}\n\nfn main() {\n let origin = foo::Point { x: 0, y: 0 }; // this is not fine\n}\n</code></pre>\n<p>We can’t use <code>struct</code> literal synatx in <code>main</code> because <code>x</code> and <code>y</code> are also not public. But within the same module, we have access, so it works. So how would we let <code>main</code> instantiate a <code>Point</code> if we can’t use the literal syntax? Well, our <code>foo</code> function does this, so we could expose it. More conventionally, we’d make it an associated function and call it <code>new</code>:</p>\n<pre><code>mod foo {\n pub struct Point {\n x: i32,\n y: i32,\n }\n \n impl Point {\n pub fn new(x: i32, y: i32) -> Point {\n Point { x: x, y: y } // this is fine, as we're in the same module\n }\n }\n}\n\nfn main() {\n let origin = foo::Point::new(0, 0);\n}\n</code></pre>\n<p>Great. But what if we wanted <code>x</code> and <code>y</code> to be public for some reason, yet we still wanted to force people to use the <code>new</code> function to create <code>Point</code>s? Maybe the initial creation does some sort of side effect that’s important. If we changed our code to this:</p>\n<pre><code>mod foo {\n pub struct Point {\n pub x: i32,\n pub y: i32,\n }\n \n impl Point {\n pub fn new(x: i32, y: i32) -> Point {\n Point { x: x, y: y } // this is fine, as we're in the same module\n }\n }\n}\n\nfn main() {\n let origin = foo::Point::new(0, 0);\n\n // but so does this:\n let origin = foo::Point { x: 0, y: 0 };\n}\n</code></pre>\n<p>By making all of the elements of <code>Point</code> public, we’ve re-enabled the literal syntax, which isn’t what we wanted. So what do we do?</p>\n<p>Fixing this requires two insights. The first is “zero-sized types”. In Rust, certain types only have values that don’t require any storage. Take, for example, <code>()</code>, the “unit” type. It only has one possible value, also <code>()</code>. Since it only has one value, there’s no need to actually store anything in memory to represent <code>()</code>; if we have a valid value, we already know what it is. That means that once we compile down to actual assembly, <code>()</code> just goes away entirely. So we can do this:</p>\n<pre><code>mod foo {\n pub struct Point {\n pub x: i32,\n pub y: i32,\n _secret: (),\n }\n \n impl Point {\n pub fn new(x: i32, y: i32) -> Point {\n Point { x: x, y: y, _secret: () }\n }\n }\n}\n\nfn main() {\n let origin = foo::Point::new(0, 0);\n}\n</code></pre>\n<p>Now, we have a new, non-public field, <code>_secret</code>. I’ve given it a name that starts with an underscore because we don’t intend to use it for anything, and so Rust won’t warn us about it. <code>_secret</code> has the type of <code>()</code>, and so it’s entirely a compile-time construct; it doesn’t materially affect <code>Point</code>’s representation. But it being private <em>does</em> affect how we’re allowed to construct our <code>Point</code>s. <code>main</code> can no longer use the <code>struct</code> literal syntax, since not all of the fields are public.</p>\n<p>However, remember that privacy is a module-level thing in Rust. Therefore, we can still use the <code>struct</code> literal syntax inside of the <code>foo</code> module:</p>\n<pre><code>mod foo {\n pub struct Point {\n pub x: i32,\n pub y: i32,\n _secret: (),\n }\n \n impl Point {\n pub fn new(x: i32, y: i32) -> Point {\n Point { x: x, y: y, _secret: () }\n }\n }\n\n fn foo() -> Point {\n Point: { x: 0, y: 0, _secret: () } // this is still allowed!\n }\n}\n\nfn main() {\n let origin = foo::Point::new(0, 0);\n}\n</code></pre>\n<p>To prevent <code>foo</code> from being able to use the literal syntax, we need one more concept: <code>pub use</code>. Check this out:</p>\n<pre><code>mod foo {\n mod point {\n pub struct Point {\n pub x: i32,\n pub y: i32,\n _secret: (),\n }\n \n impl Point {\n pub fn new(x: i32, y: i32) -> Point {\n Point { x: x, y: y, _secret: () }\n }\n }\n }\n\n pub use foo::point::Point;\n\n fn foo() -> Point {\n Point::new(0, 0) // must use `new` here, as we're no longer in the same module!\n }\n}\n\nfn main() {\n let origin = foo::Point::new(0, 0);\n}\n</code></pre>\n<p>By giving <code>Point</code> its own module, everything that’s private to it is private. But typing <code>foo::point::Point</code> would be redundant and an ergonomic regression; <code>pub use</code> saves the day! We re-export the <code>Point</code> structure into <code>foo</code>, so we can still use <code>foo::Point</code>, but since one of its members is private, literal syntax isn’t allowed.</p>\n<p>To me, understanding things like this is when I really start to feel like I’m getting to know a language: putting three or four disparate concepts together to achieve some goal. It’s when a language stops being a bunch of disjoint parts and starts becoming a cohesive whole.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/a-eulogy-for-my-20s/",
"title": "A eulogy for my 20s",
"description": null,
"url": "https://steveklabnik.com/writing/a-eulogy-for-my-20s/",
"published": "2016-01-24T00:00:00.000Z",
"updated": "2016-01-24T00:00:00.000Z",
"content": "<p>Today is the last day of my 29th year of existence. I’ve been thinking about it a lot, which is extra strange because I don’t generally care about birthdays. I’m not much of one for holidays in general. So why do I care about this one?</p>\n<p>My 20s have been an extreme period of growth. At the start of it, I thought that I was going to live in my little farming town, get married to The Girl, go to mass every Sunday, and pretty much do exactly what everyone else I knew had done. If you’re reading this, well, you know a very different me. Before my 20s, I had only left the country once, a short trip to Toronto. I had barely even left my state. I was smart, but cocky, and incredibly sheltered. I was the very embodiment of a stereotype.</p>\n<p>I’ve come a long way.</p>\n<p>The root of my anxiety about being 30 comes from something that my brain is constantly whispering into my ear: “You don’t have enough time.” There’s sort of two sides to this; I think: the first is how much I’ve changed in this decade, and the second is a particularly toxic attitude of the industry I’ve immersed myself in.</p>\n<p>I feel pretty dumb about this post overall, but whatever. Please just take this as a brain dump from this morning, not some sort of grand, planned essay about how I know all the things or something.</p>\n<h2>Identity is a function</h2>\n<p>Have you ever played around with <a href=\"http://www.utdallas.edu/~gupta/courses/apl/lambda.pdf\">lambda calculus</a>? I myself only know the basics, I’m not a particularly math-y person. Here’s a short introduction: say we have a function. This function is very simple, it returns its single argument. In other words, the identify function:</p>\n<pre><code>fn id(x) {\n return x;\n}\n</code></pre>\n<p>(Making this syntax up, though it will look sort of like Rust without types.)</p>\n<p>We can write another function, <code>zero'()</code>, like this:</p>\n<pre><code>fn zero'(f, g) {\n return g();\n}\n</code></pre>\n<p>In other words, <code>zero'()</code> takes two functions, <code>f</code> and <code>g</code>. It then calls <code>g()</code>, ignoring <code>f()</code>. This will make sense in a moment, we just need one more function: <code>succ()</code></p>\n<pre><code>fn succ(f, g) {\n return f(g());\n}\n</code></pre>\n<p>This function, short for “successor”, applies <code>f()</code> to <code>g()</code>, one time.</p>\n<p>With these tools, we can represent all of the natural numbers. This is called <a href=\"https://en.wikipedia.org/wiki/Church_encoding#Church_numerals\">Church numerals</a>, and here’s how it works:</p>\n<pre><code>\nlet zero = zero'(succ, id); // zero any function applied to id zero times.\nlet one = succ(succ, zero); // one is succ() applied to zero\nlet two = succ(succ, one); // two is succ() applied to one\nlet three = succ(succ, two); // three is succ() applied to two\n</code></pre>\n<p>And so on. Now I have this crippling fear that I’ve messed up the notation, trying to transform it into something that looks more like “normal” programming. I’m sure that someone on the Internet is going to find some kind of flaw and call me an idiot over it.</p>\n<p>Ahem.</p>\n<p>Anyway, this is the core idea: you can count the number of times you’ve applied a function to identity, and that encodes the number you’re trying to represent. We don’t need state to represent numbers and counting, we can do it all through functions.</p>\n<p>The concept of “identity” is a really important one in philosophy. You can take all kinds of approaches here. One classic approach is called “Platonic idealism”, also known as “essentialism.” Essentialism states that you can tell the identity of something by the properties it has. For example, a chair is a chair because it has a platform for you to sit, it has four legs, and a back.</p>\n<p>The problem with essentialism is that well, there are things that you or I would reasonably call a chair that has three legs, or five or six. Not all things that have a place to sit can be called a chair, either. In other words, while it sounds reasonable at first, the real world absolutely destroys this kind of categorization, in my opinion.</p>\n<p>So what do you do? Another solution to this question is posed by a really broad family called ‘process philosophy.’ The solution to identity there is based around movement, change, becoming. Function application. A “chair” is a particular kind of object that participates in a world like other things that we call “chairs”. In other words, identity is something that comes out of <em>doing</em>, not <em>being</em>.</p>\n<p>I am not the person I was ten years ago, quite literally. My body is made up of various kinds of cells, and they die at different rates. At a cellular level, I am quite literally dying all the time, in various microscopic places, and I am also regenerating. New cells are being produced. I am quite literally not my past self at a cellular level. And once I finally die, it’s not like my cells also instantly die either. Some of them will live for a period of time after I’m dead, though not for long. Bits of my body will actually out-live me.</p>\n<p>Life is full of growth and change. But at this point, the function has been applied so many times that I barely even recognize my past self. It makes me feel like I’m starting <em>now</em>, so late into the game! I’ve wasted the earlier years of my life on all kinds of dumb and bad things.</p>\n<p>I constantly feel that I am running out of time, that I have not done enough, that I will be a failure. That my work will not be meaningful enough to be remembered, or worse, will be remembered as bad. When I express this kind of feeling, people often point this out:</p>\n<p><img src=\"https://svbtleusercontent.com/yteofannex78va_small.png\" alt=\"https://svbtleusercontent.com/yteofannex78va_small.png\" /></p>\n<p>This doesn’t make me feel better, it makes me feel worse. There are so many things that I want to do that I’m <em>not</em> working on. That hypermedia book still isn’t done. I have been a bad maintainer of Resque and all my other Ruby stuff. I didn’t listen to my friends in Greece, who told me to relax, and then the exact situation they predicted happened. I just hope I’m around long enough to make an impact on people and be a positive change in the world. We’ll see if I make it.</p>\n<p>There’s a whole ’nother rant here about Camus and Heidegger and absurd-ism and being-towards-death too. I will save that sophomoric rant for my “I’m turning 40” post.</p>\n<h2>Youth Cults</h2>\n<p>The second thing that’s making me feel terrible about turning 30 is the goddamn computer industry. It’s extra worse because historically, I have benefited from this particular terrible attitude. And that’s the worship of youth.</p>\n<p>I used to teach new programmers, and often give talks to classrooms full of them. They always want to know how I got started, and I always preface it with something like</p>\n<blockquote>\n<p>My story is one I don’t like to share, because I’m the living embodiment of a stereotype that I think is harmful to new programmers.</p>\n</blockquote>\n<p>I started with computers when I was seven. That shouldn’t matter. I’ve learned more about software in the last five years than I did from seven till I was 18. But we hold up these young genius whiz-kids (and let’s be serious, it’s not ‘kids’, it’s ‘boys’) as the epitome of what our industry can do. The heroic youth who understands computers better than those silly old people. What do they know, anyway?</p>\n<p>Age discrimination is real. It’s stupid, but it’s real. And even when you know that it’s stupid, it can still slip into your brain in a really terrible way. If I’m 30 and this is all I’ve done by now, well, why even bother?</p>\n<p>This is dumb. The attitude is dumb. That I disagree with it, yet still feel it at times, is dumb. Ideology is a motherfucker.</p>\n<p>So yeah, that’s it. My 20s are almost over. And I wasted some of the last bits of them writing this silly blog post. There’s no stopping it, just watching the clock tick forward, and doing my best as hard as I can until it’s all over. How many of these decades will I even get? My dad certainly thought he had a few more until that night he collapsed and found out that he had stage four, terminal cancer.</p>\n<p>Let’s see what I can do in the next ten years.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/docember/",
"title": "Docember",
"description": null,
"url": "https://steveklabnik.com/writing/docember/",
"published": "2015-12-01T00:00:00.000Z",
"updated": "2015-12-01T00:00:00.000Z",
"content": "<p>There are few things I appreciate more than using software projects that have good documentation. When I’m new to the project, documentation is invaluable for helping me get going. When I’m not new, good documentation helps me get the information I need to :shipit:.</p>\n<blockquote>\n<p>We say programming is hard, and we hate bad code. So we practice writing code.</p>\n</blockquote>\n<p>We say writing docs is hard, and hate bad docs…</p>\n<p>— Some(@steveklabnik) (@steveklabnik) November 30, 2015</p>\n<blockquote></blockquote>\n<p>About a month ago, I was tweeting about this topic, and <a href=\"https://mayeu.me/misc/2015/11/13/docember-one-month-to-improve-your-documentation.html\">several of us came up with an idea</a>: similar to how November is <a href=\"http://nanowrimo.org/\">National Novel Writing Month</a>, why not have a similar month for programming?</p>\n<p>Here’s the idea: in December, let’s try to work on the documentation for the projects we work on. Make it 50k words like NaNoWriMo, make it <a href=\"http://24pullrequests.com/\">24 pull requests</a>, whatever floats your boat. Your future self will love you for it.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rescuing-resque-again/",
"title": "Rescuing Resque (again)",
"description": null,
"url": "https://steveklabnik.com/writing/rescuing-resque-again/",
"published": "2015-10-28T00:00:00.000Z",
"updated": "2015-10-28T00:00:00.000Z",
"content": "<p>A long time ago, there was a big open source project, which was a huge part of the Rails world, but had almost no maintenance. This project was Resque, the go-to way to write background jobs for your Rails application. I attended a talk where the current maintainer was clearly getting burned out, and so, <a href=\"http://blog.steveklabnik.com/posts/2012-09-22-resque--let-s-do-this\">offered to help</a>. The problem is, things have changed.</p>\n<p>It wasn’t just me. It was a bunch of people. We all used Resque, and loved it, and wanted to make it better. We made plans for a big 2.0 to pay off a bunch of technical debt, simplify the codebase, and add more features. A ton of people chipped in, I had some awesome remote pairing sessions. Things were good.</p>\n<p>But slowly, we all started to drift away. One contributor changed jobs. Another decided to spend more time with family. I got discouraged, and then, there was a strange situation in which I accidentally released an entirely straight-up broken version. So I got scared, released a fix, and basically never touched Resque again.</p>\n<p>Resque is used by a <em>lot</em> of people:</p>\n<p><img src=\"https://svbtleusercontent.com/rsoq8peusw08q_small.png\" alt=\"https://svbtleusercontent.com/rsoq8peusw08q_small.png\" /></p>\n<p>Yet, it hasn’t seen a release in 18 months. The PRs are piling up. 2.0 is basically dead in the water. I’m not much of a fan of 2.0s anymore myself, at least, ones that don’t follow the “stability without stagnation” principles. I don’t use Resque myself anymore, and barely have enough time for my work, let alone my personal projects, let alone a project that I care about but only work on out of the goodness of my heart.</p>\n<p>I’m here to tell you that that’s changing a little bit. I’ve been chatting with <a href=\"https://github.com/hoffmanc/\">Chris Hoffman</a>, who’s interested in starting up on work again. I asked him what his intentions were, and what he answered with was exactly what I wanted to hear, as it turns out. So, I’m going to be doing some work again, this time, with the explicit intention of handing off maintenance to Chris and anyone else who wants to chip in. If you use Resque commercially, this is a great time to give some of your people the opportunity to give back. If you’ve never worked on an open source project before, but want to, I’d be more than happy to help mentor you with this. Resque is a big, real project, with a lot of impact. You can do a lot of good. That’s why I started working on it.</p>\n<p>Without further ado, here is The Plan, from Chris and I:</p>\n<h2>TL;DR</h2>\n<ul>\n<li>change <code>master</code> from unreleased 2.0 back to <code>1-x-stable</code></li>\n<li>clean up tests - make the development experience enjoyable</li>\n<li>integrate pending Pull Requests</li>\n<li>2.0? Never heard of it. At least, not for a while.</li>\n</ul>\n<h2>Make <code>1-x-stable</code> master again</h2>\n<p><a href=\"https://github.com/resque/resque/issues/1175\">Many</a> <a href=\"https://github.com/resque/resque/issues/976\">have</a> stated that the README on the master branch has led them astray, because the published gem is still at 1.25.2.</p>\n<p>We will plan on making this switch in the next couple of weeks. If you feel this is the wrong move, please let us know. But since active development will in fact happen on 1.x, this feels like the correct thing.</p>\n<h2>Fix up the test suite</h2>\n<p>Tests need to be fast, and clean, so that there is more incentive to contribute.</p>\n<p>One possible approach is using one of the available Redis mocking libraries (mock_redis, fakeredis) in non-forking tests, which could speed suite execution up considerably.</p>\n<h2>Review Pending <code>1-x-stable</code> PRs</h2>\n<p>This is the lion’s share of the work, as there are currently 56 (that’s <em><a href=\"https://github.com/resque/resque/pulls\">fifty six</a></em>) open pull requests…</p>\n<p><em>And that’s an awesome problem!</em> Let’s make use of them! These PRs need to be evaluated and closed, so that the project can benefit from the community.</p>\n<h2>Decide What to do with 2.0</h2>\n<p>The focus of Resque should be on stability, with an eye towards legacy support. 2.0 is off the table, at least in the near term.</p>\n<p>If there is a 2.0, it will be a “garbage collection”-style release, where only deprecated functionality is removed, and the upgrade path is clear.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-rust-programming-language-will-be-published-by-no-starch-press/",
"title": "\"The Rust Programming Language\" will be published by No Starch Press",
"description": null,
"url": "https://steveklabnik.com/writing/the-rust-programming-language-will-be-published-by-no-starch-press/",
"published": "2015-09-08T00:00:00.000Z",
"updated": "2015-09-08T00:00:00.000Z",
"content": "<p>I’m happy to bring you some news: <a href=\"http://doc.rust-lang.org/stable/book/\">“The Rust Programming Language”</a> is going to be published, in physical form, by <a href=\"https://www.nostarch.com/\">No Starch Press</a>. I am really excited to work with No Starch on this: they’ve published some of my favorite programming books, like <a href=\"https://www.nostarch.com/hemingway\">“If Hemingway Wrote JavaScript”</a>, <a href=\"https://www.nostarch.com/rum\">“Ruby Under a Microscope”</a>, and <a href=\"https://www.nostarch.com/rubywizardry\">“Ruby Wizardry”</a>.</p>\n<p>Another reason I’m keen to work with No Starch on this is that they believe in publishing freely available material: we’re going to take the existing manuscript, whip it into shape, make a nice paper book out of it, and merge the changes back into the official documentation. So the improvements will get back to everyone, even those who can’t afford a physical copy.</p>\n<p>The authorship credit is to “The Rust Project Developers,” as other people have contributed significantly: we even have an <a href=\"https://github.com/rust-lang/rust/pull/28301\">active PR to re-write an entire chapter</a>. Rust is a team effort.</p>\n<p>Oh, and one last thing: the profit from the book won’t go to me, it will go to <a href=\"http://openhatch.org/\">OpenHatch</a>, “a non-profit dedicated to matching prospective free software contributors with communities, tools, and education,” to use their words about it.</p>\n<p>I’ll let you know when we’re closer to actually shipping!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/community-versions-for-rust/",
"title": "Community Versions for Rust",
"description": null,
"url": "https://steveklabnik.com/writing/community-versions-for-rust/",
"published": "2015-08-23T00:00:00.000Z",
"updated": "2015-08-23T00:00:00.000Z",
"content": "<p><a href=\"http://www.rust-lang.org/\">Rust</a> has been through a <em>lot</em> of changes in recent years. As such, tutorials get out of date. We’ve occasionally said that including <code>rustc -v</code> in your blog posts would be helpful, but today, I saw this tweet go by:</p>\n<blockquote>\n<p>Tired of guessing what versions of Ember work with a given blog post? Check out http://t.co/6joJiNdJQ5 #feedbackplz pic.twitter.com/YToBduBVtx\n— mixonic (@mixonic) August 23, 2015</p>\n</blockquote>\n<p>After some quick hacking, I’ve modified it to work with Rust. You can find the project here: <a href=\"https://github.com/steveklabnik/rust-community-versions\">https://github.com/steveklabnik/rust-community-versions</a></p>\n<p>TL;DR: Put this into your blog post, substituting the appropriate date and title:</p>\n<pre><code><iframe\n width=\"178\" height=\"24\" style=\"border:0px\"\n src=\"http://steveklabnik.github.io/rust-community-versions/2013/10/18/pointers-in-rust-a-guide.html\">\n</iframe>\n</code></pre>\n<p>Then, make a <a href=\"https://github.com/steveklabnik/rust-community-versions/pulls\">pull request</a> adding a new post with some front-matter:</p>\n<pre><code>---\nlayout: post\nurl: http://words.steveklabnik.com/pointers-in-rust-a-guide\ntitle: \"Pointers in Rust: a Guide\"\ndate: 2013-10-18\nstart_version: 0.8\nend_version: 0.9\n---\n</code></pre>\n<p>(<code>end_version</code> is optional)</p>\n<p>And bam! You get a nice looking badge. Check out out on this blog post of mine: http://words.steveklabnik.com/pointers-in-rust-a-guide</p>\n<p>What’s nice about this approach is that, if your post ever gets out of date, someone else can modify the YAML in the repo, and your badge will automatically update to include the end date. No remembering to go back and fix it.</p>\n<p>Let’s see how this goes!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/ember-data--ds-is-not-defined/",
"title": "Ember Data: ‘DS’ is not defined",
"description": null,
"url": "https://steveklabnik.com/writing/ember-data--ds-is-not-defined/",
"published": "2015-07-05T00:00:00.000Z",
"updated": "2015-07-05T00:00:00.000Z",
"content": "<p>I’m doing a little Ember app, and if there’s one thing I’ve learned from writing software, it’s to blog about error messages. Two-years-later me has ended up finding my own posts when searching for help!</p>\n<p>So today, when getting started with Ember Data 1.13, I was trying to use the new <code>JSONAPIAdapter</code>. I saw this code snippet:</p>\n<pre><code>App.ApplicationAdapter = DS.JSONAPIAdapter.extend({\n namespace: 'v1',\n});\n</code></pre>\n<p>Using that gave me an error when <code>ember serve</code>-ing, though:</p>\n<pre><code>app.js: line 16, col 26, ‘DS’ is not defined.\n</code></pre>\n<p>Turns out, <code>DS</code> isn’t in scope by default, even though <code>ember-cli</code> installs Ember Data by default.</p>\n<p>Fixing this just means importing it at the top of <code>app/app.js</code>:</p>\n<pre><code>import DS from 'ember-data';\n</code></pre>\n<p>Easy enough!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-language-strangeness-budget/",
"title": "The language strangeness budget",
"description": null,
"url": "https://steveklabnik.com/writing/the-language-strangeness-budget/",
"published": "2015-06-26T00:00:00.000Z",
"updated": "2015-06-26T00:00:00.000Z",
"content": "<p>I’ve always loved programming languages. I’ve spent plenty of times with many of them, and am fortunate enough that <a href=\"http://blog.rust-lang.org/2014/12/12/Core-Team.html\">language design is now part of my job</a>. In discussions about building Rust, I’ve noticed myself making a particular kind of argument often in design discussions. I like to call it ‘the language strangeness budget’.</p>\n<p>When building anything, it’s important to understand why you are building it, and who you are building it for. When it comes to programming languages, <a href=\"https://github.com/steveklabnik/mojikun\">building one is easy</a>, but getting people to use it is much, much harder. If your aim is to build a practical programming language with a large community, you need to be aware of how many new, interesting, exciting things that your language is doing, and carefully consider the number of such features you include.</p>\n<p>Learning a language takes time and effort. <a href=\"http://doc.rust-lang.org/stable/book/\">The Rust Programming Language</a>, rendered as a PDF, is about 250 pages at the moment. And at the moment, it really covers the basics, but doesn’t get into many intermediate/advanced topics. A potential user of Rust needs to literally read a book to learn how to use it. As such, it’s important to be considerate of how many things in your language will be strange for your target audience, because if you put too many strange things in, they won’t give it a try.</p>\n<p>You can see us avoiding blowing the budget in Rust with many of our syntactic choices. We chose to stick with curly braces, for example, because one of our major target audiences, systems programmers, is currently using a curly brace language. Instead, we spend this strangeness budget on our major, core feature: ownership and borrowing.</p>\n<p>On the other hand, if you’re trying to accomplish something different with your language, purposefully blowing this budget can be useful too. I really enjoy Haskell, and for a long time, their slogan has been ‘<a href=\"http://www.computerworld.com.au/article/261007/a-z_programming_languages_haskell/\">avoid success at all costs</a>’:</p>\n<blockquote>\n<p>As for the second, I don’t know if you know this, but Haskell has a sort of unofficial slogan: avoid success at all costs. I think I mentioned this at a talk I gave about Haskell a few years back and it’s become sort of a little saying. When you become too well known, or too widely used and too successful (and certainly being adopted by Microsoft means such a thing), suddenly you can’t change anything anymore. You get caught and spend ages talking about things that have nothing to do with the research side of things.I’m primarily a programming language researcher, so the fact that Haskell has up to now been used for just university types has been ideal</p>\n</blockquote>\n<p>This is Simon Peyton-Jones knowing exactly what kind of language he wants to build, and in what way. Haskell’s embrace of strangeness in this sense is what makes Haskell Haskell, and actually furthers its goals.</p>\n<p>If you include <em>no</em> new features, then there’s no incentive to use your language. If you include too many, not enough people may be willing to invest the time to give it a try. Language designers should give careful thought to how strange their language is, and choose the right amount to accomplish what they’re trying to accomplish.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/ouroboros/",
"title": "Ouroboros",
"description": null,
"url": "https://steveklabnik.com/writing/ouroboros/",
"published": "2015-05-18T00:00:00.000Z",
"updated": "2015-05-18T00:00:00.000Z",
"content": "<p>One of the things that I love about open source is its ability to send improvements across projects. Sometimes, an improvement in one project ends up improving its upstream. This kind of network effect really gets to the core of it for me: steady, constant improvement.</p>\n<p>I saw an example of this today that makes me smile. Before <a href=\"http://blog.rust-lang.org/2015/05/15/Rust-1.0.html\">Rust’s 1.0 release</a>, I wrote a <a href=\"http://doc.rust-lang.org/stable/book/dining-philosophers.html\">chapter about the Dining Philosopher’s problem</a>. Incidentally, I’m really happy to use classic CS concepts in the Rust docs, hopefully exposing them to people who didn’t get the opportunity to get a degree in computer science. Anyway, towards that end, I decided to cite <a href=\"http://www.usingcsp.com/cspbook.pdf\">Tony Hoare’s classic CSP paper</a>. Other funny aspect of that: I started my implementation using channels, but ended up using mutexes instead.</p>\n<p>Anyway, today, a new contributor <a href=\"https://github.com/rust-lang/rust/pull/25575\">spotted a typo</a>. The PR message made me smile:</p>\n<blockquote>\n<p>Looks like this was an issue in the source material. I’ve let the editor know and he has told me he is correcting it there as well.</p>\n</blockquote>\n<p>In my own small way, a thing I did led to someone else improving one of the most classic and well-known CS papers ever written. And it’s just Monday.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/hope/",
"title": "Hope",
"description": null,
"url": "https://steveklabnik.com/writing/hope/",
"published": "2015-05-08T00:00:00.000Z",
"updated": "2015-05-08T00:00:00.000Z",
"content": "<p>“The people are unstoppable / Another world is possible”</p>\n<p>This has long been my favorite protest chant. For me, it gets to the heart of the matter: another world <em>is</em> possible. If I’m honest with myself, most days, I don’t really believe that the world will get better. We often present history as a kind of positive progression: the world has only been getting better, more tolerant, and less poor as the years go by. I don’t <em>feel</em> that, though. What I see is imperialism, consolidation of power, and rampant racism/sexism/etc. Liberalism presents itself as an end of history, the final stop on the line. That’s unacceptable to me, but I usually feel like the only one.</p>\n<p>The start of this post sounds depressing, but I’m only writing it because I actually feel positive, invigorated. I’m writing this post from the Barcelona airport, about to board the plane home. It took me a while to figure this out, but what I realized is this: getting out of the US every once in a while really helps me feel better. This isn’t to say that things are actually <em>better</em> in other places, that’s not my point. My point is that It’s not something like “European-style parlamentary democracy is better than a bicameral legislature” or something like that, just look at the UK election results. It’s that the street signs are different. The money isn’t the same. Familiar foods have changes. I see 漢字 instead of the Latin alphabet. Whatever is unusual about whatever part of the world I am in, it’s the smallest differences that have the largest impact on me and my state of mind.</p>\n<p>When I go somewhere else, I can <em>feel</em> the difference. And difference is possibility. And possibility is hope, for a better world, for a better humanity.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/an-introduction-to-economics-under-capitalism/",
"title": "An introduction to economics under capitalism",
"description": null,
"url": "https://steveklabnik.com/writing/an-introduction-to-economics-under-capitalism/",
"published": "2015-03-10T00:00:00.000Z",
"updated": "2015-03-10T00:00:00.000Z",
"content": "<p>The dismal science a difficult one. Many people have strong opinions about economics, and it determines many of their other political beliefs. As someone who has significantly different thoughts on this topic than many of my peers, I’m writing this post out to give a basic explanation as I see it. That doesn’t mean I’m <em>inventing</em> something here, just explaining the model that I currently prefer. I’ll give sources at the end, if you want to dig deeper. Speaking of ‘deeper,’ this is also an <em>introduction</em>, one could easily write a book or three on this topic. I’ll try to be clear where I’m making an assumption, and why I’m making it.</p>\n<p>Finally, this is very much a draft, I may make small, or even maybe major errors. Nobody’s perfect. My spelling is terrible, and there’s a bug where my browser spellcheck isn’t working… anyway.</p>\n<hr />\n<p>Let’s start from the beginning. And that beginning starts with stuff. People trade things between them. Let’s be a bit more precise: people trade commodities. They trade commodities becuase they want something they do not currently have. In other words, if I’m hungry, and you have some $FOOD, and you’re cold, and see I have a coat, you might trade me your food for my coat and we’d both be better off. This ‘commodity’ is distinct from any useful thing because we trade for it, and because it satisfies some sort of need.</p>\n<p>But how do we know how much food we should trade for a coat? What does that even mean? To make this trade equal, we need to settle on a value for food, and a value for coats. That value has two different parts to it. The first: that value represents how much something satisfies our needs. Something that’s really useful will have a lot of value to us. Something that’s not may have less value. We call this component of value ‘use value.’</p>\n<p>The second component is more complicated. It’s called ‘exchange value,’ and it’s called that becuase this is the actual value that we trade each other for. These two kinds of value are distinct, but the distinction is important. While I may find a coat midly useful, and therefore, its use value is low to me, that doesn’t mean that if I trade it to someone else, they will agree in how useful it is. We might have to sell this coat for more or less than we expected to. While the usefulness affects the exchange value in a sense, it doesn’t mandate it.</p>\n<p>Let’s give a more concrete example of this. If I can trade my single coat for four $UNITs of $FOOD, then the exchange value of one coat is four $FOODs, and the exchange value of a $FOOD is one quarter of a coat.</p>\n<p>So if the exchange value of a commodity determines how we trade, and that’s based on a subjective preference for usefulness. So if I told you there was some underlying constant that did determine exchange value, you probably wouldn’t believe me. This is, in fact, one of the root areas in which economists disagree with me. Their first mistake. They claim that this exchange is entirely based on preference, with no underlying reasoning. I (and others) disagree. Consider the equation,</p>\n<pre><code>4 * c = 1 * f\n</code></pre>\n<p>where <code>c</code> is coats and <code>f</code> is food. If we wanted to reduce these equations, we could end up with two variants:</p>\n<pre><code>(4 * c) / 1 = f\n\nc = (1 * f) / 4\n</code></pre>\n<p>Because <code>c</code> and <code>f</code> are defined in terms of each other, it’s hard to get at an underlying property of both things.</p>\n<p>Now, remember why we’re trading food for coats: they satisfy some sort of need. Therefore, these equations make sense only in a certian scope, if you will: when we’re considering how to turn some sort of commodity into the satisfaction of our need. This is what produces this relationship between food and coats. But we have a layer of abstraction here: as we said, the exchange value isn’t the actual, underlying use values. Like all abstractions, this one leaks. We’ll get to the leak later.</p>\n<p>But in order to turn some food into a satisfied need, you need to do some kind of labor. First of all, this food had to actually be produced by someone. We’re not ordering raw materials, we’re ordering a finished product. Someone, somewhere, had to obtain these raw materials, and then apply themselves. And if you’re buying raw materials, someone had to do the labor to cut down the trees, or mine the ore.</p>\n<p>This is an area where I’m going to make a slight hand-wave for now, and argue that this aspect, the labor that’s needed to make and consume a commodity, is the underlying aspect missing from the equation. It might be a little bit of labor, it might be a lot of labor, but somewhere, someone is doing some kind of labor. We can make a better argument here, but for now, I’m just going to leave it as is, because I think it makes intutive sense. Many, many people go to work, and labor to produce things that get exchanged for many other things. The common thread is that we all work. We can dig into a more rigorous argument over this point later.</p>\n<p>In other words, we’re trading an abstraction over people’s labor. When I say “I’ll trade you a coat for four food,” I’m making that estimate on how much labor I would need to aquire a coat, and estimating how much to get four food, and deeming them equal. But we’re not trading the hours directly: we’re trading objects. We objectify our labor, and then trade that.</p>\n<p>Now, you might have a problem with this. And many economists do. However, their arguments are usually based on a misunderstanding of this point, which is a very important one: If value is based on how many hours of labor someone puts into something, than isn’t a slow worker making more valuable stuff than a fast worker? How does that make any sense? And if I were to do some labor that’s useless, like for example, I dig around in my backyard and make a bunch of pies out of the mud, I did some labor, and therefore, shouldn’t I get paid?</p>\n<p>Early economists such as Adam Smith often had this problem with their models. And it’s indeed a problem. But there’s a solution: we’re not talking about just any old labor. We’re only talking about labor that produces something useful. “Useful” in this case is decided by the particular society in which this person is doing this labor. A carriage maker in 1890 does a lot of productive labor, but a carriage maker in 2015 does significantly less.</p>\n<p>Our revised statement: the common thread between all of the commodities that get exchanged is that somewhere, sometime, someone had to do some sort of labor that was socially useful to produce that commodity. We exchange commodities so that we can gain the commodities we need to fulfill our desires. We exchange them at a ratio where they’re equal in terms of how much labor went into making them, a fair trade.</p>\n<p>This leads to another problem: how do we qualify ‘how much labor’? What is this unit and magnitude? Well, many people have a 40 hour week, so naturally, we gravitate towards ‘hours’ as an answer for this question. Hours are a unit of time, so we can quantify our exchange values as ‘socially neccesary labor time.’ Time, becuse we work in hours. Labor, because that’s what we’re doing. “Socially neccesary” to explain the context of only labor that’s doing something society deems useful.</p>\n<p>But back to one of the original problems: If I’m bad at making coats, and I take a hundred hours to make a coat, and you’re good at making coats, so it takes you an hour, is my coat worth one hundred times yours? That doesn’t make any sense. So we’re going to make an assumption. When we use ‘socially neccesary labor time’ in this way, we mean that it’s an average hour, by an average skilled worker, across all of society. Because averages are all you can really reason about here, it’s impossible to compare each individual worker. It’s an approximation. And understanding that it’s just an approximation has led to many successful businesses, as they’re more efficient than the average, and therefore, make more money. For now, we’re going to set that aside, however. We’ll get there.</p>\n<p>So! To recap, we have needs and desires, and to fulfill those, we need certain commodities, which we can aquire through exchanging commodities we have with others in our society who have the commodities we want. We make equal trades, in a ratio roughly determined by the average amount of time it would take an average worker in our society to make the thing.</p>\n<hr />\n<p>Exchange values are a bit unweildy, though. For one thing, there’s a ton of them: they explode in a <code>O(n^2)</code> fashion. If we have coats and food, each commodity has a single exchange value, but if we add just one more item, each commodity can be traded for two others, and therefore has two exchange values, times the three commoddities, meaning our system has six exchange values, five more than before. Adding a fourth means each of our four commodities has three possible trades, meaning twelve exchange values, six more than before. So what’s a society that’s trying to produce use-value to do?</p>\n<p>We can add more things to our equation. We have coats and food in our economy, let’s add a third, and call it ‘coins.’ We’ve established that one coat is worth four food, and let’s say that one coin is worth two food. This means that four food is worth two coins is worth one coat:</p>\n<pre><code>4 * food = 2 * coins = 1 * coat\n</code></pre>\n<p>In another sense, one coat is worth two coins is worth four food:</p>\n<pre><code>1 * coat = 2 * coins = 4 * food\n</code></pre>\n<p>We have coins in the middle of both of these equations, which means that we can express the two halves in one system:</p>\n<pre><code>1 * coat = 2 * coins\n4 * food = 2 * coins\n</code></pre>\n<p>And when we add another commodity to our equation, we can just say what it’s worth in coins, rather than what it’s worth in both coats and food. This makes it significantly easier to conduct trade, as we only have to know what our commodities are worth in terms of this particular commodity. This commodity plays a special role in our little society: it’s the commodity that we use to determine the rates of all the other commodities. The vocabulary term for this is ‘the money commodity.’ And we call the exchange value of a commodity to the money commodity a ‘price.’</p>\n<p>At this point, we’re going to take a small detour. Because our commodity is an abstraction of socially neccsary abstract labor time, I want to talk for a moment about how that abstraction leaks. Becuase objects fulfill our needs, and trading them is such an important part of society, we place a ton of emphasis on this abstraction. In other words, we love to buy things. Why does this happen? What’s the weird obsession, or some might say, fetish, with commodities?</p>\n<p>Let’s say you’re a cobbler, and you make shoes. Hopefully, you’re proud of the shoes you make, and so, to some degree, because you’ve put hard labor into those shoes, you see them as a reflection of your own self. And when every commodity is priced equivalant to the money commodity, it’s really easy to see that your shoes trade at $50, but mine may trade at $45. This means you make better shoes, and so you have a higher social standing than I do. Your labor is more socially useful. This sort of collective desire we all have that sets the price of commodities, and can make us quite obsessed. And not just in the classical materialist way, but in a way that shapes our worldview.</p>\n<p>You can’t take a microscope to a coat and observe that the coat costs $2. There’s no ‘money atom.’ It’s not some inherent property of coats. It’s something that our society has created as a basic assumption. But it doesn’t have to be that way, we’ve just chosen it.</p>\n<hr />\n<p>Let’s talk a bit more about exchanging stuff. Let’s use the previous exchange as an example, I have a coat, you have four food. But now, we don’t know each other. What do we do? Well, I can take my coat to a market, and sell it for two money. You can take your food to market, and sell it for two money. I can then buy your food for two money, and you can buy my coat for two money. We’ve exchanged our food and coats succesfully. We both completed a trade in the same way:</p>\n<pre><code>commodity -> money -> commodity\n</code></pre>\n<p>or</p>\n<pre><code>C -> M -> C'\n</code></pre>\n<p>We both had a commodity, we traded it for the money commodity, and then we traded that money commodity for a different commodity. There’s two sides to these trades: a commodity for the money commodity, and then money commodity for the commodity.</p>\n<p>It’s important we’re making an assumption here, but it’s one that virtually all economists make: we’re assuming that both of us know the value, and that the intermediary who is holding the money agrees. This simplifying assumption is also a place where many business opportunities are formed, charging a fee to facilitate transactions like this. See the pattern yet?</p>\n<p>That’s also something new: there’s an intermediate here, ‘the market.’ We don’t have to know each other, we just have to know the market exists. Software engineers would say that by lowering coupling, we enable scale. When everyone doesn’t need to know each other, we can trade a lot more things. This is another reason why the money commodity is so damn useful.</p>\n<p>Anyway, it’s important to recognize that this is four total exchanges, which form two little circuts, which form our overall economy here. It’s a system, like any other. And so we can model it like a system. This is something that we virtually all agree upon. But this assumption is treacherous too: we assume that both ends of the trade always happen. But there are four trades here, and only some of them may occur. This causes disruption in the system, which throws it into crisis. We’ll talk about this case more later.</p>\n<p>A moment ago, I mentioned a business model for charging for transactions. This is a very natural reaction to systems like these: by intervening in each trade, by being the mediator, you can argue that you are doing socially neccesary work, and therefore, should be paid in compensation.</p>\n<p>Furthermore, since the money commodity mediates all exchange, there’s lots of good reasons to control said mediation. This is why states print currency and then ensure that transactions occur in said currency, as it gives them control over all exchange, and therefore, of what is ‘socially neccesary.’</p>\n<p>To summarize up to this point: people have needs. To satisfy those needs, some people do labor to make commodities, which have the ability to satisfy those needs. We call one of these commodities ‘money’, and people trade commodities for it, and it for other commodities. This gets them the things they want. The rate that all of this is exchanged at is based in how much time it takes to make, and how important it is to the society overall.</p>\n<p>I have yet to say what any of this has to do with <em>capitalism</em>, however. This is just people trading things. Money, and exchange, have been around longer than capitalism. It’s something a bit different. Before we can get into that, though, we have to define the ‘capital’ in ‘capitalism.’</p>\n<hr />\n<p>There is a distinction between ‘money’ and ‘capital.’ Capital is a special kind of money. In other words, it holds a special, social distinction about how it’s used. Let’s examine our little trade again:</p>\n<pre><code>commodity -> money -> commodity\n</code></pre>\n<p>Someone has to actually have that money in the middle. To simplify our story, let’s say this shop both buys and sells various commodities. I sell my coat to the shopkeeper, you sell your food to the shopkeeper, then I buy food from the shopkeeper, and you buy a coat from the shopkeeper. From the shopkeeper’s perspective, these trades are backwards:</p>\n<pre><code>money -> commodity -> money\n</code></pre>\n<p>The keeper has four dollars. They give two to me in exchange for the coat, and then they sell the coat for two dollars on the other side. Their circut here is the opposite of yours and mine: they start and end with money, rather than with another kind of commodity.</p>\n<p>It’s important to note that this difference is significant. While you are attempting to exchange a commodity for another to gain some kind of use, with the <code>M -> C -> M</code> variant, you’re attempting to gain more exchange value. This underlying need is at odds with yours and mine. No concern is given for use here, just how much something can be exchanged for.</p>\n<p>However, as we know, a shopkeeper isn’t exactly happy when they start off with four dollars and end up with four dollars. While this system is balanced, they’re not being compensated for this social function, one of shopkeeper. And in the real world, shopkeers do make money, so something is missing from our model of equal exchanges. But before we get into that, consider the implications of this point: the shopkeeper has no concern for what is useful, just for what can be exchanged. If we optimize for the needs of shopkeepers, we may not be optimizing for the usefullness of society, but instead for the pursuit of exchanging the money commodity.</p>\n<p>So, if we can engineer a situation in which we have some money, we exchange it for a commodity, and then we exchange that commodity for money, and we get <em>more</em> money out of it in the end, that extra money is called ‘capital’. In line with our earlier equations:</p>\n<pre><code>M -> C -> M'\n</code></pre>\n<p>The difference between <code>M'</code> and <code>M</code> is how much money we’re making. This is very close to our <code>M -> C -> M</code> circut, and very different than our <code>C -> M -> C</code> circut. Only shopkeepers produce capital, not those who are simply exchanging a commodity in the hopes of attaining another.</p>\n<p>But we’ve already established that all trades here are equal. So how can we get <code>M'</code>? Shoudn’t that be impossible?</p>\n<p>This is where the mistake we mentioned before rears its ugly head. If all value is subjective, and exchanges are between equals, where can this extra money come from? And if all of these exchanges are equal, then why do both people feel like they’ve won?</p>\n<hr />\n<p>In order to make this happen, our prospective capitalist shopkeeper needs to find a commodity whose use-value, the act of consuming the commodity, produces value. If they can, they can purchase that commodity, and then use it to get more value than it was exchanged for, due to the extra production. In other words, they don’t need to purchase an object, but something more abstract: a potential source of value.</p>\n<p>Previously, we mentioned that the common thread behind exchanges was labor, and so, that labor was the common element of value. Labor, when placed on a market, is called ‘labor power,’ and is a commodity that produces value. We call someone who possesses labor power a worker. They enter into the market, selling their labor power. This is colloquially known as ‘getting a job.’ In order to be willing to sell your labor on the market, though, you need to have some sort of need that is fulfilled. If you don’t need anything, there’s no reason to get a job. Furthermore, you can’t just do work yourself, you need objects to do your work, that will allow you to produce something. We call these objects the ‘means of production.’ A worker also needs enough commodities to keep themselves alive, to work another day. We call this the ‘means of subsistence.’ It’s also important to note that if a worker has other commodities, they could sell those to fulfill their needs, rather than selling their labor power. So someone selling their labor-power in the market needs something, but doesn’t have anything. The only thing they have is the labor-power their body possesses.</p>\n<p>It’s important to note that “person with a pile of money” and “person possessing nothing but their ability to work” is not the state of nature. In other words, this stuation arises due to the way we’ve organized society. We like to think that this existance is somehow ‘natural’ or inevitable, but that’s not true. This doesn’t mean older forms of organization are better than what we have now, but the possiblity for change exists. Some people try to paint a picture that those who have the piles of money are virtuous, and those who have only their labor are at fault for their predicament. But the predicament wouldn’t even exist in the first place if not for certian historical developments.</p>\n<p>So, if you’re trading labor-power in a market, how do you determine how much to sell it for? Like any other commodity, its value comes down to a socially neccesary labor time. In this case, it’s the amount of money you need for subsistence, the money to pay your rent, to eat food, to stay alive. Having some more is nice, but if you don’t go to work, you can’t pay your bills, and then bad things happen.</p>\n<p>One odd thing about labor-power as a commodity is that you don’t just hand it over in the way that you hand over a coat. You put the work in first, and then, eventually, the thing you made gets sold at a totally different time. There’s a gap. Often, you don’t actually get to see the fruits of your labor, as someone else is actually doing the sales. This can lead to some angst, but I just want to make mention of this, we’ll talk more about it later.</p>\n<hr />\n<p>This commodity, labor-power, allows us to create the trick. Here’s how the story goes:</p>\n<p>Let’s change our example slightly: a coat costs two coins, and four spindles of thread costs two coins:</p>\n<pre><code>1 * coat = 2 * coin\n4 * thread = 2 * coin\n</code></pre>\n<p>It takes two spindles of thread and an hour to make a coat. By this measure, given that a coat costs two coins, you’d imagine that cost of materials is a coin, so labor must be a coin. Which means an hour of work is worth a coin. if this is true, then the exchange looks like this:</p>\n<p>Someone wants a coat. They’re willing to pay two coins. A prospective capitalist sees that the cost of materials is a coin, so they set the price of labor-power to one hour per coin:</p>\n<pre><code>1 * coat = 2 * coin\n2 * thread = 1 * coin\n1 * hour of work = 1 * coin\n</code></pre>\n<p>I spend my one hour of labor and two threads to make a coat, which costs two coins and is sold for two coins. We’re back at an equilibrium. There’s still no capital being created here.</p>\n<p>Now the capitalist is upset. “I provided the job, I provided the thread, yet I made no money. This worker couldn’t have made that coat without me. I deserve to be paid for having the initial capital to set up this business. That’s work!”</p>\n<p>But here’s the thing: if I worked for half a coin instead of a full coin for an hour, and I worked eight of those hours a day, I’m still making more than enough money to pay my bills. Because that’s the outcome I’m shooting for, I’m happy to exchange my labor on the market for less than the labor time would be: I’m making a good salary, I can live a good life. So the <em>exchange value</em> of my labor-power is less than the value I create from it. It’s not that I’m being cheated with my wages, it’s just that the exchange value of my labor is worth less than the labor I create. With this set of equations:</p>\n<pre><code>1 * coat = 2 * coin\n2 * thread = 1 * coin\n2 * hour of work = 1 * coin\n</code></pre>\n<p>the capitalist can extract that difference. He takes his initial four coins, invests two of them in four threads. He then invests a coin in two hours of a worker’s labor. That worker then uses the threads and their labor-power to make two coats. The capitalist then sells the two coats for four dollars. Looking back, he’s seemingly played a trick: he invested three coins, two in threads and one in labor, but gained four coins at the end, for the two coats.</p>\n<p>We haven’t broken any rules of exchange here. The capitalist has paid full market price for all commodities, there’s no inherent need to pay less. Each exchange is between understanding, consenting parties. There’s no trickery. Yet an extra coin has moved from one party to another, the worker to the capitalist, just by following the rules.</p>\n<p>Once you have a machine that you can put three dollars in and get four dollars out, you can live off of this extra, rather than doing labor yourself. By having the initial capital of three dollars, and by using it in this way, a capitalist can just make money out of thin air. As they say, it takes money to make money. This is why every worker isn’t a capitalist, you need to create this capital somehow, to be able to afford to use the machine. A machine that makes four dollars out of three isn’t much use if you dont have the inital three dollars.</p>\n<hr />\n<p>In summary: people come together at a market to exchange commodities to fulfill their needs. They exchange these commodities with each other in equal values. One of the commodities which can be exchanged is labor-power, which is a special commodity, because it creates more value. Capitalists purchase this commodity, and then use it to create capital, which they can re-invest to continue to accumulate money. Those who sell their labor-power on the market must because they don’t have the capital to do this themselves.</p>\n<p>Now that we’ve established that this system is off-balance, we can figure out how that will eventually become its undoing.</p>\n<hr />\n<p>Anyway, this is the beginning. There’s obviously much more detail to go into here. If you’d like to read more, this post is basically my own little summary of <a href=\"https://www.marxists.org/archive/marx/works/1867-c1/\">Capital, Volume I</a>, chapters 1-7, by Karl Marx. These basic building blocks of understanding are not without their own problems, but hopefully, I’ve given you a foothold into the topic. I may or may not do this for additional chapters, we’ll see….</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/a-new-introduction-to-rust/",
"title": "A new introduction to Rust",
"description": null,
"url": "https://steveklabnik.com/writing/a-new-introduction-to-rust/",
"published": "2015-02-27T00:00:00.000Z",
"updated": "2015-02-27T00:00:00.000Z",
"content": "<p>Lately, I’ve been giving a lot of thought to first impressions of Rust. On May 15, <a href=\"http://blog.rust-lang.org/2015/02/13/Final-1.0-timeline.html\">we’re going to have a lot of them</a>. And you only get one chance at a first impression. So I’ve been wondering if our <a href=\"http://doc.rust-lang.org/intro.html\">Intro</a> and <a href=\"http://doc.rust-lang.org/book/basic.html\">Basics</a> are putting our best foot forward. At first I thought yes, but a few days ago, I had an idea, and it’s making me doubt it, maybe. So instead of re-writing all of our introductory material, I’m just going to write the first bit. A spike, if you will. And I’d like to hear what you think about it. This would take the same place as <a href=\"http://doc.rust-lang.org/book/variable-bindings.html\">2.4: Variable bindings</a> in the existing structure: They’ve installed Rust and gotten Hello World working.</p>\n<pre><code>$ rustc --version\nrustc 1.0.0-dev (dcc6ce2c7 2015-02-22) (built 2015-02-22)\n</code></pre>\n<h2>Hello, Ownership</h2>\n<p>Let’s learn more about Rust’s central concept: ownership. Along the way, we’ll learn more about its syntax, too. Here’s the program we’re going to talk about:</p>\n<pre><code>fn main() {\n let x = 5;\n}\n</code></pre>\n<p>This small snippit is enough to start with. First up: <code>let</code>. A <code>let</code> statement introduces a <strong>variable binding</strong>. Bindings allow you to associate a name with some sort of value.</p>\n<p>Why ‘variable binding’? Rust draws heavily from both systems languages and functional programming languages. The name “variable binding” is a great example of this. Many systems languages let you declare a variable. These variables are called by that name because they can change over time, they’re mutable. Many functional languages let you declare bindings. These bindings are called by that name because they bind a name to a value, and don’t change over time. They’re immutable.</p>\n<p>Rust’s variable bindings are immutable by default, but can become mutable, allowing them to be re-bound to something else. In other words,</p>\n<pre><code>fn main() {\n let x = 5;\n x = 6; // error: re-assignment of immutable variable\n\n let mut y = 5;\n y = 6; // just fine\n}\n</code></pre>\n<p>You won’t be typing <code>mut</code> that often.</p>\n<p>In any case, there’s one way in which <code>let</code> bindings work just like variables in other languages, but they’re the key insight into ownership. As you know, a computer program is executed line by line. At least, until you hit a control flow structure, anyway. Let’s give our program line numbers, so we can talk about it.</p>\n<pre><code>// 1\nfn main() { // 2\n // 3\n let x = 5; // 4\n // 5\n} // 6\n // 7\n</code></pre>\n<p>Line one is before our program starts. The endless void. Not actually endless, though, as line two is where we start <code>main</code>. This is the first line that is actually executed, and kicks off our program. Great. Line three is blank, so nothing happens, just like one. Line four is where the first actually interesting thing occurs: we introduce a new variable binding, <code>x</code>. We set <code>x</code>’s initial value to five, allocated on the stack. If you don’t know what that means, we’ll talk about it right after ownership. For now, <code>x</code> is five. No big deal. Line six has a closing curly brace, and so <code>main</code>, and thus, our program, is over. Line seven, the void returns.</p>\n<p>This is basically the same thing as many programming languages. But let’s point out an aspect you, as a programmer, probably take for granted. Scoping. If I asked you, “Is <code>x</code> valid on line one?” you would say “no.” “Three? Seven?” “Nope, nada. <code>x</code> is valid from line four, where it was declared, to line six, where it goes out of scope.” This illustrates the idea. There is a certain scope, a certain set of lines, where <code>x</code> is a valid identifier. That scope starts from where it was declared, and goes until the end of the block. We can look at this scope in two ways: for the first, imagine this source code printed on a piece of paper. You highlight lines four three six. In some sense, this is a distance: three lines of code, rather than three meters. But if we imagine the computer running this program, this scope represents a time: three statements of processor execution. Even though that number is actually different based on the assembly, but at our level of abstraction, three units.</p>\n<p>In Rust, we have names for these concepts, which are implicit in other languages. The thing that introduces a new scope is called the ‘owner.’ It’s in charge of the data it’s bound to, so we say that it ‘owns’ that data. The length of a scope is called a ‘lifetime,’ taken from that idea of time passing as your program executes. But you can also think of it as a segment of lines of code.</p>\n<p>So if other programs do this, why does this make Rust special? Sure, Rust gives these concepts specific names, but names themselves aren’t significantly different. The difference is that Rust takes this concept and cranks it up to 11. Where most programming languages only keep track of how long variables are in scope, Rust knows how to connect the scopes of variables that are pointing to the same thing, as well as how to know the scope of things that are more than just stack-allocated memory.</p>\n<p>Let’s talk more about this connecting of scopes. Here’s another Rust program:</p>\n<pre><code>fn main() { // 1\n let x = 5; // 2\n // 3\n { // 4\n // 5\n let y = &x; // 6\n // 7\n } // 8\n // 9\n} // 10\n</code></pre>\n<p>In line four, we use an open curly brace to create a new scope. This scope, like in many languages needs to be closed before the scope of <code>main</code> gets closed. It’s nested inside of it.</p>\n<p>In this new scope, on line six, we declare a new binding, <code>y</code>, and we set it equal to <code>&x</code>. This reads as “a reference to <code>x</code>,” as <code>&</code> is the reference operator in Rust. References are Rust’s version of ‘pointers’ from other systems programming languages. Here’s a quick introduction, if you haven’t had to deal with pointers before.</p>\n<p>By default, in Rust, data is allocated on “the stack.” Your program is given a chunk of memory, and it’s able to allocate information there. The stack starts at the bottom address of memory, and then grows upwards. For example, at the start of our program, the stack has nothing in it:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20e35b5948ec7546b7a905f4db1ea869c6.csv\">Untitled</a></p>\n<p>At line2, we introduce <code>x</code>, and it stack-allocates the number five, like this:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20935c84280787447b8fefcaaefb4575c0.csv\">Untitled</a></p>\n<p>We’ve got an address in memory, the value at that address, and the name we’ve used in our program. It starts at the bottom because remember, the stack grows upward. Finally,on line six, we have <code>y</code>:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20f7b25102df1d41e884fdac5b761fe567.csv\">Untitled</a></p>\n<p>Instead of being a value itself, <code>y</code> is a pointer to another location in memory, which holds a value. Because <code>y = &x</code>, we take the memory location of <code>x</code> and store it into <code>y</code>.</p>\n<p>On line eight, the inner scope ends, and <code>y</code> goes out of scope. Its lifetime is over. It gets deallocated. Our memory is back to this:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20daa49962f5c5408d82e28e9e3e073be7.csv\">Untitled</a></p>\n<p>Finally, on line ten, our program is over. <code>x</code> goes out of scope, and we look like this:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%2086c230f1a23e49d892a529d7988a272c.csv\">Untitled</a></p>\n<p>Everything is gone, all is well.</p>\n<p>Pointers can point other places than the stack: they can also point to ‘the heap.’ The heap starts at the top of your program’s memory, and grows down:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20007add295d434d84a281a93e7358f0d5.csv\">Untitled</a></p>\n<p>The heap starts at <code>0x00</code> and grows down, and the stack starts at <code>0xff</code> and grows up. While the stack contains the data that the programming language has allocated for our bindings, our program can request dynamic amounts of memory from the heap. Here’s how to do this in C:</p>\n<pre><code>#include<stdio.h>\n#include<stdlib.h>\n\nint main() {\n int *x = malloc(sizeof(int)); // 1\n *x = 5; // 2\n // 3\n printf(\"%i\", *x); // 4\n}\n</code></pre>\n<p>The call to <code>malloc</code> (‘memory allocate’) on line 1 requests some memory from the heap. We get a pointer, <code>int *x</code>, to that memory. Our memory looks like this:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%203e8198d0b0a5408bbcf399a9d682c779.csv\">Untitled</a></p>\n<p>We have <code>x</code>, which is a variable on the stack, as a pointer to our dynamically allocated memory, which is located on the heap at <code>0x00</code>. But at that location, we haven’t done anything, so who knows what the value is? This means that after line one is done executing, <code>x</code> will be pointing to garbage memory. Hopefully we set it equal to a value before we try to use it, or bad things could happen.</p>\n<p>On line two, we set that memory location to five. Whew! Memory looks like this:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20ccd2c85eb0a34f6483acdbfafa4c841a.csv\">Untitled</a></p>\n<p>Everything looks good. On line four, we have to print out <code>*x</code>, not <code>x</code>: <code>x</code> is the address of the memory, so we’d get <code>0x00</code> instead of <code>5</code>. After line four, <code>x</code> goes out of scope, and gets deallocated. Memory looks like this:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%200aec19c2d6a645a3b860bf65cc922853.csv\">Untitled</a></p>\n<p>… wait a minute, we never took care of that <code>5</code>! We left some memory ‘dangling.’ In this program, that’s okay, because now that it’s over, the operating system reclaims all this memory, and so we don’t have any problems. But we <em>do</em> have a bug. We just don’t know it. This bug will sit there until something changes, and the bug is found. But we’ll get to that.</p>\n<p>A slightly more responsible program looks like this:</p>\n<pre><code>#include<stdio.h>\n#include<stdlib.h>\n\nint main() {\n int *x = malloc(sizeof(int));\n *x = 5;\n\n printf(\"%i\", *x);\n\n free(x);\n}\n</code></pre>\n<p>The call to <code>free</code> releases the memory pointed to by <code>x</code>, and all is good.</p>\n<p>Let’s compare this to an equivalent Rust program:</p>\n<pre><code>fn main() {\n let x = Box::new(5);\n\n println!(\"{}\", x);\n}\n</code></pre>\n<p>It looks similar at first, with <code>Box::new</code> filling in for <code>malloc</code>, But where’s the free? Does this have the same bug?</p>\n<p>No. This Rust is in fact equivalent to the one with the <code>malloc</code> and <code>free</code>. You see, when we introduce our binding, <code>x</code>, Rust knows that <code>x</code> has ownership over the memory in the <code>Box</code>. and so, at the end of the program, when <code>x</code> goes out of scope, Rust also ensures to free the memory that the <code>Box</code> allocated. It happens automatically. The programmer can’t forget it. And the allocation itself was a little easier, because Rust used the type system to figure out how many bytes to allocate, you didn’t need to calculate it with <code>sizeof</code>. Allocation also set the value to 5 for you, which is convenient and less error-prone. When printing out <code>x</code>, you could just use the binding name itself, and not need to dereference it with <code>*</code>.</p>\n<p>C++ programmers have ways of doing this kind of thing, with something called “RAII.” As promised before, Rust knows this concept at a deeper, language level. So it can be safe in places that RAII cannot. Here’s an example, in C++:</p>\n<pre><code>#include<iostream>\n#include<vector>\n#include<string>\n\nint main() {\n std::vector<std::string> v;\n\n v.push_back(\"a\");\n\n std::string& x = v[0];\n\n v.push_back(\"b\");\n\n std::cout << x;\n}\n</code></pre>\n<p>This program creates a vector of strings, called <code>v</code>:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20b1e4993d4fc14c9abbe2c2d7deb3be25.csv\">Untitled</a></p>\n<p><code>v</code> is a ‘fat pointer,’ three pointers in one. The first is a pointer to the data, the second is the length of the array, how many elements are in it. The third is the capacity of the array, how much memory is allocated for this vector. The amount of initial capacity is up to the implementation, I’ve chosen one here. Because this is a vector of strings, we need to allocate a single string, so in line with our capacity. Strings are also a fat pointer, and so we allocate three bytes, starting at <code>0x00</code>.</p>\n<p>The program then calls <code>push_back()</code> on the vector, which pushes a string onto the end of the vector:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%205c80884c8d7d4ae290de9e2b9caa1f11.csv\">Untitled</a></p>\n<p>We’ve now replaced our initial, <code>GARBAGE</code>, string with a real one. Therefore, our data, length, and capacity are updated: <code>0x03</code> is the next free location in the heap, so it gets our signle character, <code>A</code>. We have a length of one, and a capacity of one, so the other two values are set appropriately.</p>\n<p>Next, it creates a pointer to the <code>0</code>th element of that vector:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%2031184db665934f5ea7acaea0b98ce4b9.csv\">Untitled</a></p>\n<p><code>x</code> is allocated on the stack, and points to the first element of <code>v</code>, which, as we know, is located at address <code>0x00</code>, as its data pointer, <code>0xff</code>, shows us. So <code>x</code> has the initial value of <code>0x00</code>.</p>\n<p>Next is where everything goes wrong. It tries to push a string onto the end of that vector. The problem is, we only allocated one element. So we need to allocate more. After this is done, memory looks like this:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20900c132e9e144b1e94ffb6c47d5fc068.csv\">Untitled</a></p>\n<p>Inside <code>push_back()</code>, there’s a call to <code>malloc</code>, and that is assigned to some temporary value. We’ve chosen <code>0xfb</code> and <code>(temp)</code> to identify it here. It points to the newly allocated data, at <code>0x04</code>.</p>\n<p>Now that we’ve got space for two strings, let’s copy the data from the original place to the new one, from <code>0x00</code> to <code>0x04</code>:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%208311f6138fd34b39840f66e8b047db11.csv\">Untitled</a></p>\n<p>and then free that original string:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20885b35ff100341e6819740529f0d4701.csv\">Untitled</a></p>\n<p>Then, we have to update <code>v</code> to point at the newly allocated vector, at <code>0x04</code>:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20ec53dd46647143b58bb3c7b136d66a59.csv\">Untitled</a></p>\n<p>This step is where the bug is! You see, <code>x</code> was pointing at the same data that <code>v</code> was pointing at. This is called ‘aliasing.’ Because C++ does not have the concept of ownership and lifetimes, it doesn’t understand that the validity of <code>x</code> and the validity of <code>v</code> are closely intertwined. Thus, we’re allowed to do this reallocate-and-copy dance with what <code>v</code> points to, but <code>x</code> has no idea. Look at the diagram above: it still has <code>x</code> pointing to <code>0x00</code>. But we just deallocated that memory! This is when the bug occurs, but it’s a silent bug. A time bomb, ticking away.</p>\n<p>Let’s see how it explodes. Our code goes on, oblivious.</p>\n<p>Our new string at <code>0x04</code> still says that we have one capacity, one length, and that data is at <code>0x03</code>. That’s all still accurate, so we’re all good. Next, we need to deal with our new string, “B.” We have the space, so we do so:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20cc5b92de5ac545878c67a66a878863b6.csv\">Untitled</a></p>\n<p>We’ve placed our new string at <code>0x07</code> to <code>0x09</code>, with its data pointer, length, and capacity. Our data is at <code>0x0A</code>, which has the value <code>B</code> stored at it. All is good, and so we return from <code>push_back</code>. <code>(temp)</code> is deallocated, and memory looks like this:</p>\n<p><a href=\"A%20new%20introduction%20to%20Rust%204bbc792c85e141fb81570865cbbe2f87/Untitled%20Database%20ee6def6517424e0fb6c9ad7c3ec0389e.csv\">Untitled</a></p>\n<p>On the final line, we print out the value that <code>x</code> points to. As we can see above, <code>x</code> points to <code>0x00</code>, which has nothing saved at it! <code>x</code> is invalid memory. Using <code>x</code> is a bad idea. But the programming language can’t save us, even though the bug happened many lines ago.</p>\n<p>How are you supposed to deal with this problem? Here’s <a href=\"http://en.cppreference.com/w/cpp/container/vector/push_back\">the documentation for <code>push_back</code></a>:</p>\n<blockquote>\n<p>If the new size() is greater than capacity() then all iterators and references (including the past-the-end iterator) are invalidated.</p>\n</blockquote>\n<p>In other words, when you call <code>push_back</code>, it’s your responsibility to make sure to not use anything that points to the underlying data, because they are now invalid. The documentation has explained our bug perfectly! And while this is a short example, these problems can crop up in huge codebases, and the errors can be very hard to track down.</p>\n<p>Let’s look at the same problem, in Rust:</p>\n<pre><code>fn main() {\n let mut v = vec![];\n\n v.push(\"Hello\".to_string());\n\n let x = &v[0];\n\n v.push(\"world\".to_string());\n\n println!(\"{}\", x);\n}\n</code></pre>\n<p>This program has identical semantics to the C++ version presented above. (If we weren’t trying to get the exact same semantics, we could leave off the <code>.to_string()</code> calls, and it would still work. But let’s ignore that for now.) So does it have the same bug? Well, it would, but it doesn’t compile:</p>\n<pre><code>main.rs:8:5: 8:6 error: cannot borrow `v` as mutable because it is also borrowed as immutable\nmain.rs:8 v.push(\"world\");\n ^\nmain.rs:6:14: 6:15 note: previous borrow of `v` occurs here; the immutable borrow prevents subsequent moves or mutable borrows of `v` until the borrow ends\nmain.rs:6 let x = &v[0];\n ^\nmain.rs:11:2: 11:2 note: previous borrow ends here\nmain.rs:1 fn main() {\n...\nmain.rs:11 }\n ^\n</code></pre>\n<p>When you call the method <code>push()</code> on a vector, it ‘borrows’ the vector. Just like we wrote <code>let y = &x</code> before, to borrow the value of <code>x</code>. Rust now understands that the scope of <code>x</code> and the scope of <code>v</code> are connected. There’s just one problem: because <code>push</code> allocates more memory, it can’t just do a regular borrow, it needs a mutable borrow. If <code>&x</code> borrows x, <code>&mut x</code> mutably borrows <code>x</code>. Like we discussed previously, mutability means that you can reassign the value of that binding. But when we mutably <em>borrow</em> something, we make a guarantee: this is the only reference to this memory that exists. That way, we don’t have the problem of the C++ code, where a reference became outdated. If we’re the only reference, we are free to change what we’re pointing to, becuase that won’t cause any problems.</p>\n<p>So when we create a borrow with <code>&</code>, it lasts for the scope that it exists, like this:</p>\n<pre><code>fn main() {\n let mut v = vec![];\n\n v.push(\"A\"); \n\n let x = &v[0]; // <+\n // |\n v.push(\"B\"); // |\n // |\n println!(\"{}\", x); // |\n} // <+\n</code></pre>\n<p>The line indicates the scope, or lifetime, of the borrow. But when we call push with <code>B</code>, we have to get a mutable borrow, in order to allocate the new memory. But because <code>&mut</code> is a promise that there are no other references, this is a violation of the rules. This is the point at which the bug happened in the C++ example, if you remember. And so it’s the place where the Rust compiler throws its error message. It might make more sense this time around:</p>\n<pre><code>main.rs:8:5: 8:6 error: cannot borrow `v` as mutable because it is also borrowed as immutable\nmain.rs:8 v.push(\"world\");\n ^\nmain.rs:6:14: 6:15 note: previous borrow of `v` occurs here; the immutable borrow prevents subsequent moves or mutable borrows of `v` until the borrow ends\nmain.rs:6 let x = &v[0];\n ^\nmain.rs:11:2: 11:2 note: previous borrow ends here\nmain.rs:1 fn main() {\n...\nmain.rs:11 }\n ^\n</code></pre>\n<p>We cannot borrow <code>v</code> as mutable, with <code>push()</code>, becuase it is also borrowed as immutable, by <code>x</code>. That’s exactly the problem, and Rust solves it.</p>\n<p>There’s more to both ownership and Rust syntax, so let’s cover some syntactical things, and then talk more about ownership.</p>\n<hr />\n<p>I’m fairly confident with the memory layouts involved, though the addresses are of course not exact. I’m also not sure that this mega deep detail dive is the correct way to do an introduction either, it’s also thorough, but maybe in a bad way? It’s almost the exact opposite of the way that I’ve done things at present. Also, apparently I visuallize things in reverse, with <code>0x00</code> to <code>0xFF</code> as going down, but others think the opposite, which may be confusing. Please send me a <a href=\"http://twitter.com/steveklabnik\">tweet</a> or <a href=\"mailto:[email protected]\">an email</a> with your thoughts. Thanks!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/update-github-pages-with-travis-ci/",
"title": "Update GitHub Pages with Travis CI",
"description": null,
"url": "https://steveklabnik.com/writing/update-github-pages-with-travis-ci/",
"published": "2014-12-16T00:00:00.000Z",
"updated": "2014-12-16T00:00:00.000Z",
"content": "<p>Ever wanted to have your project automatically update Pages when you have a successful build?</p>\n<p>I have a demo repository here: <a href=\"https://github.com/steveklabnik/automatically_update_github_pages_with_travis_example\">https://github.com/steveklabnik/automatically_update_github_pages_with_travis_example</a></p>\n<p>It, of course, uses Travis to update its own Github Pages, which is here: <a href=\"http://steveklabnik.github.io/automatically_update_github_pages_with_travis_example/\">http://steveklabnik.github.io/automatically_update_github_pages_with_travis_example/</a></p>\n<p>I hope that helps!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/im-going-writeonly-on-twitter-for-a-while/",
"title": "I'm going writeonly on Twitter for a while",
"description": null,
"url": "https://steveklabnik.com/writing/im-going-writeonly-on-twitter-for-a-while/",
"published": "2014-09-03T00:00:00.000Z",
"updated": "2014-09-03T00:00:00.000Z",
"content": "<p>This is something I’ve been joking about for a while, but I’ve decided for the near future, I’m just going to be posting things to Twitter, and not reading replies or anything.</p>\n<p>So, if you don’t get a fav or a reply or something, it’s nothing personal.</p>\n<p>There’s a lot going into this decision, but basically, I haven’t been actually doing a lot of things that I want to do because I’m super stressed out. Turns out Twitter is also a huge cause of stress in my life. Turns out Twitter is also a large part of my social interaction, so I’ve kept doing it even though I know it’s not great for me.</p>\n<p>So, we’ll see what a post-read/write Twitter is like. You know how to get a hold of me otherwise.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rusts-documentation-is-about-to-drastically-improve/",
"title": "Rust's documentation is about to drastically improve",
"description": null,
"url": "https://steveklabnik.com/writing/rusts-documentation-is-about-to-drastically-improve/",
"published": "2014-06-16T00:00:00.000Z",
"updated": "2014-06-16T00:00:00.000Z",
"content": "<p>Historically, <a href=\"http://rust-lang.org/\">Rust</a> has had a tough time with documentation. Such a young programming language changes a lot, which means that documentation can quickly be out of date. However, Rust is nearing a 1.0 release, and so that’s about to change. I’ve just signed a contract with Mozilla, and starting Monday, June 23rd, I will be devoting forty hours per week of my time to ensuring that Rust has wonderful documentation.</p>\n<p>A year and a half ago, I was in my hometown for Christmas. I happened upon a link: <a href=\"https://mail.mozilla.org/pipermail/rust-dev/2012-December/002787.html\">Rust 0.5 released</a>. I’ve always enjoyed learning new programming languages, and I had vaguely heard of Rust before, but didn’t really remember what it was all about. So I dug in. I loved systems programming in college, but had done web-based stuff my entire professional life, and hadn’t seriously thought about pointers as part of my day-to-day in quite some time.</p>\n<p>There was just one problem: Rust was really difficult to learn. I liked what I saw, but there was very little of it. At the same time, I had been working on some ideas for a new toolchain for my <a href=\"http://www.designinghypermediaapis.com/\">book on hypermedia APIs</a>, but wanted to try it out on something else before I took the time to port the content. And so, <a href=\"http://www.rustforrubyists.com/\">Rust for Rubyists</a> was born. I decided that the best way to teach people Rust was to mimic how I learned Rust. And so, as I learned, I wrote. It ended up at about fifty pages of two weeks of work. I never contributed it to the main repository, because for me, it was really about learning, both about my ebook toolchain as well as Rust itself. I didn’t want the burden that came with writing an official tutorial, making sure that you cover every single feature, pleasing every single Github contributor…</p>\n<p>After learning Rust, I decided that I really liked it. No other language provided such a promising successor to C++. And I really missed low-level programming. So I kept evangelizing Rust, and every so often, contributing official documentation. I figured that even if my low-level chops weren’t up to the task of writing actual patches, I could at least help with my writing skills. I’d previously contributed lots of documentation to Ruby and Rails, so it was something that was very familiar to me. I’ve often found that I start with documentation and then move into contributing code, once I get my head wrapped around everything. Writing is part of my own process of understanding.</p>\n<p>Rust for Rubyists was a great hit, even amongst non-Rubyists (damn my love of alliteration!). Six months ago, on the eve of the first anniversary of the initial release of Rust for Rubyists, I <a href=\"https://air.mozilla.org/rust-meetup-december-2013/\">gave a talk</a> at the Bay Area Rust meetup, specifically on the state of Rust’s documentation. In it, I laid out a plan for how I envisioned docs looking in the future. In the last six months, a lot has improved, but a lot hasn’t. But no more! I’m now going to be able to allocate a significant amount of my time on getting documentation done.</p>\n<p>I’m also pleased in a meta-sense. You see, by contracting someone to work on documentation full-time, Mozilla is indicating that they take Rust and its future very seriously. You can (and I do) talk a lot of trash on Microsoft, but one of the reasons that the Microsoft platform is used by so many people around the world is that Microsoft products often have excellent documentation. I often find that open source ‘products’ are technically superior, but are often much harder to use, because they’re built by a community, for free, and very few people want to write documentation for free. Combined with the work that Tilde is doing on <a href=\"https://github.com/carlhuda/cargo\">Cargo</a>, Mozilla is investing a significant amount of effort and dollars into ensuring that Rust will be a fantastic platform for those developing on it. Since I love Rust, this makes me very, very happy.</p>\n<p>Forty hours a week is a significant amount of documentation, and I have a lot of work in front of me. But my first area of focus will be on the area of Rust’s documentation that’s the most weak, and simultaneously the most important: the tutorial. I tackled the first tip of that iceberg with <a href=\"http://doc.rust-lang.org/master/intro.html\">my 30 minute introduction</a>, and I’d like to tweak it too. The main tutorial, however, is the first place where people go to <em>really</em> learn about Rust and how it works. And the current tutorial is largely the same as the 0.5 days, back when I first learned Rust. It suffers from receiving patchwork contributions from a variety of people, rather than having one unifying vision. It’s also much more of a list of features and how to use them than a coherent tutorial.</p>\n<p>I’m really excited to get to work. Let’s all make Rust 1.0 a fantastic release.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/sf/",
"title": "SF",
"description": null,
"url": "https://steveklabnik.com/writing/sf/",
"published": "2014-05-20T00:00:00.000Z",
"updated": "2014-05-20T00:00:00.000Z",
"content": "<blockquote>\n<p>@steveklabnik <small>including you?</small></p>\n</blockquote>\n<p>— Nick Quaranto (@qrush) March 5, 2014</p>\n<blockquote></blockquote>\n<p>I was always not quite sure about moving to San Francisco. I’ve heard about this place for my entire life: back in high school, during the first bubble, I would read Slashdot and wonder about life out west. But San Francisco is like a vortex for software developers. It’s got this gravitational pull that’s hard to escape. When I made the choice to move here, I wasn’t sure if I’d be able to handle it. Turns out I can’t.</p>\n<p>So I’m leaving. My job is great, but it’s not worth staying in this city. I haven’t done anything outside of work in a few months. I go to work, work hard, go home, and then go to sleep. I don’t go home late. I go home at five. It’s not work/life balance that’s the issue. I’m just not psyched for the ‘life’ part. San Francisco has sucked out all my creativity, most of my passion, and has made me increasingly frustrated with things. Elaborating further would just be <a href=\"http://tomdale.net/2013/07/san-francisco-i-love-you-but-youre-bringing-me-down/\">repeating Tom Dale</a> and <a href=\"http://programmingisterrible.com/post/50421878989/come-here-and-work-on-hard-problems-except-the-ones\">tef</a> with not much difference, so I’ll leave it at that.</p>\n<p>So, at the end of the month, I’m leaving Balanced and the city. I’ll be arriving in Brooklyn, a place I’ve been trying to live for the past few years. I’m going to deeply miss my friends and my (ex-)coworkers, but I have a lot of friends in NYC, too. Every hour I spend in New York, I feel excited, and every hour I spend in San Francisco, I feel drained.</p>\n<p>So here’s to new adventures. I can’t wait to be happy again.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/hypermedia-fizzbuzz/",
"title": "Hypermedia FizzBuzz",
"description": null,
"url": "https://steveklabnik.com/writing/hypermedia-fizzbuzz/",
"published": "2014-05-02T00:00:00.000Z",
"updated": "2014-05-02T00:00:00.000Z",
"content": "<p>I read a really great blog post last week: <a href=\"http://smizell.com/weblog/2014/solving-fizzbuzz-with-hypermedia\">Solving FizzBuzz with Hypermedia</a>. In it, Stephen Mizell builds a FizzBuzz service using <a href=\"https://github.com/kevinswiber/siren\">Siren</a>, and then shows how client code evolves. I wanted to explain exactly why I think this example is amazing, because I’m not sure it’s exactly obvious.</p>\n<h2>FizzBuzz? Really?</h2>\n<p>The first thing that makes this post brilliant is that it uses FizzBuzz. While FizzBuzz has historically been a simple interview question to make sure that you can actually write some basic code, in this case, it works well because it’s a very, very simple programming problem. When writing examples, there’s always tension between real-world and straw-man examples. If you make it too real-world, all sorts of incidental details creep in and distract from your main point. If you make a trivial example, it can be claimed that it works for your simple example, but not for ‘real’ examples. Now, this may be true of FizzBuzz, but what I like about it is that it’s an example that everyone is already familiar with. It has <em>just enough</em> actual computing that it straddles that line.</p>\n<h2>Haters gonna HATEOAS</h2>\n<p>The real insight of this post, however, is demonstrating one of the hardest concepts to grok about the hypermedia approach: the way in which hypermedia reduces duplication, by moving application entirely to the server. This is the core of that dreaded “HATEOAS” concept, and it’s where people get tripped up. Here’s the sample client in the hypermedia style:</p>\n<pre><code>from siren import SirenResource as Hyperclient\n\nBASE_URL = \"http://fizzbuzzaas.herokuapp.com\"\n\ndef fizzbuzz(resource):\n \"\"\"\n Prints the fizzbuzz value and follows \"next\" links\n \"\"\"\n print resource.properties[\"value\"]\n\n if resource.has_link(\"next\"):\n fizzbuzz(resource.follow_link(\"next\"))\n\ndef begin_fizzbuzz(resource):\n \"\"\"\n Follows the first link, then hands off to fizzbuzz\n \"\"\"\n if resource.has_link(\"first\"):\n fizzbuzz(resource.follow_link(\"first\"))\n\nroot_resource = Hyperclient(BASE_URL, path=\"/\")\nbegin_fizzbuzz(root_resource)\n</code></pre>\n<p>Let’s talk about what this client’s goal is: to print a list of numbers. This is a classic recursive list traversal algorithm: we start at the head of the list (<code>first</code>), and then follow the links until we get to the end. (<code>next</code>) This client <em>does not actually calculate fizzbuzz</em>. It just so happens that the service we’re pointing it at calculates FizzBuzz. If we pointed it at, say, a list of search results, where the same link relations and Siren were used, it would still work!</p>\n<p>Compare this to the non-hypermedia client:</p>\n<pre><code>import requests\n\nBASE_URL = \"http://fizzbuzzaas.herokuapp.com\"\n\ndef fizzbuzz(params):\n url = BASE_URL + \"/fizzbuzz\"\n response = requests.get(url, params=params)\n response_json = response.json()\n return response_json[\"properties\"][\"value\"]\n\nfor number in range(1, 101):\n print fizzbuzz({\"number\": number })\n</code></pre>\n<p>This client doesn’t fully calculate FizzBuzz on its own, but it does have some of the logic for doing so embedded inside. We explicitly loop over the range FizzBuzz needs, and we print out each value.</p>\n<p>Why does this matter? It seems like a pretty subtle distinction. Well, this logic already exists on the server. We’re duplicating that logic here. This duplication creates coupling across the client/server boundary. For example, if we pointed this client at a “search results” resource, we would get 100 search results, even if there are less. We’re not letting the server guide our interactions, and we’re coding business logic into the client.</p>\n<p>This comes back to bite us when we try to change the client. As Stephen’s next example shows:</p>\n<pre><code>import requests\n\nBASE_URL = \"http://fizzbuzzaas.herokuapp.com\"\nSTARTS_AT = 4\nENDS_AT = 20\nADD = 2\n\ndef fizzbuzz(params):\n url = BASE_URL + \"/fizzbuzz\"\n response = requests.get(url, params=params)\n response_json = response.json()\n print response_json[\"properties\"][\"value\"]\n\n params[\"number\"] += ADD\n\n if params[\"number\"] <= ENDS_AT:\n fizzbuzz(params)\n\nfizzbuzz({ \"number\": STARTS_AT })\n</code></pre>\n<p>This version uses different start, end, and skip parameters. And so we had to throw out our entire old client, because it was too tightly coupled to the notion of how to calculate FizzBuzz. We’re now manually doing the skip and end calculations, even further coupling this client. Our fundamental algorithm even changed: the last client was iterative, but this client is recursive. We could have still made it iterative, but that still would have been a lot of change.</p>\n<p>Now , the hypermedia version:</p>\n<pre><code>from siren import SirenResource as Hyperclient\n\nBASE_URL = \"http://fizzbuzzaas.herokuapp.com\"\n\ndef fizzbuzz(resource):\n \"\"\"\n Prints the fizzbuzz value and follows \"next\" links\n \"\"\"\n print resource.properties[\"value\"]\n\n if resource.has_link(\"next\"):\n fizzbuzz(resource.follow_link(\"next\"))\n\ndef begin_fizzbuzz(resource):\n \"\"\"\n Follows the first link, then hands off to fizzbuzz\n \"\"\"\n if resource.has_link(\"first\"):\n fizzbuzz(resource.follow_link(\"first\"))\n\ndef custom_fizzbuzz(root_resource, params):\n \"\"\"\n Submits actions for custom fizzbuzz\n \"\"\"\n resource = root_resource.take_action(\"custom-fizzbuzz\", params)\n begin_fizzbuzz(resource)\n\nroot_resource = Hyperclient(BASE_URL, path=\"/\")\nparams = { \"startsAt\": 4, \"endsAt\": 20, \"add\": 2 }\ncustom_fizzbuzz(root_resource, params)\n</code></pre>\n<p>Very little has changed between the last client and this one. We just added an extra method to submit our custom start point, but the rest of the code is all identical: we’re still traversing the list, with the same code. This worked because we chose the right behavior for our client, and let the server handle all of the business logic.</p>\n<h2>Traversal from the root</h2>\n<p>The last thing this post does is demonstrate how an actual hypermedia API starts from the API root and progresses from there. Lots of people hear this, and assume that there will be way more API calls than in an API in a different style. That’s because they assume that the client will be written in the same way as they’re used to: function calls over HTTP. As you can see here, we don’t start each call from the root of the API, we start <em>our initial interaction with the service</em> from the root of the API. Each step is just one more call after that, just like in any other API.</p>\n<p>This is the other part of the hypermedia constraint that trips people up. Navigating a state machine of your API’s business process has a very different feel than making function calls over HTTP. It requires a different kind of mindset and approach to building clients. This is the area of hypermedia research that’s been least published about. The server-side story has been fleshed out, but the real frontier in hypermedia theory and practice is client building guidelines, and that’s why I like this example so much.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/how-to-be-an-open-source-gardener/",
"title": "How to be an open source gardener",
"description": null,
"url": "https://steveklabnik.com/writing/how-to-be-an-open-source-gardener/",
"published": "2014-04-14T00:00:00.000Z",
"updated": "2014-04-14T00:00:00.000Z",
"content": "<p>I do a lot of work on open source, but my most valuable contributions haven’t been code. Writing a patch is the easiest part of open source. The truly hard stuff is all of the <em>rest</em>: bug trackers, mailing lists, documentation, and other management tasks. Here’s some things I’ve learned along the way.</p>\n<p>It was RailsConf 2012. I sat in on a panel discussion, and the number of issues open on <a href=\"https://github.com/rails/rails\">rails/rails</a> came up. There were about 800 issues at the time, and had been for a while. Inquiring minds wished to know if that number was ever going to drop, and how the community could help. It was brought up that there was an ‘Issues team,’ whose job would be to triage issues. I enthusiastically volunteered.</p>\n<p>But what does ‘issue triage’ <em>mean</em>, exactly? Well, on a project as large as Rails, there are a ton of issues that are incomplete, stale, need more information… and nobody was tending to them. It’s kind of like a garden: you need someone to pull weeds, and do it often and regularly.</p>\n<p>But before we talk about how to pull the weeds, let’s figure out what kind of garden we even have on our hands!</p>\n<h2>What are Issues?</h2>\n<p>The very first thing your project needs to do is to figure out what Issues are supposed to be for. Each project is different. For example, in Rails, we keep Issues strictly for bugs only. Help questions go to Stack Overflow, and new feature discussion and requests go to the rails-core mailing list. For Rust, we have issues for feature requests, meta-issues… everything. For some repositories, closing all of the issues is not feasible, and for others, you’re shooting for zero. (If you don’t believe that this is even possible, check out <a href=\"https://github.com/jeremyevans/sequel/issues\">Sequel</a>. Issues are rarely even open for more than a few days!)</p>\n<p>My personal favorite is to follow the Rails way. Ideally, you’d be at zero defects, and you can still have a place to discuss features. But really, having <em>some</em> plan is a necessary first step here.</p>\n<h2>Regular tending</h2>\n<p>So how do you tackle 800 issues? The only way I knew how: read all of them. Yep. Here’s what I did: I took a Saturday (and a Sunday), and I went to <a href=\"https://github.com/rails/rails/issues?state=open\">the list of open Issues</a>, then control-clicked on each one in turn to open them in a new tab. Finally, I also control-clicked on page 2. Then I closed this tab. Now I had 31 open tabs: 30 issues, and the next page. I read through the whole issue, including comments. When I got to the last tab, I was ready to repeat the process: open 30 issues, open page 3, click close. Next!</p>\n<p>See, people think working on open source is glamorous, but it’s actually not. Working on open source is reading 800 issues over the course of a weekend.</p>\n<p>Anyway, once I read all of those issues, I was significantly more informed about the kinds of problems Rails was facing. I had a whole bunch of common questions, comments, and problems.</p>\n<p>The next step was to do it all again.</p>\n<p>Wait, again? Why? Well, now that I had a handle on things, I could actually take on the task of triage-ing the issues. If I’d tried to do it before I had the context, I might not have seen the duplicate issues, I wouldn’t know what the normal kinds of comments were on issues, I wouldn’t have known some common questions that maintainers had on pull requests, and in general, things would have just been worse.</p>\n<p>This time, when reading the issue, I went through a little algorithm to sort them out. It looked a little like this:</p>\n<ol>\n<li>Is this issue a feature request? If so, copy/paste an answer I wrote that pointed them to the mailing list, and click close.</li>\n<li>Is this issue a request for help? If so, copy/paste an answer I wrote that pointed them to StackOverflow, and click close.</li>\n<li>Was this issue for an older version of Rails than is currently supported? If so, copy/paste an answer I wrote that asks if anyone knows if this affects a supported version of Rails.</li>\n<li>Did this issue provide enough information to reproduce the error? If no, copy/paste an answer I wrote that asks if they can provide a reproduction.</li>\n<li>If the issue has a reproduction, and it wasn’t on the latest Rails, try it against HEAD. If it still happened, leave a comment that it was still an issue.</li>\n<li>If we got to this point, this issue was pretty solid. Leave a comment that I had triaged it, and cc the maintainer of that relevant sub-system of Rails, so they could find issues that pertain to the things they work on.</li>\n</ol>\n<p>At the same time I did this, I clicked this button on the GitHub interface:</p>\n<p><img src=\"https://svbtleusercontent.com/e2rhsedvszk54q_small.png\" alt=\"https://svbtleusercontent.com/e2rhsedvszk54q_small.png\" /></p>\n<p>And then set up a Gmail filter to filter all of the emails into their own tag, and to skip my inbox:</p>\n<p><img src=\"https://svbtleusercontent.com/oyuljxmbfqieia_small.png\" alt=\"https://svbtleusercontent.com/oyuljxmbfqieia_small.png\" /></p>\n<p>Why do this? Well, I didn’t do all 800 immediately. I decided to do one page per day. This kept it a bit more manageable, rather than taking up entire days of my time. I need these emails and filters for the important second part of the process: tending to the garden regularly.</p>\n<p>Each morning, before I go to work, I pour a cup of coffee and check my emails. I don’t handle all of them before work, but I made an effort to tackle Rails’ emails first. There would usually be about 20 or 25 new emails each morning, and since it was largely just one new comment, they’d be pretty fast to get through. 15 minutes later, I was back to current on all issues. At lunch, I’d do it again: ten minutes to handle the ten or so emails by lunch, and then, before I’d go to bed, I’d do it again: 15 more minutes to handle the next 20 notifications. Basically, I was spending a little under an hour each day, but by doing it <em>every</em> day, it never got out of hand.</p>\n<p>Once I got through all of the issues, we were down to more like 600. A whole fourth of the issues shouldn’t even have been open in the first place. Two weeks in is when the next big gain kicked in. Why two weeks? Well, two weeks is the grace period we decided before marking an issue as stale. Why two weeks? Well, that’s kind of arbitrary, but two weeks feels like enough time for someone to respond if they’re actively interested in getting an issue fixed. See, issues often need the help of the reporter to truly fix, as there just isn’t enough information in many bug reports to be able to reproduce and fix the problem.</p>\n<p>So, after two weeks, I did one more thing each evening: I filtered by ‘least recently updated,’ and checked to see if any of those issues were stale. You just go back until they say ‘two weeks,’ and then, if you haven’t heard from the reporter, mention that it’s stale and give the issue a close. This is one of the other things I had to kind of let go of when working on a real project: closing an issue isn’t forever. You can always re-open the issue later if it turns out you were wrong. So when trying to get a handle on 800 open issues, I defaulted to ‘guilty until proven innocent.’ Terminate issues with extreme prejudice. Leaving old, inconclusive issues doesn’t help anyone. If it’s a real bug that matters to someone, they’ll come along and help reproduce it. If not, maybe someone else will later.</p>\n<p>After a month or two, keeping on it, we got down to 450 or so issues. Members of the core team joked that they had to set up extra email filters from me, because they could tell exactly when I was doing triage. Slow and steady wins the race!</p>\n<p>At this point, I knew enough about Rails to actually start writing some patches. And I happened to be familiar with basically every open bug. So it was easy to start picking some of them and try to reproduce them locally. So I’d do that, and then try to write a patch. If I couldn’t, I’d at least upload my reproduction of the issue, and then leave a note on the Issue, pointing to my reproduction. That way, another team member could simply clone my repository and get to it. The only thing better than reproduction instructions are when those instructions say <code>git clone</code>.</p>\n<p>But I managed to get a few patches in, and then a few more. Doing all of this janitorial work directly led the way towards attaining a commit bit on Rails. It was a long slog at first, but it just got easier the more I did it. A lot of work in open source is this way: it’s really easy once you’ve done it a lot, but is hard for newbies. I’m not yet sure how to tackle this problem…</p>\n<p>I’ve since taken this approach on basically every repository I’ve worked on, and it’s worked really well. But it only works if you keep at it: if you don’t tend your garden, you’ll get weeds. I haven’t had as much time for Rails over the last few months, and it’s back to 800 issues again. I’m not sure if these are real issues or not, as I’ve stopped tending. But without someone actively paying attention, it’s only a matter of time before things get unseemly. If you’re looking to help out an open source project, it’s not a glamorous job, but all it takes is a little bit of work, and developing a habit.</p>\n<p>(Oh, and I should take a moment to mention <a href=\"http://www.codetriage.com/\">Code Triage</a> here. It’s super useful, and can also help you find projects that need help.)</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/is-npm-worth-26mm/",
"title": "Is npm worth 26MM?",
"description": null,
"url": "https://steveklabnik.com/writing/is-npm-worth-26mm/",
"published": "2014-02-12T00:00:00.000Z",
"updated": "2014-02-12T00:00:00.000Z",
"content": "<p>Yesterday, npm, Inc. announced that it had <a href=\"http://blog.npmjs.org/post/76320673650/funding\">closed a $2.6MM round of funding</a>. This was met with a bunch of derision on Twitter. I feel this is not only unwarranted, but backwards: I think this investment signals a nascent trend within the VC community, and a more honest view of how open source actually works.</p>\n<blockquote>\n<p>Further disclaimer: I work for a company which has received investment from people mentioned in this article. I have never talked to those firms directly either, and I have no privileged information as to why they invested in us, either. It all happened before I came on board.</p>\n</blockquote>\n<p>Let’s talk about VC, and then about some odd investments, and then we’ll get around to npm. I apologize for the length of this post, I didn’t have time to make a shorter one.</p>\n<h2>How VC investment works</h2>\n<p>First, a quick refresher on the actual mechanics of VC investment: <a href=\"http://en.wikipedia.org/wiki/Betteridge's_law_of_headlines\">the answer to this headline is ‘no.’</a>. npm, Inc (‘npm’ for the rest of this post) received an investment of $2.6MM, which means that their company is ‘worth’ some multiple of that. At least, it is to <a href=\"http://www.trueventures.com/\">True Ventures</a> and friends.</p>\n<p>As the saying goes, “follow the money.” A VC firm is, in many ways, a marketplace: they take money from investors (“limited partners,” or “lp’s”) and give it to entrepreneurs, and take a cut off the top. Let’s go through an example:</p>\n<p>The Vulture Capital company is a venture capital firm. They announce a new fund. This fund, like many, has a mission statement: to invest in new SaaS technologies. They go to various wealthy individuals, insurance companies, funds-of-funds, endowments, and other companies, and pitch them on this investment: “The VC firm is going to invest $100MM into new SaaS technologies. Our last fund turned $10MM into $20MM. We think we can turn this $100MM into $300MM in the next ten years. We just need your money.” This motley band of the 1% scrapes together $100MM total out of their couch, and this new fund is good to go! The fund is now ‘closed.’ The VC firm now goes to work: the individuals who work for the firm now figure out who to actually give the money to. This generally takes three to five years. In this story, to make the math easy, VC invests in ten companies, $9.75MM each. For this, they get some stake in each company: we’ll call it 30% here. Since they purchased 30% of the company for $9.75MM, each company is ‘worth’ $29.25MM. In order to actually monetize this stake, generally, these companies need to experience a liquidity event: someone has to purchase them, or they need to go public. Because early-stage investing is exceedingly risky, when we check in on Vulture Capital ten years later, six companies have gone bankrupt, three sold for $67MM, and one went public and investors sold off $100MM. 3 * 67MM + 100MM = 300MM: success! The firm has turned $100MM into $300MM. The firm takes a chunk of this money, and the rest is given to those individuals that put up the money in the first place. A new fund is announced, and the cycle repeats itself.</p>\n<p>There are variations at all points in this story, but that’s basically how it works. I’ve also ignored that VC firms generally have multiple funds going at once, that there are different kinds of employees in VC funds with different roles, and a whole ton of other things. This is a simplification, but a representative one.</p>\n<p>Two big things matter here: It’s a numbers game. Over half of our example companies bit the dust, but the firm still made their money. Things have been successful overall. Secondly, companies <em>must</em> exit. A business that’s making $1MM a year in profit off of our $9.75MM investment will still make money eventually, but it won’t in our ten year period, which is when our limited partners want their money back. In another context, a million dollars of profit a year is a success, but for us, it’s a failure.</p>\n<p>We’ll come back to these economics in a bit.</p>\n<h2>Open Source and $$$</h2>\n<p>It’s time for some #realtalk about open source. Open Source is big business. And it’s always been that way. Someday I will write more about this, but for now, I’ll just make that statement and not really back it up. For now, let’s trace the flow of labor in the open source world, and how it relates to businesses.</p>\n<p>Open Source is pretty much the de-facto way to run a startup company these days. Even people on the Microsoft stack are increasingly utilizing open source. And there’s a good reason: the initial monetary cost of open source is zero. It’s also a form of DRY: a bug fixed in an open source component fixes a bug in every firm that uses that particular component.</p>\n<p>Here’s a (sorta kinda) real world example:</p>\n<p>A company, “<a href=\"http://adequatehq.com/\">Adequate HQ</a>” builds a web application (<a href=\"http://www.recruiterspam.com/\">RecruiterSpam</a>) in a hot new language, “<a href=\"https://github.com/tenderlove/phuby\">Phuby</a>.” They decide to release a bunch of the code from this application as open source, and call it “<a href=\"http://www.youtube.com/watch?v=lsWKjS6Vufw\">Phuby on Phails</a>.” There are a number of reasons to do this, but here’s a big one: bugs fixed in Phails will be fixed in RecruiterSpam, at no cost to Adequate! Furthermore, there’s a market opportunity here: by controlling Phuby on Phails, Adequate secures its reputation as a leader in the space, which leads to an invaluable amount of publicity, book deals, and eventually, pro subscriptions to RecruiterSpam. Adequate’s CEO, <a href=\"https://twitter.com/gorbypuff\">Gorbachev “Puff Puff” Thunderhorse</a>, can just lay around all day and post photos of himself to Instagram. Other companies see the success of Adequate, and also decide to build their application on Phails. After all, Phails is pretty awesome, and is (truly) significantly better than all the other frameworks out there. Their new use exposes bugs in Phails, so they fix them and submit patches. Eventually, there’s an entire Phails ecosystem, with conferences, books, ninjas, and rockstars. Tons of gravy for everyone, and everyone rides their sports cars into their various sunsets.</p>\n<p><img src=\"/img/2014-02-12/zonda-hh.jpg\" alt=\"pagani zonda hh\" /></p>\n<p>This is a great story. Everyone wins. Everyone helps each other out. Costs go down, quality goes up. But there’s something that needs to be acknowledged here: Adequate had to exist. Someone had to do the initial work of actually building Phails (and all the other related tools I didn’t explicitly mention: deployment tools, testing tools, dependency management tools, the list goes on and on). In this case, it was because they had a kick-ass product and made boatloads of cash.</p>\n<p>But fundamentally, programmers must pay rent and eat. Yes, they may not <em>directly</em> work on open source for free. But someone is paying them, and it has to come from <em>somewhere</em>. There is no such thing as a free lunch.</p>\n<p>There are, however, different ways to get the money to buy lunch.</p>\n<h2>Infrastructure investment</h2>\n<p>So we have this trend of open source in general, and that for each language, we need an ecosystem to support building software. That ecosystem has to be bootstrapped somehow. Previous communities effectively got lucky, or put in long, long hours, only to be forgotten. I still cry tears for CPAN every once in a while. But I digress. The real question is this:</p>\n<p>I’m a VC. I need a large number of companies to exist to make my money. I also need them to get as big as possible to make my money. I need costs to be low and profits to be high. I need a healthy open source ecosystem to make this happen. So what do I do?</p>\n<p>So here’s what I’m seeing slowly happen: VCs are realizing this story, and are starting to invest in infrastructure. Remember, failure is built into their model. If you had to make ten investments, and I told you that I would take one of them, and you’d lose money on me, but I’d build an ecosystem that would help be a multiplier on all your other investments, would you take it? Remember, you just need one really, really big payday. And a bunch of your investments <em>will</em> already fail.</p>\n<p>This is what I see happening with npm. And it’s not just npm: there’s a number of investments in the last few years which fit this pattern. I’m pretty sure that <a href=\"http://en.wikipedia.org/wiki/Andreessen_Horowitz\">a16z</a> is the only VC firm that explicitly understands this model, and True Ventures may have stumbled upon it by accident.</p>\n<p>Because Rails (If the sarcasm wasn’t heavy enough, Phails is Rails) isn’t exactly repeatable. It’s a fluke. A wonderful fluke that’s changed my life (I have a Ruby tattoo), but I wouldn’t bet on it happening that way again. If I told you, “I have a design firm. No programming skill. We’re going to hire an offshore contractor to build an application and let them do whatever they want.” Would you invest? I wouldn’t.</p>\n<p>If you want to repeatably manufacture an open source ecosystem, you need capital to do so. And a firm that’s progressive enough to understand the indirect payoffs of investing in infrastructure is poised to have a huge advantage.</p>\n<p>While my VC story above is a generic, average one, there are also oddball investments. Before we get to NPM exactly, I want to talk about a few others. The first one is not actually a VC investment itself, but it is a total curveball that caused a lot of waves. Let’s talk about 37signals and Jeff Bezos. In many ways, 37signals is the anti-VC. A boostrapped, incredibly profitable company. So why take money from an investor? Well, money isn’t the only things investors have to offer. From <a href=\"http://signalvnoise.com/archives2/bezos_expeditions_invests_in_37signals.php\">the announcement</a>:</p>\n<blockquote>\n<p>Since we launched Basecamp we’ve been contacted by nearly 30 different VC firms. We’ve never been interested in the typical traditional VC deal. With a few exceptions, all the VCs could offer us was cash and connections. We’re fine on both of those fronts. We don’t need their money to run the business and our little black book is full. We’re looking for something else.What we’ve been looking for is the wisdom of a very special entrepreneur who’s been through what we’re going through. Someone who sees things a little differently and makes us feel right at home. Someone with a long term outlook, not a build-to-flip mentality. We found a perfect match in Jeff. Jeff is our kinda guy.</p>\n</blockquote>\n<p>37signals themselves have said many, many times that they would never sell the company. So where’s Bezos going to get his liquidity event from? Jeff doesn’t actually need Basecamp (which 37signals abruptly re-branded to last week, so let’s abruptly re-brand them here) to experience a liquidity event to get what he wants out of this investment. Bezos <a href=\"http://www.crunchbase.com/financial-organization/bezos-expeditions\">invests in all kinds of things</a>. And, as Basecamp mentions, his advice and connections are invaluable.</p>\n<p>The next two investments that fall in this category of “force multiplier for other things I’m doing that I don’t have to worry about getting my money back directly” are both from Andreessen Horowitz. a16z is in my opinion the most progressive and interesting VC firm in the valley. They make a lot of weird, oddball, long-shot investments, and they tend to pay off. There are two (and a half) that directly fall into the category of investment I’m talking about in this post, and that’s GitHub and Meteor (and CoinBase). Let’s talk Meteor first.</p>\n<p><a href=\"https://www.meteor.com/blog/2012/07/25/meteors-new-112-million-development-budget\">Meteor Development Group got $11.2MM in a round led by a16z</a>. While MDG claims they’ll eventually release some sort of commercial product, I don’t think that’s what really matters. If Meteor is the new Rails, then a16z gets the most inside track possible to that entire new ecosystem. And given that most VC investments fail anyway, this is a really good deal for a16z. They’re purchasing a lottery ticket, but it’s more than that: it’s a lottery ticket that makes future lottery tickets worth more money. They can invest in future Meteor-based startups, knowing that Meteor development will continue for a long time. They’ll always have their nose to the ground on any promising companies built around Meteor, earlier than everyone else. Knowledge and information is power.</p>\n<p>Secondly is GitHub. <a href=\"http://peter.a16z.com/2012/07/09/software-eats-software-development/\">GitHub raised $100MM from a16z, and a16z alone</a>. At the time, I was super mad, because I said exactly what everyone has been saying about the NPM investment. As time goes on, though, I don’t think that GitHub <em>needs</em> to experience a liquidity event. While we don’t know the valuation of GitHub, I <em>do</em> know that an acquisition doesn’t make any sense, and I’m increasingly of the opinion that an IPO doesn’t either. This is based primarily on subjective opinions that I’ve gleaned from being friends with so many GitHubbers over the years, and I don’t think an IPO is <em>impossible</em>, but unlikely. So why invest?</p>\n<p><a href=\"http://online.wsj.com/news/articles/SB10001424053111903480904576512250915629460\">a16z famously believes that software is eating the world</a>. They even reference this article in the title of the GitHub announcement. Investing in GitHub makes a ton of sense when viewed through this lens. In fact, if I had to guess, I’d bet that a16z got a small enough chunk that GitHub’s valuation is <em>insane</em>. (And I try not to use that word.) This means that a16z and GitHub management are protected from a acquisition, and can retain control of GitHub. Why does that matter? Well:</p>\n<p>“Scumbag Steve image saying”Open source infrastructure should be free\" / “pushes all code to GitHub”\" Apparently memegenerator got hacked, and it was giving this page a warning.</p>\n<p>Increasingly, GitHub is where all the open source code is hosted. And open source is the way business is going. And GitHub makes money off of the code that’s not open. In many ways, the open source stuff is a loss leader, so that you get hooked on the GitHub interface and then use it for everything, even your closed source code. This is a great business strategy, but it also places GitHub at the center of this new universe. So for similar reasons, a16z needs GitHub to be awesome to bootstrap <em>every</em> open source ecosystem that will exist into the future. It’s the infrastructure for the infrastructure. The primordial ooze of the 10x return. And a16z has the money to ‘throw away’ on something they won’t get a direct return out of, because they’re smart enough to invest some of their fund in ecosystem development.</p>\n<p>The extra half-investment I’m talking about is CoinBase. <a href=\"http://techcrunch.com/2013/12/12/coinbase-raises-25m-from-andreessen-horowitz-to-build-its-bitcoin-wallet-and-merchant-services/\">They got $25MM from a16z and others</a>, and <a href=\"http://dealbook.nytimes.com/2014/01/21/why-bitcoin-matters/\">Andreesen himself has written about BitCoin</a>. By way of analogy:</p>\n<p>“Software will eat the world” : GitHub :: “Why Bitcoin Matters” : CoinBase</p>\n<p>I only give them a half in this case because it’s not clear to me that CoinBase will be dominant in the way that GitHub is dominant, and that Bitcoin will build a new payments ecosystem the way that open source has built a new ecosystem. But <em>some</em> entity like CoinBase is necessary for Bitcoin to succeed. More on that in a future post, someday.</p>\n<h2>npm?</h2>\n<p>So by now, I hope it’s clear how I view this NPM investment: True Ventures puts in money because the ecosystem around Node.js is still young, and a package manager is one of the biggest parts that <em>requires</em> money. Someone has to pay that S3 (I guess CouchDB) and EC2 bills. So they bootstrap the ecosystem, and also give a multiplier on their future investments. They get insight and access at unprecedented levels into the future of Node, at the cost of something they can already afford to lose.</p>\n<p>We’ll see if I’m right in ten years.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/announcing-emoji-1-0/",
"title": "Announcing Emoji 1.0",
"description": null,
"url": "https://steveklabnik.com/writing/announcing-emoji-1-0/",
"published": "2014-02-10T00:00:00.000Z",
"updated": "2014-02-10T00:00:00.000Z",
"content": "<p>A long time ago, I wanted an emoji gem. So I set out to make one. Turns out, emoji are damn complex. <a href=\"http://words.steveklabnik.com/emoji-licensing\">Emoji Licensing</a> was one of the results. Who’d have guessed the hard part of making a gem was the legal part! Next up was that the ‘emoji’ gem was waaaay out of date: it was a library for converting between Japanese telecoms’ emoji, which was important years ago, but not now that they’re just UTF-8. So I contacted the owner, who graciously gave me the gem.</p>\n<p>Well, after doing the legal and social work, it turns out that some other people were interested in the idea as well! In the end, <a href=\"https://github.com/steveklabnik/emoji/commits?author=steveklabnik\">I didn’t write very much actual code</a>, and <a href=\"https://github.com/wpeterson\">Winfield</a> really stepped up and made the code side happen.</p>\n<p>So today, I’m very happy to have just released version 1.0 of the emoji gem, and to officially hand off maintainership to Winfield. Thanks for all your hard work!</p>\n<p>Install it with</p>\n<pre><code>$ gem install emoji\n</code></pre>\n<p>And find the source on GitHub: <a href=\"https://github.com/steveklabnik/emoji\">https://github.com/steveklabnik/emoji</a></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/a-30-minute-introduction-to-rust/",
"title": "A 30 minute introduction to Rust",
"description": null,
"url": "https://steveklabnik.com/writing/a-30-minute-introduction-to-rust/",
"published": "2014-01-11T00:00:00.000Z",
"updated": "2014-01-11T00:00:00.000Z",
"content": "<p>I recently gave a <a href=\"https://air.mozilla.org/rust-meetup-december-2013/\">proposal for Rust’s documentation</a>. An important component of my overall proposal is a short, simple introduction for people who may have heard of Rust, so that they can figure out if Rust is right for them. The other day, I saw <a href=\"http://www.youtube.com/watch?v=gfCtbGiHcg0\">this excellent presentation</a>, and thought it might serve as a great basis for this introduction. Consider this an RFC for such an introduction. Feedback very welcome on <a href=\"https://mail.mozilla.org/pipermail/rust-dev/2014-January/007903.html\">the rust-dev mailing list</a> or <a href=\"https://twitter.com/steveklabnik\">on Twitter</a>.</p>\n<h2>This tutorial has now <a href=\"http://static.rust-lang.org/doc/master/intro.html\">become the official one</a>.</h2>\n<p>Rust is a systems programming language that focuses on strong compile-time correctness guarantees. It improves upon the ideas other systems languages like C++, D, and Cyclone by providing very strong guarantees and explicit control over the life cycle of memory. Strong memory guarantees make writing correct concurrent Rust code easier than in other languages. This might sound very complex, but it’s easier than it sounds! This tutorial will give you an idea of what Rust is like in about thirty minutes. It expects that you’re at least vaguely familiar with a previous ‘curly brace’ language. The concepts are more important than the syntax, so don’t worry if you don’t get every last detail: the <a href=\"http://static.rust-lang.org/doc/master/tutorial.html\">tutorial</a> can help you out with that later.</p>\n<p>Let’s talk about the most important concept in Rust, “ownership,” and its implications on a task that programmers usually find very difficult: concurrency.</p>\n<h2>Ownership</h2>\n<p>Ownership is central to Rust, and is one of its more interesting and unique features. “Ownership” refers to which parts of your code are allowed to modify various parts of memory. Let’s start by looking at some C++ code:</p>\n<pre><code>int *dangling(void)\n{\n int i = 1234;\n return &i;\n}\n\nint add_one(void)\n{\n int *num = dangling();\n return *num + 1;\n}\n</code></pre>\n<p>This function allocates an integer on the stack, and stores it in a variable, <code>i</code>. It then returns a reference to the variable <code>i</code>. There’s just one problem: stack memory becomes invalid when the function returns. This means that in the second line of <code>add_one</code>, <code>num</code> points to some garbage values, and we won’t get the effect that we want. While this is a trivial example, it can happen quite often in C++ code. There’s a similar problem when memory on the heap is allocated with <code>malloc</code> (or <code>new</code>), then freed with <code>free</code> (or <code>delete</code>), yet your code attempts to do something with the pointer to that memory. More modern C++ uses RAII with constructors/destructors, but it amounts to the same thing. This problem is called a ‘dangling pointer,’ and it’s not possible to write Rust code that has it. Let’s try:</p>\n<pre><code>fn dangling() -> &int {\n let i = 1234;\n return &i;\n}\n\nfn add_one() -> int {\n let num = dangling();\n return *num + 1;\n}\n</code></pre>\n<p>When you try to compile this program, you’ll get an interesting (and long) error message:</p>\n<pre><code>temp.rs:3:11: 3:13 error: borrowed value does not live long enough\ntemp.rs:3 return &i;\n\ntemp.rs:1:22: 4:1 note: borrowed pointer must be valid for the anonymous lifetime #1 defined on the block at 1:22...\ntemp.rs:1 fn dangling() -> &int {\ntemp.rs:2 let i = 1234;\ntemp.rs:3 return &i;\ntemp.rs:4 }\n \ntemp.rs:1:22: 4:1 note: ...but borrowed value is only valid for the block at 1:22\ntemp.rs:1 fn dangling() -> &int { \ntemp.rs:2 let i = 1234; \ntemp.rs:3 return &i; \ntemp.rs:4 } \nerror: aborting due to previous error\n</code></pre>\n<p>In order to fully understand this error message, we need to talk about what it means to “own” something. So for now, let’s just accept that Rust will not allow us to write code with a dangling pointer, and we’ll come back to this code once we understand ownership.</p>\n<p>Let’s forget about programming for a second and talk about books. I like to read physical books, and sometimes I really like one and tell my friends they should read it. While I’m reading my book, I own it: the book is in my possession. When I loan the book out to someone else for a while, they “borrow” it from me. And when you borrow a book, it’s yours for a certain period of time, and then you give it back to me, and I own it again. Right?</p>\n<p>This concept applies directly to Rust code as well: some code “owns” a particular pointer to memory. It’s the sole owner of that pointer. It can also lend that memory out to some other code for a while: the code “borrows” it. It borrows it for a certain period of time, called a “lifetime.”</p>\n<p>That’s all there is to it. That doesn’t seem so hard, right? Let’s go back to that error message: <code>error: borrowed value does not live long enough</code>. We tried to loan out a particular variable, <code>i</code>, using Rust’s borrowed pointers: the <code>&</code>. But Rust knew that the variable would be invalid after the function returns, and so it tells us that: <code>borrowed pointer must be valid for the anonymous lifetime #1... but borrowed value is only valid for the block</code>. Neat!</p>\n<p>That’s a great example for stack memory, but what about heap memory? Rust has a second kind of pointer, a ‘unique’ pointer, that you can create with a <code>~</code>. Check it out:</p>\n<pre><code>fn dangling() -> ~int {\n let i = ~1234;\n return i;\n}\n\nfn add_one() -> int {\n let num = dangling();\n return *num + 1;\n}\n</code></pre>\n<p>This code will successfully compile. Note that instead of a stack allocated <code>1234</code>, we use an owned pointer to that value instead: <code>~1234</code>. You can roughly compare these two lines:</p>\n<pre><code>// rust\nlet i = ~1234;\n\n// C++\nint *i = new int;\n*i = 1234;\n</code></pre>\n<p>Rust is able to infer the size of the type, then allocates the correct amount of memory and sets it to the value you asked for. This means that it’s impossible to allocate uninitialized memory: Rust does not have the concept of null. Hooray! There’s one other difference between this line of Rust and the C++: The Rust compiler also figures out the lifetime of <code>i</code>, and then inserts a corresponding <code>free</code> call after it’s invalid, like a destructor in C++. You get all of the benefits of manually allocated heap memory without having to do all the bookkeeping yourself. Furthermore, all of this checking is done at compile time, so there’s no runtime overhead. You’ll get (basically) the exact same code that you’d get if you wrote the correct C++, but it’s impossible to write the incorrect version, thanks to the compiler.</p>\n<p>You’ve seen one way that ownership and lifetimes are useful to prevent code that would normally be dangerous in a less-strict language, but let’s talk about another: concurrency.</p>\n<h2>Concurrency</h2>\n<p>Concurrency is an incredibly hot topic in the software world right now. It’s always been an interesting area of study for computer scientists, but as usage of the Internet explodes, people are looking to improve the number of users a given service can handle. Concurrency is one way of achieving this goal. There is a pretty big drawback to concurrent code, though: it can be hard to reason about, because it is non-deterministic. There are a few different approaches to writing good concurrent code, but let’s talk about how Rust’s notions of ownership and lifetimes can assist with achieving correct but concurrent code.</p>\n<p>First, let’s go over a simple concurrency example in Rust. Rust allows you to spin up ‘tasks,’ which are lightweight, ‘green’ threads. These tasks do not have any shared memory, and so, we communicate between tasks with a ‘channel’. Like this:</p>\n<pre><code>fn main() {\n let numbers = [1,2,3];\n\n let (port, chan) = Chan::new();\n chan.send(numbers);\n\n do spawn {\n let numbers = port.recv();\n println!(\"{:d}\", numbers[0]);\n }\n}\n</code></pre>\n<p>In this example, we create a vector of numbers. We then make a new <code>Chan</code>, which is the name of the package Rust implements channels with. This returns two different ends of the channel: a channel and a port. You send data into the channel end, and it comes out the port end. The <code>spawn</code> function spins up a new task. As you can see in the code, we call <code>port.recv()</code> (short for ‘receive’) inside of the new task, and we call <code>chan.send()</code> outside, passing in our vector. We then print the first element of the vector.</p>\n<p>This works out because Rust copies the vector when it is sent through the channel. That way, if it were mutable, there wouldn’t be a race condition. However, if we’re making a lot of tasks, or if our data is very large, making a copy for each task inflates our memory usage with no real benefit.</p>\n<p>Enter Arc. Arc stands for ‘atomically reference counted,’ and it’s a way to share immutable data between multiple tasks. Here’s some code:</p>\n<pre><code>extern mod extra;\nuse extra::arc::Arc;\n\nfn main() {\n let numbers = [1,2,3];\n\n let numbers_arc = Arc::new(numbers);\n\n for num in range(0, 3) {\n let (port, chan) = Chan::new();\n chan.send(numbers_arc.clone());\n\n do spawn {\n let local_arc = port.recv();\n let task_numbers = local_arc.get();\n println!(\"{:d}\", task_numbers[num]);\n }\n }\n}\n</code></pre>\n<p>This is very similar to the code we had before, except now we loop three times, making three tasks, and sending an <code>Arc</code> between them. <code>Arc::new</code> creates a new Arc, <code>.clone()</code> makes a new reference to that Arc, and <code>.get()</code> gets the value out of the Arc. So we make a new reference for each task, send that reference down the channel, and then use the reference to print out a number. Now we’re not copying our vector.</p>\n<p>Arcs are great for immutable data, but what about mutable data? Shared mutable state is the bane of the concurrent programmer. You can use a mutex to protect shared mutable state, but if you forget to acquire the mutex, bad things can happen.</p>\n<p>Rust provides a tool for shared mutable state: <code>RWArc</code>. This variant of an Arc allows the contents of the Arc to be mutated. Check it out:</p>\n<pre><code>extern mod extra;\nuse extra::arc::RWArc;\n\nfn main() {\n let numbers = [1,2,3];\n\n let numbers_arc = RWArc::new(numbers);\n\n for num in range(0, 3) {\n let (port, chan) = Chan::new();\n chan.send(numbers_arc.clone());\n\n do spawn {\n let local_arc = port.recv();\n\n local_arc.write(|nums| {\n nums[num] += 1\n });\n\n local_arc.read(|nums| {\n println!(\"{:d}\", nums[num]);\n })\n }\n }\n}\n</code></pre>\n<p>We now use the <code>RWArc</code> package to get a read/write Arc. The read/write Arc has a slightly different API than <code>Arc</code>: <code>read</code> and <code>write</code> allow you to, well, read and write the data. They both take closures as arguments, and the read/write Arc will, in the case of write, acquire a mutex, and then pass the data to this closure. After the closure does its thing, the mutex is released.</p>\n<p>You can see how this makes it impossible to mutate the state without remembering to aquire the lock. We gain the efficiency of shared mutable state, while retaining the safety of disallowing shared mutable state.</p>\n<p>But wait, how is that possible? We can’t both allow and disallow mutable state. What gives?</p>\n<h2>A footnote: unsafe</h2>\n<p>So, the Rust language does not allow for shared mutable state, yet I just showed you some code that has it. How’s this possible? The answer: <code>unsafe</code>.</p>\n<p>You see, while the Rust compiler is very smart, and saves you from making mistakes you might normally make, it’s not an artificial intelligence. Because we’re smarter than the compiler, sometimes, we need to over-ride this safe behavior. For this purpose, Rust has an <code>unsafe</code> keyword. Within an <code>unsafe</code> block, Rust turns off many of its safety checks. If something bad happens to your program, you only have to audit what you’ve done inside <code>unsafe</code>, and not the entire program itself.</p>\n<p>If one of the major goals of Rust was safety, why allow that safety to be turned off? Well, there are really only three main reasons to do it: interfacing with external code, such as doing FFI into a C library, performance (in certain cases), and to provide a safe abstraction around operations that normally would not be safe. Our Arcs are an example of this last purpose. We can safely hand out multiple references to the <code>Arc</code>, because we are sure the data is immutable, and therefore it is safe to share. We can hand out multiple references to the <code>RWArc</code>, because we know that we’ve wrapped the data in a mutex, and therefore it is safe to share. But the Rust compiler can’t know that we’ve made these choices, so <em>inside</em> the implementation of the Arcs, we use <code>unsafe</code> blocks to do (normally) dangerous things. But we expose a safe interface, which means that the Arcs are impossible to use incorrectly.</p>\n<p>This is how Rust’s type system allows you to not make some of the mistakes that make concurrent programming difficult, yet get the efficiency of languages such as C++.</p>\n<h2>That’s all, folks</h2>\n<p>I hope that this taste of Rust has given you an idea if Rust is the right language for you. If that’s true, I encourage you to check out <a href=\"http://static.rust-lang.org/doc/0.9/tutorial.html\">the tutorial</a> for a full, in-depth exploration of Rust’s syntax and concepts.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/2013--year-in-review/",
"title": "2013: year in review",
"description": null,
"url": "https://steveklabnik.com/writing/2013--year-in-review/",
"published": "2013-12-31T00:00:00.000Z",
"updated": "2013-12-31T00:00:00.000Z",
"content": "<p>Every year I try to do a “year in Review” blog post, and every year I realize I didn’t keep good notes. Sigh.</p>\n<p>Overall, this was a pretty… meh year, to be honest. :/ Don’t get me wrong, a lot of really, really good things happened, but as far as my grand strategic life goals go, this year was pretty much treading water, and in some cases, regressive. Oh well.</p>\n<p>So instead of doing as much of a review, I’m going to instead discuss what I’d like 2014 to be like:</p>\n<h3>Better tracking</h3>\n<p>In general, I don’t think I accomplished all I wanted to because I didn’t measure it at all. I’ve found that <a href=\"http://beeminder.com/\">Beeminder</a> has helped me track one or two of the things that I’ve really wanted to accomplish, and so I plan on making lots of use of that. It’s not perfect for everything, but it helps.</p>\n<h3>Reading</h3>\n<p>I have so much I need to read. I think I’m going to start by recording chapters in Beeminder; books are too granular. I don’t think that a book every two weeks is an unreasonable goal, and I may turn it up if I get in the swing of things.</p>\n<h3>Writing</h3>\n<p>I really need to finish off the hypermedia book, and the Rails 4 in Action book. R4IA should be easy, the hypermedia book will be harder.</p>\n<h3>Software</h3>\n<p>I want to contribute to Tor this year. I’ve been putting off contributing to Tor due to my paranoia about NSA stuff, but really, at this point, there’s nothing to worry about. I’m probably on all of those lists anyway. They’re watching everyone.</p>\n<p>In order to do that, I need to up my crypto game. I’d like to finish off the Matasano Crypto Challenge, and read a bunch more things.</p>\n<p>I also want to make lots of contributions to Rust this year, if not the language itself, the ecosystem. There’s so much work to be done.</p>\n<h3>Body</h3>\n<p>I’ve been trying to stay relatively fit, and it’s been okay, but I really need to get back on that train. I got a copy of “Starting Strength” for Christmas.</p>\n<h3>Languages</h3>\n<p>I’m consistently embarassed that I only speak one language. I’ve been working on some German, and it’s been okay, but I need to take it more seriously. I’d also like to pick up the basic Japanese I had before.</p>\n<h3>Conclusion</h3>\n<p>I think that’s it, at least for now. Let’s hope this next year is a good one.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rust-is-surprisingly-expressive/",
"title": "Rust is surprisingly expressive",
"description": null,
"url": "https://steveklabnik.com/writing/rust-is-surprisingly-expressive/",
"published": "2013-12-28T00:00:00.000Z",
"updated": "2013-12-28T00:00:00.000Z",
"content": "<p>Do you remember the first time you saw Rails’ ActiveSupport library? I do: it totally blew my mind. The only dynamic language I had used up to that point was Perl, and it had a similarly mind-blowing effect on my young self. I thought that dynamic typing was mostly a requirement towards making this kind of expressiveness possible, but it turns out Rust can be just as expressive, while retaining type safety and static dispatch.</p>\n<p>There was a time in the early days of Rails when its evangelists would show off snippets of what it let you do. These were often ActiveSupport extensions to the core language itself. It’d go something like this:</p>\n<blockquote>\n<p>Hey, have you tried Rails? Check this out:</p>\n</blockquote>\n<pre><code>irb(main):002:0> Time.now - 2.days\n=> 2009-12-26 09:57:02 -0800\nirb(main):003:0> 2.days.ago\n=> 2009-12-26 09:57:04 -0800\n</code></pre>\n<blockquote>\n<p>Did I just blow your mind???</p>\n</blockquote>\n<p>At the time, I enthusiastically replied “Yes, yes you did!” and immediately typed <code>rails new foo</code>. And all was well and good in the world.</p>\n<p>Fast forward a few years. I now know that static typing is not equivalent to <code>public static void main(String [] args) {</code>. I’ve seen the <a href=\"http://www.cafepress.com/mf/2498088/what-part-of-types-dont-you-understand_tshirt?shop=skicalc&productId=6225368\">Hindley-Milner</a> light. Yet, for all practical purposes, I’ve still been hacking in Ruby for these past few years. For the most part, I’d found the complaints about dynamic typing from the static typing camp to be very FUD-y. I very rarely get <code>TypeError</code>s in my Ruby code. Refactoring generally isn’t very difficult.</p>\n<p>But slowly, this has started to change. I think that it’s that I tend to do more infrastructure and plumbing work now. I’ve run into more and more situations where types would be helpful, or picking the mutable state path has caused headaches.</p>\n<p>Then, I wake up to an IM from <a href=\"http://twitter.com/wycats\">Yehuda</a>:</p>\n<blockquote>\n<p>bro, did you see my tweets?</p>\n</blockquote>\n<p>Basically, this:</p>\n<pre><code>➜ active_support git:(master) cat main.rs\nextern mod active_support;\nuse active_support::Period;\nuse active_support::Time;\n\nfn main() {\n let time = Time::now();\n println!(\"{:?}\", time);\n println!(\"{:?}\", 2.days().from_now());\n println!(\"{:?}\", 2.weeks().from_now());\n println!(\"{:?}\", 2.months().from_now());\n println!(\"{:?}\", 2.years().from_now());\n}\n➜ active_support git:(master) ./main \nactive_support::time::Time{date: active_support::date::Date{year: 2013, month: 12u, day: 28u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 242647081u}\nactive_support::time::Time{date: active_support::date::Date{year: 2013, month: 12u, day: 30u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243287552u}\nactive_support::time::Time{date: active_support::date::Date{year: 2014, month: 1u, day: 11u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243347757u}\nactive_support::time::Time{date: active_support::date::Date{year: 2013, month: 2u, day: 28u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243388962u}\nactive_support::time::Time{date: active_support::date::Date{year: 2015, month: 12u, day: 28u}, hours: 0u, minutes: 59u, seconds: 6u, nanos: 243427393u}\n</code></pre>\n<p>Whaaaat? Yup. To compare side-by-side:</p>\n<pre><code># Ruby\n2.days.from_now\n// Rust\n2.days().from_now()\n</code></pre>\n<p>Awesome. Please note that in Rust, this is fully type-checked, safe, and <em>statically dispatched</em>. No slow reflection techniques used here!</p>\n<p>Now, I should say that I almost <em>never</em> use <code>2.days</code>, but the point is that it’s a benchmark of expressiveness: if you can write it, you can do all sorts of other flexible things. Thinking back to my ‘libraries vs. application code’ confusion over types, I’m reminded by something <a href=\"https://twitter.com/garybernhardt\">Gary Bernhardt</a> once said about Ruby vs. Python. I’m paraphrasing, but as I remember it:</p>\n<blockquote>\n<p>Python is a beautiful, clean language. But the same restrictions that make it nice and clean mean that it’s hard to write beautiful, clean libraries. Ruby, on the other hand, is a complicated, ugly language. But that complexity allows you to write really clean, nice, easy-to-use libraries. The end result is that I prefer Python the language, but I find myself preferring programming with Ruby.</p>\n</blockquote>\n<p>I’m starting to wonder if maybe I feel the same way about static vs. dynamic typing: I like the simplicity and the speed of programming in a dynamic language, but maybe, given the right static language, I’ll end up preferring to program in $LANGUAGE better.</p>\n<p>And that language might just be Rust.</p>\n<hr />\n<p>You can check out Yehuda’s <a href=\"https://github.com/wycats/rust-activesupport\">rust-activesupport on GitHub</a>, if you’d like to see how this is accomplished. He’s also told me that he will write a post explaining its implementation in technical detail, I’ll link to it here when he’s written it.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/how-dogecoin-changed-my-perspective-on-cryptocurrency/",
"title": "How Dogecoin changed my perspective on cryptocurrency",
"description": null,
"url": "https://steveklabnik.com/writing/how-dogecoin-changed-my-perspective-on-cryptocurrency/",
"published": "2013-12-21T00:00:00.000Z",
"updated": "2013-12-21T00:00:00.000Z",
"content": "<p>I have long been a Bitcoin hater, for many reasons. But then, a fortnight ago, something happened. <a href=\"http://dogecoin.com/\">Dogecoin</a> came into existence, and for some reason, I just couldn’t get mad about it. After some thoughtful self-evaluation this week, I’ve realized that Dogecoin has actually reversed my position on cryptocurrency. Here’s why: basically, I’ve come to see Dogecoin as an absurdist art project, mocking the entire global financial system.</p>\n<blockquote>\n<p>Investment Disclaimer: This webpage is provided for general information only and nothing contained in the material constitutes a recommendation for the purchase or sale of any security. Although the statements of fact in this report are obtained from sources that I consider reliable, I do not guarantee their accuracy and any such information may be incomplete or condensed. Also views expressed in this Report are based on research materials available from sources considered reliable. Views are subject to change on the basis of additional or new research, new facts or developments. The investment risks described herein are not purported to be exhaustive, any person considering an investment should seek independent advice on the suitability or otherwise of the particular investment. Investment products are not insured by government or governmental agencies. Investment and Treasury products are subject to Investment risk, including possible loss of principal amount invested. Past performance is not indicative of future results: prices can go up or down. Investors investing in investments and/or treasury products denominated in foreign (non-local) currency should be aware of the risk of exchange rate fluctuations that may cause loss of principal when foreign currency is converted to the investors home currency. I personally only hold about 40,000 DOGE.</p>\n</blockquote>\n<p>Okay, maybe I’m taking something that’s silly way too seriously. Do I really believe that sentence? This question was key for my own understanding of Dogecoin and where it fits into the world. It’s the first question many people ask when they first hear about Dogecoin:</p>\n<blockquote>\n<p>Wait. Is Dogecoin a real thing?</p>\n</blockquote>\n<p>— horse >>= js (@jcoglan) December 18, 2013</p>\n<blockquote></blockquote>\n<p>Searching for <a href=\"https://twitter.com/search?q=is%20dogecoin%20real&src=typd&f=realtime\">“is dogecoin real” on Twitter</a> reveals a lot of people who harbor the same confusion. Here’s the thing: if you are confused if Dogecoin is ‘real’ or not, you’re basically on a wild ride. Critically investigating the ‘realness’ of Dogecoin leads you to realize that it’s basically just as ‘real’ as ‘real money,’ whatever that means. And if Dogecoin is absurd, then so is the entire financial system.</p>\n<blockquote>\n<p>Dogecoin as symptom of the postmodern condition.</p>\n</blockquote>\n<p>— Steve Klabnik (@steveklabnik) December 19, 2013</p>\n<blockquote></blockquote>\n<p>Wait, let’s back up here. What do I mean when I say Dogecoin is ‘just as real’ as, let’s say, Euro? And in economics, most questions end up leading to “What is the meaning of ‘value’?” It’s one of the most fundamental questions of the field. And there are many different positions. For the purposes of this post, I’m going to explain things from the perspective of the “subjective theory of value,” which is arguably the most dominant explanation of value in the field today. I’m painting this concept with a pretty wide brush. Please note two things: I’m not an economist by training, I just enjoy reading lots about it, and I don’t subscribe to the subjective theory of value. That said, I am trying to represent it as accurately as possible in this post.</p>\n<p>The simplest formulation of the subjective theory of value is this: something has whatever value a person says it has. Put another way, my coffee cost $3 because a bunch of people all agree that it’s worth $3. I, as an individual, have goals that I want to accomplish. In this case, it’s to write this blog post. To accomplish this, I made a rational decision: make some coffee at home, or go to the coffee shop. I have various resources that allow me to accomplish this goal, namely, that I have three United States dollars. So I take all the information that’s available to me, and make a decision: three dollars is a pretty small amount of money to me (I’m lucky in this regard), I don’t have anything else I need to do at home, the coffee shop hasn’t changed its prices at all in the last week, so I can be pretty sure it’s still $3. So I decide that getting out of the house is worth at least three dollars to me, and I got to exchange my 3 USD for one latte. You can see how endlessly complex this is: if I had chronic pain, for example, I may not want to walk down the steps, so that changes my decision. Maybe I needed to do laundry at home, so staying at home and drinking regular coffee makes more sense. I can’t make a latte at home, so maybe I get out of the house anyway because I’d prefer it to black coffee today.</p>\n<p>Anyway, that’s it. Seems pretty simple, right? So what about Dogecoin?</p>\n<p>Well, if you think about my example more, I actually have a number of commodities that I’m using to exchange with others to achieve my goal. For example, I also ate a sandwich here at the coffee shop, so I actually spent $6, and that’s partially because I didn’t have any food at home. The coffee shop I went to has good sandwiches, so I chose it over other places that may also have decent lattes. So in my example, I’m trading $6 USD for a latte and a sandwich, and for the right to sit here and use up a seat and some wifi. The woman working behind the counter is trading her time and labor for a wage paid by the company who owns the shop, and the company is trading some coffee beans, a bagel, and some other things with me, through the woman, in exchange for my money.</p>\n<p>TL;DR: buying a cup of coffee is surprisingly complicated.</p>\n<p>Okay, so now that you’ve seen there are way more commodities involved in this transaction than just USD, why is USD special? If this scenario played out in Paris, I would have been using Euro instead of USD. There’s no <em>inherent</em> reason that a piece of paper has a certain amount of value. There is a <em>subjective</em> reason, though: namely, that a large enough number of people agree that USD is a worthwhile commodity. They believe that for a number of reasons, including that our Army will go off and fucking kill anybody who threatens to break the spell that makes everyone agree that USD are worth your time to accept.</p>\n<p>Anyway.</p>\n<p>But really, that’s the issue, right? Everyone agrees that dollars are worth something, and we trade them all the time. They’re really easy to trade. I don’t know of any coffee shop near me that takes Euros, so even though 1€ is “worth more” than $1 USD, the Euro note I have in my backpack is basically useless for me to get some coffee. Economists call this property ‘liquidity,’ and it basically means ‘how easy is it for me to get what I want with this commodity.’ USD is very liquid. Euro is very not liquid, unless I deposit it with my bank, which will exchange those Euro for me for USD, which I can then trade for my coffee. Most discussions around the ‘realness’ of cryptocurrency are really arguments about its liquidity. Bitcoins aren’t real because I can’t buy anything with them. Really, liquidity is the aspect that matters to most people when it comes to basic financial transactions.</p>\n<p>This brings me to <a href=\"https://www.mtgox.com/\">MtGox</a> and <a href=\"https://coinbase.com/\">CoinBase</a>. These two companies are the biggest things to happen in cryptocurrency, and that’s because they significantly improve the liquidity of Bitcoins. MtGox is a place where you can trade Bitcoins for USD. As soon as you have an exchange like this, your Bitcoins now become incredibly more liquid: you can always trade some BTC for USD, and then trade USD with… anything. Coinbase is an online wallet that allows you to store Bitcoins, but more importantly, it provides liquidity to the BTC market because it allows merchants to ‘accept Bitcoins’ without exposing themselves to the risk of Bitcoin’s price fluctuations (this is called ‘volatility’). As a merchant, I can integrate with Coinbase, and someone can, upon checkout, pay with their Coinbase account. It’s sort of similar, from the perspective of the buyer, as paying with your PayPal account. Anyway, here’s the crucial bit: Coinbase then instantly converts the Bitcoins to USD, and then gives the USD to the merchant. This is <em>incredibly</em> important for the future of Bitcoin. By allowing merchants to not worry about the fact that their customers are paying in BTC, it means that more of them will ‘accept Bitcoins,’ which increases Bitcoins’ liquidity in two ways: first, that they can buy more things in BTC ‘directly,’ as well as adding a healthy amount of orders on the various marketplaces, which helps stabilize the price.</p>\n<p>I haven’t said the word “Dogecoin” in a while, so you’re probably wondering where I’m going with this. So here’s the thing: as long as BTC and USD can be exchanged easily, it allows for a significant opening for all other cryptocurrencies. This is because BTC is incredibly liquid when it comes to other cryptocurrencies: there are a number of exchanges that will trade whatever kind of the cambrian explosion of cryptocurrencies you want for BTC. Basically, BTC is sort of the ‘world reserve currency’ of cryptocurrencies. So as long as the link to the ‘real world currencies’ exists, they can cross the gap. Imagine a “Dogecoin VISA”, a credit card where I hold a balance in DOGE. When I go to buy something with my Dogecoin VISA, the backing bank would query the DOGE-> BTC -> USD rate, and then ship the merchant that many USD, and debit me that many DOGE. What if I want to trade EURO for DOGE? EURO -> USD -> BTC -> DOGE. Done. The weakest and most tenuous link here is the BTC <-> USD bridge. As long as it’s maintained, cryptocurrency is liquid, and therefore, ‘real.’ Furthermore, the entire financial system can be re-built on the cryptocurrency side of the bridge, as well. Imagine a site that allowed you to “Dogecoin IPO,” listing your stock on the DSE (Dogecoin Stock Exchange). TWTR -> USD -> BTC -> DOGE -> MYCOMPANY. First step is on the <a href=\"https://www.google.com/finance?cid=32086821185414\">NYSE</a>, second step is on MtGox, third step is on <a href=\"https://coinedup.com/\">Coinedup</a>, fourth step is on the DSE. It all works. Here’s the truly scary thing: we heavily regulate the finance industry, because when we don’t, bad things happen. Hell, when we do, bad things happen. As long as cryptocurrency isn’t taken seriously, it too, will remain unregulated, which allows for a massive extraction of wealth. The cryptocurrency community is largely made up of people who have no idea what they’re doing, and people who <em>do</em> know what they’re doing are starting to notice. It’s all of the worst, most predatory aspects of the global capitalist system, without any consumer protection.</p>\n<p>Regardless, there’s an initial ‘bootstrapping’ phase for anything, and I think Bitcoin is past that point now. It doesn’t matter if I think Bitcoin is a terrible idea, it’s going to be a thing regardless, so time to figure out what that actually means.</p>\n<p>So the statement that Dogecoin poses to us is this: If things really are worth whatever anyone says they’re worth, so are memes. Do we really want to live in a world where a dumb joke about dogs that can’t talk has the same legitimacy as money? Because the more people that take Dogecoin seriously, the more serious it becomes… and the very nature of its silliness encourages us to think seriously about the nature of money and therefore value.</p>\n<p>Before, I just laughed. Now, thanks to a joke, I’m scared.</p>\n<hr />\n<p>You can tip me in Dogecoin at this address: <code>DEJf4pmo7a87FeZP1fZAT8jPjDHnKL9r8C</code> or in Bitcoin at this address: <code>13BHd8tYA41JMuoBpLTGjyXfrnc6Xi6ma5</code></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/git-history-modification-and-libuv/",
"title": "git, history modification, and libuv",
"description": null,
"url": "https://steveklabnik.com/writing/git-history-modification-and-libuv/",
"published": "2013-11-30T00:00:00.000Z",
"updated": "2013-11-30T00:00:00.000Z",
"content": "<p>There has been a recent kerfuffle over a <a href=\"https://github.com/joyent/libuv/pull/1015\">pull request to libuv</a>: it was rejected, applied, reverted, and re-applied. There was some question as to the authorship of that commit, and I wanted to show you why that was, because it illustrates how git handles history.</p>\n<p>At first, the pull request was closed. Then, it was committed in <a href=\"https://github.com/joyent/libuv/commit/47d98b64c45db8335bf7e065351e385cae32323d\">47d98b6</a>. Then <a href=\"https://github.com/joyent/libuv/commit/804d40ee14dc0f82c482dcc8d1c41c14333fcb48\">804d40ee</a> reverted that 47d98b6. But when you look at 804d40ee on GitHub, you’ll see no indication of which branch it’s on. That’s because it’s not on any branch. If you clone down libuv, and try to find the commit, you’ll see it’s not there:</p>\n<pre><code>~/libuv(master)$ git log 804d40e \nfatal: ambiguous argument '804d40e': unknown revision or path not in the working tree. \n</code></pre>\n<p>What gives?</p>\n<p>Let’s make a test repository:</p>\n<pre><code>$ mkdir test \n$ cd test \n$ git init \nInitialized empty Git repository in /home/action/test/.git/ \n$ touch HELLO.md \n$ git add HELLO.md \n$ git commit -m \"initial commit\" \n[master (root-commit) 646567b] initial commit \n 1 file changed, 0 insertions(+), 0 deletions(-) \n create mode 100644 HELLO.md \n</code></pre>\n<p>Okay, now let’s make a second commit:</p>\n<pre><code>$ touch WHATEVER.md \n$ git add WHATEVER.md \n$ git commit -m \"adding whatever\" \n[master 7c232cc] adding whatever \n 1 file changed, 0 insertions(+), 0 deletions(-) \n create mode 100644 WHATEVER.md \n</code></pre>\n<p>Actually, that commit was a mistake. Since we haven’t pushed anywhere yet, let’s just use <code>git reset --hard</code> to just throw it out.</p>\n<pre><code>$ git reset --hard HEAD~1 \nHEAD is now at 646567b initial commit \n</code></pre>\n<p>But what about that commit? Where is it?</p>\n<pre><code>$ git show 7c232cc \ncommit 7c232cceb94a2e7cdd95c526de785efe08da2325 \nAuthor: Steve Klabnik <[email protected]> \nDate: Sat Nov 30 20:19:26 2013 +0000 \n \n adding whatever \n \ndiff --git a/WHATEVER.md b/WHATEVER.md \nnew file mode 100644 \nindex 0000000..e69de29 \n</code></pre>\n<p>It’s still in the repository. We can use <code>git reflog</code> to see our changes:</p>\n<pre><code>$ git reflog \n646567b HEAD@{0}: reset: moving to HEAD~1 \n7c232cc HEAD@{1}: commit: adding whatever \n646567b HEAD@{2}: commit (initial): initial commit \n</code></pre>\n<p>Git calls these kinds of commits ‘unreachable’ because, while they exist in your repository, you can’t find them unless you know their SHA. They’re not connected to any tag or branch. We can use <code>git fsck</code> to find these for us automatically:</p>\n<pre><code>$ git fsck --no-reflogs --unreachable \nChecking object directories: 100% (256/256), done. \nunreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5 \nunreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325\n</code></pre>\n<p>Some Git commands will run <code>git gc</code>, the Git garbage collector, as part of their normal operation. However, if we run <code>git gc</code> ourselves, it doesn’t look like anything happens:</p>\n<pre><code>$ git gc \nCounting objects: 5, done. \nDelta compression using up to 8 threads. \nCompressing objects: 100% (3/3), done. \nWriting objects: 100% (5/5), done. \nTotal 5 (delta 0), reused 0 (delta 0) \n$ git fsck --no-reflogs --unreachable \nChecking object directories: 100% (256/256), done. \nChecking objects: 100% (5/5), done. \nunreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5 \nunreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325 \n</code></pre>\n<p>What gives? Well, <code>git gc</code> has some settings that indicate how long it will let an unreachable commit lay around in your repository. The <code>gc.pruneExpire</code> configuration variable controls this behavior, and it defaults to 14 days. So even if we try to throw away a commit, and even if we manually run the garbage collector, we’ll still have the commit for two weeks.</p>\n<p>We can, of course, tell <code>git gc</code> to ignore the setting:</p>\n<pre><code>$ git gc --prune=now \nCounting objects: 5, done. \nDelta compression using up to 8 threads. \nCompressing objects: 100% (3/3), done. \nWriting objects: 100% (5/5), done. \nTotal 5 (delta 0), reused 5 (delta 0) \n$ git fsck --no-reflogs --unreachable \nChecking object directories: 100% (256/256), done. \nChecking objects: 100% (5/5), done. \nunreachable tree 1536f028d8051a63f7f39951f06b7180a96faff5 \nunreachable commit 7c232cceb94a2e7cdd95c526de785efe08da2325 \n</code></pre>\n<p>Uhhhh what? It turns out that <code>git gc</code> won’t touch commits that are still in our reflog. So let’s clear that:</p>\n<pre><code>$ git reflog expire --expire=now --all \n$ git reflog \n$\n</code></pre>\n<p>Good. And now, let’s take out the garbage:</p>\n<pre><code>$ git gc --prune=now \nCounting objects: 3, done. \nWriting objects: 100% (3/3), done. \nTotal 3 (delta 0), reused 3 (delta 0) \n$ git fsck --no-reflogs --unreachable \nChecking object directories: 100% (256/256), done. \nChecking objects: 100% (3/3), done. \n</code></pre>\n<p>Easy! When you fetch from a remote repository, Git does not include unreachable commits. That’s why when we cloned down libuv earlier, we didn’t get the orphaned commit.</p>\n<p>Anyway, as you can see, it’s actually really hard to lose your data with Git, even if you explicitly try to throw it away. This is why you can rebase with… well, maybe not reckless abandon, but you don’t have to worry about your data. All it takes is a quick <code>git reset --hard HEAD@{1}</code> (read: reset me to the last entry in the reflog) and you’re back to where you were before that operation you screwed up.</p>\n<p>So, what happened with that commit on GitHub? Basically, one committer pushed 804d40ee, and then another did a <code>git reset --hard</code> to remove it, and then force pushed up to the repository. That commit still lives on GitHub, and is still viewable in their interface, but in some time, when GitHub runs <code>git gc</code> on the repository, it will go away.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/why-im-partnering-with-balanced/",
"title": "Why I'm partnering with Balanced",
"description": null,
"url": "https://steveklabnik.com/writing/why-im-partnering-with-balanced/",
"published": "2013-11-21T00:00:00.000Z",
"updated": "2013-11-21T00:00:00.000Z",
"content": "<p>My next venture will be joining up with <a href=\"https://www.balancedpayments.com/\">Balanced</a>, an API for marketplace payments, and I’m really excited about it. To the tax man, of course, the title of this post is “Why I’m going to be an employee of Balanced,” but I like to think about it as a partnership.</p>\n<h3>Why I Balanced</h3>\n<p>I first heard of Balanced almost exactly a year ago. You see, my friend Chad had started this project called <a href=\"https://www.gittip.com/\">Gittip</a>. He was using Stripe to handle payments, but Gittip is a marketplace, and Stripe didn’t support marketplaces. So they gave Gittip a month to find another processor. So <a href=\"https://github.com/gittip/www.gittip.com/pull/137\">a Balanced employee came along and submitted a pull request</a>. I thought this was super awesome. So much so, that I tweeted it:</p>\n<blockquote>\n<p>@jordanmessina It was like ‘hmm we can't use stripe, who should we use?’ “oh hey, just swung by to integrate you with us. <3”</p>\n</blockquote>\n<p>— Steve Klabnik (@steveklabnik) December 1, 2012</p>\n<blockquote></blockquote>\n<p>I still told people that story, but didn’t really think a whole lot more about Balanced than that. But every so often, I’d bump into them again. The next time Balanced came across my radar was when they <a href=\"http://blog.balancedpayments.com/benefits-open-company-balanced/\">declared themselves an Open Company</a>. Whoah. Now, don’t get me wrong: it’s not exactly socialism, but it <em>is</em> super interesting. The idea of doing feature development and discussion out in the open is really interesting to me. For example, if you’re interested in seeing when Balanced supports international payments, you can check out <a href=\"https://github.com/balanced/balanced-api/issues/23\">this Issue</a>, and when the feature happens, you’ll know. That feature is obvious, but for less obvious ones, like <a href=\"https://github.com/balanced/balanced-api/issues/204\">Bitcoin support</a>, you can see that there’s real discussion, where the users who want a feature can help explain their use cases, Balanced will ask questions, and (hopefully) eventually it’ll get implemented.</p>\n<p>Anyway, so when I was looking for something new, Balanced seemed like a good fit. I dropped by their offices a few times, and met with the team. I was impressed when they included their intern, Matthew, in one of our meetings. Most companies I’ve seen treat their interns really poorly. When I mentioned that I was a Python newbie, Balanced’s CTO, <a href=\"https://twitter.com/mahmoudimus\">Mahmoud</a>, replied that that was perfect: if they only hired Python people, they’d only get Python answers, and monoculture is bad. During that discussion, I remembered my own startup, CloudFab, and remembered my cofounder’s issues getting us a bank account, because we were a marketplace. I had felt the pain Balanced is trying to solve before. At that point, I was in, but Mahmoud thanking me for my role in helping fight for equality in tech was extra icing on the cake.</p>\n<h3>My role</h3>\n<p>Anyway, that’s why I’m pumped about Balanced. So what exactly will I be doing? A few things, taken straight from my offer letter:</p>\n<ol>\n<li>Design Balanced’s API based on the needs of our customers and the feedback of the team</li>\n<li>Lead Balanced’s path towards HTTP and JSON-API compliance</li>\n<li>Move standards forward to support Balanced’s innovation</li>\n<li>Build tools to allow Balanced and others to create standards compliant APIs</li>\n<li>Promote and evangelize the benefits of the Open Company philosophy</li>\n</ol>\n<p>To expand on these five steps:</p>\n<p>You can reach me at <a href=\"mailto:[email protected]\">[email protected]</a>. I have uploaded <a href=\"http://pgp.mit.edu:11371/pks/lookup?op=vindex&search=0x1782FE756ACC3706\">a PGP key</a> to the MIT keyserver if that’s your jam.</p>\n<p>If you’re a Balanced customer and you have thoughts, good or bad, about Balanced, please get in touch.</p>\n<p>If you’re <em>not</em> a Balanced customer, I’d love to hear why. Send me an email.</p>\n<p>I’ve joined the <a href=\"http://www.w3.org/2001/tag/\">W3C TAG mailing list</a>, and will be getting even more involved in the web standards process than I already am, to make sure that Balanced is keeping abreast of the latest standards, and to make sure that Balanced’s use-cases are properly advocated for as part of that process.</p>\n<p>I’ll be working hard on <a href=\"http://jsonapi.org/\">JSON API</a>, both with Balanced’s customers, our team, and with others who want to use the standard. Some tickets have been a bit stagnant: it’s now a part of my job to resolve them.</p>\n<p>I’ll be working on some tooling, hopefully with other companies that care about APIs, to build tooling that we all can use. If you’re in San Fransisco, I’d love to swing by your office and say hello. If not, email works too.</p>\n<p>If you want me to come speak to your usergroup, meetup, or conference about APIs, web standards, Open Companies, or payment systems, let me know, I really enjoy it.</p>\n<p>Expect to see a bunch of writing on the <a href=\"http://blog.balancedpayments.com/\">Balanced blog</a> from me on all of those topics, as well.</p>\n<p>I’m excited for the future. There’s so much work to do. Let’s all build some awesome APIs!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/on-leftist-infighting/",
"title": "On leftist infighting",
"description": null,
"url": "https://steveklabnik.com/writing/on-leftist-infighting/",
"published": "2013-11-18T00:00:00.000Z",
"updated": "2013-11-18T00:00:00.000Z",
"content": "<p>There used to be a post here: <a href=\"https://web.archive.org/web/20131118145002/http://words.steveklabnik.com/on-leftist-infighting\">https://web.archive.org/web/20131118145002/http://words.steveklabnik.com/on-leftist-infighting</a></p>\n<p>After having some conversations on Twitter, I was made aware of some context around ‘infighting’ that made it problematic. I decided that the post was unworkable, as I certainly didn’t mean some of the things it implied, and so decided it was better to simply delete it.</p>\n<p>I’ve been told that some people have been trying to cite this post as me saying that I believe any and all dissent should not be public, or that I believe in covering up incidents of wrongdoing because they may ‘harm the movement’ or something. This is wrong. I wrote this post after trying a number of different tactics to get other men to stop tone policing several women, specifically Shanley. “If you don’t like the way she says things, why not distract from the issue at hand and instead say nothing, and focus on the misogyny instead” was one strategy that actually worked. Hence the post. But since I didn’t just straight-up come out and say that, it failed.</p>\n<p>Wrongdoing should be called out as wrong, and tone policing is bad. End of story.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/abstract-vs-concrete/",
"title": "Abstract vs concrete",
"description": null,
"url": "https://steveklabnik.com/writing/abstract-vs-concrete/",
"published": "2013-11-14T00:00:00.000Z",
"updated": "2013-11-14T00:00:00.000Z",
"content": "<p>When I talk to people about systemic inequalities, one of the hardest things for them to understand is the concrete nature of how these relations affect the day-to-day lives of others. For many, this topic is an abstract discussion. For many others, this is life, day to day.</p>\n<p>In the interest of helping bridge this gap, I want to share a story with you from a few weeks ago. The setting: the West Village, Manhattan. The night: Halloween. In the early morning hours of All Saint’s Day, I got a text message from someone I care about. Here’s some screenshots of the conversation. She has seen this post and said it’s okay to share, for the same reasons I’m interested in sharing this with you. Also, if reading a story about a bad situation or a slur or two will make you upset, you may want to read another blog post.</p>\n<p>I’m not sure how to convince random people on Twitter that yes, in 2013, in the West Villiage, in the largest city in the country, on a holiday night when everyone is out, you still can’t let your guard down. Even if you’re not actually a gay man.</p>\n<p>And this is just one story.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/pointers-in-rust-a-guide/",
"title": "Pointers in Rust, a guide",
"description": null,
"url": "https://steveklabnik.com/writing/pointers-in-rust-a-guide/",
"published": "2013-10-18T00:00:00.000Z",
"updated": "2013-10-18T00:00:00.000Z",
"content": "<p>Rust’s pointers are one of its more unique and compelling features. Pointers are also one of the more confusing topics for newcomers to Rust. They can also be confusing for people coming from other languages that support pointers, such as C++. This tutorial will help you understand this important topic.</p>\n<h1>You don’t actually need pointers</h1>\n<p>I have good news for you: you probably don’t need to care about pointers, especially as you’re getting started. Think of it this way: Rust is a language that emphasizes safety. Pointers, as the joke goes, are very pointy: it’s easy to accidentally stab yourself. Therefore, Rust is made in a way such that you don’t need them very often.</p>\n<p>“But tutorial!” you may cry. \"My co-worker wrote a function that looks like this:</p>\n<pre><code>fn succ(x: &int) -> int { *x + 1 }\n</code></pre>\n<p>So I wrote this code to try it out:</p>\n<pre><code>fn main() {\n let number = 5;\n let succ_number = succ(number);\n println!(\"{}\", succ_number);\n}\n</code></pre>\n<p>And now I get an error:</p>\n<pre><code>error: mismatched types: expected `&int` but found `<VI0>` (expected &-ptr but found integral variable)\n</code></pre>\n<p>What gives? It needs a pointer! Therefore I have to use pointers!\"</p>\n<p>Turns out, you don’t. All you need is a reference. Try this on for size:</p>\n<pre><code>fn main() {\n let number = 5;\n let succ_number = succ(&number);\n println!(\"{}\", succ_number);\n}\n</code></pre>\n<p>It’s that easy! One extra little <code>&</code> there. This code will run, and print <code>6</code>.</p>\n<p>That’s all you need to know. Your co-worker could have written the function like this:</p>\n<pre><code>fn succ(x: int) -> int { x + 1 }\n\nfn main() {\n let number = 5;\n let succ_number = succ(number);\n println!(\"{}\", succ_number);\n}\n</code></pre>\n<p>No pointers even needed. Then again, this is a simple example. I assume that your real-world <code>succ</code> function is more complicated, and maybe your co-worker had a good reason for <code>x</code> to be a pointer of some kind. In that case, references are your best friend. Don’t worry about it, life is too short.</p>\n<p>However.</p>\n<p>Here are the use-cases for pointers. I’ve prefixed them with the name of the pointer that satisfies that use-case:</p>\n<ol>\n<li>Owned: ~Trait must be a pointer, becuase you don’t know the size of the object, so indirection is mandatory.</li>\n<li>Owned: You need a recursive data structure. These can be infinite sized, so indirection is mandatory.</li>\n<li>Owned: A very, very, very rare situation in which you have a <em>huge</em> chunk of data that you wish to pass to many methods. Passing a pointer will make this more efficient. If you’re coming from another language where this technique is common, such as C++, please read “A note…” below.</li>\n<li>Managed: Having only a single owner to a piece of data would be inconvenient or impossible. This is only often useful when a program is very large or very complicated. Using a managed pointer will activate Rust’s garbage collection mechanism.</li>\n<li>Borrowed: You’re writing a function, and you need a pointer, but you don’t care about its ownership. If you make the argument a borrowed pointer, callers can send in whatever kind they want.</li>\n</ol>\n<p>Five exceptions. That’s it. Otherwise, you shouldn’t need them. Be skeptical of pointers in Rust: use them for a deliberate purpose, not just to make the compiler happy.</p>\n<h2>A note for those proficient in pointers</h2>\n<p>If you’re coming to Rust from a language like C or C++, you may be used to passing things by reference, or passing things by pointer. In some languages, like Java, you can’t even have objects without a pointer to them. Therefore, if you were writing this Rust code:</p>\n<pre><code>struct Point {\n x: int,\n y: int,\n}\n\nfn main() {\n let p0 = Point { x: 5, y: 10};\n let p1 = transform(p0);\n println!(\"{:?}\", p1);\n}\n</code></pre>\n<p>I think you’d implement <code>transform</code> like this:</p>\n<pre><code>fn transform(p: &Point) -> Point {\n Point { x: p.x + 1, y: p.y + 1}\n}\n\n// and change this:\nlet p1 = transform(&p0);\n</code></pre>\n<p>This does work, but you don’t need to create those references! The better way to write this is simply:</p>\n<pre><code>struct Point {\n x: int,\n y: int,\n}\n\nfn transform(p: Point) -> Point {\n Point { x: p.x + 1, y: p.y + 1}\n}\n\nfn main() {\n let p0 = Point { x: 5, y: 10};\n let p1 = transform(p0);\n println!(\"{:?}\", p1);\n}\n</code></pre>\n<p>But won’t this be inefficent? Well, that’s a complicated question, but it’s important to know that Rust, like C and C++, store aggregate data types ‘unboxed,’ whereas languages like Java and Ruby store these types as ‘boxed.’ For smaller structs, this way will be more efficient. For larger ones, it may be less so. But don’t reach for that pointer until you must! Make sure that the struct is large enough by performing some tests before you add in the complexity of pointers.</p>\n<h1>Owned Pointers</h1>\n<p>Owned pointers are the conceptually simplest kind of pointer in Rust. A rough approximation of owned pointers follows:</p>\n<ol>\n<li>Only one owned pointer may exist to a particular place in memory. It may be borrowed from that owner, however.</li>\n<li>The Rust compiler uses static analysis to determine where the pointer is in scope, and handles allocating and de-allocating that memory. Owned pointers are not garbage collected.</li>\n</ol>\n<p>These two properties make for three use cases.</p>\n<h2>References to Traits</h2>\n<p>Traits must be referenced through a pointer, becuase the struct that implements the trait may be a different size than a different struct that implements the trait. Therefore, unboxed traits don’t make any sense, and aren’t allowed.</p>\n<h2>Recursive Data Structures</h2>\n<p>Sometimes, you need a recursive data structure. The simplest is known as a ‘cons list’:</p>\n<pre><code>enum List<T> {\n Nil,\n Cons(T, ~List<T>),\n}\n \nfn main() {\n let list: List<int> = Cons(1, ~Cons(2, ~Cons(3, ~Nil)));\n println!(\"{:?}\", list);\n}\n</code></pre>\n<p>This prints:</p>\n<pre><code>Cons(1, ~Cons(2, ~Cons(3, ~Nil)))\n</code></pre>\n<p>The inner lists <em>must</em> be an owned pointer, becuase we can’t know how many elements are in the list. Without knowing the length, we don’t know the size, and therefore require the indirection that pointers offer.</p>\n<h2>Efficiency</h2>\n<p>This should almost never be a concern, but because creating an owned pointer boxes its value, it therefore makes referring to the value the size of the box. This may make passing an owned pointer to a function less expensive than passing the value itself. Don’t worry yourself with this case until you’ve proved that it’s an issue through benchmarks.</p>\n<p>For example, this will work:</p>\n<pre><code>struct Point {\n x: int,\n y: int,\n}\n\nfn main() {\n let a = Point { x: 10, y: 20 };\n do spawn {\n println(a.x.to_str());\n }\n}\n</code></pre>\n<p>This struct is tiny, so it’s fine. If <code>Point</code> were large, this would be more efficient:</p>\n<pre><code>struct Point {\n x: int,\n y: int,\n}\n\nfn main() {\n let a = ~Point { x: 10, y: 20 };\n do spawn {\n println(a.x.to_str());\n }\n}\n</code></pre>\n<p>Now it’ll be copying a pointer-sized chunk of memory rather than the whole struct.</p>\n<h1>Managed Pointers</h1>\n<p>Managed pointers, notated by an <code>@</code>, are used when having a single owner for some data isn’t convenient or possible. This generally happens when your program is very large and complicated.</p>\n<p>For example, let’s say you’re using an owned pointer, and you want to do this:</p>\n<pre><code>struct Point {\n x: int,\n y: int,\n}\n \nfn main() {\n let a = ~Point { x: 10, y: 20 };\n let b = a;\n println(b.x.to_str());\n println(a.x.to_str());\n}\n</code></pre>\n<p>You’ll get this error:</p>\n<pre><code>test.rs:10:12: 10:13 error: use of moved value: `a`\ntest.rs:10 println(a.x.to_str());\n ^\ntest.rs:8:8: 8:9 note: `a` moved here because it has type `~Point`, which is moved by default (use `ref` to override)\ntest.rs:8 let b = a;\n ^\n</code></pre>\n<p>As the message says, owned pointers only allow for one owner at a time. When you assign <code>a</code> to <code>b</code>, <code>a</code> becomes invalid. Change your code to this, however:</p>\n<pre><code>struct Point {\n x: int,\n y: int,\n}\n \nfn main() {\n let a = @Point { x: 10, y: 20 };\n let b = a;\n println(b.x.to_str());\n println(a.x.to_str());\n}\n</code></pre>\n<p>And it works:</p>\n<pre><code>10\n10\n</code></pre>\n<p>So why not just use managed pointers everywhere? There are two big drawbacks to managed pointers:</p>\n<ol>\n<li>They activate Rust’s garbage collector. Other pointer types don’t share this drawback.</li>\n<li>You cannot pass this data to another task. Shared ownership across concurrency boundaries is the source of endless pain in other languages, so Rust does not let you do this.</li>\n</ol>\n<h1>Borrowed Pointers</h1>\n<p>Borrowed pointers are the third major kind of pointer Rust supports. They are simultaneously the simplest and the most complicated kind. Let me explain: they’re called ‘borrowed’ pointers because they claim no ownership over the data they’re pointing to. They’re just borrowing it for a while. So in that sense, they’re simple: just keep whatever ownership the data already has. For example:</p>\n<pre><code>use std::num::sqrt;\n\nstruct Point {\n x: float,\n y: float,\n}\n\nfn compute_distance(p1: &Point, p2: &Point) -> float {\n let x_d = p1.x - p2.x;\n let y_d = p1.y - p2.y;\n\n sqrt(x_d * x_d + y_d * y_d)\n}\n\nfn main() {\n let origin = @Point { x: 0.0, y: 0.0 };\n let p1 = ~Point { x: 5.0, y: 3.0 };\n\n println!(\"{:?}\", compute_distance(origin, p1));\n}\n</code></pre>\n<p>This prints <code>5.83095189</code>. You can see that the <code>compute_distance</code> function takes in two borrowed pointers, but we give it a managed and unique pointer. Of course, if this were a real program, we wouldn’t have any of these pointers, they’re just there to demonstrate the concepts.</p>\n<p>So how is this hard? Well, because we’re igorning ownership, the compiler needs to take great care to make sure that everything is safe. Despite their complete safety, a borrowed pointer’s representation at runtime is the same as that of an ordinary pointer in a C program. They introduce zero overhead. The compiler does all safety checks at compile time.</p>\n<p>This theory is called ‘region pointers,’ and involve a concept called ‘lifetimes’. Here’s the simple explanation: would you expect this code to compile?</p>\n<pre><code>fn main() {\n println(x.to_str());\n let x = 5;\n}\n</code></pre>\n<p>Probably not. That’s becuase you know that the name <code>x</code> is valid from where it’s declared to when it goes out of scope. In this case, that’s the end of the <code>main</code> function. So you know this code will cause an error. We call this duration a ‘lifetime’. Let’s try a more complex example:</p>\n<pre><code>fn main() {\n let mut x = ~5;\n if(*x < 10) {\n let y = &x;\n println!(\"Oh no: {:?}\", y);\n return;\n }\n *x = *x - 1;\n println!(\"Oh no: {:?}\", x);\n}\n</code></pre>\n<p>Here, we’re borrowing a pointer to <code>x</code> inside of the <code>if</code>. The compiler, however, is able to determine that that pointer will go out of scope without <code>x</code> being mutated, and therefore, lets us pass. This wouldn’t work:</p>\n<pre><code>fn main() {\n let mut x = ~5;\n if(*x < 10) {\n let y = &x;\n *x = *x - 1;\n\n println!(\"Oh no: {:?}\", y);\n return;\n }\n *x = *x - 1;\n println!(\"Oh no: {:?}\", x);\n}\n</code></pre>\n<p>It gives this error:</p>\n<pre><code>test.rs:5:8: 5:10 error: cannot assign to `*x` because it is borrowed\ntest.rs:5 *x = *x - 1;\n ^~\ntest.rs:4:16: 4:18 note: borrow of `*x` occurs here\ntest.rs:4 let y = &x;\n ^~\n</code></pre>\n<p>As you might guess, this kind of analysis is complex for a human, and therefore hard for a computer, too! There is an entire <a href=\"http://static.rust-lang.org/doc/0.8/tutorial-borrowed-ptr.html\">tutorial devoted to borrowed pointers and lifetimes</a> that goes into lifetimes in great detail, so if you want the full details, check that out.</p>\n<h1>Returning Pointers</h1>\n<p>We’ve talked a lot about funtions that accept various kinds of pointers, but what about returning them? Here’s the rule of thumb: only return a unique or managed pointer if you were given one in the first place.</p>\n<p>What does that mean? Don’t do this:</p>\n<pre><code>fn foo(x: ~int) -> ~int {\n return ~*x;\n}\n\nfn main() {\n let x = ~5;\n let y = foo(x);\n}\n</code></pre>\n<p>Do this:</p>\n<pre><code>fn foo(x: ~int) -> int {\n return *x;\n}\n\nfn main() {\n let x = ~5;\n let y = ~foo(x);\n}\n</code></pre>\n<p>This gives you flexibility, without sacrificing performance. For example, this will also work:</p>\n<pre><code>fn foo(x: ~int) -> int {\n return *x;\n}\n\nfn main() {\n let x = ~5;\n let y = @foo(x);\n}\n</code></pre>\n<p>You may think that this gives us terrible performance: return a value and then immediately box it up?!?! Isn’t that the worst of both worlds? Rust is smarter than that. There is no copy in this code. <code>main</code> allocates enough room for the <code>@int</code>, passes it into <code>foo</code> as <code>x</code>, and then <code>foo</code> writes the value into the new box. This writes the return value directly into the allocated box.</p>\n<p>This is important enough that it bears repeating: pointers are not for optimizing returning values from your code. Allow the caller to choose how they want to use your output.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/new-beginnings/",
"title": "New Beginnings",
"description": null,
"url": "https://steveklabnik.com/writing/new-beginnings/",
"published": "2013-10-01T00:00:00.000Z",
"updated": "2013-10-01T00:00:00.000Z",
"content": "<p>TL;DR:</p>\n<blockquote>\n<p>WANTED: Proletarian/petite bourgeoisie looking for capitalist/industrialist/haute bourgeoisie. I’ve got the labor power, you’ve got the money commodity. Let’s exchange commodities at full market value in accordance with the Law of Value; I know you’ll extract some surplus and alienate my species-being, but I really need to regenerate and reproduce myself so that I can be around to labor tomorrow.</p>\n</blockquote>\n<p>Yes, that’s right, I am officially looking for work. I loved my job at <a href=\"http://jumpstartlab.com/\">Jumpstart Lab</a>, but all good things must come to an end. So here we are. There’s a <a href=\"http://jumpstartlab.com/news/archives/2013/10/01/a-fond-farewell\">farewell post</a> on the JSL blog too.</p>\n<p>So, what’s next? To be honest, I’m not fully sure. I’m open to suggestions, and I give bonus points for interesting.</p>\n<p>Things I want out of life:</p>\n<ol>\n<li>Travel. I still like doing this. I just don’t want to do it <em>constantly</em>.</li>\n<li>Related: some degree of location independence. I’ll only accept positions that require me to be in a seat that’s in San Francisco or New York City. Remote would be even better. Probably.</li>\n<li>Piles of money. I have student loan debt I’d prefer not to have.</li>\n<li>Side project ability. I am highly motivated by doing side projects that are different than I do all day.</li>\n</ol>\n<p>… I’m not saying that Ruby is a minus, but I’m not saying it’s a plus, either.</p>\n<p>I’m also open to re-locating out of the country. That certainly counts as interesting.</p>\n<p>So, if you think we might work well together, please, <a href=\"mailto:[email protected]\">send me an email</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/keep-saturdays-sacred/",
"title": "Keep Saturdays sacred",
"description": null,
"url": "https://steveklabnik.com/writing/keep-saturdays-sacred/",
"published": "2013-09-28T00:00:00.000Z",
"updated": "2013-09-28T00:00:00.000Z",
"content": "<p>There are obviously a lot of factors that have contributed to my career as a programmer, but I think the most important one can be summed up in this sentence: Keep Saturdays sacred.</p>\n<p>You see, in college, my friends and I were moderately obsessed with writing software. We sorta stumbled onto a tradition of making sure that every Saturday, we got together and wrote code. At some point, we realized that this was the formula:</p>\n<ol>\n<li>If none of us set our alarms, we’d all wander into the computer lab at 1pm Saturday.</li>\n<li>We then went and got some burritos from Chipotle.</li>\n<li>Once we were appropriately stuffed, we’d go back to the lab and hack on whatever.</li>\n<li>In our town, food was half off at certain restaurants after 11pm. So we’d code until then, and then go get some food, and then go home later.</li>\n</ol>\n<p>Many people sifted in and out of this group over time, but a core was <em>always</em> there. Every week. For two or three years. I was one of those people.</p>\n<p>Looking back, I think that time was where I grew the most as a software developer. I’ve been coding since I was very young, but didn’t really know what I was doing until that point. It’s where I tried out new stuff. Argued about methodology. Learned something about something.</p>\n<p>We weren’t even always working on the same projects. They were often quite different, and changed all the time. But it was a safe place to try things, a safe space to fail. A place to get support and help. To socialize and figure out just what the hell we were even doing.</p>\n<p>It was also an incredible privilege. Over the last few years, I’ve been too busy to keep that going. Life gets in the way. It’s hard when you don’t have that group. Maybe the solution, then, isn’t that it’s all day on a Saturday. Maybe it’s just finding <em>some</em> kind of gap time, but doing it every week.</p>\n<p>Today, I’m doing work on http://www.designinghypermediaapis.com/ and the Rust compiler. What about you?</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/thankfulness/",
"title": "Thankfulness",
"description": null,
"url": "https://steveklabnik.com/writing/thankfulness/",
"published": "2013-09-24T00:00:00.000Z",
"updated": "2013-09-24T00:00:00.000Z",
"content": "<p>I recently attended GoGaRuCo, which was a great conference. My favorite part, however, wasn’t directly attatched to the conf itself: it was a few moments of thankfulness.</p>\n<p>A lot of what I’ve been experiencing lately has been profoundly negative. That is what it is; there’s a lot of realy negative things in the world. But I’ve been a bit… overloaded lately.</p>\n<p>The two moments tie into each other:</p>\n<p>I’m at the speaker dinner. I bumped into someone I hadn’t met or said hi to yet. “Oh hey Steve, I’m <a href=\"https://twitter.com/samsaffron\">Sam</a>.” “Oh hi! You don’t know how many times in the last few weeks I’ve said ‘Sam Saffron makes me so happy.’” You see, Sam works on <a href=\"http://www.discourse.org/\">Discourse</a>, and it’s a Rails app. Sam’s work on Discourse has identified some bugs and performance regressions in the Rails codebase, and Sam has been submitting some great pull requests upstream. Lately, I’ve been feeling like nobody who uses Rails actually cares how the sausage is made, and so Sam has been a bright spot in my day. So I told him so, straight to his face. It felt really good to say thanks, and I hope I made Sam’s day a bit brighter, as well.</p>\n<p>A couple times at the conference, someone would walk up to me and say basically the same thing for some other reason. It felt really good to have my work appreciated; the day-to-day grind of GitHub issue triaging can really make you feel crappy. Basically:</p>\n<p><img src=\"http://s3.amazonaws.com/theoatmeal-img/comics/making_things/14.jpg\" alt=\"http://s3.amazonaws.com/theoatmeal-img/comics/making_things/14.jpg\" /></p>\n<p>negative comments</p>\n<p><a href=\"http://theoatmeal.com/comics/making_things\">The Oatmeal</a> is often problematic, but this one really rings true.</p>\n<p>Anyway, I’m not trying to fish for compliments here. I have a fairly high profile, and I hear this more than most. What I <em>do</em> want to emphasize, though, is that I’m sure it’s not just me that feels this way.</p>\n<p>So, if someone does something that you appreciate, please take a moment to say thank you. <em>Especially</em> if they’re not someone who happened to get up on a stage and give a talk. Be specific: a generic ‘thanks for what you do’ is nice, but ‘oh man I lost two hours on that bug and your patch saved me’ is even nicer.</p>\n<p>You never know. They might have been having a bad day too, your thanks might turn their day around.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-closure-companion/",
"title": "The CLOSURE companion",
"description": null,
"url": "https://steveklabnik.com/writing/the-closure-companion/",
"published": "2013-09-19T00:00:00.000Z",
"updated": "2013-09-19T00:00:00.000Z",
"content": "<p>Today is the fourth <a href=\"http://whyday.org/\">_why day</a>. So I thought I’d say a few words about his final gift to the rest of us, <a href=\"http://words.steveklabnik.com/closure\">CLOSURE</a>.</p>\n<h2>Be Skeptical</h2>\n<p>First, a few words about the words I’m about to write. Software developers (though you don’t have to be one to read this) love meta, so surely you’ll indulge me for these next few paragraphs. Don’t worry, it’s only a little bit of meta-data.</p>\n<blockquote>\n<p>And if you don’t understand some of my language, things like “Open GL” and “fighting NULL” and so on, you can skip it and be fine I think. - _why, CLOSURE</p>\n</blockquote>\n<blockquote>\n<p>Nowadays people are actually somewhat jaded by the term ``postmodern’’. Well, perhaps jaded is an understatement. Nauseated might be more like it. - Larry Wall, “Perl, the first post-modern programming language”</p>\n</blockquote>\n<blockquote>\n<p>I define postmodern as incredulity toward meta-narratives, - Lyotard, “The Postmodern Condition: a report on knowledge”</p>\n</blockquote>\n<p>_why’s work has always taken on a strong element of the postmodern, and CLOSURE is no different. So in order to really get CLOSURE, you need to get a little postmodern. So let’s breifly examine this statement by Lyotard.</p>\n<p>A ‘narrative’, to Lyotard, is a story that organizes tribal knowledge, the structure of our life and society. Therefore, a meta-narrative is a narrative about a narrative. As an example, let’s say you want to make sense of programming languages. A meta-narrative about programming languages would be that in the beginning, we started off with assembly language. Then we moved forward to structured programming, with Pascal and C and such. Next came object oriented languages, which is where we’re mostly at today. We’re starting to finally lean functional.</p>\n<p>Now, this narrative works. It’s not <em>wrong</em>. But it also isn’t really <em>true</em>, either. After all, Lisp existed waaaay before the contemporary moment, yet it’s often functional. Actually, presenting Lisp as functional is in of itself not exactly true. So this meta-narrative, while it helps us understand a bit about programming languages and their history, also has a bunch of lies. It’s not black and white, it’s quite gray.</p>\n<blockquote>\n<p>\"Well, it doesn’t matter – no one’s going to listen to either of us. I can say whatever I want at this point. You can tell the truth and no one would care.“That’s amazing.”“I’m going to change my story weekly – the more confusion the better. The Jonathan story is locked.”_why, CLOSURE, HELLOYES (p56)</p>\n</blockquote>\n<p>So what I’m saying is this: I’m about to present you with a meta-narrative about CLOSURE. So be skeptical of me! What I’m about to tell you is a lie, I swear. Don’t take what’s about to come as the final word, some sort of actual truth. It’s just my own little anti-memory that I’d like to share with you.</p>\n<h2>What CLOSURE is</h2>\n<p>CLOSURE is a few different things. 90% of what you need to know is located in <a href=\"https://github.com/steveklabnik/CLOSURE/blob/master/PDF/DISCLAIMER.pdf\">DISCLAIMER</a>, which is page 13 of CLOSURE. In it, _why mentions a few things:</p>\n<ol>\n<li>It’s ‘perilous to communicate this way’</li>\n<li>Everything he’s done belongs to us</li>\n<li>He likes what we’ve done so far</li>\n<li>This will be the last thing from him.</li>\n</ol>\n<p>#4 is a big deal. No more waiting for _why to come back. That’s it. No more questions, no more wondering, no more guesses. This is all we have.</p>\n<p>For me, #3 was the most important. It brought me to tears, to be honest. That’s what I personally really needed to hear. It’s very strange to take up the work of someone you’ve never met, and make their life’s work your own. I spent quite a bit of time really wondering if keeping some of his stuff alive was the correct thing to do, and so this was a big relief. It was all gravy after that. #2 is related, it’s all public domain. That’s good to know.</p>\n<h2>Identity</h2>\n<p>#1 is interesting. It could just be flavor, but I think it’s something more than that. One of the major themes of CLOSURE is identity, and _why / Jonathan ’s struggles with it. On page 19:</p>\n<blockquote>\n<p>Now, could you please tell me the point of this ridiculous anonymity exercise, hmm?As it turns out, oddly enough, your real self is just an unknown programmer from Utah. The myth is that easily dispelled. Why not making something of your real self? (Of course I know why and can tell you: Because your fear of the world has clouded your ability to do things to improve your situation. You are stuck there in Draper, Utah, until you can cut through the paranoia!)Please, Mr. Gillette, come on in. The water’s fine. ;)Emery Pestus, CLOSURE</p>\n</blockquote>\n<p>While many hold _why as some kind of superhero, it’s really important to remember that he’s human. We, the Ruby community, placed an indescribable amount of pressure on a human being to play a character for us, without consideration for his feelings. Think about this for a moment: could Jonathan ever write Ruby code? As Jonathan? What if it had unit tests? If it was just some old, normal, boring gem that added a little feature to Rails?</p>\n<p>Could we let him just be him? I’m not sure. I’m not sure he could let him just be him, either.</p>\n<p>Once, I met someone for dinner in Chicago. This was the first time we’d met, and so we were asking each other a lot of questions. One of my answers yielded an interesting response: “Oh, you’re like actually for-real. All that twitter stuff isn’t an act. That’s really you.”</p>\n<h2>Impermanance</h2>\n<p>Impermanence is possibly the biggest question raised in CLOSURE.</p>\n<blockquote>\n<p>kafka would be a lot harder to get into if the trial only ran on a power pc. - one of _why’s last tweets</p>\n</blockquote>\n<p>This tweet was really confusing, until CLOSURE. _why reveals that one of his biggest problems is what we call ‘bitrot’: you can’t just write a program, it must be updated. I have one of the first C programs I ever wrote, from when I was 12, and it wouldn’t compile on a modern system, due to a ‘clear screen’ function I wrote that interacted with the hardware that I owned at the time. I don’t have any of the GW-BASIC programs that I wrote as a child, because there was no way to transfer the source off of the computer: I didn’t have a disk drive, they were too expensive.</p>\n<p>And so it is with Kafka. Right before he died, Kafka asked his friend Max Brod to burn everything he ever wrote. Brod published it instead. _why brings up that if The Trial was written for the PowerPC, he wouldn’t have needed Brod to burn it: it would have just naturally gone away.</p>\n<p>Our industry is constantly changing, and that’s great. But we have no institutional memory. We keep making the same mistakes, over and over again. _why talks about fighting NULL, and how that was his biggest struggle as a programmer. The guy who invented null pointers, Tony Hoare, calls it <a href=\"http://www.infoq.com/presentations/Null-References-The-Billion-Dollar-Mistake-Tony-Hoare\">“my billion dollar mistake”</a>. Yet Go has null pointers.</p>\n<p>It’s really easy to burn out. I won’t lie, when _why deleted himself, I thought it was a terribly silly idea. But the more stuff I do, the bigger my projects and responsibilities get, and the more of a public person I am, the more it sounds appealing.</p>\n<p>When I had a shitty band in high school, I wrote a dumb song after a girl dumped me called “The Only Constant is Change.” I thought it was clever at the time.</p>\n<h2>The Software Industry</h2>\n<p>Speaking of the industry, and burnout, that’s the last part of CLOSURE, and the hardest. The entire end of the book is about it, and it’s a very obvious, very ridiculous allegory.</p>\n<p>There is a cult of Steve Jobs, and they speak incorrect French. They obsess over these flutes, and the flutes can make more flutes, and when one of them gets old and dies, they forget him immediately. When a new person is born, they start worshipping him, and induct him into the cult as quickly as possible, by clothing him in blue jeans and a black turtleneck. The cult members never eat any food.</p>\n<p>The cult is programmers, the flutes are programming. It all falls out of that. Songs on the flute always start with C#, which I found amusing.</p>\n<blockquote>\n<p>It occured to me that they could use the flutes to describe a whole landscape, to transfer a map from one man’s mind to another._why, CLOSURE, page 84: SPAWN</p>\n</blockquote>\n<p>There’s also a really fantastic comparison between Steve Jobs and Bill Gates, as well as an angel investor who wants to invest in _why: I highly recommend that part.</p>\n<blockquote>\n<p>For instance, many people struggle with acceptance, feeling like they aren’t accepted by other people. But what I deal with is primarily hatred of entrepreneurs. But it’s something that I’m always working on and I’ve gotten much better.</p>\n</blockquote>\n<p>I’ve come to realize that I’m starting to become a grumpy old man. These damn kids never learn from their elders, we solved all the world’s problems back in the 60s and 70s, and they don’t know anything. I wish we had a better way to transfer institutional knowledge and stop repeating ourselves.</p>\n<h2>Some final thoughts</h2>\n<p>There is so much more that’s enjoyable in CLOSURE. Please check out the full thing, even if you don’t get all of it. For example, the part on Neil Gaiman is amazing.</p>\n<p>I hope these jumping off points helps you in your journey. Let’s all have some fun with programming, okay?</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/resque-1-25-0-has-been-released/",
"title": "Resque 1.25.0 has been released",
"description": null,
"url": "https://steveklabnik.com/writing/resque-1-25-0-has-been-released/",
"published": "2013-09-16T00:00:00.000Z",
"updated": "2013-09-16T00:00:00.000Z",
"content": "<p>I’ve just released Resque 1.25.0! Resque is the most stable, widely-used job queue for Ruby. It uses Redis as its backing store.</p>\n<p>This release is thanks to <a href=\"https://github.com/adelcambre\">adelcambre</a>, who took the time to get it over the finish line! <3.</p>\n<p>A total of 19 contributors helped out with this release, with 77 commits. Thank you to everyone!</p>\n<p>This release contains no changes from the 1.25.0.pre release that was put out a few weeks ago, so if you’re using it, you can upgrade with no problems.</p>\n<h2>CHANGES</h2>\n<p>You can see the full CHANGELOG <a href=\"https://github.com/resque/resque/blob/v1.25.0/HISTORY.md\">here</a>.</p>\n<p>This release had a pre-release because of one big change: <a href=\"https://github.com/resque/resque/pull/1017/files\">a refactor to the forking code</a>. This was introduced to fix a problem: the exit syscall raises a SystemExit Exception in ruby, its exit is treated as a failure, thus deregistering the worker. After the refactor + fix, things should be peachy. But please give the prerelease a try.</p>\n<p>In addition, it has been discovered that we have accidentally introduced a SemVer violation back in v1.23.1: https://github.com/resque/resque/issues/1074</p>\n<p>I take SemVer seriously, and apologize for this change. SemVer states that at this time, I should basically do whatever. Since this fixes a bug, I’m inclined to leave it in. If this introduces a significant problem for you, please let me know. Since others might now be relying on this new behavior, I’m not 100% sure what the best option is. Your feedback helps.</p>\n<h2>INSTALLING</h2>\n<p>To get 1.25.0 installed, just use <code>gem install</code>:</p>\n<pre><code>$ gem install resque\n</code></pre>\n<p>Or include it in your <code>Gemfile</code>:</p>\n<pre><code>gem 'resque', '~> 1.25.0'\n</code></pre>\n<h2>SOURCE</h2>\n<p>You can find the source <a href=\"https://github.com/resque/resque\">on GitHub</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/just-the-regularly-scheduled-apocalypse/",
"title": "Just the regularly scheduled apocalypse",
"description": null,
"url": "https://steveklabnik.com/writing/just-the-regularly-scheduled-apocalypse/",
"published": "2013-09-10T00:00:00.000Z",
"updated": "2013-09-10T00:00:00.000Z",
"content": "<p>Every once in a while, I unfollow EVERYONE on Twitter. I did it again last night. Feels good, like a fresh start.</p>\n<p>Part of the problem is that I feel like I should be following everyone who I know in real life. This causes Twitter to be absolutely unmanageable, as I’m following far too many. So then I just dip into the stream every so often, but then I miss some people’s tweets entirely. So why follow them all in the first place?</p>\n<p>So, if I’ve unfollowed you, don’t worry, I still <3 you. And even as I start following people back, I’m doing it slowly, and at semi-random. Trying to get some people I hadn’t seen in a while. People from outside my own community. People that’ll make me a little less caremad.</p>\n<p>Yeah, like that’ll last.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/emoji-licensing/",
"title": "Emoji licensing",
"description": null,
"url": "https://steveklabnik.com/writing/emoji-licensing/",
"published": "2013-08-06T00:00:00.000Z",
"updated": "2013-08-06T00:00:00.000Z",
"content": "<p>I recently decided to make an emoji gem. Of course, as with any project, I first look into the relevant licensing issues. What I found made me <code>:cry:</code>. Here’s the low-down on emoji and intellectual property law.</p>\n<h2>A history</h2>\n<p>So what <em>are</em> emoji, anyway? The Japanese spelling is 絵文字: 絵 (e) means ‘picture’ and 文字 (moji) means ‘letter.’ Picture letters. Simple.</p>\n<p>Back in the day, three different Japanese phone carriers created their own versions of these smilies: docomo, au, and SoftBank. Codes would be sent down the (nonexistant) wire, which would then be turned into the final character. However, the three carriers used different characters and different codes.</p>\n<p>Google and Apple petitioned the Unicode Consortium to include a number of these characters, and version 6.0 of the Unicode Standard ended up with 722 emoji.</p>\n<h2>The font</h2>\n<p>Apple ended up implementing a font named Apple Color Emoji that included these symbols. It came first with OS X Lion and iOS 5. These are the characters that you’re probably familliar with. Apple actually implements the font with a proprietary extension to OpenType, but that’s not particularly important. What <em>is</em> important is two things: typing a character that’s not on a normal keyboard, and displaying custom fonts on the web. These two problems have led to the current emoji IP situation.</p>\n<h3>Typing extended characters</h3>\n<p>On an iPhone, you can type emoji easily by enabling the extra keyboard. That’s not hard. But what about on a computer? Web sites that wanted to use emoji needed an easy way for you to type these characters in an input box.</p>\n<p>I’m not sure who was first, but the first I remember is GitHub: with their <a href=\"https://github.com/blog/816-emoji\">announcement post</a>, showed off a slick way to input emojis: use a set of characters that would get replaced with the emoji. So <code>:heart:</code> becomes a heart. Easy!</p>\n<h3>Custom fonts in browsers</h3>\n<p>That takes care of input, but what about output? If you don’t have the proper font installed, you’ll get those terrible boxes. That’s not very useable. Also, even if you do, <a href=\"http://code.google.com/p/chromium/issues/detail?id=90177\">Chrome won’t display them</a>.</p>\n<p>So what do you do? You use your good buddy <code><img></code>.</p>\n<p>If you’re building a web application, you are already telling users to type <code>:heart:</code> rather than the character, you can just make <code>:heart:</code> be rendered as <code><img src=\"/assets/heart.png\"></code>. So that’s easy…. but where do you get the PNGs?</p>\n<h2>Apple Color Emoji</h2>\n<p>Remember that extension? <a href=\"https://color-emoji.googlecode.com/git/specification/v1.html\">The spec was published</a>, and is being worked in to <a href=\"http://google-opensource.blogspot.de/2013/05/open-standard-color-font-fun-for.html\">FreeType</a>.</p>\n<p>Anyway, the extension works by embedding PNGs into the font. I think that you can see where this is going… just rip the images out of the font, and you’re golden. Well, other than that pesky IP law…</p>\n<h2>Extensions</h2>\n<p>Furthermore, now that you’re not constrained by the unicode standard, you can make anything between two colons an emoji. Like <code>:shipit:</code>. Several services do this. <a href=\"https://github.com/arvida/emoji-cheat-sheet.com/blob/master/LICENSE\">http://emoji-cheat-sheet.com lists the licenses as such</a>:</p>\n<pre><code>octocat, squirrel, shipit\nCopyright (c) 2012 GitHub Inc. All rights reserved.\n\nbowtie\nCopyright (c) 2012 37signals, LLC. All rights reserved.\n\nneckbeard\nCopyright (c) 2012 Jamie Dihiansan. Creative Commons Attribution 3.0 Unported\n\nfeelsgood, finnadie, goberserk, godmode, hurtrealbad, rage 1-4, suspect\nCopyright (c) 2012 id Software. All rights reserved.\n\ntrollface\nCopyright (c) 2012 whynne@deviantart. All rights reserved.\n\nAll other emoji images\nCopyright (c) 2012 Apple Inc. All rights reserved.\n</code></pre>\n<p>I made some inquries into all of these extended emoji, and GitHub said they’d get back to me, but didn’t. Therefore, their three are All Rights Reserved.</p>\n<p>37Signals had said <code>:neckbeard:</code> is CC licensed, and in an email, said they’d be happy to license <code>:bowtie:</code> the same way.</p>\n<p>The inventor of Trollface has yet to return my email. Go figure. Also All Rights Reserved.</p>\n<p>Finally, while DOOM had its code released under the GPL, the assets were very clearly not. All Rights Reserved there too.</p>\n<p>I didn’t even bother emailing Apple.</p>\n<h2>What’s this mean for me?</h2>\n<p>Basically, <strong>if you’re distributing the images from Apple Color emoji, you are violating copyright law</strong>. And if you’re not one of the above mentioned entities, you’re violating their licenses, as well.</p>\n<p>So that’s a big downer. What to do?</p>\n<h2>Phantom Open Emoji</h2>\n<p>At some point, there was a kickstarter for an open emoji set. And it worked! And so <a href=\"https://github.com/Genshin/PhantomOpenEmoji\">Phantom Open Emoji</a> was born.</p>\n<p>Phantom Open Emoji basically gives you images to use as all of the glyphs used in Apple Open Emoji, but you have a license to use them however you’d like. So that’s pretty awesome.</p>\n<hr />\n<p>I wrote this post because I was writing a <a href=\"http://rubygems.org/gems/emoji\">Ruby gem for emoji</a>, and needed to know if I could distribute the emoji with the gem. It’s not done yet (it’s barely started), but if you want to use emoji with your app, it will be the simplest and safest way to make sure you don’t run afoul of this kind of thing.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/redis-namespace-1-3-1--security-release/",
"title": "redis-namespace 1.3.1 security release",
"description": null,
"url": "https://steveklabnik.com/writing/redis-namespace-1-3-1--security-release/",
"published": "2013-08-03T00:00:00.000Z",
"updated": "2013-08-03T00:00:00.000Z",
"content": "<p>TL;DR: if you use <a href=\"https://rubygems.org/gems/redis-namespace\">redis-namespace</a> and you use <code>send</code>, <code>Kernel#exec</code> may get called. Please upgrade to 1.0.4, 1.1.1, 1.2.2, or 1.3.1 immediately.</p>\n<p>Link to the fix: <a href=\"https://github.com/resque/redis-namespace/commit/6d839515e8a3fdc17b5fb391500fda3f919689d6\">https://github.com/resque/redis-namespace/commit/6d839515e8a3fdc17b5fb391500fda3f919689d6</a></p>\n<h2>The Problem</h2>\n<p>Redis has an EXEC command. We handle commands through <code>method_missing</code>. This works great, normally:</p>\n<pre><code> r = Redis::Namespace.new(\"foo\") r.exec # => error, not in a transaction, whatever\n</code></pre>\n<p>Here’s the problem: <code>Kernel#exec</code>. Since this is on every object, when you use <code>#send</code>, it skips the normal visibility, and calls the private but defined <code>Kernel#exec</code> rather than the <code>method_missing</code> verison:</p>\n<pre><code> r = Redis::Namespace.new(\"foo\") r.send(:exec, \"ls\") # => prints out your current directory\n</code></pre>\n<p>We fix this by not proxying <code>exec</code> through <code>method_missing</code>.</p>\n<p>You are only vulnerable if you do the always-dangerous ‘send random input to an object through <code>send</code>.’ And you probably don’t. A cursory search through GitHub didn’t find anything that looked vulnerable.</p>\n<p>However, if you write code that wraps or proxies Redis in some way, you may do something like this.</p>\n<p>The official Redis gem does not use <code>method_missing</code>, so it should not have any issues: https://github.com/redis/redis-rb/blob/master/lib/redis.rb#L2133-L2147</p>\n<p>I plan on removing the <code>method_missing</code> implementation in a further patch, but since this is an immediate issue, I wanted to make minimal changes.</p>\n<p>Testing this is hard, <code>#exec</code> replaces the current running process. I have verified this fix by hand, but writing an automated test seems difficult.</p>\n<p>:heart::cry:</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/beware-subclassing-ruby-core-classes/",
"title": "Beware subclassing Ruby core classes",
"description": null,
"url": "https://steveklabnik.com/writing/beware-subclassing-ruby-core-classes/",
"published": "2013-07-24T00:00:00.000Z",
"updated": "2013-07-24T00:00:00.000Z",
"content": "<p>TL;DR: Subclassing core classes in Ruby can lead to unexpected side effects. I suggest composition over inheritance in all these cases.</p>\n<h2>Subclassing Review</h2>\n<p>If you’re familiar with the concept of subclassing, skip down to “The Problem.”</p>\n<p>In Ruby, you can make your own classes:</p>\n<pre><code>class List\nend\n</code></pre>\n<p>You can also make subclasses of those classes:</p>\n<pre><code>class OrderedList < List\nend\n\nputs OrderedList.new.kind_of?(List) # => true\n</code></pre>\n<p>Now, subclassing represents an “is a” relationship. This means that our <code>OrderedList</code> should be a <code>List</code> in every respect, but with some added behavior. The <a href=\"http://en.wikipedia.org/wiki/Liskov_substitution_principle\">Liskov Substitution Principle</a> is one formulation of this idea.</p>\n<h2>The Problem</h2>\n<p>Ruby has two major bits of code that it provides for your use: the core library and the standard library. <a href=\"http://www.ruby-doc.org/core-2.0/\">The core library can be found here</a>, and contains cllasses that you know and love, like <code>String</code>, <code>Hash</code>, and <code>Array</code>. <a href=\"http://www.ruby-doc.org/stdlib-2.0/\">The standard library can be found here</a>, and contains your favorite hits, like <code>CSV</code>, <code>JSON</code>, and <code>Logger</code>.</p>\n<p>One way to think about the difference between core and the standard library is that core is written in C, while the standard library is written in Ruby. Core are the classes that are used the most, so they’re implemented in as low-level a fashion as possible. They’ll be in every single Ruby program, so might as well make them fast! The standard library only gets pulled in by bits and pieces; another way of thinking about the difference is that you need to <code>require</code> everything in the standard library, but nothing in core.</p>\n<p>What do you think this code should do?</p>\n<pre><code>class List < Array\nend\n\nputs List.new.to_a.class\n</code></pre>\n<p>If you said “it prints <code>Array</code>,” you’d be right. This behavior really confuses me, though, because <code>List</code> is already an <code>Array</code>; in my mind, this operation shouldn’t suddenly change the class.</p>\n<p>Why does this happen? Let’s check out the implementation of <a href=\"https://github.com/ruby/ruby/blob/trunk/array.c#L2064-L2082\">Array#to_a</a>:</p>\n<pre><code>static VALUE\nrb_ary_to_a(VALUE ary)\n{\n if (rb_obj_class(ary) != rb_cArray) {\n VALUE dup = rb_ary_new2(RARRAY_LEN(ary));\n rb_ary_replace(dup, ary);\n return dup;\n }\n return ary;\n}\n</code></pre>\n<p>If the class is not an <code>Array</code>, (represented by <code>rb_cArray</code>), then we make a new array of the same length, call <code>replace</code> on it, and then return the new array. If this C scares you, here’s a direct port to pure Ruby:</p>\n<pre><code>def array_to_a(ary)\n if ary.class != Array\n dup = []\n dup.replace(ary)\n return dup\n end\n return ary\nend\n\narray_to_a(List.new).class # => Array\n</code></pre>\n<p>So why do this? Well, again, this class will be used all over the place. For example, I made a brand new Rails 4 application, generated a controller and view, and put this in it:</p>\n<pre><code>ObjectSpace.count_objects[:T_ARRAY]: <%= ObjectSpace.count_objects[:T_ARRAY] %>\n</code></pre>\n<p>ObjectSpace allows you to inspect all of the objects that exist in the system. Here’s the output:</p>\n<p><img src=\"http://i.imgur.com/PVwdvyY.png\" alt=\"http://i.imgur.com/PVwdvyY.png\" /></p>\n<p>rails arrays</p>\n<p>That’s a lot of arrays! This kind of shortcut is generally worth it: 99.99% of the time, this code is perfect.</p>\n<p>That last 0.01% is the problem. If you don’t know exactly how these classes operate at the C level, you’re gonna have a bad time. In this case, this behavior is odd enough that someone was kind enough to document it.</p>\n<p>Here’s the Ruby version of what I’d expect to happen:</p>\n<pre><code>def array_to_a2(ary)\n return ary if ary.is_a?(Array)\n dup = []\n dup.replace(ary)\n dup\nend\n\narray_to_a2(List.new).class # => List\n</code></pre>\n<p>This has the exact same behavior except when we’re already dealing with an Array, which is what I’d expect.</p>\n<p>Let’s take another example: reverse.</p>\n<pre><code>l = List.new\nl << 1\nl << 2\nputs l.reverse.class # => Array\n</code></pre>\n<p>I would not expect that calling <code>#reverse</code> on my custom <code>Array</code> would change its class. Let’s <a href=\"https://github.com/ruby/ruby/blob/trunk/array.c#L2138-L2161\">look at the C</a> again:</p>\n<pre><code>static VALUE\nrb_ary_reverse_m(VALUE ary)\n{\n long len = RARRAY_LEN(ary);\n VALUE dup = rb_ary_new2(len);\n\n if (len > 0) {\n const VALUE *p1 = RARRAY_RAWPTR(ary);\n VALUE *p2 = (VALUE *)RARRAY_RAWPTR(dup) + len - 1;\n do *p2-- = *p1++; while (--len > 0);\n }\n ARY_SET_LEN(dup, RARRAY_LEN(ary));\n return dup;\n}\n</code></pre>\n<p>We get the length of the array, make a new blank array of the same length, then do some pointer stuff to copy everything over, and return the new copy. Unlike <code>#to_a</code>, this behavior is <em>not</em> currently documented.</p>\n<p>Now: you could make the case that this behavior is expected, in both cases: after all, the point of the non-bang methods is to make a copy. However, there’s a difference to me between “make a new array with this stuff in it” and “make a new copy with this stuff in it”. Most of the time, I get the same class back, so I expect the same class back in these circumstances.</p>\n<p>Let’s talk about a more pernicious issue: Strings.</p>\n<p>As you know, the difference between interpolation and concatenation is that interpolation calls <code>#to_s</code> implicitly on the object it’s interpolating:</p>\n<pre><code>irb(main):001:0> \"foo\" + 2\nTypeError: no implicit conversion of Fixnum into String\n from (irb):1:in `+'\n from (irb):1\n from /opt/rubies/ruby-2.0.0-p195/bin/irb:12:in `<main>'\nirb(main):002:0> \"foo#{2}\"\n=> \"foo2\"\nirb(main):001:0> class MyClass\nirb(main):002:1> def to_s\nirb(main):003:2> \"yup\"\nirb(main):004:2> end\nirb(main):005:1> end\n=> nil\nirb(main):006:0> \"foo#{MyClass.new}\"\n=> \"fooyup\"\n</code></pre>\n<p>So what about a custom <code>String</code>?</p>\n<pre><code>class MyString < String\n def to_s\n \"lol\"\n end\nend\n\ns = MyString.new\ns.concat \"Hey\"\n\nputs s\nputs s.to_s\nputs \"#{s}\"\n</code></pre>\n<p>What does this print?</p>\n<pre><code>$ ruby ~/tmp/tmp.rb HeylolHey\n</code></pre>\n<p>That’s right! With <code>String</code>s, Ruby doesn’t call <code>#to_s</code>: it puts the value in directly. How does this happen?</p>\n<p>Well, dealing with string interpolation deals with the parser, so let’s check out the bytecode that Ruby generates. Thanks to <a href=\"http://twitter.com/tenderlove\">Aaron Patterson</a> for suggesting this approach. <3</p>\n<pre><code>irb(main):013:0> x = RubyVM::InstructionSequence.new(%q{puts \"hello #{'hey'}\"})\n=> <RubyVM::InstructionSequence:<compiled>@<compiled>>\nirb(main):014:0> puts x.disasm\n== disasm: <RubyVM::InstructionSequence:<compiled>@<compiled>>==========\n0000 trace 1 ( 1)\n0002 putself\n0003 putstring \"hello hey\"\n0005 opt_send_simple <callinfo!mid:puts, argc:1, FCALL|ARGS_SKIP>\n0007 leave\n=> nil\nirb(main):015:0> x = RubyVM::InstructionSequence.new(%q{puts \"hello #{Object.new}\"})\n=> <RubyVM::InstructionSequence:<compiled>@<compiled>>\nirb(main):016:0> puts x.disasm\n== disasm: <RubyVM::InstructionSequence:<compiled>@<compiled>>==========\n0000 trace 1 ( 1)\n0002 putself\n0003 putobject \"hello \"\n0005 getinlinecache 12, <ic:0>\n0008 getconstant :Object\n0010 setinlinecache <ic:0>\n0012 opt_send_simple <callinfo!mid:new, argc:0, ARGS_SKIP>\n0014 tostring\n0015 concatstrings 2\n0017 opt_send_simple <callinfo!mid:puts, argc:1, FCALL|ARGS_SKIP>\n0019 leave\n=> nil\n</code></pre>\n<p>You can see with a string, the bytecode actually puts the final concatenated string. But with an object. it ends up calling <code>tostring</code>, and then <code>concatstrings</code>.</p>\n<p>Again, 99% of the time, this is totally fine, and much faster. But if you don’t know this trivia, you’re going to get bit.</p>\n<p><a href=\"https://rails.lighthouseapp.com/projects/8994/tickets/6023-unable-to-find-by-subclasses-of-string-in-activerecord#ticket-6023-5\">Here</a> is an example from an older version of Rails. Yes, you might think “Hey idiot, there’s no way it will store your custom <code>String</code> class,” but the whole idea of subclassing is that it’s a drop-in replacement.</p>\n<p>I know that there’s some case where Ruby will not call your own implementation of <code>#initialize</code> on a custom subclass of <code>String</code>, but I can’t find it right now. This is why this problem is so tricky: most of the time, things are fine, but then occasionally, something strange happens and you wonder what’s wrong. I don’t know about you, but my brain needs to focus on more important things than the details of the implementation.</p>\n<p>Since I first wrote this post, <a href=\"http://twitter.com/jeg2\">James Edward Gray II</a> helped me remember what this example is. One of the early exercises in <a href=\"http://exercism.io/\">http://exercism.io/</a> is based on making a DNA type, and then doing some substitution operations on it. Many people inherited from <code>String</code> when doing their answers, and while the simple case that passes the tests works, this case won't:</p>\n<pre><code>class Dna < String\n def initialize(*)\n super\n puts \"Building Dna: #{inspect}\"\n end\nend\n\nresult = Dna.new(\"CATG\").tr(Dna.new(\"T\"), Dna.new(\"U\"))\np result.class\np result\n</code></pre>\n<p>This prints:</p>\n<pre><code>Building Dna: \"CATG\"\nBuilding Dna: \"T\"\nBuilding Dna: \"U\"\nDna\n\"CAUG\"\n</code></pre>\n<p>It never called our initializer for the new string. Let's check <a href=\"https://github.com/ruby/ruby/blob/trunk/string.c#L5484-L5525\">the source of <code>#tr</code></a>:</p>\n<pre><code>static VALUE\nrb_str_tr(VALUE str, VALUE src, VALUE repl)\n{\n str = rb_str_dup(str);\n tr_trans(str, src, repl, 0);\n return str;\n}\n</code></pre>\n<p><code>rb_str_dup</code> has a pretty simple definition:</p>\n<pre><code>VALUE\nrb_str_dup(VALUE str)\n{\n return str_duplicate(rb_obj_class(str), str);\n}\n</code></pre>\n<p>and so does <code>str_duplicate</code>:</p>\n<pre><code>static VALUE\nstr_duplicate(VALUE klass, VALUE str)\n{\n VALUE dup = str_alloc(klass);\n str_replace(dup, str);\n return dup;\n}\n</code></pre>\n<p>So there you have it: MRI doesn't go through the whole initialization process when duplicating a string: it just allocates the memory and then replaces the contents.</p>\n<p>If you re-open <code>String</code>, it's also weird:</p>\n<pre><code>class String\n alias_method :string_initialize, :initialize\n\n def initialize(*args, &block)\n string_initialize(*args, &block)\n puts \"Building MyString: #{inspect}\"\n end\nend\n\nresult = String.new(\"CATG\").tr(\"T\", \"U\") # => Building MyString: \"CATG\"\np result.class # => String\np result # => \"CAUG\"\n</code></pre>\n<p>Again, unless you know exactly how this works at a low level, surprising things happen.</p>\n<h2>The Solution</h2>\n<p>Generally speaking, subclassing isn’t the right idea here. You want a data structure that <em>uses</em> one of these core classes internally, but isn’t exactly like one. Rather than this:</p>\n<pre><code>class Name < String\nend\n</code></pre>\n<p>do this:</p>\n<pre><code>require 'delegate'\n\nclass Name < SimpleDelegator\n def initialize\n super(\"\")\n end\nend\n</code></pre>\n<p>This allows you to do the same thing, but without all of the pain:</p>\n<pre><code>class Name\n def to_s\n \"hey\"\n end\nend\n\n\"#{Name.new}\" # => \"hey\"\n</code></pre>\n<p>However, this won’t solve all problems:</p>\n<pre><code>require 'delegate'\n\nclass List < SimpleDelegator\n def initialize\n super([])\n end\nend\n\nl = List.new\nl << 1\nl << 2\nputs l.reverse.class # => Array\n</code></pre>\n<p>In general, I’d prefer to delegate things manually, anyway: a <code>Name</code> is not actually a drop-in for a <code>String</code> it’s something different that happens to be a lot like one:</p>\n<pre><code>class List\n def initialize(list = [])\n @list = list\n end\n\n def <<(item)\n @list << item\n end\n\n def reverse\n List.new(@list.reverse)\n end\nend\n\nl = List.new\nl << 1\nl << 2\nputs l.reverse.class # => List\n</code></pre>\n<p>You can clean this up by using <code>Forwardable</code> to only forward the messages you want to forward:</p>\n<pre><code>require 'forwardable'\n\nclass List\n extend Forwardable\n def_delegators :@list, :<<, :length # and anything else\n\n def initialize(list = [])\n @list = list\n end\n\n def reverse\n List.new(@list.reverse)\n end\nend\n\nl = List.new\nl << 1\nl << 2\nputs l.reverse.class # => List\n</code></pre>\n<p>Now you know! Be careful out there!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/resque-1-25-0-pre-has-been-released/",
"title": "Resque 1.25.0.pre has been released",
"description": null,
"url": "https://steveklabnik.com/writing/resque-1-25-0-pre-has-been-released/",
"published": "2013-07-23T00:00:00.000Z",
"updated": "2013-07-23T00:00:00.000Z",
"content": "<p>I’ve just released Resque 1.25.0.pre! Resque is the most stable, widely-used job queue for Ruby. It uses Redis as its backing store.</p>\n<p>This release is thanks to <a href=\"https://github.com/adelcambre\">adelcambre</a>, who took the time to get it over the finish line! <3</p>\n<p>A total of 19 contributors helped out with this release, with 77 commits. Thank you to everyone!</p>\n<h2>CHANGES</h2>\n<p>You can see the full CHANGELOG <a href=\"https://github.com/resque/resque/blob/v1.25.0.pre/HISTORY.md\">here</a>.</p>\n<p>I am releasing this version as ‘pre’ due to one big change: <a href=\"https://github.com/resque/resque/pull/1017/files\">a refactor to the forking code</a>. This was introduced to fix a problem: the exit syscall raises a SystemExit Exception in ruby, its exit is treated as a failure, thus deregistering the worker. After the refactor + fix, things should be peachy. But please give the prerelease a try.</p>\n<p>In addition, it has been discovered that we have accidentally introduced a SemVer violation back in v1.23.1: https://github.com/resque/resque/issues/1074</p>\n<p>I take SemVer seriously, and apologize for this change. SemVer states that at this time, I should basically do whatever. Since this fixes a bug, I’m inclined to leave it in. If this introduces a significant problem for you, please let me know. Since others might now be relying on this new behavior, I’m not 100% sure what the best option is. Your feedback helps.</p>\n<h2>INSTALLING</h2>\n<p>To get 1.25.0.pre installed, just use <code>gem install</code>:</p>\n<pre><code>$ gem install resque --pre\n</code></pre>\n<h2>SOURCE</h2>\n<p>You can find the source <a href=\"https://github.com/resque/resque\">on GitHub</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/removing-turbolinks-from-rails-4/",
"title": "Removing Turbolinks from Rails 4",
"description": null,
"url": "https://steveklabnik.com/writing/removing-turbolinks-from-rails-4/",
"published": "2013-06-25T00:00:00.000Z",
"updated": "2013-06-25T00:00:00.000Z",
"content": "<p>If you don’t want to use Turbolinks with your Rails 4 application, it’s easy! Just do this:</p>\n<ol>\n<li>Remove the <code>gem 'turbolinks'</code> line from your Gemfile.</li>\n<li>Remove the <code>//= require turbolinks</code> from your <code>app/assets/javascripts/application.js</code>.</li>\n<li>Remove the two <code>\"data-turbolinks-track\" => true</code> hash key/value pairs from your <code>app/views/layouts/application.html.erb</code>.</li>\n</ol>\n<p>Done!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/twitter-ghosts/",
"title": "Twitter ghosts",
"description": null,
"url": "https://steveklabnik.com/writing/twitter-ghosts/",
"published": "2013-06-22T00:00:00.000Z",
"updated": "2013-06-22T00:00:00.000Z",
"content": "<p>I’ve been thinking about death more than usual lately; my father died just before Father’s Day a year ago, and so I remembered, then was constantly reminded again on that day. But there’s another death that’s recently happened that really bothers me. The Internet is a really strange place.</p>\n<p>A few days ago, I was reading Twitter as I often do, and I saw a tweet that mentioned a death. I don’t remember which one it was, specifically, but I didn’t think a whole lot of it: death happens all the time, and this one referred to an accident. It basically just said something like “Whoah, Michael, it was way too soon to lose you. :(” I didn’t think much of it, but a few minutes later, I saw a few more tweets: “@mmhastings, amazing reporter, what a great guy, how tragic.” If a few of my followers all knew this dude, then he must be a good guy. So I clicked through to his Twitter profile:</p>\n<p><img src=\"http://i.imgur.com/uy8cRbB.png\" alt=\"http://i.imgur.com/uy8cRbB.png\" /></p>\n<p>Michael Hastings’ Twitter Profile</p>\n<p>Here’s what I noticed:</p>\n<ol>\n<li>At least his last tweet is a nice one.</li>\n<li>He has 20,000 followers. Nice. And he’s only following 1,400 people, so it’s probably a legit number, not spammy.</li>\n<li>Oh, how weird, he’s from Hollywood. I lived there for half of last year, and am still in LA.</li>\n<li>Wait, he’s following <em>me</em>? And I’m not following him?</li>\n</ol>\n<p>As the day went on, more and more articles came out, and I read them all. Apparently, Michael Hastings was a really awesome guy. Apparently he was one of the few real reporters, the ones who dig in deep, aren’t afraid to speak about what they find, and never backed down. <a href=\"http://www.rollingstone.com/politics/news/michael-hastings-rolling-stone-contributor-dead-at-33-20130618\">The story about his death in Rolling Stone</a> says “Hastings’ hallmark as reporter was his refusal to cozy up to power.”</p>\n<p>So here’s what weirds me out about this situation:</p>\n<p>First of all, wow, life is short. At any given time, if I just got hit by a car and died, I probably wouldn’t be all that proud of my last tweet. Isn’t it weird, that when someone’s life gets cut short, we can see what they were up to right before their death? I can now read the words of a man I’d never spoken to, and see what he was up to and what he cared about. This isn’t new news, there’s even a 4chan meme (from back in the early days of /b/) about a kid who committed suicide and all of his friends writing eulogies on his MySpace page. This is one of the first times it’s (in)directly affected me, though, and so it’s taken on a different tone.</p>\n<p>Second, why didn’t I know who this was? Apparently this was a man who was a really great person, the kind I would have liked. And apparently he saw enough value in my random ramblings on Twitter to pay attention to me. But why didn’t I reciprocate? We lived in the same place. It wouldn’t have been hard to go grab a beer sometime. I walked past the intersection that he died many, many times. Would I have been friends with this person in some alternate universe?</p>\n<p>One of the things that I always said about my father’s death was that at least he had two months where he knew he was going to die. While it was an incredible stress on everyone involved, at least he was able to say everything to everyone he wanted to say something to, wrap up all his unfinished business, and make peace with the God he deeply believed in. Sudden deaths have their own entire element of terrible, where you don’t get that chance. I mean, I’ve been thinking about this situation basically nonstop for the last few days, and I didn’t even really know this person: I can’t imagine how horribly tragic this is for all of the people who were actually close to Michael. I saw what a toll my father’s death took on my mother, I can’t even imagine the grief his wife must be dealing with right now.</p>\n<p>So many questions. So few answers. So sad. All we can do is look at these digital remnants, wonder, and remember.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/returning-to-free-software-a-guide/",
"title": "Returning to Free Software, a guide",
"description": null,
"url": "https://steveklabnik.com/writing/returning-to-free-software-a-guide/",
"published": "2013-06-15T00:00:00.000Z",
"updated": "2013-06-15T00:00:00.000Z",
"content": "<p>A month or two ago, I made a conscious decision that I would return to using as much Free Software as possible. The PRISM debacle of the last week confirmed my fears and reasoning, and so I made the decision to accelerate the schedule.</p>\n<p>Here’s what I did, and how you can, too.</p>\n<h2>My GNU/Linux Background</h2>\n<p>The first computer I ever used was an Apple machine, and at various times in my life I’ve used GNU/Linux as my main OS. So I’m already pretty familiar with what’s involved.</p>\n<p>That said, it’s only gotten better over the years. Especially if you buy hardware that is known to work well, using GNU/Linux as your main OS shouldn’t be a big deal. One of the nice things about the rise of OS X in the developer community is that they’ve come to rely on the standard GNU tools, and so much of what you’re already working with will probably translate straight over. Of course, if you work on iOS or something, this won’t be the case, but for your run-of-the-mill web developer, there isn’t a whole lot of difference.</p>\n<p>Well, other than that whole ‘knowing exactly what’s on your machine’ thing.</p>\n<h2>Hardware: X1 Carbon</h2>\n<p>IBM’s ThinkPad series of laptops have always enjoyed fantastic Linux support. When they were purchased by Lenovo, some were worried that they’d ruin this fantastic hardware line. That didn’t really happen, though, and they’re still fantastic computers to this day. When asking my Twitter followers what they thought, everyone who owned a ThinkPad loved it, whichever model they had.</p>\n<p>I happened to get an <a href=\"http://shop.lenovo.com/us/en/laptops/thinkpad/x-series/x1-carbon/index.html\">X1 Carbon</a>. I was coming from a 13\" MacBook Air 13“, and the Carbon is very similar. It’s 14” though, which is a really nice size; it still fits in the same pocket in my bag as the MBA did, but I get a little more screen real estate. A note for those who give presentations: the X1 Carbon uses a MiniDisplay port, so save your Apple connectors!</p>\n<p>All of the hardware I use has Just Worked; I hear that the thumbprint reader may not, but I don’t plan on using it, so that doesn’t bother me.</p>\n<p>If you’re not in the market for new hardware, you can generally run GNU/Linux on your Apple hardware as well, which is a starting point, at least.</p>\n<h2>Operating System: Crunchbang GNU/Linux</h2>\n<p>I’ve used a lot of different GNU/Linux distros, and have been a happy Arch Linux user for years. However, this time around, I decided to go with <a href=\"http://crunchbang.org/\">CrunchBang</a>. Crunchbang is a Debian derivative, but includes <a href=\"http://openbox.org/\">OpenBox</a> as the default window manager. The result is an OS that’s lightweight, yet very, very functional. I’ve been really happy so far with Crunchbang’s level of polish.</p>\n<p>Okay, so actually <em>installing</em> Crunchbang: the X1 Carbon comes with Windows by default, and so we need to get Crunchbang on there somehow. Luckily, you can just use USB. <code>dd</code> is your friend here, which you can use on your Mac, of course. You can grab a copy of Crunchbang <a href=\"http://crunchbang.org/download/\">from their download page</a>, and they have a link to making the USB stick right on that page.</p>\n<p>After you’ve made the USB stick, just reboot your computer with it on, and press down all of your F keys. I was simply too lazy to look up which one would actually let me choose a boot device. ;) The Crunchbang installer was very straightforward, just follow the prompts and you’ll be good to go.</p>\n<p>One thing that I did choose was the ‘encrypted lvm’ option. Basically, your entire disk is encrypted except a small boot partition. On boot, it asks you for a passphrase to decrypt the disk. It’s just a matter of choosing the right option, in practice, this has worked and worked well for me.</p>\n<p>EDIT: One of the reasons I chose Crunchbang was because I thought that it was solely configuration on top of Debian, which includes 100% Free Software. Checking my <code>sources.list</code>, there is a Crunchbang-specific repository, and I haven’t evaluated it for Free-ness. So if you’re trying for 100% purity, straight-up Debian may be better.</p>\n<h2>Software</h2>\n<p>Most of the tools that I use every day are already either Free Software or open source, and so my core experience hasn’t changed: <code>bash</code>, <code>vim</code>, <code>git</code>, and friends aren’t any different.</p>\n<p>I did, however, install <a href=\"https://www.torproject.org/\">tor</a>. Tor is not a privacy panacea, however, it does help in many cases.</p>\n<p>I’ve added <a href=\"http://www.ghostery.com/\">Ghostery</a>, <a href=\"http://noscript.net/\">NoScript</a>, and <a href=\"http://adblockplus.org/\">AdBlock Plus</a> to the built-in Iceweasel browser, which is for all intents and purposes Firefox without the Mozilla branding. These allow me to control exactly what information leaks out a little bit more. They also have the nice side effect of making pages load a bit faster, since a lot of the usual bullshit doesn’t even get loaded. (EDIT: it’s been brought to my attention that Ghostery isn’t actually free. Oops!)</p>\n<p>I plan on setting up backups with <a href=\"http://www.tarsnap.com/\">Tarsnap</a> later in the week, which is an excellent backup service that encrypts all of your data locally before saving it off in The Cloud. It has some pretty good (I hear) de-duplication features, and you only pay for the bytes used, so it’s pretty inexpensive. The author, Colin Percival, has said that he has some customers doing a muti-gig daily backup of their home directories, and they’re paying ~$10/month. (I should mention that <a href=\"http://www.tarsnap.com/legal.html\">Tarsnap is not Free</a>, but it is shared source. I don’t plan on using it for anything else, so I’m okay with this tradeoff at this time.)</p>\n<p>Most of my development work at the moment is in Ruby, and getting it set up was a breeze. I use the <a href=\"https://github.com/sstephenson/ruby-build\">ruby-build</a> tool to compile my Rubies, and the most excellent <a href=\"https://github.com/postmodern/chruby\">chruby</a> to switch between them.</p>\n<p>One thing that I have yet to explore (but I’d like to) is to use a tool like Chef to set up my personal machine, so that I can periodically wipe everything and re-build from scratch. That’s still on the to-do list, though.</p>\n<p>I personally re-map my caps lock key to control. To do this, make a <code>.Xmodmap</code> file in your home directory, and put this in it:</p>\n<pre><code>keycode 66 = Control_L\nclear Lock\nadd control = Control_L\n</code></pre>\n<p>I figured this out via a few simple searches. There’s a small chance that your caps lock is not code 66, and if it’s not, you can run the <code>xev</code> command to check it out. Just press your caps lock and make note of the number that shows up in the console.</p>\n<p>I had to install <code>grandr</code> in order to share my screen to give a presentation. It’s easy enough to get: <code>sudo apt-get install grandr</code>.</p>\n<p>I started giving presentations with <a href=\"http://rabbit-shocker.org/\">rabbit</a>, which is what many Japanese Rubyists use. It’s pretty fun; you write plain text files and it makes your presentation for you, as cairo/pango application. You can also use it to generate PDFs of slides. It’s not for everyone, but I’m enjoying it so far.</p>\n<p>Oh, and <em>incredibly</em> important: I typed <code>sudo apt-get install ttf-ancient-fonts</code> to get Emoji support. They’re not as pretty as the Apple ones, but they’re not empty boxes either.</p>\n<h2>Conclusion</h2>\n<p>I’m really, really happy with this setup. Very little of my day-to-day has changed, but I have a bit more privacy, and can feel good that I know exactly what software is on my machine. No NSA backdoors here!</p>\n<p>I still have improvements to make, but what I have works well so far. Please let me know what you think, and suggest ways to make this setup even better.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rust--y-scraps--iterating-with-a-step/",
"title": "Rust-y Scraps: iterating with a step",
"description": null,
"url": "https://steveklabnik.com/writing/rust--y-scraps--iterating-with-a-step/",
"published": "2013-05-30T00:00:00.000Z",
"updated": "2013-05-30T00:00:00.000Z",
"content": "<p>This blog post works with <a href=\"http://rust-lang.org/\">Rust</a> v 0.6, and may change in the future :).</p>\n<p>A common need in various algorithms is to iterate with some sort of step. A simple way to do this in Rust is with <code>unit::range_step</code>:</p>\n<pre><code>let nums = [1,2,3,4,5,6,7];\nfor uint::range_step(0, bytes.len() - 1, 2) |i| {\n println(fmt!(\"%d & %d\", nums[i], nums[i+1]));\n}\n</code></pre>\n<p>This yields</p>\n<pre><code>1 & 2\n3 & 4\n5 & 6\n\n</code></pre>\n<p>Of course, this is awkward: I wish I had some code to do something like:</p>\n<pre><code>let nums = [1,2,3,4,5,6,7];\n\nfor nums.in_groups |i, j| {\n println(fmt!(\"%d & %d\", i, j));\n}\n</code></pre>\n<p>But I think this is just out of my abilities. The issue is that the closure needs to take the same number of arguments as the step, and I’m not sure that I have the Rust-fu to make it work yet.</p>\n<p>After a discussion in the IRC room, Rust doesn’t have the Rust-fu yet either; varargs aren’t there, so this style is a no-go for now.</p>\n<hr />\n<p>Oh, and since there were no examples of this yet, <a href=\"https://github.com/mozilla/rust/pull/6841\">I added some to the docs</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rails-4-0-0-beta1-to-rails-4-0-0-rc1/",
"title": "Rails 4.0.0-beta1 to Rails 4.0.0-rc1",
"description": null,
"url": "https://steveklabnik.com/writing/rails-4-0-0-beta1-to-rails-4-0-0-rc1/",
"published": "2013-05-21T00:00:00.000Z",
"updated": "2013-05-21T00:00:00.000Z",
"content": "<p>I’m writing <a href=\"http://www.manning.com/bigg2/\">Rails 4 in Action</a> along with Ryan Bigg and Yehuda Katz. In updating the book to Rails 4, I started early: the sample app (<a href=\"https://github.com/steveklabnik/ticketee\">ticketee</a>) was done with <code>4.0.0.beta1</code>.</p>\n<p>Then <code>4.0.0.rc1</code> came out. I didn’t upgrade immediately, but last night, I took the plunge. There were only three errors, and the whole process took me something like 40 minutes.</p>\n<p>Here’s what I had to do:</p>\n<h2>Cookie Stores</h2>\n<p>Error:</p>\n<pre><code>/Users/steve/.gem/ruby/2.0.0/gems/railties-4.0.0.rc1/lib/rails/application/configuration.rb:144:in `const_get': uninitialized constant ActionDispatch::Session::EncryptedCookieStore (NameError)\n</code></pre>\n<p>The EncryptedCookieStore went away. Now, the regular cookie store handles these kinds of things.</p>\n<p>Fix:</p>\n<pre><code>diff --git a/config/initializers/session_store.rb b/config/initializers/session_store.rbindex c512cda..b4681fd 100644--- a/config/initializers/session_store.rb+++ b/config/initializers/session_store.rb@@ -1,3 +1,3 @@ # Be sure to restart your server when you modify this file.-Ticketee::Application.config.session_store :encrypted_cookie_store, key: '_ticketee_session'+Ticketee::Application.config.session_store :cookie_store, key: '_ticketee_session'\n</code></pre>\n<h2>Routes</h2>\n<p>Error:</p>\n<pre><code>/Users/steve/.gem/ruby/2.0.0/gems/actionpack-4.0.0.rc1/lib/action_dispatch/routing/route_set.rb:408:in `add_route': Invalid route name, already in use: 'signin' (ArgumentError)\nYou may have defined two routes with the same name using the `:as` option, or you may be overriding a route already defined by a resource with the same naming. For the latter, you can restrict the routes created with `resources` as explained here:\nhttp://guides.rubyonrails.org/routing.html#restricting-the-routes-created\n</code></pre>\n<p>You can’t name two routes the same thing any more. In my case, just dropping the <code>:as</code> allowed me to get to where I needed to go:</p>\n<p>Fix:</p>\n<pre><code>diff --git a/config/routes.rb b/config/routes.rbindex f010adb..d573e89 100644--- a/config/routes.rb+++ b/config/routes.rb@@ -2,8 +2,8 @@ Ticketee::Application.routes.draw do root to: \"projects#index\"- get \"/signin\", to: \"sessions#new\", as: \"signin\"- post \"/signin\", to: \"sessions#create\", as: \"signin\"+ get \"/signin\", to: \"sessions#new\"+ post \"/signin\", to: \"sessions#create\" delete \"/signout\", to: \"sessions#destroy\", as: \"signout\"\n</code></pre>\n<h2>Clean dbs</h2>\n<p>Error:</p>\n<p>Tons of test failures relating to what was essentially “I don’t delete the db between test runs.”</p>\n<p>Fix:</p>\n<p>Upgrade to rspec-rails 2.13.2. Done!</p>\n<h2>has_secure_password</h2>\n<p>Error:</p>\n<pre><code> 1) User passwords needs a password and confirmation to save\n Failure/Error: expect(u).to_not be_valid\n expected #<User id: 1, name: \"steve\", email: \"[email protected]\", password_digest: \"$2a$04$9D7M2iPvYavvvpb3rlLzz.vqMJdGBAQVBSJ6L.UIoXP6...\", created_at: \"2013-05-21 23:59:44\", updated_at: \"2013-05-21 23:59:44\", admin: false> not to be valid\n</code></pre>\n<p>This test failed. The test looked like this:</p>\n<pre><code>it \"needs a password and confirmation to save\" do u = User.new(name: \"steve\", email: \"[email protected]\") u.save expect(u).to_not be_valid u.password = \"password\" u.password_confirmation = \"\" u.save expect(u).to_not be_valid u.password_confirmation = \"password\" u.save expect(u).to be_validend\n</code></pre>\n<p>Turns out, <a href=\"https://github.com/rails/rails/pull/10694\">it’s a regression</a>! This will get fixed for RC2, but for now, just commenting out that middle section works. The final test is good enough for now.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/ditching-google--chat-with-xmpp/",
"title": "Ditching Google Chat with XMPP",
"description": null,
"url": "https://steveklabnik.com/writing/ditching-google--chat-with-xmpp/",
"published": "2013-05-16T00:00:00.000Z",
"updated": "2013-05-16T00:00:00.000Z",
"content": "<p>Last week, I decided that it was time to migrate one of my services away from Google: XMPP for IM. The nice thing about XMPP is that you don’t have to totally drop support for your old identity: since Google still uses it (<a href=\"http://eschnou.com/entry/whats-next-google--dropping-smtp-support--62-24930.html\">for now</a>), you can get online with both.</p>\n<p>Here’s how:</p>\n<hr />\n<p>I set mine up with <a href=\"http://digitalocean.com/\">Digital Ocean</a>, because my friends told me it was rad. It was pretty damn easy.</p>\n<p>Once you have a server, you need to set up all the usual stuff: a non-privledged user account, yadda, yadda. I’m assuming you can do that.</p>\n<h2>Install ejabberd</h2>\n<p>I picked <a href=\"http://www.ejabberd.im/\">ejabberd</a> for my server software because it’s used by a lot of people.</p>\n<p>EDIT: I just got a tweet from <a href=\"http://twitter.com/jamesgolick\">@jamesgolick</a>, who I trust to know stuff about stuff. He says:</p>\n<blockquote>\n<p>ejabberd is buggy. Use the erlangsolutions fork instead. - <a href=\"https://twitter.com/jamesgolick/status/335136677364396032\">@jamesgolick</a></p>\n</blockquote>\n<p>So, take that for what it’s worth.</p>\n<p>Anyway, since I use Arch Linux, installing was as easy as</p>\n<pre><code>$ sudo pacman -S ejabberd\n</code></pre>\n<p>After that was over, time to edit the config. <code>ejabberd</code> is written in Erlang, and so it uses Erlang for its config. Don’t worry! It’s easy. Just copy and paste these bits in, or change them if there’s something like them in your config:</p>\n<pre><code>{hosts, [\"localhost\", \"xmpp.steveklabnik.com\"]}.{acl, admin, {user, \"steve\", \"xmpp.steveklabnik.com\"}}.\n</code></pre>\n<p>These change my host to <code>xmpp.steveklabnik.com</code> as well as <code>localhost</code> and set my admin user to <code>[email protected]</code>.</p>\n<p>Next up, we boot the node:</p>\n<pre><code>sudo ejabberdctl start\n</code></pre>\n<p>And then register a way to administer things via the web interface:</p>\n<pre><code>sudo ejabberdctl register steve xmpp.steveklabnik.com $password\n</code></pre>\n<p>Of course, change those to be your values, and change $password to your password. Now that you’ve typed a password into the command line, it’s good to clean out your bash history. When I did this, typing <code>history</code> showed this:</p>\n<pre><code> 35 sudo ejabberdctl register ....\n</code></pre>\n<p>So, do this:</p>\n<pre><code>$ history -d 35$ history -w\n</code></pre>\n<p>Or, to nuke <em>everything</em>:</p>\n<pre><code>$ history -c$ history -w\n</code></pre>\n<p>Anywho, now that that’s over, you need to go to your domain registrar and insert an A record pointing at your server. I use <a href=\"https://www.namecheap.com/\">NameCheap</a>.</p>\n<p>Now that that’s done, hit this page in your browser:</p>\n<pre><code>http://xmpp.steveklabnik.com:5280/admin/\n</code></pre>\n<p>The trailing slash is important, and obviously, you need to change it to be your server. Given that the DNS isn’t cached wrong, you should get a popup asking you to log in. Do so with the credentials you typed in on the command line up there. If you can log in, great! Everything should be working.</p>\n<p>Let’s turn off registration, though. Go back to your <code>ejabberd</code> config, <code>/etc/ejabberd/ejabberd.cfg</code>:</p>\n<pre><code>{access, register, [{deny, all}]}.\n</code></pre>\n<p>You probably have one that says <code>allow</code>, so just change it. Check in your admin panel on the site that it’s the same.</p>\n<hr />\n<p>Bam! That’s it. This whole process took me about an hour, including writing this blog post. I do have some experience with VPSes, though.</p>\n<p>If I can help free you of Google in any way, ping me at <a>[email protected]</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-profile-link-relation-and-you/",
"title": "The profile link relation and you",
"description": null,
"url": "https://steveklabnik.com/writing/the-profile-link-relation-and-you/",
"published": "2013-05-06T00:00:00.000Z",
"updated": "2013-05-06T00:00:00.000Z",
"content": "<p>I was quite pleased when <a href=\"http://tools.ietf.org/html/rfc6906\">RFC 6906</a> was finalized. It’s a really useful pattern that people are using to enhance documentation of their APIs.</p>\n<p>Let’s start with an example. I tweet something like this:</p>\n<blockquote>\n<p>“Oh man, the example.com API is super awesome. You should check it out!” - <a href=\"https://twitter.com/steveklabnik\">@steveklabnik</a> seconds ago from web</p>\n</blockquote>\n<p>You don’t know anything about this API or what it offers. So you fire up <code>curl</code>:</p>\n<pre><code>$ curl -i http://example.com\nHTTP/1.1 200 OK\nContent-Type: application/json\nContent-Length: 273\n\n{\n \"wtl\": \"MjAxMy0wNS0wNiAxMjo1Nzo1MyAtMDcwMA==\\n\",\n \"grobb34s\": [\n {\n \"flog\": \"Top 100 foobars\",\n \"zilch\": \"http://example.com/foo/bar?baz=qux\"\n },\n {\n \"flog\": \"Mega Troll Title\",\n \"zilch\": \"http://example.com/exploit.exe/foobar\"\n }\n ]\n}\n</code></pre>\n<p>To be blunt, this makes absolutely no sense. You tell me so, and the next day, I tell you to check it out again. You grumble, and get back to the <code>curl</code>:</p>\n<pre><code>$ curl -i http://example.com\nHTTP/1.1 200 OK\nContent-Type: application/json\nContent-Length: 273\n Link: <http://example.com/profile>; rel=\"profile\"\n\n{\n \"wtl\": \"MjAxMy0wNS0wNiAxMjo1Nzo1MyAtMDcwMA==\\n\",\n \"grobb34s\": [\n {\n \"flog\": \"Top 100 foobars\",\n \"zilch\": \"http://example.com/foo/bar?baz=qux\"\n },\n {\n \"flog\": \"Mega Troll Title\",\n \"zilch\": \"http://example.com/exploit.exe/foobar\"\n }\n ]\n}\n</code></pre>\n<p>Oh, wait. “Profile”. Let’s see what that’s about:</p>\n<pre><code>$ curl -i http://example.com/profile\nHTTP/1.1 200 OK\nContent-Type: text/plain\nContent-Length: 548\n\nThe Example.com API\n===================\n\nExample.com provides access to our blog through an API.\n\nIn the API, you'll see two major things of interest: `wtl` and `grobb34s`.\n\n## wtl\n\nThe value provided under the `wtl` key is the time the latest blog post\nwas posted, in \"%Y-%m-%d %H:%M:%S %z\" format. This value is then Base64\nencoded.\n\n## grobb34s\n\nThe `grobb34s` key will hold an array of blog posts. These posts are\nrepresented by a JSON object with two keys. `flog` has the title, suitable for\ndisplay, and `zilch` contains a link to the post.\n</code></pre>\n<p>Oh. You don’t care about some blog with such terrible titles, so you go about your day.</p>\n<h2>self-descriptive-ness</h2>\n<p>You may consider this a highly contrived example, but it’s not as contrived as you think. In Fielding’s thesis, he describes a ‘self-described messages’ constraint, which means that any API call should be able to be understood alone, without additional context.</p>\n<p>In HTTP, this information is generally provided via the <code>Content-Type</code> header. It describes a media type (such as <code>application/json</code> in this example) that describes the rules for processing the response. When writing a HTTP client, you fetch the response, check the <code>Content-Type</code>, and then invoke the correct parser based on the type provided.</p>\n<p>But many times are very general. Take <code>application/json</code>, for example. It says nothing about blog posts. A user-agent that only knows about <code>application/json</code> is very much like you as a human seeing gibberish keys and values; it’s not smart enough to make assumptions based on the text of keys and values it may see in JSON. However, with the added context of a profile, we have enough information to make sense of this strange API. And any user-agent that sees a profile that it recognizes can act on those new semantics, too.</p>\n<h2>JSON API</h2>\n<p>This is one of the reasons that we’re working on <a href=\"http://jsonapi.org/\">JSON API</a>, a standard media type for APIs that use JSON. Generic JSON has no standard semantics that are useful to API authors, since it’s a very generic format. By declaring a new media type with common features to many APIs, we can write generic tools that handle the 80% of cases that come up during API development.</p>\n<p>And if the generic case doesn’t suit your requirements, you can always extend it with profiles. In fact, it’d be nice if many people just added a <code>Link</code> header to their existing API documentation; that’d start alleviating this problem.</p>\n<hr />\n<p>If you enjoyed this article, you might want to check out my in-progress book on building APIs that respect standards and HTTP, <a href=\"http://www.designinghypermediaapis.com/\">Designing Hypermedia APIs</a>. This post also serves as <a href=\"http://www.designinghypermediaapis.com/blog/the-profile-link-relation-and-you.html\">its first blog post</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/closure/",
"title": "Closure",
"description": null,
"url": "https://steveklabnik.com/writing/closure/",
"published": "2013-04-19T00:00:00.000Z",
"updated": "2013-04-19T00:00:00.000Z",
"content": "<p>I spent six weeks of last December/January in my hometown. It was the first Christmas without my father, and it was cheap to stay, so I went back. It was my first time back in Pittsburgh since I’d moved away from the only place I’d ever lived.</p>\n<p>The trip was incredibly emotional for many reasons, but one of the best was when I decided to walk down the street to get a pizza. I had to wait ten minutes for it to cook, and so I decided to open up my web browser. Then I saw this:</p>\n<p><img src=\"https://svbtleusercontent.com/steveklabnik_24595149844116_small.png\" alt=\"https://svbtleusercontent.com/steveklabnik_24595149844116_small.png\" /></p>\n<p>Whoah.</p>\n<p>You see, I’ve always had a complex relationship with _why. I had been dabbling in Ruby when he was still alive, and planned on dropping by this ART && CODE thing I’d heard about over at CMU, but I was out of town that weekend, so I shrugged and didn’t think twice.</p>\n<p>Then he disappeared.</p>\n<p>I started looking into _why and what he was actually about. What I found was amazing, and you already know that part of the story. I thought Hackety Hack was a masterpiece, and decided to help maintain it. Unfortunately for me, nobody else did. I hadn’t done a whole lot of open source work before, and now I was maintaining a big, important project by one of the most well-known Rubyists of all time. Whoah.</p>\n<p>I cared about Hackety enough and felt inspired enough by _why that I actually quit my startup to spend more time working on it.</p>\n<p>Here’s the weird thing about someone disappearing: you can’t ask them questions. I had so many things that I wanted to ask, so many unresolved questions, so much I wanted to know. It’s really weird to take someone’s life’s work and make it your own. I spent a lot of that first year with crippling doubts about what I was doing, if _why would like it, if that even mattered, wondering why he’d abandoned something so important, why he’d abandoned <em>us</em>. And I felt terrible that I <em>needed</em> this from him. Why does he owe us all that? Who am I to impose?</p>\n<p>So I marked my calendar for the 18th, and waited. That was yesterday.</p>\n<p><img src=\"http://distilleryimage6.s3.amazonaws.com/a8767b30a88f11e2896922000a1fbe1a_7.jpg\" alt=\"http://distilleryimage6.s3.amazonaws.com/a8767b30a88f11e2896922000a1fbe1a_7.jpg\" /></p>\n<p><img src=\"http://distilleryimage11.s3.amazonaws.com/c22b37d2a8bc11e287e222000aaa0aa2_7.jpg\" alt=\"http://distilleryimage11.s3.amazonaws.com/c22b37d2a8bc11e287e222000aaa0aa2_7.jpg\" /></p>\n<p>It was… great. Just great. The stuff _why sent is basically everything I’d ever hoped. It’s got so much buried inside. An IF game, commentary on Jobs vs. Gates, a very person struggle with identity, a comment that he likes what we’ve done… it’s great. And it’s finally brought me some closure.</p>\n<p><a href=\"https://github.com/steveklabnik/CLOSURE\">So, I’ve saved it all for <em>you</em>. <3</a></p>\n<p><img src=\"https://svbtleusercontent.com/steveklabnik_24595264609650_small.jpg\" alt=\"https://svbtleusercontent.com/steveklabnik_24595264609650_small.jpg\" /></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/upgrading-from-rust-0-5-to-0-6/",
"title": "Upgrading from Rust 0.5 to 0.6",
"description": null,
"url": "https://steveklabnik.com/writing/upgrading-from-rust-0-5-to-0-6/",
"published": "2013-04-07T00:00:00.000Z",
"updated": "2013-04-07T00:00:00.000Z",
"content": "<p>Here are some error messages that I got when updating <a href=\"http://www.rustforrubyists.com/\">Rust for Rubyists</a> from Rust 0.5 to the new 0.6, and how to fix those errors.</p>\n<hr />\n<p>Error:</p>\n<pre><code>rust.rs:1:4: 1:16 error: unresolved name\nrust.rs:1 use task::spawn;\n ^~~~~~~~~~~~\nrust.rs:1:4: 1:16 error: failed to resolve import: task::spawn\nrust.rs:1 use task::spawn;\n ^~~~~~~~~~~~\nerror: failed to resolve imports\nerror: aborting due to 3 previous errors\nmake: *** [all] Error 101\n</code></pre>\n<p>or</p>\n<pre><code>rust.rs:1:4: 1:16 error: unresolved name\nrust.rs:1 use io::println;\n ^~~~~~~~~~~~\nrust.rs:1:4: 1:16 error: failed to resolve import: io::println\nrust.rs:1 use io::println;\n ^~~~~~~~~~~~\nerror: failed to resolve imports\nerror: aborting due to 3 previous errors\nmake: *** [all] Error 101\n</code></pre>\n<p>The fix:</p>\n<p>Imports are now more explicit, so you can’t just import things from <code>core</code> without specifying that you want to any more.</p>\n<pre><code>+ use core::task::spawn;- use task::spawn;\n</code></pre>\n<p>or</p>\n<pre><code>+ use core::io::println;- use io::println;\n</code></pre>\n<p>etc.</p>\n<hr />\n<p>Error:</p>\n<pre><code>error: main function not found\nerror: aborting due to previous error\n</code></pre>\n<p>The fix:</p>\n<hr />\n<p>When I used to write TDD’d Rust, I wouldn’t often write a <code>main</code> function until I had done a bunch of tests. This was okay, but now, you need one.</p>\n<pre><code>+ fn main() {\n+ }\n</code></pre>\n<hr />\n<p>Error:</p>\n<pre><code>rustc rust.rs --test\nrust.rs:5:2: 5:6 error: unresolved name: `fail`.\nrust.rs:5 fail;\n ^~~~\nerror: aborting due to previous error\nmake: *** [build_test] Error 101\n</code></pre>\n<p>The fix:</p>\n<p><code>fail</code> was turned into a macro. It now needs to be passed an owned pointer to a string:</p>\n<pre><code>- fail;\n+ fail!(~\"Fail!\");\n</code></pre>\n<hr />\n<p>Error:</p>\n<pre><code>rust.rs:5:18: 5:26 error: unresolved name: `int::str`.\nrust.rs:5 println(int::str(num))\n ^~~~~~~~\nerror: aborting due to previous error\nmake: *** [build] Error 101\n</code></pre>\n<p>The fix:</p>\n<p><code>int::str</code> is now <code>int::to_str</code></p>\n<pre><code>- int::str\n+ int::to_str\n</code></pre>\n<hr />\n<p>Error:</p>\n<pre><code>rust.rs:5:12: 5:18 error: cannot determine a type for this local variable: unconstrained type\nrust.rs:5 let mut answer;\n ^~~~~~\nerror: aborting due to previous error\nmake: *** [build] Error 101\n</code></pre>\n<p>The fix:</p>\n<p>You have to tell it what kind it is. In my case, it was a string:</p>\n<pre><code>- let mut answer;\n+ let mut answer = \"\";\n</code></pre>\n<hr />\n<p>Error:</p>\n<pre><code>rust.rs:3:11: 3:21 error: expected `;` or `}` after expression but found `is_fifteen`\nrust.rs:3 assert is_fifteen(15)\n ^~~~~~~~~~\nmake: *** [build_test] Error 101\n</code></pre>\n<p>The fix:</p>\n<p>This happens because <code>assert</code> is now a macro.</p>\n<pre><code>- assert is_fifteen(15)\n+ assert!(is_fifteen(15))\n</code></pre>\n<hr />\n<p>Error:</p>\n<pre><code>rustc rust.rs\nrust.rs:11:10: 11:24 error: the type of this value must be known in this context\nrust.rs:11 chan.send(10);\n^~~~~~~~~~~~~~\nerror: aborting due to previous error\nmake: *** [build] Error 101\n</code></pre>\n<p>The fix:</p>\n<p>Rust 0.6 got rid of ‘capture clauses.’ Don’t worry about it.</p>\n<pre><code>- do spawn |chan| {\n+ do spawn {\n</code></pre>\n<hr />\n<p>Error:</p>\n<pre><code>rust.rs:10:17: 10:21 error: expected `,` but found `chan`\nrust.rs:10 do spawn |move chan| {\n ^~~~\nmake: *** [build] Error 101\n</code></pre>\n<p>The fix:</p>\n<p><code>move</code> is gone. Just remove it.</p>\n<pre><code>- do spawn |move chan| {\n+ do spawn |chan| {\n</code></pre>\n<p>Note that this diff wouldn’t actually make it work, you’d still run into the issue above. But it gets rid of the <code>move</code> error.</p>\n<hr />\n<p>Error:</p>\n<pre><code>rust.rs:13:6: 13:12 error: unexpected token: `static`\nrust.rs:13 static fn count() {\n ^~~~~~\nmake: *** [build] Error 101\n</code></pre>\n<p>The fix:</p>\n<p><code>static</code> was removed. Any method that doesn’t take <code>self</code> is static.</p>\n<pre><code>- static fn count() {\n+ fn count() {\n</code></pre>\n<hr />\n<p>Error:</p>\n<pre><code>rust.rs:11:30: 11:37 error: obsolete syntax: colon-separated impl syntax\n</code></pre>\n<p>The fix:</p>\n<p>As it says, the colon syntax is gone. Replace it with <code>for</code>:</p>\n<pre><code>- impl float: Num\n+ impl Num for float\n</code></pre>\n<hr />\n<p>Error:</p>\n<pre><code>rust.rs:5:18: 5:22 error: use of undeclared type name `self`\nrust.rs:5 fn new() -> self;\n ^~~~\n</code></pre>\n<p>The fix:</p>\n<p>This happens when you’re making a trait, and you want the implementations to return whatever type they are. The type name is now <code>Self</code>:</p>\n<pre><code>- fn new() -> self;\n+ fn new() -> Self;\n</code></pre>\n<hr />\n<p>I hope you found that informative! Happy hacking!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/deleuze-for-developers-will-smooth-spaceopen-source-suffice-to-save-us/",
"title": "Deleuze for Developers: will smooth space/open source suffice to save us?",
"description": null,
"url": "https://steveklabnik.com/writing/deleuze-for-developers-will-smooth-spaceopen-source-suffice-to-save-us/",
"published": "2013-03-14T00:00:00.000Z",
"updated": "2013-03-14T00:00:00.000Z",
"content": "<p>If you truly want to understand technology today, then you should at least be familiar with the philosophy of Gilles Deleuze. Unfortunately for technologists, Deleuze is rooted firmly in a philosophical tradition and a writing style that they probably find opaque. In this blog series, I plan on explaining Deleuze’s philosophy in terms that programmers can understand. This is the third in the series. You can find the first <a href=\"/deleuze-for-developers-assemblages\">here</a>. Enjoy.</p>\n<hr />\n<p>Deleuze and Guattari employ the notion of ‘smooth space’ quite a bit in <em>A Thousand Plateaus</em>. I saw this quote about it the other day:</p>\n<blockquote>\n<p>never believe that a smooth space will suffice to save us</p>\n</blockquote>\n<p>I’ve recently been giving a lot of thought to <a href=\"http://en.wikipedia.org/wiki/Cultural_hegemony\">‘cultural hegemony’</a> and its applicability to the web, specifically along the lines of Google. Basically, cultural hegemony is the idea that you don’t just have dominance through laws or economics, but through culture. So, for example, Google submitted SPDY to become HTTP 2.0: it’s going through the standards process, it’s ‘open,’ but it also gives Google a lot of cultural leverage.</p>\n<p>So there’s this connection there between smooth space and open source, and the battle between the Nomads and the State Apparatus. We’ll cover that second bit in a while, but for now: what is ‘smooth space’?</p>\n<h2>The spatial metaphor</h2>\n<p>The first thing to understand about these concepts is that Deleuze and Guattari is (I think of them as a singular entity…) a big fan of spatial metaphors. Essentially, we can project our problem domain onto a 2 or 3 (or more) dimensional space, and then use tools to analyze that space. I was first introduced to this technique by Manuel de Landa, who has a great lecture called “Deleuze and the use of the genetic algorithm in archetecture” (<a href=\"http://www.youtube.com/watch?v=50-d_J0hKz0\">video</a>)(<a href=\"http://www.cddc.vt.edu/host/delanda/pages/algorithm.htm\">text</a>). A quick rundown:</p>\n<p>So, let’s talk about the <a href=\"http://en.wikipedia.org/wiki/Eight_queens_puzzle\">N queens problem</a>. We have a chess board with one space, and we place one queen on it:</p>\n<p>(warning, lots of ASCII (+ unicode) art to follow)</p>\n<pre><code>.-.\n|♕|\n.-.\n</code></pre>\n<p>This board is legal. It’s a solution to the ‘1 queen problem’: one queen, a 1x1 board. What about 2 queens? We have no solutions. (2 and 3 don’t. :/ All other natural numbers do.) First we place a queen:</p>\n<pre><code>.---.\n|♕| |\n-----\n| | |\n.---.\n</code></pre>\n<p>… but there’s no legal place for the second one. Bummer. We can determine that there’s no solution by using a simple brute force with backtracking: place the queen in the upper right, then try to place the second below it. That’s not legal, so we backtrack and place it on the right. That’s not legal, so we backtrack and put it in the bottom corner. Oh no! That didn’t work, so let’s move our first queen: now it’s in the bottom right, and we try to place the second in the upper right: fail! So we backtrack….</p>\n<p>As you can see, this is a lot of steps for a small board. Once you get up to a ‘real’ chess board, there are 4,426,165,368 possible placements, but only 92 solutions. A needle in a haystack! And we’re using a lot of steps to determine if there’s even one possible placement. We need something better.</p>\n<p>One answer is genetic algorithms. So we can take our board’s current state and assign it a score. When we place a new queen, if it makes our score better, we keep it, and if it doesn’t, we lose it. Now we’ve taken our 2x2 board and projected it onto a 2-axis linear space. Imagine a plot where queens are on the x axis and score is on the y axis:</p>\n<pre><code> .dAHAd.\n .adAHHHAbn.\n .dAHAd. .adAHHHHHHAbn.\n .adAHHHAbn. .adAHHHHHHHHAbn.\n dHHHHHHHHHHHHHHHHHHHHHHHHHHHHHb\n dHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHb\n</code></pre>\n<p>So we want to keep going with our solution as long as it slopes up: we’ve found that our answers are at the maximum of each curve. Yay calculus! But here’s the thing: this curve only has two possible maxima: we will have more. And it might not be in 2-d space, it might be in 5-d space. And we can only ‘use math’ to find the maximum if we generate the <em>entire</em> curve, which seems computationally out of our reach. So how do we find the maximum without generating the whole curve? Genetic algorithms!</p>\n<p>One such is ‘<a href=\"http://en.wikipedia.org/wiki/Simulated_annealing\">simulated annealing</a>.’ Without getting too into it, let’s just say that there’s a ‘cooling factor’ that controls how tolerant we are of going back down a slope. So at first, we wildly go all over the search space, but then, as we progress, we tighten up our cooling factor, and we stop being so wild. Eventually, we’ll arrive at a solution that’s very likely to be the true global maxima/minima. Neat!</p>\n<h2>Discrete vs. Continuous</h2>\n<blockquote>\n<p>If we know that the enemy is open to attack, and also know that our men are in a condition to attack, but are unaware that the nature of the ground makes fighting impracticable, we have still gone only halfway towards victory.Sun Tzu, “the Art of War”</p>\n</blockquote>\n<p>Another interesting feature of this particular projection is that it transforms our problem from a discrete problem into a continuous one. One great tactic for when you’re losing: change the battlefield. Our genetic algorithm tool needs a continuous space to operate, but our problem is that our chess board is discrete. What do I mean by this?</p>\n<p>“<a href=\"http://en.wikipedia.org/wiki/Discrete_space\">discrete space</a>” is one in which the points are separated from one another in some way. As an example, ‘integers’ form a discrete topology: there’s a ‘gap’ between 1, 2, 3, and 4. You can see this by drawing a <a href=\"http://en.wikipedia.org/wiki/Number_line\">number line</a>:</p>\n<pre><code><--(-2)--(-1)--(0)--(1)--(2)-->\n</code></pre>\n<p>The real numbers form a <a href=\"http://en.wikipedia.org/wiki/Continuity_%28topology%29\">continuous topology</a> rather than a discrete one, there is no space between them. A ‘<a href=\"http://en.wikipedia.org/wiki/Real_line\">real line</a>’:</p>\n<pre><code><------------------------->\n</code></pre>\n<p>Our ‘scoring’ mechanism allows us to change the battlefield, it forms a function (isomorphism) to convert our discrete topology into a continuous one. We can now bring our continuous tooling to bear on a problem that was previously inaccessible.</p>\n<h2>Striated vs Smooth Space</h2>\n<blockquote>\n<p>Military tactics are like unto water; for water in its natural course runs away from high places and hastens downwards. So in war, the way is to avoid what is strong and to strike at what is weak. Like water, taking the line of least resistance. Water shapes its course according to the nature of the ground over which it flows; the soldier works out his victory in relation to the foe whom he is facing. Therefore, just as water retains no constant shape, so in warfare there are no constant conditions.Sun Tzu, “the Art of War”</p>\n</blockquote>\n<p>Okay. NOW we’re ready to talk about smooth and striated space. They have a number of names for this concept, and the one that’s most direct from where we currently are is ‘Riemann space / Euclidean space’. Smooth -> Riemann, Euclidean -> Striated. Another isomorphism. ;)</p>\n<p><a href=\"http://en.wikipedia.org/wiki/Euclidean_geometry\">Euclidean geometry</a> is pretty much the foundation of a metric ton of our math. It’s what’s now known as algebra and geometry: lots of integers, discrete spaces. This cube starts at (1,2,4) and has a side length of 5. Cartesian coordinates.</p>\n<p>Along comes <a href=\"http://en.wikipedia.org/wiki/Carl_Friedrich_Gauss\">Gauss</a>, who was incredibly intelligent. He had this student named <a href=\"http://en.wikipedia.org/wiki/Bernhard_Riemann\">Riemann</a>, who ran with the ideas Gauss had and started <a href=\"http://en.wikipedia.org/wiki/Riemannian_geometry\">Riemannian Geometry</a>. Riemann’s geometry is <a href=\"http://en.wikipedia.org/wiki/Non-Euclidean_geometry\">non-Euclidean</a>: if Euclid described lines, Riemann described curves. Einstein would later base the theory of relativity on Riemannian geometry.</p>\n<p>So, who cares? Well, think about it this way: You’re in a car, driving down a straight road. It goes for miles, with no turns or other streets. You’re operating in a smooth space: you can easily go from one place to another. Now, someone comes along and puts up stop signs every block. You must (for now) stop at these signs. Now you’re operating in a striated space: your movement is restricted. Now you’re moving from point to point on a line. It’s a very different experience, and you also didn’t have anything to do with the stop signs….</p>\n<p>According to D&G, ‘the state machine’ (one instance of which is the State) attempts to take space that is smooth and striate it. This is often a means towards imperialism, as the striation happens through quantification. Take, for example, the sea: a smooth space, you can impose latitude and longitude on top of it, simultaneously quantifying and striating it. This allows you to navigate the waters, and conquer the sea.</p>\n<p>This is also the position that many web startups take: ‘friendship’ was a smooth space before social networks came along. They quantified and striated that space, turning “I have many friends” into “I have 200 friends and you have 100 friends.” This quantification allows for commodification: now Facebook can sell ads. It’s also why startups don’t often have a ‘business model’ at first: they first need to conquer and striate their space before they find the commodity. Sometimes you just hit dirt and go bust, sometimes you find a more precious commodity, then refine and sell it.</p>\n<p>‘nomads’ are entities which navigate and live in smooth spaces. Maybe my twitter bio makes a bit more sense now. ;)</p>\n<h2>How do you create a smooth space?</h2>\n<p>You might remember Riemann from the ‘<a href=\"http://en.wikipedia.org/wiki/Riemann_sum\">Riemann sum</a>’ in Calc I. The brilliance of the Riemann sum is that it first striates, then re-smooths the space. You start off first with a curve, and then approximate the area under it by dividing it into a number of evenly-sized bits. This first bit was necessary in a world without calculus: we didn’t have the tooling or concepts to actually tackle the real area, so we mapped that problem to one we did know how to solve. As the number of bits goes up, and their width goes down, more and more of the space under the curve is captured by our algorithm. Finally, once we’re able to take that first step away from (striated) algebra and move into (smooth) calculus, we’re able to move about on the curve for reals. We’re back to smooth space again.</p>\n<p>This interplay between smooth and striated spaces often happens, and there’s really no part of our world today that’s completely smooth or entirely striated. D&G posit that the left should be attempting to create as much smooth space as possible, and that capitalism is constantly attempting to striate space. That said, smooth space is necessary, but not sufficient: capitalism, in its hunger and lust for acquisition and totalitization, has managed to navigate some smooth spaces. I’ll just quote straight from ‘a thousand plateaus’, 492. It’s a little hard, but this is getting long enough as is. ;)</p>\n<blockquote>\n<p>Not only does the user as such tend to be an employee, but capitalism operates less on a quantity of labor than by a complex qualitative process bringing into play modes of transportation, urban models, the media, the entertainment industries, ways of perceiving and feeling – every semiotic system. It is though, at the outcome of the striation that capitalism was able to carry to an unequaled point of perfection, circulating capital necessarily recreated, reconstituted, a sort of smooths pace in which the destiny of human beings is recast. … at the level of world capitalism, a new smooth space is produced in which capital reaches its ‘absolute’ speed, based on machinic components rather than the human component of labor. The multinationals fabricate a kind of deterritorialized smooth space in which points of occupation as well as poles of exchange become quite independent of the classical paths to striation. .. the essential thing is instead the distinction between striated capital and smooth capital, and the way in which the former gives rise to the latter through complexes that cut across territories and States, and even the different types of States.</p>\n</blockquote>\n<p>The first part about ‘users’ is very much drawing a parallel to “If you’re not paying for the product, you are the product.” When they talk about the ‘speed’ of capital, think of HFT. Multinational corporations have managed to overcome the striation that a State has imposed. McDonalds is only partially affected by US laws. They operate ‘outside’ of states.</p>\n<p>In this way, simply being smooth will not suffice to save us.</p>\n<hr />\n<p>If you liked this blog post, you may enjoy <a href=\"https://groups.google.com/forum/#!forum/philosophy-in-a-time-of-software\">\"Philosophy in a time of software</a>, a moderated Google Group to discuss the intersection of philosophy and technology. This post started out as an email to that list.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/travis-build-matrix-for-rails/",
"title": "Travis build matrix for Rails",
"description": null,
"url": "https://steveklabnik.com/writing/travis-build-matrix-for-rails/",
"published": "2013-03-10T00:00:00.000Z",
"updated": "2013-03-10T00:00:00.000Z",
"content": "<p>Do you have a gem that needs to test against multiple versions of Rails? Doing it all can be complex, and while I have lots to say about this topic, but here’s one of the pieces: Travis build matrix.</p>\n<p>Here’s what you want:</p>\n<pre><code>language: ruby\nrvm:\n - 1.8.7\n - 1.9.2\n - 1.9.3\n - 2.0.0\n - ruby-head\nenv:\n - \"RAILS_VERSION=3.2\"\n - \"RAILS_VERSION=3.1\"\n - \"RAILS_VERSION=3.0\"\n - \"RAILS_VERSION=master\"\nmatrix:\n allow_failures:\n - rvm: ruby-head\n - env: \"RAILS_VERSION=master\"\n exclude:\n - rvm: 2.0.0\n env: \"RAILS_VERSION=3.0\"\n - rvm: 2.0.0\n env: \"RAILS_VERSION=3.1\"\n - rvm: 1.8.7\n env: \"RAILS_VERSION=master\"\n - rvm: 1.9.2\n env: \"RAILS_VERSION=master\"\n - rvm: ruby-head\n env: \"RAILS_VERSION=3.0\"\n - rvm: ruby-head\n env: \"RAILS_VERSION=3.1\"\n</code></pre>\n<p>Here’s what all this does:</p>\n<ol>\n<li>1.8.7 and 1.9.2 is supported across all of the 3.x series, but not master</li>\n<li>2.0.0 is supported on Rails 3.2 (as of the upcoming 3.2.13, but it will probably work with your gem with 3.2.12) and master, but not 3.0 or 3.1.</li>\n<li>We want to allow failures on ruby-head, because it’s not likely to be stable</li>\n<li>We also want to allow failures for Rails master, because sometimes it’s not stable either</li>\n<li>ruby-head should only be built against master</li>\n</ol>\n<p>What do you think? <a href=\"http://twitter.com/steveklabnik\">Tweet at me</a> if you have suggestions for improving this matrix.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/announcing-securityreleasepractice/",
"title": "Announcing security_release_practice",
"description": null,
"url": "https://steveklabnik.com/writing/announcing-securityreleasepractice/",
"published": "2013-03-08T00:00:00.000Z",
"updated": "2013-03-08T00:00:00.000Z",
"content": "<p>Security is hard. One of the skills any OSS maintainer needs is how to do releases, and security releases are a special kind of release that needs special git-fu to make it work.</p>\n<p>I’ve created a repository so that you can practice this particular skill. It’s called ‘security_release_practice’.</p>\n<h2>The problem</h2>\n<p>‘security_release_practice’ provides a binary, <code>omg_insecure</code>, which has a security issue. Basically, the <code>super_secure_calculation</code> method converts user input to a symbol. Since Ruby does not garbage collect symbols, the longer you run <code>omg_insecure</code>, the more memory it will use, until your computer runs out of memory and the box grinds to a halt. Seems bad.</p>\n<p>So, just fix <code>super_secure_calculation</code> and release, right? Well, here’s the problem: the last release of <code>security_release_practice</code> was 1.0.0. Since then, we’ve had a new feature, and a backwards incompatible change with <code>super_secure_calculation</code>. You can see the two commits <a href=\"https://github.com/steveklabnik/security_release_practice/compare/v1.0.0...master\">here</a>.</p>\n<p>This is a problem: if we fix the issue and release, people who are relying on the <code>+ 5</code> behavior can’t upgrade: they’ll now be getting <code>+ 6</code>. Also, the new feature (<code>another_new_calculation</code>) may have conflicts or weirdness with their code. That’s bad! So what we really want is a relase that’s exactly the same as 1.0.0, but with the security fix applied.</p>\n<p>Let’s give that a shot.</p>\n<h2>The answer</h2>\n<p>If you think you’re good with <code>git</code>, you can try this out right now. If you’ve done it correctly, you should end up with the following:</p>\n<ol>\n<li>A 1-0-stable branch</li>\n<li>That branch should contain a new commit that fixes the issue</li>\n<li>That branch should contain a new tag, v1.0.1 that fixes the issue</li>\n<li>Master should have a backported version of the commit in #2.</li>\n</ol>\n<p>The repository <a href=\"https://github.com/steveklabnik/security_release_practice\">as it exists</a> has all of this stuff, so check your work against it!</p>\n<h2>Practice!</h2>\n<p>If you <em>don’t</em> know how to do this, or you get stuck, you’ve come to the right place! Here’s what you need to do:</p>\n<p>First, some setup work. Fork the repository and clone it down. Or, just clone mine, whatever:</p>\n<pre><code>$ git clone https://github.com/steveklabink/security_release_practice\n$ cd security_release_practice\n</code></pre>\n<p>Next, since this repository has the backported fix involved, you need to remove that commit:</p>\n<pre><code>$ git reset --hard HEAD~1\n</code></pre>\n<p>This basically backs our branch out by one commit. Now we’re ready to go.</p>\n<p>The first thing in actually doing the work is to check out the tag that we last released from. In our case, that tag is <code>v1.0.0</code>. So let’s do that now:</p>\n<pre><code>$ git checkout v1.0.0\n</code></pre>\n<p><code>git</code> will give you a message:</p>\n<pre><code>Note: checking out 'v1.0.0'.\n\nYou are in 'detached HEAD' state. You can look around, make experimental\nchanges and commit them, and you can discard any commits you make in this\nstate without impacting any branches by performing another checkout.\n\nIf you want to create a new branch to retain commits you create, you may\ndo so (now or later) by using -b with the checkout command again. Example:\n\n git checkout -b new_branch_name\n\nHEAD is now at 47a8bfb... Initial release.\n</code></pre>\n<p>We want to take <code>git</code>’s advice: let’s make a new branch. Since it’s going to be all our fixes for <code>1.0.x</code>, let’s call it <code>1-0-stable</code>:</p>\n<pre><code>git checkout -b 1-0-stable\n</code></pre>\n<p>Now we have our stable branch. Awesome! Master is the work that will go into <code>1.1.x</code>, and this branch will be for <code>1.0.x</code>.</p>\n<p>Next, we need to fix our bug. You need to remove this one line:</p>\n<pre><code>--- a/lib/security_release_practice.rb+++ b/lib/security_release_practice.rb@@ -3,7 +3,7 @@ require \"security_release_practice/version\" module SecurityReleasePractice def super_secure_calculation(input)- input.to_sym input.to_i + 5 end def another_new_calculation(input)\n</code></pre>\n<p>We don’t even use that symbol, what a waste! Commit this, with a descriptive message. You can <a href=\"https://github.com/steveklabnik/security_release_practice/commit/168d5f756221ed43b0c67569ac82429f0b391504\">find mine here</a>. Note the first few characters of the hash: mine is <code>168d5f756221</code>.</p>\n<p>Next, we need to release. Go ahead and increment the version number in <code>lib/security_release_practice/version.rb</code>, commit that, and then try <code>rake install</code>. Everything should work. Great! If you were actually releasing this gem, you’d be running <code>rake release</code> instead, but it’s my gem, not yours. ????</p>\n<p>Okay, now we’ve released a version with our fix, but <code>master</code> still has a vulnerability: we need to port the fix. So let’s go back to master:</p>\n<pre><code>$ git checkout master\n</code></pre>\n<p>And then cherry-pick our fix over:</p>\n<pre><code>$ git cherry-pick 168d5f756221\n</code></pre>\n<p>There will be a conflict. The diff is a little weird, such is life. Go ahead and fix the conflict, then commit:</p>\n<pre><code>$ git commit\n</code></pre>\n<p>And you’re done!</p>\n<h3>Other considerations</h3>\n<p>If we had a CHANGELOG, we’d need to udpate that as appropriate, including creating new sections for our <code>1.0.1</code> release.</p>\n<p>Sometimes it’s easier to fix things on master first, then backport to the new branch. I prefer to do it the way I showed here.</p>\n<p>You should probably try to get as many people to know that you’ve fixed the bug. Tweet, blog, rabble-rouse, and possibly <a href=\"https://groups.google.com/forum/#!forum/ruby-security-ann\">post to the ruby-security-ann mailing list</a>, which was created to help Rubyists know about security releases of their gems.</p>\n<p>If your gem is really widely used, you may want to actually register a CVE. You can find information on this process <a href=\"https://groups.google.com/forum/#!forum/ruby-security-ann\">here</a>. I made one <a href=\"https://groups.google.com/d/msg/ruby-security-ann/TDXOlIVjS54/Ty_9PXYNr3AJ\">here</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/going-vimgan/",
"title": "Going vimgan",
"description": null,
"url": "https://steveklabnik.com/writing/going-vimgan/",
"published": "2013-03-06T00:00:00.000Z",
"updated": "2013-03-06T00:00:00.000Z",
"content": "<p>I’m going vim-gan. Basically, I don’t want to consume any source code which doesn’t come from vim, or products that contain non-vim source code.</p>\n<p>While I won’t, some people I’ve talked to will also consume code produced in Eclipse, since recent scientific studies show that Eclipse users can’t actually feel pain. This is enough of an ethical gray area that I’m staying away from it, but I can see that perspective.</p>\n<p>I just can’t stay silent while the software industry brutalizes source code like this. Not cool, everyone.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/using-puma-on-heroku/",
"title": "Using puma on Heroku",
"description": null,
"url": "https://steveklabnik.com/writing/using-puma-on-heroku/",
"published": "2013-02-24T00:00:00.000Z",
"updated": "2013-02-24T00:00:00.000Z",
"content": "<p>I’m a big fan of <a href=\"http://puma.io/\">Puma</a> these days. Today, I converted my blog over to use Puma, and I figured I’d share how I did it. This blog uses Sinatra.</p>\n<p>If you want to look ahead, <a href=\"https://github.com/steveklabnik/blog/commit/3cd0f04ed29a25df6d6dfeeffccd2e12548c05cf\">here</a> is the commit.</p>\n<h2>Gemfile</h2>\n<p>Installing Puma is really easy: just add <code>gem 'puma'</code> to your Gemfile. Done!</p>\n<p>This section is way too easy. I’m sorry Puma isn’t more complicated. ;)</p>\n<h2>Procfile</h2>\n<p>Heroku attempts to guess how you want to run your site, but sometimes it guesses wrong. I prefer to be a bit more explicit, and in Puma’s case, it will guess wrong and use Webrick, so we need to do this step.</p>\n<p>Heroku uses a gem called <code>foreman</code> to run your application. Basically, <code>foreman</code> gives you a ‘Procfile’ that lets you specify different processes and their types, and then foreman manages running them for you.</p>\n<p>To use <code>foreman</code>, add it to your Gemfile:</p>\n<pre><code>gem 'foreman'\n</code></pre>\n<p>And then make a <code>Procfile</code>:</p>\n<pre><code>web: rackup -s puma -p $PORT\n</code></pre>\n<p>This says “I want my web processes to run this command.” Simple. We tell <code>rackup</code> to use Puma with the <code>-s</code> flag, for ‘server.’</p>\n<p>To start up your application, normall you’d run <code>rackup</code> yourself, but now, you use <code>foreman</code>:</p>\n<pre><code>$ bundle exec foreman start\n</code></pre>\n<p>It’ll print out a bunch of messages. Mine looks like this:</p>\n<pre><code>$ bundle exec foreman start\n15:05:05 web.1 | started with pid 52450\n15:05:06 web.1 | Puma 1.6.3 starting...\n15:05:06 web.1 | * Min threads: 0, max threads: 16\n15:05:06 web.1 | * Environment: development\n15:05:06 web.1 | * Listening on tcp://0.0.0.0:5000\n</code></pre>\n<p>Now you can <code>open http://localhost:5000</code> in another terminal, and bam! Done.</p>\n<h2>Other Rubies</h2>\n<p>If you really want Puma to scream, you should run with Rubinius or JRuby. I haven’t done this yet, but once I do, I’ll update this post with instructions.</p>\n<h2>Rails</h2>\n<p>If you’re using Rails, it’s the same process, but the Procfile line should be</p>\n<pre><code>web: bundle exec puma -p $PORT\n</code></pre>\n<p>Easy!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/i-m-writing--rails-4-in-action--/",
"title": "I'm writing \"Rails 4 in Action\"!",
"description": null,
"url": "https://steveklabnik.com/writing/i-m-writing--rails-4-in-action--/",
"published": "2013-02-13T00:00:00.000Z",
"updated": "2013-02-13T00:00:00.000Z",
"content": "<p>Last night, I took a big step forward in my writing career: I’ll be building upon the work Ryan Bigg was doing to take “Rails 3 in Action” and move it to Rails 4. I signed the contract a few weeks ago, but last night I actually did the first bit of editing, and successfully got everything through the system.</p>\n<p>Here’s the <a href=\"http://www.manning.com/bigg2/\">early access edition</a> link, where you can check out what I’m doing. Of course, Rails 4 isn’t actually out yet, but since I’m on the Rails team, I already keep abreast of changes daily. I’ll be doing a first pass to take the book from 3.2 to edge over the next few weeks, and then keeping it current until the actual release of Rails 4.</p>\n<p>Of course, updating the content is much easier than writing it in the first place. Thanks for all of your hard work on previous editions, Yehuda and Ryan!</p>\n<hr />\n<p>Of course, this doesn’t mean I’ll stop working my my two self-published books, <a href=\"http://www.designinghypermediaapis.com/\">Designing Hypermedia APIs</a> and <a href=\"http://www.rustforrubyists.com/\">Rust for Rubyists</a>. It just means I’ll be doing even more writing!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-next-iteration-of-designing-hypermedia-apis/",
"title": "The next iteration of \"Designing Hypermedia APIs\"",
"description": null,
"url": "https://steveklabnik.com/writing/the-next-iteration-of-designing-hypermedia-apis/",
"published": "2013-02-12T00:00:00.000Z",
"updated": "2013-02-12T00:00:00.000Z",
"content": "<p>I sent out an email today to everyone who’d previously purchased <a href=\"http://www.designinghypermediaapis.com/\">Designing Hypermedia APIs</a>. Here’s the text:</p>\n<pre><code>Hey there,\n\nFirst of all, I want to apologize for sending you an email. I try to\nkeep these to a minimum. But this one is important.\n\nI haven't been uploading new content lately for a few reasons.\nHolidays are complicated, I wasn't feeling inspired... but this is the\nreal reason: I've totally moved the site to something different. I\njust updated it, so the DNS might need an update, but\nhttp://www.designinghypermediaapis.com/ is all-new. Here's the deal:\n\n1. Gone is logging in and reading it online. PDF/ePUB/MOBI now.\n2. Because you purchased a copy before, you get the big bundle. Thank\nyou for your support over the last year.\n3. You'll be getting another email with a link to download the files.\nWhen I send an update, you'll get a new email with a new link.\n4. The current written content is basically an appendix: I'm going to\nbe writing a more traditional book, from start to finish. It'll walk\nyou through building an application, both client and server.\n5. I want this to be of the highest quality, so I'm not going to\ncommit to a schedule.\n\nIf you aren't a part of the mailing list, we're doing a reading club\nover the next few months, \"Building Hypermedia APIs with HTML5 and\nNode.\" You can join by emailing [email protected].\n\nThank you once again for your support of this project, I really\nappreciate it. As always, any and all feedback welcome: I bet I can do\nsome nicer formatting of the PDF/ePUB/MOBI now that I have them\ngenerating.\n</code></pre>\n<p>Wooo!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/ruby-on-rails-maintenance-policy/",
"title": "Ruby on Rails maintenance policy",
"description": null,
"url": "https://steveklabnik.com/writing/ruby-on-rails-maintenance-policy/",
"published": "2013-02-11T00:00:00.000Z",
"updated": "2013-02-11T00:00:00.000Z",
"content": "<p>Recently, the Rails team has committed to a specific policy related to release maintenance. Due to the rapid pace of recent releases, it’s good to understand how your apps relate to this policy.</p>\n<p>The policy was originally posted on Google Groups, here: <a href=\"https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-security/G4TTUDDYbNA\">https://groups.google.com/forum/?fromgroups=#!topic/rubyonrails-security/G4TTUDDYbNA</a>.</p>\n<p>Here’s a copy if you don’t want to read Google Groups:</p>\n<hr />\n<p>Since the most recent patch releases there has been some confusion about what versions of Ruby on Rails are currently supported, and when people can expect new versions. Our maintenance policy is as follows.</p>\n<p>Support of the Rails framework is divided into four groups: New features, bug fixes, security issues, and severe security issues. They are handled as follows:</p>\n<h2>New Features</h2>\n<p>New Features are only added to the master branch and will not be made available in point releases.</p>\n<h2>Bug fixes</h2>\n<p>Only the latest release series will receive bug fixes. When enough bugs are fixed and its deemed worthy to release a new gem, this is the branch it happens from.</p>\n<p>Currently included series: 3.2.x</p>\n<h2>Security issues:</h2>\n<p>The current release series and the next most recent one will receive patches and new versions in case of a security issue.</p>\n<p>Currently included series: 3.2.x, 3.1.x</p>\n<h2>Severe security issues:</h2>\n<p>For severe security issues we will provide new versions as above, and also the last major release series will receive patches and new versions. The classification of the security issue is judged by the core team.</p>\n<p>Currently included series: 3.2.x, 3.1.x, 2.3.x</p>\n<h2>Unsupported Release Series</h2>\n<p>When a release series is no longer supported, it’s your own responsibility to deal with bugs and security issues. We may provide back-ports of the fixes and publish them to git, however there will be no new versions released. If you are not comfortable maintaining your own versions, you should upgrade to a supported version.</p>\n<p>You should also be aware that Ruby 1.8 will reach End of Life in June 2013, no further ruby security releases will be provided after that point. If your application is only compatible ruby 1.8 you should upgrade accordingly.</p>\n<p>– Cheers,</p>\n<p>Koz</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/emberjs-and-accessibility/",
"title": "Ember.js and accessibility",
"description": null,
"url": "https://steveklabnik.com/writing/emberjs-and-accessibility/",
"published": "2013-02-07T00:00:00.000Z",
"updated": "2013-02-07T00:00:00.000Z",
"content": "<p>Hey everyone! I made a video today about Ember.js and accessibility. I had always been repeating what I’d heard about screen readers: they can’t work with JavaScript. Turns out that’s not exactly true. The video is the best way to show you:</p>\n<p><a href=\"http://vimeo.com/59124303\">VoiceOver, Ember.js, and WAI-ARIA</a> from <a href=\"http://vimeo.com/steveklabnik\">Steve Klabnik</a> on <a href=\"http://vimeo.com/\">Vimeo</a>.</p>\n<p>Here’s a rough transcript, if you don’t want to watch it:</p>\n<hr />\n<p>Hey everyone. I recently was checking out <a href=\"http://emberjs.com/\">Ember.js</a>, and more specifically, <a href=\"http://discourse.org/\">Discourse</a>. Discourse is supposed to be a next-generation discussion platform, and it’s an Ember app backed by a Rails-based JSON API. I mentioned to Ember co-creator <a href=\"http://twitter.com/wycats\">Yehuda Katz</a> that I wished Ember had ‘a better accessibility story’. He asked me to elaborate, and I mentioned how my friends on Twitter were saying that Discourse wouldn’t able to be inclusive because JavaScript-heavy apps can’t be used by screenreaders. So he said, “Hit ⌘-F5.”</p>\n<p>Whoah.</p>\n<p>Hitting that key combination activates <a href=\"http://www.apple.com/accessibility/voiceover/\">VoiceOver</a>, a feature of OS X I didn’t know existed. Basically, VoiceOver is able to interact with applications and allow keyboard navigation for those who are blind or sight-impaired. Neat! But what about in-browser? Turns out VoiceOver handles JavaScript heavy pages just fine.</p>\n<p>I tried it out on <a href=\"http://try.discourse.org/\">http://try.discourse.org/</a>, and it works pretty well! Not perfect, but you can manage. Yehuda also pointed me at <a href=\"http://www.w3.org/WAI/intro/aria.php\">WAI-ARIA</a>, which is a W3C initiative. I’ll let the site explain:</p>\n<blockquote>\n<p>WAI-ARIA, the Accessible Rich Internet Applications Suite, defines a way to make Web content and Web applications more accessible to people with disabilities. It especially helps with dynamic content and advanced user interface controls developed with Ajax, HTML, JavaScript, and related technologies.</p>\n</blockquote>\n<p>Neat! Basically, you can add markup to your HTML, and it will help applications like VoiceOver allow people to use your site more easily, even if it uses a lot of Ajax. This is super cool, and for me, mitigates one of my main complaints against using SPAs. Of course, WAI-ARIA isn’t really Ember’s job; you’ll have to make the HTML be output in the right way yourself. But it makes me really happy to know that people with disabilities don’t need to be left out of the rich-client portion of the web.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/how-to-not-rely-on-rubygemsorg-for-deployment/",
"title": "How to not rely on rubygems.org for development",
"description": null,
"url": "https://steveklabnik.com/writing/how-to-not-rely-on-rubygemsorg-for-deployment/",
"published": "2013-01-31T00:00:00.000Z",
"updated": "2013-01-31T00:00:00.000Z",
"content": "<p>Due to the recent <a href=\"https://status.heroku.com/incidents/489\">situation with Rubygems.org</a>, a lot of people noticed that they rely on Rubygems.org when deploying. A lot of people advocate <a href=\"http://ryan.mcgeary.org/2011/02/09/vendor-everything-still-applies/\">“vendor everything”</a>, and while that’s one approach, I actually don’t think it’s necessary. I think a lot of people <em>think</em> they rely on Rubygems.org more than they actually do, or, at least, aren’t deploying with Bundler correctly. So let’s talk about it.</p>\n<h2>Deploying with Bundler</h2>\n<p>Bundler has a <code>--deployment</code> flag to help you deploy code. <a href=\"http://gembundler.com/v1.2/deploying.html\">Here’s the documentation</a>. If you use Bundler, and you don’t use <code>--deployment</code> when deploying, you are probably Doing It Wrong. Let’s try a brand new app, and see what happens:</p>\n<pre><code>$ rails _3.2.11_ new foo --skip-bundle\n create \n create README.rdoc\n create Rakefile\n create config.ru\n create .gitignore\n create Gemfile\n create app\n create app/assets/images/rails.png\n create app/assets/javascripts/application.js\n create app/assets/stylesheets/application.css\n create app/controllers/application_controller.rb\n create app/helpers/application_helper.rb\n create app/views/layouts/application.html.erb\n create app/mailers/.gitkeep\n create app/models/.gitkeep\n create config\n create config/routes.rb\n create config/application.rb\n create config/environment.rb\n create config/environments\n create config/environments/development.rb\n create config/environments/production.rb\n create config/environments/test.rb\n create config/initializers\n create config/initializers/backtrace_silencers.rb\n create config/initializers/inflections.rb\n create config/initializers/mime_types.rb\n create config/initializers/secret_token.rb\n create config/initializers/session_store.rb\n create config/initializers/wrap_parameters.rb\n create config/locales\n create config/locales/en.yml\n create config/boot.rb\n create config/database.yml\n create db\n create db/seeds.rb\n create doc\n create doc/README_FOR_APP\n create lib\n create lib/tasks\n create lib/tasks/.gitkeep\n create lib/assets\n create lib/assets/.gitkeep\n create log\n create log/.gitkeep\n create public\n create public/404.html\n create public/422.html\n create public/500.html\n create public/favicon.ico\n create public/index.html\n create public/robots.txt\n create script\n create script/rails\n create test/fixtures\n create test/fixtures/.gitkeep\n create test/functional\n create test/functional/.gitkeep\n create test/integration\n create test/integration/.gitkeep\n create test/unit\n create test/unit/.gitkeep\n create test/performance/browsing_test.rb\n create test/test_helper.rb\n create tmp/cache\n create tmp/cache/assets\n create vendor/assets/javascripts\n create vendor/assets/javascripts/.gitkeep\n create vendor/assets/stylesheets\n create vendor/assets/stylesheets/.gitkeep\n create vendor/plugins\n create vendor/plugins/.gitkeep\n\nsteve at thoth in ~/tmp\n$ cd foo\n\nsteve at thoth in ~/tmp/foo\n$ cat .bundle/config \ncat: .bundle/config: No such file or directory\n</code></pre>\n<p>We have no configuration for bundler. Makes sense, we never bundled. If we try to use <code>--deployment</code> now, we get an error:</p>\n<pre><code>steve at thoth in ~/tmp/foo\n[1] $ bundle install --deployment ✘\nThe --deployment flag requires a Gemfile.lock. Please make sure you have checked your Gemfile.lock into version control before deploying.\n</code></pre>\n<p>This is because <code>--deployment</code> checks the lock. So let’s make one:</p>\n<pre><code>steve at thoth in ~/tmp/foo\n[16] $ bundle ✘\nFetching gem metadata from https://rubygems.org/...........\nFetching gem metadata from https://rubygems.org/..\nUsing rake (10.0.3) \nUsing i18n (0.6.1) \nUsing multi_json (1.5.0) \nUsing activesupport (3.2.11) \nUsing builder (3.0.4) \nUsing activemodel (3.2.11) \nUsing erubis (2.7.0) \nUsing journey (1.0.4) \nUsing rack (1.4.4) \nUsing rack-cache (1.2) \nUsing rack-test (0.6.2) \nUsing hike (1.2.1) \nUsing tilt (1.3.3) \nUsing sprockets (2.2.2) \nUsing actionpack (3.2.11) \nUsing mime-types (1.19) \nUsing polyglot (0.3.3) \nUsing treetop (1.4.12) \nUsing mail (2.4.4) \nUsing actionmailer (3.2.11) \nUsing arel (3.0.2) \nUsing tzinfo (0.3.35) \nUsing activerecord (3.2.11) \nUsing activeresource (3.2.11) \nUsing bundler (1.3.0.pre.5) \nUsing coffee-script-source (1.4.0) \nUsing execjs (1.4.0) \nUsing coffee-script (2.2.0) \nInstalling rack-ssl (1.3.3) \nUsing json (1.7.6) \nUsing rdoc (3.12) \nUsing thor (0.17.0) \nUsing railties (3.2.11) \nUsing coffee-rails (3.2.2) \nUsing jquery-rails (2.2.0) \nUsing rails (3.2.11) \nUsing sass (3.2.5) \nUsing sass-rails (3.2.6) \nUsing sqlite3 (1.3.7) \nUsing uglifier (1.3.0) \nYour bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.\n</code></pre>\n<p>This of course does hit the network, and does all the usual things. Your application normally has a lock already and has it checked in. We still don’t have a config:</p>\n<pre><code>steve at thoth in ~/tmp/foo\n$ cat .bundle/config\ncat: .bundle/config: No such file or directory\n</code></pre>\n<p>So, this is the state our app would be in before deploying. At this point, it’s just like we did a <code>git pull</code> or an <code>scp</code> to our server. So let’s install for deployment:</p>\n<pre><code>steve at thoth in ~/tmp/foo\n$ bundle install --deployment\nFetching gem metadata from https://rubygems.org/.........\nFetching gem metadata from https://rubygems.org/..\nInstalling rake (10.0.3) \nInstalling i18n (0.6.1) \nInstalling multi_json (1.5.0) \nInstalling activesupport (3.2.11) \nInstalling builder (3.0.4) \nInstalling activemodel (3.2.11) \nInstalling erubis (2.7.0) \nInstalling journey (1.0.4) \nInstalling rack (1.4.4) \nInstalling rack-cache (1.2) \nInstalling rack-test (0.6.2) \nInstalling hike (1.2.1) \nInstalling tilt (1.3.3) \nInstalling sprockets (2.2.2) \nInstalling actionpack (3.2.11) \nInstalling mime-types (1.19) \nInstalling polyglot (0.3.3) \nInstalling treetop (1.4.12) \nInstalling mail (2.4.4) \nInstalling actionmailer (3.2.11) \nInstalling arel (3.0.2) \nInstalling tzinfo (0.3.35) \nInstalling activerecord (3.2.11) \nInstalling activeresource (3.2.11) \nInstalling coffee-script-source (1.4.0) \nInstalling execjs (1.4.0) \nInstalling coffee-script (2.2.0) \nInstalling rack-ssl (1.3.3) \nInstalling json (1.7.6) \nInstalling rdoc (3.12) \nInstalling thor (0.17.0) \nInstalling railties (3.2.11) \nInstalling coffee-rails (3.2.2) \nInstalling jquery-rails (2.2.0) \nUsing bundler (1.3.0.pre.5) \nInstalling rails (3.2.11) \nInstalling sass (3.2.5) \nInstalling sass-rails (3.2.6) \nInstalling sqlite3 (1.3.7) \nInstalling uglifier (1.3.0) \nYour bundle is complete! It was installed into ./vendor/bundle\nPost-install message from rdoc:\nDepending on your version of ruby, you may need to install ruby rdoc/ri data:\n\n<= 1.8.6 : unsupported\n = 1.8.7 : gem install rdoc-data; rdoc-data --install\n = 1.9.1 : gem install rdoc-data; rdoc-data --install\n>= 1.9.2 : nothing to do! Yay!\n\n</code></pre>\n<p>We still hit Rubygems.org, but note now that it said “Installing” rather than using. That’s because we now have everything vendored:</p>\n<pre><code>$ cat .bundle/config\n---\nBUNDLE_FROZEN: '1'\nBUNDLE_PATH: vendor/bundle\nBUNDLE_DISABLE_SHARED_GEMS: '1'\n\nsteve at thoth in ~/tmp/foo\n$ ls vendor/bundle/ruby/1.9.1\nbin cache doc gems specifications\n\nsteve at thoth in ~/tmp/foo\n$ ls vendor/bundle/ruby/1.9.1/gems\nactionmailer-3.2.11 multi_json-1.5.0\nactionpack-3.2.11 polyglot-0.3.3\nactivemodel-3.2.11 rack-1.4.4\nactiverecord-3.2.11 rack-cache-1.2\nactiveresource-3.2.11 rack-ssl-1.3.3\nactivesupport-3.2.11 rack-test-0.6.2\narel-3.0.2 rails-3.2.11\nbuilder-3.0.4 railties-3.2.11\ncoffee-rails-3.2.2 rake-10.0.3\ncoffee-script-2.2.0 rdoc-3.12\ncoffee-script-source-1.4.0 sass-3.2.5\nerubis-2.7.0 sass-rails-3.2.6\nexecjs-1.4.0 sprockets-2.2.2\nhike-1.2.1 sqlite3-1.3.7\ni18n-0.6.1 thor-0.17.0\njourney-1.0.4 tilt-1.3.3\njquery-rails-2.2.0 treetop-1.4.12\njson-1.7.6 tzinfo-0.3.35\nmail-2.4.4 uglifier-1.3.0\nmime-types-1.19\n</code></pre>\n<p>Okay! Neat. So we’ve Vendored Everything… what happens if we <code>bundle</code> again?</p>\n<pre><code>steve at thoth in ~/tmp/foo\n$ bundle\nUsing rake (10.0.3) \nUsing i18n (0.6.1) \nUsing multi_json (1.5.0) \nUsing activesupport (3.2.11) \nUsing builder (3.0.4) \nUsing activemodel (3.2.11) \nUsing erubis (2.7.0) \nUsing journey (1.0.4) \nUsing rack (1.4.4) \nUsing rack-cache (1.2) \nUsing rack-test (0.6.2) \nUsing hike (1.2.1) \nUsing tilt (1.3.3) \nUsing sprockets (2.2.2) \nUsing actionpack (3.2.11) \nUsing mime-types (1.19) \nUsing polyglot (0.3.3) \nUsing treetop (1.4.12) \nUsing mail (2.4.4) \nUsing actionmailer (3.2.11) \nUsing arel (3.0.2) \nUsing tzinfo (0.3.35) \nUsing activerecord (3.2.11) \nUsing activeresource (3.2.11) \nUsing coffee-script-source (1.4.0) \nUsing execjs (1.4.0) \nUsing coffee-script (2.2.0) \nUsing rack-ssl (1.3.3) \nUsing json (1.7.6) \nUsing rdoc (3.12) \nUsing thor (0.17.0) \nUsing railties (3.2.11) \nUsing coffee-rails (3.2.2) \nUsing jquery-rails (2.2.0) \nUsing bundler (1.3.0.pre.5) \nUsing rails (3.2.11) \nUsing sass (3.2.5) \nUsing sass-rails (3.2.6) \nUsing sqlite3 (1.3.7) \nUsing uglifier (1.3.0) \nYour bundle is complete! It was installed into ./vendor/bundle\n\n</code></pre>\n<p>The <code>bundle</code> command uses our config settings in <code>.bundle/config</code>, which re-runs with the vendored bundle.</p>\n<h2>Wait, I thought you said NO to Vendor Everything!!!</h2>\n<p>Well, here’s the deal: you can Vendor Everything <em>on your server</em>, which means that we’re not committing gems into source control, and then pushing that huge mess over the network.</p>\n<p>Let’s re-examine <code>--deployment</code> in the context of our two strategies: <code>scp</code> and <code>git</code>.</p>\n<h3>git</h3>\n<p>If you deploy with git, it’s two ways: <code>ssh</code>ing into the server and running a <code>git pull</code>, or by doing a <code>git push</code>. In both cases, you’ll have some sort of post-deploy hook that manages the rest of the deploy.</p>\n<p>This scenario is exactly as shown above: just make sure that your first <code>bundle</code> on deploy is using <code>--deployment</code>, and you’re Good To Go. Each next deploy won’t hit the network. Rock on.</p>\n<h3>scp</h3>\n<p>If you use scp in some way to deploy, then you’re getting a new copy of the application every time, so that bundle full of gems won’t work. You need one more flag: <code>--path</code></p>\n<pre><code>steve at thoth in ~/tmp/foo\n$ rm -rf vendor \n\nsteve at thoth in ~/tmp/foo\n$ bundle install --deployment --path ../vendor/bundle\nFetching gem metadata from https://rubygems.org/.........\nFetching gem metadata from https://rubygems.org/..\nInstalling rake (10.0.3) \nInstalling i18n (0.6.1) \nInstalling multi_json (1.5.0) \nInstalling activesupport (3.2.11) \nInstalling builder (3.0.4) \nInstalling activemodel (3.2.11) \nInstalling erubis (2.7.0) \nInstalling journey (1.0.4) \nInstalling rack (1.4.4) \nInstalling rack-cache (1.2) \nInstalling rack-test (0.6.2) \nInstalling hike (1.2.1) \nInstalling tilt (1.3.3) \nInstalling sprockets (2.2.2) \nInstalling actionpack (3.2.11) \nInstalling mime-types (1.19) \nInstalling polyglot (0.3.3) \nInstalling treetop (1.4.12) \nInstalling mail (2.4.4) \nInstalling actionmailer (3.2.11) \nInstalling arel (3.0.2) \nInstalling tzinfo (0.3.35) \nInstalling activerecord (3.2.11) \nInstalling activeresource (3.2.11) \nInstalling coffee-script-source (1.4.0) \nInstalling execjs (1.4.0) \nInstalling coffee-script (2.2.0) \nInstalling rack-ssl (1.3.3) \nInstalling json (1.7.6) \nInstalling rdoc (3.12) \nInstalling thor (0.17.0) \nInstalling railties (3.2.11) \nInstalling coffee-rails (3.2.2) \nInstalling jquery-rails (2.2.0) \nUsing bundler (1.3.0.pre.5) \nInstalling rails (3.2.11) \nInstalling sass (3.2.5) \nInstalling sass-rails (3.2.6) \nInstalling sqlite3 (1.3.7) \nInstalling uglifier (1.3.0) \nYour bundle is complete! It was installed into /Users/steve/tmp/vendor/bundle\nPost-install message from rdoc:\nDepending on your version of ruby, you may need to install ruby rdoc/ri data:\n\n<= 1.8.6 : unsupported\n = 1.8.7 : gem install rdoc-data; rdoc-data --install\n = 1.9.1 : gem install rdoc-data; rdoc-data --install\n>= 1.9.2 : nothing to do! Yay!\n\nsteve at thoth in ~/tmp/foo\n$ cat .bundle/config \n---\nBUNDLE_FROZEN: '1'\nBUNDLE_PATH: ../vendor/bundle\nBUNDLE_DISABLE_SHARED_GEMS: '1'\n\nsteve at thoth in ~/tmp/foo\n$ ls vendor\nls: vendor: No such file or directory\n\nsteve at thoth in ~/tmp/foo\n[1] $ ls ../vendor ✘\nbundle\n\nsteve at thoth in ~/tmp/foo\n$ bundle\nUsing rake (10.0.3) \nUsing i18n (0.6.1) \nUsing multi_json (1.5.0) \nUsing activesupport (3.2.11) \nUsing builder (3.0.4) \nUsing activemodel (3.2.11) \nUsing erubis (2.7.0) \nUsing journey (1.0.4) \nUsing rack (1.4.4) \nUsing rack-cache (1.2) \nUsing rack-test (0.6.2) \nUsing hike (1.2.1) \nUsing tilt (1.3.3) \nUsing sprockets (2.2.2) \nUsing actionpack (3.2.11) \nUsing mime-types (1.19) \nUsing polyglot (0.3.3) \nUsing treetop (1.4.12) \nUsing mail (2.4.4) \nUsing actionmailer (3.2.11) \nUsing arel (3.0.2) \nUsing tzinfo (0.3.35) \nUsing activerecord (3.2.11) \nUsing activeresource (3.2.11) \nUsing coffee-script-source (1.4.0) \nUsing execjs (1.4.0) \nUsing coffee-script (2.2.0) \nUsing rack-ssl (1.3.3) \nUsing json (1.7.6) \nUsing rdoc (3.12) \nUsing thor (0.17.0) \nUsing railties (3.2.11) \nUsing coffee-rails (3.2.2) \nUsing jquery-rails (2.2.0) \nUsing bundler (1.3.0.pre.5) \nUsing rails (3.2.11) \nUsing sass (3.2.5) \nUsing sass-rails (3.2.6) \nUsing sqlite3 (1.3.7) \nUsing uglifier (1.3.0) \nYour bundle is complete! It was installed into /Users/steve/tmp/vendor/bundle\n</code></pre>\n<p>The <code>--path</code> flag controls where the bundle is located. In this case, we store it one directory up. Now, when we copy new versions of the code over, it will use the bundle location that stays the same, and all is peachy keen.</p>\n<p>I am told by several people that this is what the Capistrano/bundler recipe does by default, so if you’re using that, you’re already doing this.</p>\n<h2>One Tiny Weakness</h2>\n<p>There is one small weakness of this approach compared to Vendor Everything: as an engineer, it’s your tradeoff to make.</p>\n<p>This way of using bundler <em>will</em> hit the network <em>when you deploy for the first time after updating your bundle</em>. The cache on the server has to update in this case, so it will go fetch the new gems. So here’s the scenario:</p>\n<ol>\n<li>I update my bundle. It gets the new versions of the gems.</li>\n<li>Rubygems.org goes down.</li>\n<li>I need to deploy a branch that has the new versions in it.</li>\n</ol>\n<p>In this case, if you had Vendored Everything, the hit to Rubygems.org would have happened during step 1, and so things would work. If you used this strategy, it would have hit locally, so you could have developed, but then when deploying, it’d hit again to update the bundle on the server, and so it wouldn’t.</p>\n<p>In these situations, you can temporarily switch to Vendor Everything, since you have the bundle installed locally: just copy your local gems over to <code>vendor/bundle</code> and you’re done. This may or may not be too much of a hassle. When I examine the downtime of Rubygems.org, I think it’s worth it to not mess up my git repository with all that gem code. You might not. Do whatever you need to do, but now you know how to not rely on Rubygems.org for every deployment.</p>\n<hr />\n<h3>An addendum</h3>\n<p>I got an email mentioning one more issue with the ‘vendor everything’ strategy:</p>\n<pre><code>$ ls vendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/*.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_document.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_element_description.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_entity_lookup.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_sax_parser_context.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/html_sax_push_parser.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/nokogiri.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_attr.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_attribute_decl.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_cdata.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_comment.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_document.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_document_fragment.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_dtd.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_element_content.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_element_decl.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_encoding_handler.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_entity_decl.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_entity_reference.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_io.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_libxml2_hacks.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_namespace.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_node.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_node_set.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_processing_instruction.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_reader.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_relax_ng.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_sax_parser.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_sax_parser_context.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_sax_push_parser.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_schema.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_syntax_error.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_text.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xml_xpath_context.o\nvendor/bundle/ruby/1.9.1/gems/nokogiri-1.5.6/ext/nokogiri/xslt_stylesheet.o\n</code></pre>\n<p>Basically, your compiled files get vendored, so if you’re, say, developing on Mac and deploying on Linux, you might end up with compiled native extensions for the totally wrong architecture.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/node/",
"title": "Node",
"description": null,
"url": "https://steveklabnik.com/writing/node/",
"published": "2013-01-23T00:00:00.000Z",
"updated": "2013-01-23T00:00:00.000Z",
"content": "<p>Soooo there’s this thing: <a href=\"http://harthur.wordpress.com/2013/01/24/771/\">http://harthur.wordpress.com/2013/01/24/771/</a></p>\n<p>First of all, unequivocally, I said it before, and I’ll say it again: I’m sorry, and feel terrible that I made someone feel terrible. @harthur, I felt bad yesterday, and I’m feeling bad again. Not that it helps you that I feel bad. But I don’t like making other people feel bad. I am sorry.</p>\n<h2>tl;dr</h2>\n<p>TL;DR: I am sorry. I feel terrible that I made someone else feel terrible. I feel even worse because I didn’t mean to say that they were terrible, but accidentally did.</p>\n<h2>the why</h2>\n<p>Sooooooo I wrote up <em>why</em> I said what it was that I said, because what I was trying to say was totally unrelated to <code>replace</code>. I was trying to explain why I don’t like Node, and it basically came out wrong. But, explaining what I said had the affect of feeling like I was trying not to apologize. So, I’ll just cut to the chase and say this, cut straight from the chat log:</p>\n<pre><code>SK: i dont want people to think i'm saying \"i got caught being an asshole so i'm saying sorry\"\nSK: i want to say \"I was accidentally being an asshole so i'm sorry\"\n</code></pre>\n<p>So, I’ll just leave it at that. Twitter makes it so hard not to accidentally be an asshole.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/why-is-a-role-not-a-person/",
"title": "_why is a role, not a person",
"description": null,
"url": "https://steveklabnik.com/writing/why-is-a-role-not-a-person/",
"published": "2013-01-16T00:00:00.000Z",
"updated": "2013-01-16T00:00:00.000Z",
"content": "<p>http://whytheluckystiff.net/ has re-ignited a discussion about _why, and while I have tons and tons and tons of things to say on the subject, this is all I can give you for now: _why is a role, not a person.</p>\n<p>For a long time, I was talking about a separation between _why and the person who played him. I still have not yet figured out how to refer to that person, and so for now, I’ve settled on Subject. So I’ll be using that description for the rest of this post.</p>\n<h2>All the world’s a stage</h2>\n<blockquote>\n<p>All the world’s a stage, And all the men and women merely players:Shakespeare, “As You Like It,” Act II Scene VII</p>\n</blockquote>\n<p>One of the things I’ve learned as I’ve gotten a little older (21 days till 27!) is that I’m a different person in different situations. This may seem painfully obvious, but let’s expand on this a bit: when I’m on stage at a Ruby conference, I’m one person, when I’m by myself at home, I’m a different person, online, I’m a different person, and when I’m spending time with someone I care about, I’m someone else. Each of these contexts produces different behavior, mannerisms, and vocabulary. When I’m speaking with someone who’s read French Postmodernism, I use very different terms than when I’m speaking with my Republican ’Murican relatives.</p>\n<p>There’s absolutely nothing wrong with this, but it really suggests that ‘essentialism’ is not a thing that can properly model the world. “Essentialism” is a position that states that there is some set of attributes that one can possess to ‘be’ something. So, for example, let’s take a Zebra: some essential attributes may be:</p>\n<ol>\n<li>Four legs</li>\n<li>Black with White Stripes</li>\n</ol>\n<p>http://www.flickr.com/photos/kliski/4816615762/</p>\n<p>Shit.</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24435086611428_raw.jpg\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24435086611428_raw.jpg\" /></p>\n<p>Double shit.</p>\n<p>So what makes a zebra a zebra then? I don’t want to get into it, but basically, I take the position that you can’t define something by what it is, you have to define it by what it <em>does</em>. I’m punting on why this is, again, I want to talk about _why, so please give me a charitable reading and take this as axiomatic for now.</p>\n<p>If we’re defined by what we do, then it makes sense that we can contextualize what we do: we can take on different roles at different times. When I program in Ruby, I am a Rubyist; when I ride a bike, I am a cyclist, when I have a child, I am a father, when I love and am loved, I am a lover.</p>\n<p>When you treat programming as an art, when you treat software as an artistic medium, when you spread joy and whimsy with what you do, you are _why.</p>\n<p>It just so happens that Subject was the person who created this particular role, as was the one who best played it. That doesn’t mean that we can’t also try to take that role on from time to time, try out out, see if it fits, wiggle our toes a bit.</p>\n<h2>_why vs. Subject</h2>\n<p>This is one of the reasons I think the concept of _why is important: Subject was trying to teach us something. They were creating an abstraction, or, maybe, an indirection. By keeping themselves private, they put an emphasis on the distance between _why-as-role and _why-as-person, since the second doesn’t exist. The focus was, as it should be, on the role.</p>\n<p>This is also why it’s wrong to ‘track _why down.’ This doesn’t make any sense. Subject is not _why, they just played them on TV. I also think that’s why it’s not wrong to keep talking about _why, as long as we make sure that we’re actually talking about _why, and not about Subject. Subject deserves their privacy.</p>\n<p>I also don’t mean that I want someone to literally ‘take up the mask’ of _why and publish under that name. What I mean is that if we want to take on the lessons that Subject taught us through their persona, we should periodically have becoming-_why moments.</p>\n<h2>On Sloppiness and blog posts written in haste</h2>\n<blockquote>\n<p>@steveklabnik hand write it, take a picture of it. leave the mistakes in.</p>\n</blockquote>\n<p>— tef (@tef) January 6, 2013</p>\n<blockquote></blockquote>\n<h2>Further reading</h2>\n<p>To send you further down a rabbit-hole: http://en.wikipedia.org/wiki/Body_without_organs</p>\n<p>Enjoy.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/draper-1-0-0-released/",
"title": "Draper 1.0.0 released",
"description": null,
"url": "https://steveklabnik.com/writing/draper-1-0-0-released/",
"published": "2013-01-14T00:00:00.000Z",
"updated": "2013-01-14T00:00:00.000Z",
"content": "<p>Today, I released Draper 1.0.0! Draper is a gem that gives you decorators/view models for Rails.</p>\n<ul>\n<li>Rubygems: <a href=\"https://rubygems.org/gems/draper\">https://rubygems.org/gems/draper</a></li>\n<li>GitHub: <a href=\"https://github.com/drapergem/draper\">https://github.com/drapergem/draper</a></li>\n<li>README: <a href=\"https://github.com/drapergem/draper/blob/master/README.md\">https://github.com/drapergem/draper/blob/master/README.md</a></li>\n</ul>\n<p>You can get it with Rubygems:</p>\n<pre><code>$ gem install draper\n</code></pre>\n<p>or by adding this to your Gemfile:</p>\n<pre><code>gem \"draper\", \"~>1.0.0\"\n</code></pre>\n<p>Draper supports all versions of Rails 3, and follows <a href=\"http://semver.org/\">SemVer</a>, so you can be confident that your application will work well with Draper in the future. Thank you to everyone who tried out our six (six!!!11) beta releases and gave us patches and feedback.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rails-has-two-default-stacks/",
"title": "Rails has two default stacks",
"description": null,
"url": "https://steveklabnik.com/writing/rails-has-two-default-stacks/",
"published": "2013-01-14T00:00:00.000Z",
"updated": "2013-01-14T00:00:00.000Z",
"content": "<p>Rails’ greatest strength is “Convention over Configuration.” Basically, Rails makes a ton of default choices for you about how things should be named, where they should go, and other things. This is why working with Rails is so productive for an experienced developer: just follow the Golden Path and everything comes together quickly. It’s second greatest strength is its ‘full stack’ nature: Rails includes everything that you need to get going. It does not attempt to be ‘minimal’ in any way. Put these two things together, and a Rails app makes lots of assumptions about <strong>everything</strong> in your app.</p>\n<p>But what happens when a significant number of people don’t use those defaults?</p>\n<p>First, let’s talk about the actual default stack. Since <a href=\"http://david.heinemeierhansson.com/2012/rails-is-omakase.html\">Rails is omakase</a>, I’m going to call this the ‘Omakase Stack.’ Roughly, this stack contains:</p>\n<ul>\n<li>ERB for view templates</li>\n<li>MySQL for databases</li>\n<li>MiniTest for testing</li>\n<li>Fat Models, Skinny Controllers</li>\n</ul>\n<p>There has been a ‘second default stack’ brewing for a while now. I’m going to call this the “<a href=\"http://en.wikipedia.org/wiki/Prime_(symbol)%23Use_in_mathematics.2C_statistics.2C_and_science\">Prime</a> stack”. This stack’s approximate choices are:</p>\n<ul>\n<li>Haml for view templates</li>\n<li>PostgreSQL for databases</li>\n<li>Rspec/Cucumber for testing</li>\n<li>Skinny models, controllers, and a service layer</li>\n</ul>\n<p>This is great! As David says:</p>\n<blockquote>\n<p>That doesn’t mean patrons can’t express exceptions. Substitutions are allowed, within reason. So you don’t like test/unit? No problem, bring your own rspec. Don’t care for CoffeeScript as a dessert? Delete a single line from your Gemfile.</p>\n</blockquote>\n<p>A considerable minority uses a stack like this. It’s important that the Prime Stack isn’t exact: you might not use Cucumber at all, for example, or maybe you don’t have a service layer. The important part, though, is that this is the new default you build preferences off of: a big group of people assumes that you don’t use MiniTest, you use Rspec, and the choice is Cukes or Capybara. This creates the ‘secondary default’ effect.</p>\n<h2>Dual Stack Ecology</h2>\n<p>So where’s the beef? Well, before I get into it, I want to make one thing explicit: I am <em>not</em> taking a position on which stack, if any, is ‘better.’ I’m not interested, at least in this post, at placing them into any kind of order. That’s for another time. What I want to talk about here is about the effect that a second default stack has on the overall Rails ecosystem.</p>\n<p>Looking at history, the Prime Stack was born from the Omakase Stack. A certain group of people noticed that they were changing a significant amount of the stack most of the time, and so the Prime Stack came into existence. Since this is a historical progression, many blog posts got written describing the delta: take “<a href=\"http://robots.thoughtbot.com/post/159807023/waiting-for-a-factory-girl\">Waiting for a Factory Girl</a>”, for example.</p>\n<blockquote>\n<p>Here at thoughtbot, we’ve had it with fixtures.</p>\n</blockquote>\n<p>But wait! Then, years later, you get a discussion on how factories are bad. My own “<a href=\"http://blog.steveklabnik.com/posts/2012-07-14-why-i-don-t-like-factory_girl\">Why I don’t like factory_girl</a>” is an example:</p>\n<blockquote>\n<p>So that’s what it really boils down to: the convenience of factories has set Rails testing strategies and software design back two years.</p>\n</blockquote>\n<p>Maybe a bit strongly worded. Anyway, the people who have created the Prime Stack also happen to be those who are vocal, blog a lot, and speak at a lot of conferences. Everybody loves a polemic. If I submitted a talk to RailsConf called “Why Fixtures are :metal:” this year, people would assume that I was being sarcastic. People always assume that things are slowly getting better: If the original advice was “use fixtures” and the new advice is “use Factories” and the <em>newest</em> advice is “use neither”… there’s no way that the “right answer” is “use fixtures,” right? Isn’t that so 2007?</p>\n<p>For those of us who’ve been around Rails for a long time, this discussion is good, and interesting. I was able to search for that Factory Girl post by memory, because I remember reading it when it was posted. I’ve written tons of Rails apps in the past 6ish years, and formed my own opinions on what tools I want to use due to that experience.</p>\n<p>But what about someone starting Rails today?</p>\n<h2>Beginners lose out</h2>\n<p>Someone told me earlier today that “Starting Rails today is like starting to watch a soap opera in the 7th season.” Like most witty statements, there’s both truth and falsehood here. But, as a newcomer, how do you pick between the Omakase Stack and the Prime Stack?</p>\n<p>This is the crux of the matter for me. If today, you tweeted that you were building a Rails app with fixtures, people would tell you to <a href=\"http://www.youtube.com/watch?v=n-Tej0297wk\">Stop It</a>. I saw it happen this morning. This is again, because a very vocal minority of Rails developers use the Prime Stack, and are really passionate about their choice over the Omakase Stack. That’s great.</p>\n<p>The problem is this: it puts a pretty heavy burden on a new developer: they have to learn the Omakase Stack <strong>and</strong> the delta. At the moment when someone is new, and barely knows anything, we throw them into making <em>more</em> choices. This is the moment when they’re least likely to be able to properly evaluate which option they need to choose. It’s not just “You’re new to Ruby, now learn Rails,” it’s now “You’re new to Ruby, now learn Rails, and RSpec, and Cucumber, and use this advanced modeling technique that we agree Rails discourages. Oh, and your only help is all these blog posts. Have fun!”</p>\n<p>As I said at the beginning of this post, Convention over Configuration is Rails’ greatest strength. So when a new person starts, the first thing we do is tell them to do a bunch of configuration. Any reasonable developer could easily ask “If these options really are so superior, why do I have to include them? And which do I choose? I’m drowning!”</p>\n<h2>Helping People Build Better Apps</h2>\n<p>We all want to write better, more maintainable apps. And many people (including me) are trying to push the state of the art forward in this regard. But we’re leaving the majority in the dust. Rather than trying to push the boundaries by choosing new techniques, maybe we should be helping people get better at existing ones, too. Because we need both approaches. There’s already so much to learn when building a Rails application, adding more gems and techniques to the pile just isn’t going to help.</p>\n<p>Once those beginners become intermediate, then they can figure out how new techniques solve the particular pains that they feel while going with the default flow. But skipping straight to the tool that solves my pain without feeling that pain the first place doesn’t help, it just confuses.</p>\n<p>Let’s keep pushing those boundaries, but let’s not <em>require</em> someone to catch up on a few hundred episodes before they can understand what’s going on.</p>\n<hr />\n<p>If you read this post previously, the “Omakase Stack” was called the “37signals stack,” but David has pointed out <a href=\"https://twitter.com/dhh/status/293108811219230720\">that even if he didn’t work with 37signals, he’d use this stack, so calling it that would be wrong</a>. I’ve since changed the name.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/2012-year-in-review/",
"title": "2012: year in review",
"description": null,
"url": "https://steveklabnik.com/writing/2012-year-in-review/",
"published": "2012-12-29T00:00:00.000Z",
"updated": "2012-12-29T00:00:00.000Z",
"content": "<p>I always enjoy reading <a href=\"http://www.kalzumeus.com/2012/12/29/bingo-card-creator-and-other-stuff-year-in-review-2012/\">patio11’s year in review posts</a>, and while I didn’t take enough notes for this year to do a GOOD one, I figure that I should at least leave some notes so that when I do one next year, I’ll have something to benchmark against.</p>\n<h2>Ruby</h2>\n<p>This has been a whirlwind year for my career. My work with Jumpstart Lab is incredibly fulfilling, and makes sure my bills get paid.</p>\n<ul>\n<li>I flew all over the world. I’ve always wanted to travel, and while I complain about it a lot, it’s truly wonderful to be able to go places. I now have the second highest Delta frequent flyer tier memebership. If you’d told me that three years ago, I’d have laughed at you.</li>\n<li>I spoke at a <em>lot</em> of conferences, and gave my first keynote. More lined up for next year already. Speaking is something I love to do.</li>\n<li>Teaching classes has been fantastic, and I’d love to do even more of it next year.</li>\n<li>I now have commit access to Rails. I put in a lot of work over the last few months, and this means a lot to me. Last step: core. We’ll see if I ever get there.</li>\n<li>I now have 22 gems, with Draper being my largest project, at almost 200,000 downloads.</li>\n</ul>\n<p>Unrelated to Ruby, I was able to contribute to a wide variety of projects in a few languages this year, and now have <a href=\"http://github.com/steveklabnik\">116 repos on GitHub</a>. A lot of those are abandoned or old, but still, they’re all something I was doing <em>something</em> with this year.</p>\n<h2>Writing</h2>\n<p>I finally made the <a href=\"http://designinghypermediaapis.com/\">hypermedia book</a> become a reality, and it’s done really well. Many people have told me that it has opened their eyes to a new way of doing things, and a bunch of startups have told me they’ll be releasing hypermedia APIs in 2013. Sales have been brisk: I don’t want to disclose exact numbers, but I will tell you that it’s in the 5 figure range. This has allowed me to start taking chunks out of my student loans: once they’re paid off, I will be super lean financially.</p>\n<p>I made it to the Svbtle blog network, which is invite-only. I’ve gotten mostly good feedback about everything I’ve done there, which is pleasing.</p>\n<p>I started <a href=\"http://www.rustforrubyists.com/\">a book to learn Rust programming</a> last week, as a way of helping myself learn Rust, making some more money, and testing out a publishing workflow that might influence the next iteration of Designing Hypermedia APIs. I like it a lot.</p>\n<h2>Reading</h2>\n<p>I read a lot, though not as much as I’d like. I want to read 30 books this next year, and finally got into the habit of using GoodReads.</p>\n<h2>Etc</h2>\n<p>I had a really good year in other regards: fulfilling relationships, fun times, good food… not everything was spectacular, but overall, really good. I moved out of my hometown, finally, and that’s been really great. I’m back now for the holidays, and it feels so small…</p>\n<p>The only really bad thing was the death of my father. That still has interesting effects on my life, but mostly in non-obvious ways. That’s all I’ll say about that.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/most-things-i-do-are-interconnected/",
"title": "Most things I do are interconnected",
"description": null,
"url": "https://steveklabnik.com/writing/most-things-i-do-are-interconnected/",
"published": "2012-12-28T00:00:00.000Z",
"updated": "2012-12-28T00:00:00.000Z",
"content": "<p>I work on a <em>lot</em> of different things. Sometimes I get asked <a href=\"http://words.steveklabnik.com/how-do-you-find-the-time\">how I find the time</a>, but there’s another aspect not related to time management: most of my projects build off of each other.</p>\n<h2>Draper and RequestStore</h2>\n<p><a href=\"https://github.com/drapergem/draper/issues/390\">This issue</a> got filed in Draper. What Draper does isn’t important, but it did have this code in it:</p>\n<pre><code>def self.current\n Thread.current[:current_view_context] ||= build_view_context\nend\n\ndef self.current=(context)\n Thread.current[:current_view_context] = context\nend\n</code></pre>\n<p>Basically, storing stuff in a <code>Thread</code> local variable. This is great, normally, except when you run one of those new hot servers like Thin or Puma. They have a different concurrency model, and so while you’d normally expect this <code>Thread</code> local to be <code>nil</code> on each request, it will persist between requests.</p>\n<p>I <em>could</em> have just fixed this bug in Draper, but I figured that it might be useful to someone else, so I whipped up a little gem to do it: <a href=\"https://rubygems.org/gems/request_store\">request_store</a>, and used the gem in Draper. I tweeted about it.</p>\n<blockquote>\n<p>@steveklabnik Holy shit. I love you.</p>\n</blockquote>\n<p>— Dominic Dagradi (@dddagradi) December 17, 2012</p>\n<blockquote></blockquote>\n<p>Turns out that one of my friends had <em>exactly</em> this problem, and wasn’t sure how to solve it. Now a bunch of people at Heroku are using my gem, and it’s helping them in their work.</p>\n<p>I wouldn’t have bothered building it if it didn’t scratch my itch, but by making it easy for other people to use, I helped them scratch their itch too.</p>\n<h2>Rust for Rubyists and Designing Hypermedia APIs</h2>\n<p>I’ve been working on a new iteration of <a href=\"http://designinghypermediaapis.com/\">Designing Hypermedia APIs</a> that is more linear and like a traditional book. The issue is that I’d built the entire platform for the project myself, and now spitting out ePUB and PDFs would be hard.</p>\n<p>Enter <a href=\"http://www.rustforrubyists.com/\">Rust for Rubyists</a>: I’d been learning Rust, and there are very few docs since it’s a new programming language. I had also seen Zed Shaw present at RuPy recently, and he showed off <a href=\"http://orkestrix.org/\">Orkestrix</a>, which let him generate a site and a book for music stuff.</p>\n<p>So I decided to write up what I’d learned as <a href=\"http://www.rustforrubyists.com/\">a tutorial on programming in Rust</a>, and built it using the Orkestrix gear. I also tried out <a href=\"http://getdpd.com/\">DPD</a> after seeing others use it. It’s only been up for two days, but I really like the combo, and today I pushed my first update out to the people who bought a copy already.</p>\n<p>This has given me the confidence that the tooling process is good enough to use for DHAs, so I’ll be doing all of that soon.</p>\n<h2>Rails and Resque</h2>\n<p>I started working on Resque because we needed to have a few production implementations of the new ActiveQueue system going before Rails 4 was released. I was already working on Rails to help get 4 out the door, and Terence was obviously swamped by all the things he works on. So helping out with Resque would help out with Rails.</p>\n<h2>bring_back_snowman and configuration</h2>\n<p>When I don’t know how to do something, I often look at a gem that does what I’m interested in doing, and copy how it does it. I had never built a gem that allowed you to configure it using the <code>config.something</code> pattern before, so I peeked at a few gems to learn how.</p>\n<p>I had also been thinking a lot about the Rails 4 release, and what was new since Rails 3. I was teaching a new batch of people Rails with <a href=\"http://jumpstartlab.com/\">Jumpstart Lab</a>, my employer, and was explaining why we have <code>form_for</code> generate a ✔. This gave me an idea: What if you could configure what Rails used in this case?</p>\n<p>So I built <a href=\"https://github.com/steveklabnik/bring_back_snowman\">bring_back_snowman</a>. It’s not very <em>useful</em>, but it is cute and fun, and more importantly, <a href=\"https://github.com/steveklabnik/bring_back_snowman/blob/master/lib/bring_back_snowman.rb#L5-L13\">it serves as a concise example of how to build a configurable option into your gems</a>. I can always look at that code if I forget how this works, or point others to it if they ask.</p>\n<h2>Many, many more</h2>\n<p>These are just the most recent examples I can think of. I’d really encourage you to try this sometime: if you want to learn something, make a new little project. Make it small. Make it focused. Make it help you out with something bigger.</p>\n<p>Pretty soon, you’ll feel mega-productive too.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/i-invented-hypermedia-apis-by-accident/",
"title": "I invented hypermedia APIs by accident",
"description": null,
"url": "https://steveklabnik.com/writing/i-invented-hypermedia-apis-by-accident/",
"published": "2012-12-21T00:00:00.000Z",
"updated": "2012-12-21T00:00:00.000Z",
"content": "<p>Long long ago, I got an internship in college. Back then, I didn’t know anything about web development. My college professor said “GET and POST are the same thing, it’s just that GET requests show parameters in the URL bar.” But a friend of mine was working at this post-acquisition startup. They still operated as an autonomous unit within the parent company, so they were still pretty fun to work for. People would upload us MP3s and we’d give them a podcast. They’d get a blog, the necessary RSS, we’d help get them listed in iTunes, all that jazz.</p>\n<p>Then the iPhone came out, and you could make apps for it.</p>\n<p>Bizdev decided that we should sell apps to our clients, who could then sell them to their audience and help finance the show. We’d do all the dev work, we’d all make money, it’d be a good time. So they asked me to write an iPhone app generator. We picked a big feature set, and people could give us some images, pick their individual features, and set some other config options. We’d take that file, and compile in them as defaults. All seemed good.</p>\n<p>Then the first app got rejected: it turns out that Apple has some restrictions on the bitrate of MP3s that you could download. I suspected that AT&T wasn’t very happy with you using lots of data back then. Anyway, we had to get around this issue: we wanted to serve up quality audio. What to do?</p>\n<p>I don’t remember who came up with it, but we decided that the first thing the app would do would be to fetch the config file from the server, to see if any of the defaults were changed. It was just XML, so the iPhone could parse it easily, and then change whatever was different. So we’d have something like this:</p>\n<pre><code><?xml version=\"1.0\" encoding=\"UTF-8\"?><config> <link href=\"http://example.com/low.rss\" rel=\"podcast_url\" /></config>\n</code></pre>\n<p>I don’t remember the exact schema, but you get the idea.</p>\n<p>Anyway, so we’d link to the low quality feed while the app was in review, and then later, we’d change the response to this:</p>\n<pre><code><?xml version=\"1.0\" encoding=\"UTF-8\"?><config> <link href=\"http://example.com/high.rss\" rel=\"podcast_url\" /></config>\n</code></pre>\n<p>Now the client would fetch the high quality feed. No client code needed to change! We’d managed to sneak around a restriction in the review process.</p>\n<p>But why stop here? Once we saw this flexibility, we started taking full advantage. People could set up a bunch of different configuration options, and that would change the UI based on their choices. So, for example, they could choose to be contacted by email, they could put in a website or two, and the page would change. Here’s a mockup of the web page:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24409213486218_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24409213486218_raw.png\" /></p>\n<p>This would cause our app to serve up the following XML:</p>\n<pre><code><?xml version=\"1.0\" encoding=\"UTF-8\"?><config> <link href=\"http://example.com/high.rss\" rel=\"podcast_url\" /> <about> <links> <link href=\"http://example.com\">Homepage</link> <link href=\"http://example.com/about\">About Us</link> </links> <phone>5558675309</phone> <email>[email protected]</email> </about></config>\n</code></pre>\n<p>And that would make this appear in the “about” section of the app:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24409223801964_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24409223801964_raw.png\" /></p>\n<p>Neat, eh? We’d turn the email into a <code>mailto:</code> link and the phone into a <code>tel:</code> one, as well.</p>\n<p>Anyway, maybe later they’d go back and unset their phone number, and the about us link. So then we’d generate this XML:</p>\n<pre><code><?xml version=\"1.0\" encoding=\"UTF-8\"?><config> <link href=\"http://example.com/high.rss\" rel=\"podcast_url\" /> <about> <links> <link href=\"http://example.com\">Homepage</link> </links> <email>[email protected]</email> </about></config>\n</code></pre>\n<p>But check out what the app would do:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24409247735934_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24409247735934_raw.png\" /></p>\n<p>Whoah! Isn’t that rad? The UI would change based on the config.</p>\n<p>Here’s what’s even rad-er: because the app would read the config on each load, changes would get propagated very quickly. If you took your email away, everyone’s apps would no longer have it, as if by magic. Next time they loaded up the app, it just wouldn’t be there any more.</p>\n<p>Here’s what’s <strong>even rad-er</strong>: You wouldn’t need to go through the App Store approval process to push this change out to the users. It Just Worked. If you’re an app developer, and you’ve been forced to sit through long waits to push out a release, you know how painful it can be. I don’t know how fast it is these days, but back then, it could take a month. With this approach, we could, say, remove a feature, and it’d be gone immediately. No oversight. If we added a new feature, older apps would still work, because they’d just ignore the new part of the config, and people who got the new version would get the newest features.</p>\n<p>Years later, after doing tons of research on Hypermedia APIs, I realized that that thing I did long ago was a form of it, and it provided us with a pretty massive benefit. So of course I wasn’t the first one to come up with it; my point is that I implemented a system in line with hypermedia principles because it made sense to do so, even though I had no idea that Roy Fielding was a PhD candidate. And I certainly had no way of articulating it as a ‘style,’ it was just a neat hack.</p>\n<hr />\n<p>If you liked this story, you may care to learn about what else I’ve learned along the way with Hypermedia APIs. I wrote an e-‘book’ about it called <a href=\"http://designinghypermediaapis.com/\">Designing Hypermedia APIs</a>. It’s only $20, and you get updates forever! I’m actively working on adding new content, and I’ll be doing a total redux in the new year.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/an-early-christmas-present-for-you/",
"title": "An early Christmas present for you",
"description": null,
"url": "https://steveklabnik.com/writing/an-early-christmas-present-for-you/",
"published": "2012-12-19T00:00:00.000Z",
"updated": "2012-12-19T00:00:00.000Z",
"content": "<p>Normally, I’m a bit of a grinch. I’ll admit it, I really hate Christmas. It reminds me of all of the worst parts of America: You know, how the baby Jesus commands us to <a href=\"http://www.ajc.com/news/news/crime-law/alleged-shoplifter-dies-after-being-subdued-by-wal/nTFPx/\">literally shop until we kill each other</a>…</p>\n<p>Anyway, today, I was teaching a bunch of brand new Rails devs, and I had to explain the mysterious hidden input that <code>form_for</code> generates:</p>\n<pre><code><input name=\"utf8\" type=\"hidden\" value=\"✔\">\n</code></pre>\n<p>That, combined with the impending release of Rails 4, made me think about the charming little Snowman that Rails originally had: ☃.</p>\n<h2>So?</h2>\n<p>And what happened, then? Well, in Whoville they say - that @steveklabnik’s small heart grew three sizes that day. So he decided to make you a present:</p>\n<pre><code>gem install bring_back_snowman\n</code></pre>\n<p>Just do it, and your forms will submit <code>_snowman=☃</code> to ensure that your data is encoded in UTF-8.</p>\n<p>It is configurable, if you prefer something else. Add this to your <code>application.rb</code>:</p>\n<pre><code>config.snowman = {:emoji => \"😢\"}\n</code></pre>\n<p>You may also need to add the ‘magic comment’ at the top of the file:</p>\n<pre><code># encoding: UTF-8\n</code></pre>\n<p>Please remember that if you don’t use something that’s UTF-8, this won’t actually work to fix the bug.</p>\n<p>You can find the <a href=\"https://github.com/steveklabnik/bring_back_snowman\">source of bring_back_snowman on GitHub</a>, of course.</p>\n<p>Merry Christmas!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/announcing-requeststore/",
"title": "Announcing request_store",
"description": null,
"url": "https://steveklabnik.com/writing/announcing-requeststore/",
"published": "2012-12-17T00:00:00.000Z",
"updated": "2012-12-17T00:00:00.000Z",
"content": "<p>Last night I had some insomnia, so I wrote a gem.</p>\n<p>Here it is: <a href=\"https://github.com/steveklabnik/request_store\">https://github.com/steveklabnik/request_store</a>.</p>\n<p>TL;DR:</p>\n<p>If you’re using <code>Thread.current</code> in your Rails app to store global-ish data, don’t do it! If you use Thin or Puma or a threaded web server, it won’t get reset between requests, and you’ll end up with subtle bugs. So do this instead:</p>\n<pre><code>gem \"request_store\"\n</code></pre>\n<p>and replace</p>\n<pre><code>Thread.current[:foo] = 1\n</code></pre>\n<p>with</p>\n<pre><code>RequestStore.store[:foo] = 1\n</code></pre>\n<p>And things will be peachy keen.</p>\n<h3>No Rails?</h3>\n<p>If you’re not using Rails, <code>use RequestStore::Middleware</code> to make it work. If you are using Rails, a Railtie takes care of it for you.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/getting-started-with-chruby/",
"title": "Getting started with chruby",
"description": null,
"url": "https://steveklabnik.com/writing/getting-started-with-chruby/",
"published": "2012-12-13T00:00:00.000Z",
"updated": "2012-12-13T00:00:00.000Z",
"content": "<p>If you’re looking for crazy simplicity in your ‘switch between multiple Rubies’ life, you may want to check out <a href=\"https://github.com/postmodern/chruby\">chruby</a>. Written by Postmodern, it’s basically the simplest possible thing that can work. As in, <a href=\"https://github.com/postmodern/chruby/blob/master/share/chruby/chruby.sh\">76 lines of shell script</a>.</p>\n<p>For that, you get:</p>\n<h3>Features</h3>\n<ul>\n<li>Updates <code>$PATH</code>.\n<ul>\n<li>Also adds RubyGems <code>bin/</code> directories to <code>$PATH</code>.</li>\n</ul>\n</li>\n<li>Correctly sets <code>$GEM_HOME</code> and <code>$GEM_PATH</code>.\n<ul>\n<li>Users: gems are installed into <code>~/.gem/$ruby/$version</code>.</li>\n<li>Root: gems are installed directly into <code>/path/to/$ruby/$gemdir</code>.</li>\n</ul>\n</li>\n<li>Additionally sets <code>$RUBY</code>, <code>$RUBY_ENGINE</code>, <code>$RUBY_VERSION</code> and <code>$GEM_ROOT</code>.</li>\n<li>Optionally sets <code>$RUBYOPT</code> if second argument is given.</li>\n<li>Calls <code>hash -r</code> to clear the command-lookup hash-table.</li>\n<li>Fuzzy matching of Rubies by name.</li>\n<li>Defaults to the system Ruby.</li>\n<li>Supports [bash] and [zsh].</li>\n<li>Small (~80 LOC).</li>\n<li>Has tests.</li>\n</ul>\n<h3>Anti-Features</h3>\n<ul>\n<li>Does not hook <code>cd</code>.</li>\n<li>Does not install executable shims.</li>\n<li>Does not require Rubies be installed into your home directory.</li>\n<li>Does not automatically switch Rubies upon login or when changing directories.</li>\n<li>Does not require write-access to the Ruby directory in order to install gems.</li>\n</ul>\n<p>Kinda crazy, eh?</p>\n<h2>Installing</h2>\n<p>Most of the time, I install things from <code>homebrew</code>, but I actually prefered to run the <code>setup.sh</code> script:</p>\n<pre><code>wget https://github.com/downloads/postmodern/chruby/chruby-0.2.3.tar.gz\ntar -xzvf chruby-0.2.3.tar.gz\ncd chruby-0.2.3/\n./scripts/setup.sh\n</code></pre>\n<p>You can see the source for that script <a href=\"https://github.com/postmodern/chruby/blob/master/scripts/setup.sh\">here</a>. You’ll end up with an MRI, JRuby, and Rubinius all installed in <code>/opt/rubies</code>, and after restarting your shell, you should be good to go!</p>\n<h2>Usage</h2>\n<p>To see what Rubies you’ve got, just <code>chruby</code>:</p>\n<pre><code>$ chruby\n ruby-1.9.3-p327\n jruby-1.7.0\n rubinius-2.0.0-rc1\n</code></pre>\n<p>to pick one:</p>\n<pre><code>$ chruby 1.9.3\n</code></pre>\n<p>Pretty damn simple.</p>\n<h2>Getting more Rubies?</h2>\n<p>The setup script will install <a href=\"https://github.com/sstephenson/ruby-build#readme\">ruby-build</a>, so just use that to get more rubies:</p>\n<pre><code>ruby-build 1.9.3-p327 /opt/rubies/ruby-1.9.3-p327\n</code></pre>\n<p>You’ll then need to update your list of Rubies in your <code>.bashrc</code> or <code>.zshrc</code>:</p>\n<pre><code>RUBIES=(\n # other rubies here\n /opt/rubies/ruby-1.9.3-p327\n)\n</code></pre>\n<p>If you find this tedius, you can just glob them:</p>\n<pre><code>RUBIES=(/opt/rubies/*)\n</code></pre>\n<h2>Automatic switching?</h2>\n<p><a href=\"https://github.com/postmodern/chruby/issues/40\">Not yet</a>. This is pretty much the only thing keeping me from saying ‘zomg use this for sure now.’</p>\n<h2>How good is it?</h2>\n<p>I don’t know, I’ve been using it for less than 24 hours. Seems good.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/deleuze-for-developers-deterritorialization/",
"title": "Deleuze for developers: deterritorialization",
"description": null,
"url": "https://steveklabnik.com/writing/deleuze-for-developers-deterritorialization/",
"published": "2012-12-07T00:00:00.000Z",
"updated": "2012-12-07T00:00:00.000Z",
"content": "<p>If you truly want to understand technology today, then you should at least be familiar with the philosophy of Gilles Deleuze. Unfortunately for technologists, Deleuze is rooted firmly in a philosophical tradition and a writing style that they probably find opaque. In this blog series, I plan on explaining Deleuze’s philosophy in terms that programmers can understand. This is the second in the series. You can find the first <a href=\"/deleuze-for-developers-assemblages\">here</a>, and the next one <a href=\"http://words.steveklabnik.com/deleuze-for-developers-will-smooth-spaceopen-source-suffice-to-save-us\">here</a>. Enjoy.</p>\n<hr />\n<p>Let’s re-examine this diagram of the assemblage:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388447125816_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388447125816_raw.png\" /></p>\n<p>What’s up with this line?</p>\n<p>Well, just because assemblages are amorphous doesn’t mean that there’s no way to demarcate what’s in the assemblage and what is not. This particular assemblage has a ‘territory,’ of sorts: everything that’s inside the line is part of the territory, and everything that’s not is outside. Think of a fort, or a castle:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388459420680_raw.jpg\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388459420680_raw.jpg\" /></p>\n<p>Or, since we’re not just talking about physical spaces, a social circle or group:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388506430020_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388506430020_raw.png\" /></p>\n<p>Of course, this would extend around all the members of Rails Core, hence it being cut off at the bottom. But the pink line would designate the boundaries of the assemblage that makes up Rails Core.</p>\n<h2>Deterritorialization</h2>\n<p>So what happens when these boundaries get crossed? Well, your castle gets invaded. A new member joins the team. New servers are brought online. This process is called ‘deterritorialization.’ Healing the lines, repairing them, and re-containing the boundary is called ‘reterritorialization.’ I recently came across a really interesting symbol of deterritorialization: the Open Source logo:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388521022692_raw.gif\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388521022692_raw.gif\" /></p>\n<p>Check it out: visually, this logo communicates the deterritorializing effect of opening up your source code: the private internals have now been exposed internally. The walls have been breeched!</p>\n<p>Another example, from building a web service: we start off with just our service:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388526532744_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388526532744_raw.png\" /></p>\n<p>We have here an application server, a database, and our user’s identity. They form the assemblage of our system. Remember, abstract concepts are objects just as code and servers are! Our identity notion is stored within the database, therefore, they’re connected.</p>\n<p>Next, we decide to implement a ‘log in with Twitter’ feature.</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388533077814_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388533077814_raw.png\" /></p>\n<p>It’s the primary way that our users sign up and use our site. Now, Twitter’s assemblage has deterritorialized ours:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388535026314_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388535026314_raw.png\" /></p>\n<p>We can minimize the effects (or reterritorialize our service) by making sure to have our own concept of identity within the system, and making sure that we have our own notion of identity within the system, and just connecting Twitter’s notion of identity to our own:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388627455810_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388627455810_raw.png\" /></p>\n<p>Now, by containing the Twitter-assemblage entirely within our service, I don’t mean that it actually is. Obviously, the Twitter-assemblage is interconnected with a ton of other assemblages that represent other services. But from our perspective, they are now a part of our assemblage. The decisions they make affect us. While our code is separated, we’re not totally separate anymore: updates and policies of Twitter have a direct effect on us.</p>\n<p>There’s also a more sublte, secondary re/de-territorialization going on here: our code from our service. These used to be isomorphic, but now our code has claimed a territory of its own, and is now just one assemblage within our system-assemblage, instead of <em>being</em> our system-assemblage.</p>\n<h2>A git example</h2>\n<p>The previous notion of deterritorialization largely relied on the notion of dependencies as the mechanism which we drew our diagrams. But it doesn’t have to be that way. Let’s take another example: git.</p>\n<p>Every GitHub pull request is an act of deterritorialization, and every merge is one of re-territorialization. Consider this small repository, with three commits:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388550633502_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388550633502_raw.png\" /></p>\n<p>You do a git clone:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388551164862_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388551164862_raw.png\" /></p>\n<p>Make a new commit, adding a new object into your repo-assemblage:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388553101194_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388553101194_raw.png\" /></p>\n<p>Then you send me an email, asking me to pull from your repository:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388554578454_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388554578454_raw.png\" /></p>\n<p>You like the change, so you do a <code>git fetch</code>:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388557943410_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388557943410_raw.png\" /></p>\n<p>de-territorialized!</p>\n<p>Now it makes the local copy:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388560627354_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388560627354_raw.png\" /></p>\n<p>and your repository has been re-territorialized. These steps happen so quickly that you probably don’t even think about them, but conceptually, this is what’s happening.</p>\n<h2>A social example</h2>\n<p>One last example that’s even less related to code: entering a new social group. You’re at a conference, and there are four people standing around talking:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388564936014_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388564936014_raw.png\" /></p>\n<p>They form an assemblage we call a ‘conversation.’ It’s in relation with an object known as “refinements” from Ruby 2.0, which means this assemblage is likely to be particularly vibrant. Heh. Anyway, you decide that your opinion matters, so you de-territorialize the assemblage, and (assuming they don’t ignore you) it re-territorializes around you:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24388566884064_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24388566884064_raw.png\" /></p>\n<p>Even the language we use here implies something similar: you ‘enter the conversation.’ Like entering a door, into a space you previously were not.</p>\n<h2>Conclusion: Diagrams</h2>\n<p>We can call these drawings ‘diagrams’ or ‘abstract machines.’ They can represent these kinds of conceptual relationships in a visual way, which I find really useful for understanding. Programmers call abstract machines ‘design patterns.’</p>\n<p>Now that you’ve seen this process that assemblages use to relate to one another, I hope that you find this particular diagram useful. Because of its level of abstraction, it’s applicable in a wide variety of situations.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/deleuze-for-developers-assemblages/",
"title": "Deleuze for developers: assemblages",
"description": null,
"url": "https://steveklabnik.com/writing/deleuze-for-developers-assemblages/",
"published": "2012-12-03T00:00:00.000Z",
"updated": "2012-12-03T00:00:00.000Z",
"content": "<p>If you truly want to understand technology today, then you should at least be familiar with the philosophy of Gilles Deleuze. Unfortunately for technologists, Deleuze is rooted firmly in a philosophical tradition and a writing style that they probably find opaque. In this blog series, I plan on explaining Deleuze’s philosophy in terms that programmers can understand. This is the first in the series.</p>\n<p>Enjoy.</p>\n<hr />\n<p>The ancient Greeks thought that atoms were it. ἄτομος, Wikipedia will tell you, comes from ‘ἀ’- meaning “not” and ‘τέμνω’, meaning ‘I cut.’ Cells are the smallest thing that there is. You can’t cut them in half. Everything is composed of a bunch of atoms.</p>\n<p>Well, guess what? Modern science has found things smaller than atoms. Dammit! What we <em>do</em> know for sure, though, is that some things are made up of composites of other things. That’s for sure. We’re still not clear on what that fundamental stuff is, but composites are a pretty safe bet. Even if atoms are made up of other things, we know that other things are made up of a bunch of atoms.</p>\n<p>So how can we talk about composites? Well, a programmer’s fundamental tool is abstraction. So we can black-box any particular composite and treat it as a whole entity, without worrying about what we’ve encapsulated inside. That’s good, and that’s useful, but there’s a problem: when you start thinking of a collection of things as just one thing, you miss out on a lot of important details. As we know, all abstractions are leaky, so how does a black box leak?</p>\n<p>Let’s think about your body: it’s a composite, you’re made up of (among other things) a whole pile of organs, and one (skin) wrapping the rest of them up in a nice black box. If we had lived a few hundred years ago, thinking of the body as a black box would be a pretty fair metaphor: after all, if you take out someone’s heart, they die. The box needs all of its parts inside. Today, though, that’s not true: we can do heart transplants. Further, we have artificial hearts. A person with an artificial heart is not the exact same as a person without one, yet they are both still people. Thinking of a body as a totality leaks as an abstraction in modern times.</p>\n<p>Let’s move away from biology for a bit and into web applications instead. Here’s an architecture diagram of a typical web service:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24382217466576_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24382217466576_raw.png\" /></p>\n<p>Looks pretty normal, right? Let’s modify it a bit:</p>\n<p><img src=\"https://svbtleusercontent.com/inline_steveklabnik_24382219810338_raw.png\" alt=\"https://svbtleusercontent.com/inline_steveklabnik_24382219810338_raw.png\" /></p>\n<p>This is the same web service, right? We’ve introduced a few database slaves and a reverse proxy, which allowed us to knock out one of our application servers. In a certain sense, this service is equivalent; in another, it’s different: It’s got another extra component, and one component has developed more complexity. We might need to hire a DBA and someone who’s good at writing Varnish configs. Different skills are needed.</p>\n<p>So what do we call this thing? An ‘assemblage.’ It’s made up of smaller assemblages as well; after all, our application servers may use the Reactor pattern, the worker model of Unicorn, or the threaded model of the JVM. To quote Levi Bryant, “Assemblages are composed of heterogeneous elements or objects that enter into relations with one another.” In the body metaphor, these are organs, and in the systems diagram, they’re the application servers, databases, and job queues. Assemblages are also ‘objects’ in this context, so you can think of any of those objects as being made up of other ones. But to think of the formations of objects as a total whole or black box is incorrect, or at best, a leaky abstraction.</p>\n<p>Let’s get at one other excellent example of an assemblage: the internet. Computers connect, disconnect, reconnect. They may be here one moment, gone the next, or stay for an extended period of time. The internet isn’t any less an internet when you close your laptop and remove it, nor when you add a new server onto your rack. Yet we still talk about it as though it were a whole. It’s not, exactly: it’s an assemblage.</p>\n<p>Furthermore, there’s a reason that Bryant uses the generic ‘objects’ or ‘elements’ when talking about assemblages: objects can be ‘people’ or ‘ideas’ as well as physical parts. This is where the ‘systems’ idea of computer science diverges from ‘assemblage’: depending on how you want to analyze things, your users may be part of an assemblage diagram as well.</p>\n<p>So what’s the point? Well, this is a basic term that can be used to think about problems. A design pattern of thought. A diagram. In future posts, I’ll actually use this pattern to address a particular problem, but you need to understand the pattern before you can understand how it’s used.</p>\n<p>(and to give you a comparison of complexity of explanation, here’s the blog post by Levi Bryant on assemblages where I got that quote from. He quotes Deleuze directly and extensively: http://larvalsubjects.wordpress.com/2009/10/08/deleuze-on-assemblages/ )</p>\n<hr />\n<p>The next article in the series is on <a href=\"/deleuze-for-developers-deterritorialization\">‘deterritorialization’</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/draper-1-0-0-beta2-release/",
"title": "Draper 1.0.0.beta2 release",
"description": null,
"url": "https://steveklabnik.com/writing/draper-1-0-0-beta2-release/",
"published": "2012-12-03T00:00:00.000Z",
"updated": "2012-12-03T00:00:00.000Z",
"content": "<p>I’ve relased the second beta for Draper 1.0.0! Many thanks to all of you who gave the first beta a shot. We’ve fixed some issues, and I’d apprecaite you giving this one a try. This solves all known issues with Draper, including every one we’ve found in 1.0.0.beta1.</p>\n<p>You can get it by installing it from Rubygems:</p>\n<pre><code>$ gem install draper --pre\n</code></pre>\n<p>or by putting it in your Gemfile:</p>\n<pre><code>gem \"draper\", \"1.0.0.beta2\"\n</code></pre>\n<h2>CHANGELOG</h2>\n<p>These are the changes since beta1. To find out what’s new in beta1, please <a href=\"/posts/2012-11-30-draper-1-0-0-beta1-release\">see my previous post</a>.</p>\n<ul>\n<li><code>has_finders</code> is now <code>decorates_finders</code>. <a href=\"https://github.com/haines/draper/commit/33f18aa062e0d3848443dbd81047f20d5665579f\">https://github.com/haines/draper/commit/33f18aa062e0d3848443dbd81047f20d5665579f</a></li>\n<li>If a finder method is used, and the source class is not set and cannot be inferred, an <code>UninferrableSourceError</code> is raised. <a href=\"https://github.com/haines/draper/commit/8ef5bf2f02f7033e3cd4f1f5de7397b02c984fe3\">https://github.com/haines/draper/commit/8ef5bf2f02f7033e3cd4f1f5de7397b02c984fe3</a></li>\n<li>Class methods are now properly delegated again. <a href=\"https://github.com/haines/draper/commit/731995a5feac4cd06cf9328d2892c0eca9992db6\">https://github.com/haines/draper/commit/731995a5feac4cd06cf9328d2892c0eca9992db6</a></li>\n<li>We no longer <code>respond_to?</code> private methods on the source. <a href=\"https://github.com/haines/draper/commit/18ebac81533a6413aa20a3c26f23e91d0b12b031\">https://github.com/haines/draper/commit/18ebac81533a6413aa20a3c26f23e91d0b12b031</a></li>\n<li>Rails versioning relaxed to support Rails 4 <a href=\"https://github.com/drapergem/draper/commit/8bfd393b5baa7aa1488076a5e2cb88648efaa815\">https://github.com/drapergem/draper/commit/8bfd393b5baa7aa1488076a5e2cb88648efaa815</a></li>\n</ul>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/draper-1-0-0-beta1-release/",
"title": "Draper 1.0.0.beta1 release",
"description": null,
"url": "https://steveklabnik.com/writing/draper-1-0-0-beta1-release/",
"published": "2012-11-30T00:00:00.000Z",
"updated": "2012-11-30T00:00:00.000Z",
"content": "<p>I’m happy to announce the release of Draper 1.0.0.beta1 today. If you use Draper, I’d appreciate you checking out the beta release in your app, kicking the tires, and letting me know what’s up.</p>\n<p>You can get it by installing it from Rubygems:</p>\n<pre><code>$ gem install draper --pre\n</code></pre>\n<p>or by putting it in your Gemfile:</p>\n<pre><code>gem \"draper\", \"1.0.0.beta1\"\n</code></pre>\n<p>Notable changes include:</p>\n<ul>\n<li>Renaming <code>Draper::Base</code> to <code>Draper::Decorator</code>. This is the most significant change you’ll need to upgrade your application. <a href=\"https://github.com/drapergem/draper/commit/025742cb3b295d259cf0ecf3669c24817d6f2df1\">https://github.com/drapergem/draper/commit/025742cb3b295d259cf0ecf3669c24817d6f2df1</a></li>\n<li>Added an internal Rails application for integration tests. This won’t affect your application, but we’re now running a set of Cucumber tests inside of a Rails app in both development and production mode to help ensure that we don’t make changes that break Draper. <a href=\"https://github.com/drapergem/draper/commit/90a4859085cab158658d23d77cd3108b6037e36f\">https://github.com/drapergem/draper/commit/90a4859085cab158658d23d77cd3108b6037e36f</a></li>\n<li>Add <code>#decorated?</code> method. This gives us a free RSpec matcher, <code>is_decorated?</code>. <a href=\"https://github.com/drapergem/draper/commit/834a6fd1f24b5646c333a04a99fe9846a58965d6\">https://github.com/drapergem/draper/commit/834a6fd1f24b5646c333a04a99fe9846a58965d6</a></li>\n<li><code>#decorates</code> is no longer needed inside your models, and should be removed. Decorators automatically infer the class they decorate. <a href=\"https://github.com/drapergem/draper/commit/e1214d97b62f2cab45227cc650029734160dcdfe\">https://github.com/drapergem/draper/commit/e1214d97b62f2cab45227cc650029734160dcdfe</a></li>\n<li>Decorators do not automatically come with ‘finders’ by default. If you’d like to use <code>SomeDecorator.find(1)</code>, for example, simply add <code>#has_finders</code> to the decorator to include them. <a href=\"https://github.com/drapergem/draper/commit/42b6f78fda4f51845dab4d35da68880f1989d178\">https://github.com/drapergem/draper/commit/42b6f78fda4f51845dab4d35da68880f1989d178</a></li>\n<li>To refer to the object being decorated, <code>#source</code> is now the preferred method. <a href=\"https://github.com/drapergem/draper/commit/1e84fcb4a0eab0d12f5feda6886ce1caa239cb16\">https://github.com/drapergem/draper/commit/1e84fcb4a0eab0d12f5feda6886ce1caa239cb16</a></li>\n<li><code>ActiveModel::Serialization</code> is included in Decorators if you’ve requred <code>ActiveModel::Serializers</code>, so that decorators can be serialized. <a href=\"https://github.com/drapergem/draper/commit/c4b352799067506849abcbf14963ea36abda301c\">https://github.com/drapergem/draper/commit/c4b352799067506849abcbf14963ea36abda301c</a></li>\n<li>Properly support Test::Unit <a href=\"https://github.com/drapergem/draper/commit/087e134ed0885ec11325ffabe8ab2bebef77a33a\">https://github.com/drapergem/draper/commit/087e134ed0885ec11325ffabe8ab2bebef77a33a</a></li>\n</ul>\n<p>And many small bug fixes and refactorings.</p>\n<p>Before the actual release of 1.0.0, I want to improve documentation and handle a few other things, but we currently have no confirmed and one possible bug in Draper as it stands, so your feedback as I clean up these last few things would be excellent. Please file issues on the tracker if you find anything.</p>\n<p>Thank you! <3 <3 <3</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/introducing-the-rails-api-project/",
"title": "Introducing the rails-api project",
"description": null,
"url": "https://steveklabnik.com/writing/introducing-the-rails-api-project/",
"published": "2012-11-22T00:00:00.000Z",
"updated": "2012-11-22T00:00:00.000Z",
"content": "<p>Ruby on Rails is a great tool to build websites incredibly quickly and easily. But what about applications that aren’t websites? Rails is still a first-class choice for this use-case, but why settle for good when you could be the best?</p>\n<p>That’s why I’m happy to introduce Rails API: a set of tools to build excellent APIs for both heavy Javascript applications as well as non-Web API clients.</p>\n<h2>What</h2>\n<p>Rails’ greatest feature is making trivial choices for you. By following conventions, you’re able to avoid bikeshedding and only focus on the actual decisions that matter for your application, while automatically building in best-practices. Rails API will do this for the API use-case.</p>\n<p>Here are some initial thoughts on what Rails API will do for you:</p>\n<h3>A simpler stack</h3>\n<p>First, we can remove many parts of Rails that aren’t important in an API context: many middleware don’t make sense, and all of ActionView can go away.</p>\n<p>To check out exactly what we mean, look at <a href=\"https://github.com/rails-api/rails-api#choosing-middlewares\">this part</a> of the README, which explains which middlewares we include. The default stack has 22, we have 14.</p>\n<p>Similarly, the structure of ActionController::Base is interesting: it includes a ton of modules that implement various features. This means that we can build an alternate ‘controller stack’ that doesn’t include all of the ones that are in Base. Check out <a href=\"https://github.com/rails-api/rails-api#choosing-controller-modules\">this portion</a> of the README for a demonstration of the modules we do include. Base includes 29 modules, we include 11.</p>\n<h3>Consistent Javascript output</h3>\n<p>One of the biggest issues when building a distributed system is messages that are sent between components. In most Rails-based APIs, this means the JSON that your server side emits, and the JSON that your client consumes. By forming best practices around what JSON is sent around, we can simplify things for server-side and client-side developers.</p>\n<p>The generation of this JSON should be as transparent and simple as possible on both sides. <a href=\"https://github.com/rails-api/active_model_serializers\">ActiveModel::Serializers</a> is key here. Many new APIs are settling on HAL as a JSON variant that is important to them, and so we’ll be considering it heavily.</p>\n<h3>Authentication</h3>\n<p>APIs don’t need to handle cookie-based auth, and so a lot of that infrastructure isn’t important. Handling other kinds of auth in a simple way is incredibly important.</p>\n<h3>JavaScript as a first-class citizen</h3>\n<p>JavaScript is just as important as Ruby for these kinds of applications, and should be equally as important with respect to tooling, directory layout, and documentation.</p>\n<p>This also means that it should be easy to use the various Javascript frameworks and testing tools. We have representatives from the Ember team on-board, but using other frameworks should be equally fantastic.</p>\n<h3>Hypermedia</h3>\n<p>Many people are interested in building Hypermedia APIs on top of Rails, and so building best-practices around this style should be taken care of, and building them should be easy.</p>\n<h2>Where</h2>\n<p>You can find the organization on GitHub at <a href=\"https://github.com/rails-api\">https://github.com/rails-api</a></p>\n<p>Originally, these two repositories were located at <a href=\"https://github.com/spastorino/rails-api\">https://github.com/spastorino/rails-api</a> and <a href=\"https://github.com/josevalim/active_model_serializers\">https://github.com/josevalim/active_model_serializers</a>.</p>\n<p>We have a Google Group for discussion here: <a href=\"https://groups.google.com/forum/?fromgroups#!forum/rails-api-core\">https://groups.google.com/forum/?fromgroups#!forum/rails-api-core</a></p>\n<h2>Getting Started</h2>\n<p>To get started, check out the README here: <a href=\"https://github.com/rails-api/rails-api#railsapi\">https://github.com/rails-api/rails-api#railsapi</a>.</p>\n<h2>Who</h2>\n<p>Currently,</p>\n<ul>\n<li>Carlos Antonio</li>\n<li>Rafael França</li>\n<li>Santiago Pastorino</li>\n<li>Steve Klabnik</li>\n<li>Yehuda Katz</li>\n<li>Tom Dale</li>\n<li>José Valim</li>\n</ul>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/narratives/",
"title": "Narratives",
"description": null,
"url": "https://steveklabnik.com/writing/narratives/",
"published": "2012-11-18T00:00:00.000Z",
"updated": "2012-11-18T00:00:00.000Z",
"content": "<p>Occasionally in this big wide world, things happen. When things happen, we like to tell each other about it. When you take a bunch of connected events and tell someone about it, you’ve formed a <em>narrative</em>. Narratives are important, because they tell the story! The details of the account that one presents can really shape how others view those events later.</p>\n<p>Due to our lived experiences, we often have our own ‘internal narratives.’ You can think of them as pre-packaged ways that we tend to think about a given series of events. Human brains are wonderful pattern-matching machines, and so we see similarities in situations, and then apply the same narrative.</p>\n<p>Recently, I was playing around with Unicode and Ruby, and was trying to make some funny class names. I ended up figuring some stuff out, and tweeted this, which at the time of this writing, it has 157 retweets and 48 favorites:</p>\n<blockquote>\n<p>This is valid Ruby:class ಠ_ಠendಠ_ಠ.new</p>\n</blockquote>\n<p>— Steve Klabnik (@steveklabnik) November 17, 2012</p>\n<blockquote></blockquote>\n<p>This tweet is factually incorrect. Yet, only two people pointed this out to me. Why?</p>\n<p>Ruby is kind of a crazy language. That’s what I love about it; it’s got lots of wacky little corners, nooks, and crannies. I’ve also been doing Ruby for a long time, and tweet about it often. So here’s the internal narrative when you see this tweet:</p>\n<ol>\n<li>Steve knows a lot about Ruby.</li>\n<li>Ruby is kinda weird.</li>\n<li>That’s really funny.</li>\n</ol>\n<p>This particular situation fits perfectly into this internal narrative. Hence, a retweet. No checking. Just tweeting.</p>\n<p>These narratives can play off of each other really easily. Now that we’ve seen one more way in which Ruby is kinda crazy, we assume that it’s even more crazy in the future. This tweet has reinforced our internal narrative about Ruby’s wackiness.</p>\n<hr />\n<p>So #britruby.</p>\n<p>I’m sure you’re expecting me to say something specific about this situation, but I’m not going to give you what you want. What I <em>do</em> want to do is analyze the narrative of this situation, and see how it compares to actual events.</p>\n<p>Josh Susser tweets this:</p>\n<blockquote>\n<p>Nice speaker lineup for @britruby. Except for the 100% white guys part.</p>\n</blockquote>\n<p>— Josh Susser (@joshsusser) November 17, 2012</p>\n<blockquote></blockquote>\n<p>and James Rosen tweets this:</p>\n<blockquote>\n<p>By what metric is @britruby “one of Europe’s… most diverse Ruby conferences”? Every speaker is a white man.</p>\n</blockquote>\n<p>— James A Rosen (@jamesarosen) November 17, 2012</p>\n<blockquote></blockquote>\n<p>That’s it. If you examine those Twitter pages, you can see some conversation; both were conducted pretty cordially. Then, BritRuby decides to cancel their conference.</p>\n<p>Okay. That’s the situation. Now let’s look at two versions of the narrative. First up, <a href=\"http://2013.britruby.com/\">the cancellation announcement</a>. I’m going to excerpt the most important parts, with my commentary afterwards.</p>\n<blockquote>\n<p>We started this conference to build a community within the United Kingdom. Our mission statement was to encourage Ruby developers to unite and create a community, which would allow such to network, exchange ideas and provoke innovation for the future of Ruby. We wanted to encourage jobs and allow companies the opportunity to meet the community and primarily boost the UK developer industry.</p>\n</blockquote>\n<p>We wanted to build a great conference that would help UK Rubyists tremendously.</p>\n<blockquote>\n<p>Our selection process was the content and nothing more. Not the individuals gender, race, age or nationality.</p>\n</blockquote>\n<p>This is trivially disproven, as everyone has biases. But the message is “We only accepted the best speakers, regardless of who they are.”</p>\n<blockquote>\n<p>It’s about community. It’s about helping one another to strive for success and drive budding developers to do the same. We contacted universities in a hope that we would be able to empower young minds and show them what a great language Ruby really is. They are the future.</p>\n</blockquote>\n<p>We are team players.</p>\n<blockquote>\n<p>The Ruby community has been battling with issues of race and gender equality. We at Brit Ruby were well aware of this fundamental and important issue. This was one of the reasons why we encouraged everyone to submit a speaker proposal. Sadly, BritRuby was used as the arena to air these issues on Twitter and this has fundamentally destroyed any chance we had of addressing these issues.</p>\n</blockquote>\n<p>BritRuby was ‘used to air these issues,’ meaning, ‘others had an agenda and took advantage of us to promote it.’</p>\n<blockquote>\n<p>Instead the community should have worked together and allowed us to bring these issues to light at the conference.</p>\n</blockquote>\n<p>These individuals are divisive, and we are inclusive.</p>\n<blockquote>\n<p>How can the community address these issues if every time someone tries they are shot down and accused of such awful things?</p>\n</blockquote>\n<blockquote>\n<p>The British Ruby team although very small consisted of a diverse mix of nationality and gender.</p>\n</blockquote>\n<p>We cannot be tainted by accusations of monoculture, as we are diverse ourselves.</p>\n<blockquote>\n<p>This was a non-profit conference being run by simple developers.</p>\n</blockquote>\n<p>We were not even trying to make money.</p>\n<blockquote>\n<p>The team has been working so hard in their own time to bring a unique conference to you all. Thank you to those of you that dedicated your time, skills and to their families for putting up with the long hours.</p>\n</blockquote>\n<p>We, nay, you and us together all worked really hard, so you should cut us some slack.</p>\n<p>Anyway, so there you go. Same events, different story: We are a collective that was trying to do our best. Outsiders took advantage of us to promote their agenda and divide all of us. Therefore, we’re taking our ball and going home.</p>\n<p>Here’s the funniest thing about the cancellation: They don’t give an actual reason. The closest they get is ‘was used as an arena to air these issues’ but they don’t explain why that made them cancel their conference. This narrative attempts to shift the blame from the organizers to the critics. “We wanted to give you this beautiful thing, and we worked hard, but these other people ruined it.”</p>\n<p>This particular narrative plays off of a well-known trope when discussing diversity issues: a persecution complex by the majority. Majorities fear losing their power, and often will develop internal narratives that <em>they’re</em> the ones being persecuted. Some good examples of this are the Mens Rights movement and the White Power movement. This is a milder form of that same privilege: the power of a single tweet by the ‘politically correct’ is enough to bring the most powerful of plans grinding to a halt.</p>\n<hr />\n<p>Let’s examine a few narratives involved in <a href=\"https://gist.github.com/4106776\">this gist</a>:</p>\n<blockquote>\n<p>And here it is, brought down by careless words.</p>\n</blockquote>\n<p>“These people don’t actually care about us or our conference.”</p>\n<blockquote>\n<p>Yes, gender equality and racial equality are important. But the team’s motives were to get the best speakers who were able to make it to Manchester.</p>\n</blockquote>\n<p>First of all, “but” is an amazing word. You can’t even imagine the things I’ve heard said after “I’m not racist, but.” This sentence implies two things: an absolution from blame on the part of the conference because hey, they care about equality. Their second statement implies that the best speakers all happened to be white men. The converse of this is that non-whites and/or women are not the best speakers, and the overall message is that including them would degrade the quality of the conference.</p>\n<blockquote>\n<p>Adding a token minority speaker is offensive to that speaker, it says “You’re here because you tick a box - not because you’re skilled.”</p>\n</blockquote>\n<p>Anyone who seeks out women or minorities is only doing it to ‘tick boxes,’ because, as we’ve already established, they clearly are not the best speakers.</p>\n<p>Oh, and you’re actually the sexist/racist one if you try to encourage them, because it’s patronizing.</p>\n<blockquote>\n<p>It doesn’t matter who speaks at a conference, as long as they’re capable, interesting and relevant. That’s what matters: content, not style.</p>\n</blockquote>\n<p>We cannot be at fault because we don’t see race or gender. Again, basic psychology trivially disproves this statement.</p>\n<hr />\n<p>There are a ton of other micro-narratives that occur in all of these discussions as well. Most of these are a derailing tactic, and straight-up distract from the issue. “Egalitarianism is anti-free speech,” “Feminists and their allies are irrational and hysterical,” “I don’t see race or gender,” “If they were calmer about this, it would have worked out,” “This would be better handled in private,” and many, many more. I’ve seen so many. It’s exhausting. These internal narratives get reinforced by a particular interpretation of a given situation, without examining what actually happened.</p>\n<p>Yet we still have conferences that are mostly full of white dudes.</p>\n<p>I lost a lot of respect for a lot of people today.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/how-to-squash-commits-in-a-github-pull-request/",
"title": "How to squash commits in a GitHub pull request",
"description": null,
"url": "https://steveklabnik.com/writing/how-to-squash-commits-in-a-github-pull-request/",
"published": "2012-11-08T00:00:00.000Z",
"updated": "2012-11-08T00:00:00.000Z",
"content": "<p>So you’ve contributed some code to an open source project, say, Rails. And they’d like you to squash all of the commits in your pull request. But you’re not a git wizard; how do you make this happen?</p>\n<p>Normally, you’d do something like this. I’m assuming <code>upstream</code> is a git remote that is pointing at the official project repository, and that your changes are in your ‘omgpull’ branch:</p>\n<pre><code>$ git fetch upstream\n$ git checkout omgpull \n$ git rebase -i upstream/master\n\n< choose squash for all of your commits, except the first one >\n< Edit the commit message to make sense, and describe all your changes >\n\n$ git push origin omgpull -f\n</code></pre>\n<p>GitHub will then automatically update your pull request with the new commits. Super easy!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/theory-and-practice/",
"title": "Theory and practice",
"description": null,
"url": "https://steveklabnik.com/writing/theory-and-practice/",
"published": "2012-11-06T00:00:00.000Z",
"updated": "2012-11-06T00:00:00.000Z",
"content": "<p><img src=\"http://imgs.xkcd.com/comics/purity.png\" alt=\"http://imgs.xkcd.com/comics/purity.png\" /></p>\n<p>purity</p>\n<hr />\n<p>There is always a tension between Theory and Practice. These two separate realms are connected through a process of abstraction and application. To explain this relationship by way of theory, Theory deterritorializes Practice, and Practice reterritorializes Theory: a Theory which is a becoming-Practice and a Practice which is a becoming-Theory. To explain this relationship by way of practice, Theory is abstracted Practice, and Practice is applied Theory.</p>\n<p>There’s an age-old problem with this particular relationship: those who specialize in Practice often claim that those who specialize in Theory are detached from the ‘real world,’ ie, the world of Practice. Those who specialize in Theory often claim that those who specialize in Practice have no fundamental understanding of what they do, and this leads to contradictory, incongruous practices.</p>\n<p>There’s a third kind of person, though: one that embodies the becoming, the abstraction/application process. These people are a conduit, fundamentally bridging the two worlds. There’s a certain art to explaining just the core of Theory in the words of someone who Practices, and there’s a certain art to combining the essences of Practices and presenting it to those who Theorize. Building this bridge is an act of creation, of building, an opening of space.</p>\n<hr />\n<p>Some people are great at Ivory Tower intellectual stuff, and others just don’t really care. It sucks, because those who are doing could be way better at it if they just knew some theory, and those who love to philosophize all day might be more understandable if they’d just actually <em>do</em> something sometimes.</p>\n<p>The only way you can get these two camps to talk to each other is to figure out what the theory says that provides value to those who practice. Practice-ers are ruthlessly focused on value, so to get through to them, you have to speak their language. On the flip side, theorists understand that practicers don’t care too much about the theory, but love seeing their thoughts to go good use, and can appreciate when practicers stumble across flaws in their thought. So demonstrating how practicers produce contradictions in the theory can be really useful to theorists.</p>\n<p>If you’re this kind of person, accept that you’re in many ways a jack of all trades, but a master of none. Of sorts, anyway. Theorists will sometimes hate you for not grokking every last detail and reference, and practical people will argue that you don’t do enough useful things. Don’t listen to either of them; you know that you’re part of neither camp, so it makes sense that they both find ‘flaws.’ You’re awesome because you know a bit of both, and can facilitate communication which makes both better. A force multiplier.</p>\n<hr />\n<p>You have to remember that while you’re building things, there’s an underlying set of rules that you’re implicitly following, but it’s more important to act than it is to memorize a bunch of rules, and try to analyze what you’re doing according to them. If all you do is think all day, you’ll never get anything done. Things may go wrong, but you can always fix it later. Other people can sit around and theorize about what you’re doing, leave them to it.</p>\n<p>Mental masturbation is fun and all, but when all is said and done, developing a culture of shipping is one of the most important things you can do. Those who can’t do, teach. History is written by the victors.</p>\n<p>On occasion, you’ll run into someone who can actually explain complicated theory stuff to you in an accessible way. If you find someone like this, make sure to hold onto them closely, as they’re really rare. But they can help provide you with some insight that will really boost your productivity, without having to invest all the time in figuring out all that wankery that the priests of theory love.</p>\n<hr />\n<p>“In theory, there is no difference between theory and practice. But, in practice, there is.” - Yogi Berra</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/i-got-hit-by-a-car-today/",
"title": "I got hit by a car today",
"description": null,
"url": "https://steveklabnik.com/writing/i-got-hit-by-a-car-today/",
"published": "2012-10-25T00:00:00.000Z",
"updated": "2012-10-25T00:00:00.000Z",
"content": "<p>“Humans are really squishy, messy, complicated, and contradictory. That’s what makes life interesting.”</p>\n<p>Veggie Grill is damn delicious, but it’s hard to talk and eat at the same time. I’m usually trying to shovel my Santa Fe Chickin’ sandwich into my face as hard as possible. Today, though, there was a screw-up, and I didn’t get my food for a while. So I talked instead.</p>\n<p>“Do you know anything about Hegelian dialectics?”</p>\n<p>My friends are the best. Most people don’t put up with rambling, half-formed ideas, but mine generally entertain them, even though they haven’t read the books I’m talking about. I’m really grateful for this, as describing things really helps me sort them out in my head. The blank look and continued chewing meant that this was once again one of those times. Whatever.</p>\n<p>“Basically, dialectics are a method of finding the truth in a threefold process: First, a ‘thesis’ is presented, which makes some sort of statement about the world. Then, the ‘antithesis’ contradicts it in some fashion. Finally, a ‘synthesis’ is found which resolves the tension between the two opposing sides. It’s a really interesting way of looking at the world; I’ve found it’s way easier to find things that totally contradict each other than I thought. Synthesis is really hard, though. Think about a trial, for example: the story of the plaintiff is told, then the story of the defendant, then the jury figures out what actually happened. There’s three sides to every story.”</p>\n<p>Last week, I flipped over the handlebars of a bike. I was pumping as hard as I could, trying to make the light before it turned red, and the chain couldn’t take the tension any longer. It popped off, went between the back wheel and the frame, and stopped it cold. A few seconds later my eyebrows were scraping across the pavement. Lots of blood, but no serious injuries. I’m going to have a few sweet scars, though…</p>\n<p>\"Anyway, we were talking about feelings and ways people act before, but it’s true of physical bodies as well. Sometimes, accidents happen, and before you can even react, you’re dead. Other times, you escape with very little damage. We’re simultaneously incredibly vulnerable and almost impervious. Some people get in a bike accident and their life is forever changed. Other times, like me last week, you get away with a few scrapes. Once I fell about 8 feet onto my knees. Landed right on some pavement. I walked away 5 minutes later, didn’t even have bruises. I still have no idea what the fuck happened there.</p>\n<p>I still haven’t found the synthesis yet.\"</p>\n<p>After lunch, it’s time to bike home. It’s about 15 miles, which I can do pretty much at the drop of a hat nowadays. I really enjoy biking up Santa Monica Boulevard, but I hate it, too. It’s nice and broad, with two lanes, a bunch of green in the middle, and some bike lanes. Of course, it isn’t all like this, but much of it is. Especially on a day like today, around noon, it’s really pretty in places.</p>\n<p><img src=\"http://upload.wikimedia.org/wikipedia/commons/6/62/Santa_Monica_Boulevard_at_Thayer.jpg\" alt=\"http://upload.wikimedia.org/wikipedia/commons/6/62/Santa_Monica_Boulevard_at_Thayer.jpg\" /></p>\n<p>So I’m about 8 miles into my trip, and the next thing I know, I’m on the ground. Here’s the basic problem:</p>\n<p><img src=\"http://www.examiner.com/images/blog/wysiwyg/image/car-in-bikelane-small.jpg\" alt=\"http://www.examiner.com/images/blog/wysiwyg/image/car-in-bikelane-small.jpg\" /></p>\n<p>Yeah. Bike lanes don’t help if someone wants to turn right. When alongside cars, it’s very, very important to keep alert for two things: Open doors from parked cars, and drivers who don’t use their turn signals. A few weeks ago, I almost got hit by someone in this same exact way, just a few blocks down the street. I was paying close attention, and actually guessed that they were going to turn before they did. Free driver psychoanalysis done here! But more seriously, this was in the middle of the block. This guy saw that there was a space at this parking garage, hit his breaks and turned right without looking.</p>\n<p>“YOU STUPID MOTHERFUCKER! WATCH WHERE THE FUCK YOU’RE GOING, YOU’RE GOING TO FUCKING KILL SOMEONE! JESUS!”</p>\n<p>I ran a systems diagnostic check, and I was good, but enraged. Some pedestrians were screaming, and one ran over and asked if I was okay. “I said, yes, thank you, at least until this guy fucking kills me or someone else,” got back on my bike, and drove off.</p>\n<p>At least my handlebars gave him a big long scratch the whole way up his car. Fucker.</p>\n<p>There’s also something else interesting in this story as well: Generally, in a car/human interaction, I am the squishy one, and the car is the tank. Yet, in this particular interaction, I basically escaped unscathed, while the car took some damage. Once again, life is surprising, producing an inversion of the expected circumstances.</p>\n<p>Anyway, if you drive a car, please, please, please watch what you’re doing. Pedestrians and cyclists don’t have thousands of pounds of metal surrounding them. Cyclists, please stay safe; it’s a jungle out there.</p>\n<p>Ultimately, life is made out of these little incidents. Each moment to the next, you have no idea what’s going to happen. It’s exciting, scary, and sloppy. But that’s what makes it interesting. And you also don’t know when it’s going to end.</p>\n<p>I still haven’t found that synthesis…</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/living-with-a-terrible-taste-in-music/",
"title": "Living with a terrible taste in music",
"description": null,
"url": "https://steveklabnik.com/writing/living-with-a-terrible-taste-in-music/",
"published": "2012-10-12T00:00:00.000Z",
"updated": "2012-10-12T00:00:00.000Z",
"content": "<p>Music has always been very important to me. Throughout most of my life, I’ve listened to a lot of music, and it’s played a huge role in defining my personality at the time I listened to it. In many ways, the history of the music I’ve listened to is a history of me.</p>\n<p>Here’s a look at where I’ve been, and where I’m at now.</p>\n<h2>Country</h2>\n<p>My parents essentially listen exclusively to country music, and so it’s basically all I listened to for the first decade of my life. And when I say country, I don’t mean old stuff, I mean proto-pop country:</p>\n<p>Many, many people that I know now are quite surprised by this. Country is one of those genres that brings out a certain revulsion in people. It still comes up from time to time, and my understanding of country brings out a laugh. Once in college, I was in a friend’s hometown, the Middle of Nowhere, PA. His brother rolls up in his huge pickup truck, blasting some George Straight. My friend looks at me, and rolls his eyes. With a straight face, I start saying the words as George sings them. The look of shock and confusion on his face was priceless.</p>\n<p>In many ways, that’s exactly who I was destined to be: just a country boy gettin’ down on the farm.</p>\n<p>But then, I discovered skateboarding, and punk rock.</p>\n<h2>Punk</h2>\n<p>Of course, it wasn’t even good punk music, either. See, I had always thought skateboarding was neat, but wasn’t never had one because there was nowhere to ride it. I lived on a farm, we didn’t have any pavement anywhere. So I forgot about skating. But then I got a call from a friend, he’d just gotten a new video game:</p>\n<p><img src=\"/img/2012-10-12/TonyHawksProSkaterPlayStation1.jpg\" alt=\"thps 1 cover\" /></p>\n<p>Here’s the soundtrack list:</p>\n<ul>\n<li>“Police Truck” by the Dead Kennedys</li>\n<li>“Here and Now” by The Ernies</li>\n<li>“Vilified” by Even Rude</li>\n<li>“Superman” by Goldfinger</li>\n<li>“Jerry Was a Race Car Driver” by Primus</li>\n<li>“Screamer”/“Nothing to Me” by Speedealer</li>\n<li>“Cyco Vision” by Suicidal Tendencies</li>\n<li>“New Girl” by The Suicide Machines</li>\n<li>“Committed” by Unsane</li>\n<li>“Euro-Barge” by The Vandals</li>\n</ul>\n<p>Pretty awesome. He also had this new album that came out at the same time…</p>\n<p><img src=\"/img/2012-10-12/blink-182-enema-of-the-state-front.jpg\" alt=\"enema of the state\" /></p>\n<p>From there, it was all over. All this edginess combined with my (just budding, of course) testosterone meant I was pretty much set. The funniest part about this period of my life was that I actually wasn’t really a particularly rebellious child: I maintained pretty damn good grades, save for a few semesters, I still went to church, and I was generally a good kid. I didn’t even smoke pot! Yet even the act of possessing these albums was heinously offensive. I got grounded for half a year for possessing Blink182’s “Take off your Pants and Jacket” and Limp Bizkit’s “Three Dollar Bill, Y’all$.” This constant conflict over music with my parents dominates my memories at the time. I literally printed out all the lyrics and wrote two pages explaining why I should be allowed to own a copy of the Offspring’s “Americana” even though they said ‘shit’ a few times. Punk, even third generation pop-punk, really spoke to my desire for autonomy and my attempts to be my own person.</p>\n<h2>Metal</h2>\n<p>Of course, slippery slopes are everywhere, and so I started searching for harder and harder music. Always the <a href=\"http://words.steveklabnik.com/how-do-you-find-the-time\">min/maxer</a>, I looked for loopholes in my parent’s rules. The biggest hole: anything from the Christian Family Bookstore was fair game. So I scoured it for anything awesome. That’s when I found Zao:</p>\n<p>Screw punk, metal was way more awesome. I grew my hair out long and learned to headbang.</p>\n<p>I don’t think I need to explain to you why metal is ridiculous. I mean, come on:</p>\n<p><img src=\"/img/2012-10-12/dethklokbandfeb2024_638.webp\" alt=\"dethklok\" /></p>\n<h2>Hardcore</h2>\n<p>Of course, there has to be a way to reconcile all this, right? Take two steps forward, one step back? Well, that’s hardcore.</p>\n<p>My fascination with hardcore in late high school and early college was two-fold, like ‘hardcore’ itself: there’s ‘hardcore’ and then there’s ‘hardcore punk.’ Hardcore punk is a specific sub-genre of punk rock from the 80s, and hardcore is its continuation in the modern day, after that whole emo thing happened.</p>\n<p>To recap: hardcore punk:</p>\n<p><img src=\"/img/2012-10-12/a18-MinorThreat.jpg\" alt=\"minor threat poster\" /></p>\n<p>hardcore:</p>\n<p><img src=\"/img/2012-10-12/tumblr_lpqekrWMCt1qh3e7yo1_400.gif\" alt=\"slam dancing\" /></p>\n<p>Uuhhh yeah. Hardcore punk also brought me to straightedge, which leads to silliness like this:</p>\n<p><img src=\"/img/2012-10-12/KmCWp.jpg\" alt=\"my straightedge tattoo\" /></p>\n<p>To this day, I still don’t regret this tattoo, and anyone that’s seen me at a Ruby conference knows that I’m not straightedge anymore. But this music and philosophy defined who I was for about 8 years, which was, at the end of it, about a third of my life. It changed my college experience. I’m (basically) permanently marked by it.</p>\n<p>But no, seriously:</p>\n<p><img src=\"/img/2012-10-12/tumblr_lpqekrWMCt1qh3e7yo1_400.gif\" alt=\"slam dancing\" /></p>\n<h2>Party Rock</h2>\n<p>Anyway, so nowadays, this is what I listen to:</p>\n<p><iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/edP0L6LQzZE?si=uTrbb2_tUchSk1FC\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen></iframe></p>\n<p><iframe width=\"560\" height=\"315\" src=\"https://www.youtube.com/embed/k0BWlvnBmIE?si=m4OITv0l29x0zot5\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" allowfullscreen></iframe></p>\n<p>Here’s the thing: once I finally admitted to myself that I enjoy terrible pop music and stopped trying to apologize for my taste, I realized I was way more productive. Upbeat music with a vaguely positive vibe means I get shit done. Way better than the heavy, dreary, angry stuff I listened to in the past. I listen to hours of music a day, and I’d much rather have it be happy…</p>\n<p>But that’s when I realized I’ve basically always had bad taste in music.</p>\n<p>The thing is, that’s totally okay! Music has always provided an interesting backdrop for the rest of my life, and it’s fit whatever moment I was living. Writing this post was a blast, because I’d forgotten about half of this stuff, and re-living it is a good time. I can remember debating my friends about the Casualties being street punk or hardcore, if some band had really sold out, or if straightedge still matters now that Ian MacKaye smokes pot.</p>\n<p>I wonder what I’ll be listening to in five years.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/run-rails-with-custom-patches/",
"title": "Run Rails with custom patches",
"description": null,
"url": "https://steveklabnik.com/writing/run-rails-with-custom-patches/",
"published": "2012-10-04T00:00:00.000Z",
"updated": "2012-10-04T00:00:00.000Z",
"content": "<p>I often see comments <a href=\"https://github.com/rails/rails/pull/7397#issuecomment-9132009\">like this</a> in the Rails bugtracker. Generally, someone is running an older version of Rails, and some bug they face has been fixed on edge. But they may be running a version that’s too old to recieve fixes, or need a fix that has yet to be included in an actual release. What to do?</p>\n<p>Luckily, <a href=\"http://gembundler.com/\">Bundler</a> exists. It makes it super easy to run your own Rails. Check it out:</p>\n<h2>Step 1: Fork you!</h2>\n<p>Go to GitHub, hit up <a href=\"http://github.com/rails/rails\">rails/rails</a>, and click that fork button. My own personal fork of Rails is <a href=\"https://github.com/steveklabnik/rails\">here</a>, for example, so in the rest of these examples, I’ll be using my own username. If you’re me, you can use it too, but seriously, stop being me. If you’re not me, substitute your own username.</p>\n<h2>Step 2: apply patch</h2>\n<p>Let’s get our copy going locally:</p>\n<pre><code>$ git clone [email protected]:steveklabnik/rails.git\n$ cd rails\n</code></pre>\n<p>We’ll apply the patch from the example comment above. In this case, the person wants to have this patch work on Rails 3.1.</p>\n<p>You have two options here: Rails keeps a branch open for every minor version, and also tags every release. You can pick between the stable branch and a particular release. The stable branch should work just fine, but maybe you’re on a specific Rails release for a reason. For example, the latest version of Rails 3.1 is 3.1.8, but this person says they’re on 3.1.1. If they use the 3-1-stable branch, they’ll also get all the changes from 3.1.1-3.1.8, as well as unreleased changes that may exist on that branch. That’s possibly too much for you, so we’ll just work off of 3.1.1. This is probably a bad idea, since <a href=\"http://weblog.rubyonrails.org/2012/6/12/ann-rails-3-1-6-has-been-released/\">3.1.6 included several security fixes</a>, but you’re an adult, do dumb things if you want to.</p>\n<pre><code>$ git checkout v3.1.1\n</code></pre>\n<p>You’ll see a big message about a detached HEAD. Don’t worry about it.</p>\n<p>If you want the stable branch for extra fixes,</p>\n<pre><code>$ git checkout 3-1-stable\n</code></pre>\n<p>Now, it’s a good idea to make our own branches, so that we don’t get confused with upstream. So let’s make a new branch:</p>\n<pre><code>$ git checkout -b my_patched_rails\n</code></pre>\n<p>Awesome. Now, we can grab the commit we wanted. We can do this with <code>cherry-pick</code>:</p>\n<pre><code>$ git cherry-pick 8fc8763fde2cc685ed63fcf640cfae556252809b\n</code></pre>\n<p>I found this SHA1 by checking out <a href=\"https://github.com/rails/rails/pull/7397/commits\">this page</a>, which is linked at the top of the pull request.</p>\n<p>If there are multiple commits you need, you can do one of two things: either <code>cherry-pick</code> the merge commit, in which case you’ll probably need to pass the <code>-m1</code> option, or simply <code>cherry-pick</code> them all in order.</p>\n<p>You may get conflicts. Yay backporting! If <code>git</code> whines at you, do what you normally do. Resolve the merge, then <code>git commit</code>.</p>\n<p>Finally, push it back up to your GitHub:</p>\n<pre><code>$ git push origin my_patched_rails\n</code></pre>\n<p>Congrats! You’ve got your own custom Rails. Now it’s time to use it.</p>\n<h2>Step 3: update Gemfile</h2>\n<p>Go to your Rails app, and edit this line in your Gemfile:</p>\n<pre><code>gem 'rails', \"3.1.1\"\n</code></pre>\n<p>change it to this:</p>\n<pre><code>gem 'rails', :git => \"https://github.com/steveklabnik/rails\", :branch => \"my_patched_rails\"\n</code></pre>\n<p>It’s that easy! Now bundle:</p>\n<pre><code>$ bundle update\n</code></pre>\n<p>You should see it mention something about checking out a certain copy of Rails.</p>\n<h2>Step 4: Profit!</h2>\n<p>That’s it! Congrats, you’re using Rails with your patch.</p>\n<h2>But I’m on 2.3, I’m not running Bundler!</h2>\n<p><a href=\"http://gembundler.com/v0.9/rails23.html\">Do this</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/how-do-you-find-the-time/",
"title": "How do you find the time?",
"description": null,
"url": "https://steveklabnik.com/writing/how-do-you-find-the-time/",
"published": "2012-09-27T00:00:00.000Z",
"updated": "2012-09-27T00:00:00.000Z",
"content": "<p>I’m often asked the question, “How do you find the time?” Mostly, it’s around open source, sometimes it’s about books, and occasionally, on other things. I’ve generally deflected the question with something like “I don’t know, man, I just <em>do</em> it.” But that answer isn’t very helpful. And I’ve been wondering if I can do even better. So I’ve done some reflection this week, and here’s how I find the time:</p>\n<h2>Min/max</h2>\n<p><img src=\"http://2.bp.blogspot.com/_6VPzVlB8N28/SwC5AWnireI/AAAAAAAABgA/O883tYn78As/s1600/DDplayershandbook2000.jpg\" alt=\"http://2.bp.blogspot.com/_6VPzVlB8N28/SwC5AWnireI/AAAAAAAABgA/O883tYn78As/s1600/DDplayershandbook2000.jpg\" /></p>\n<p>player’s handbook</p>\n<p>I have this personality type that is an asset to any competitive gamer: Dungeons and Dragons nerds call it “min/maxing”. The basic idea is this: you try to minimize the things that are bad, and maximize those that are good. This sounds super obvious, but there are many times when maximizing ‘winning’ isn’t what you’re going for. One great example is ‘pretending to be a fantasy adventurer.’ If you’re not shooting for strictly winning, then min/maxing doesn’t make any sense. But if you’re looking to compete, it’s incredibly important.</p>\n<p>Speaking of competition, you should really read this book:</p>\n<p><img src=\"http://ecx.images-amazon.com/images/I/41SZq3uDRVL._SS500_.jpg\" alt=\"http://ecx.images-amazon.com/images/I/41SZq3uDRVL.SS500.jpg\" /></p>\n<p>playing to win</p>\n<p><a href=\"http://www.sirlin.net/ptw/\">“Playing to Win”</a> is an excellent book by one of the best Street Fighter players in the world. It is a must-read for anyone who wishes to compete in any way.</p>\n<p>In it, he develops the theory of “Players” and “Scrubs”:</p>\n<blockquote>\n<p>A scrub is a player who is handicapped by self-imposed rules that the game knows nothing about.</p>\n</blockquote>\n<p>A great tell of someone being a scrub is when they mention ‘fairness.’ A game is an impartial, closed system. There are rules to the system that constrains the players. Simply, the more rules, the harder it is for you to achieve your win condition. If you invent extra rules, you handicap yourself. Games are not ‘fair.’ Players don’t handicap themselves in this way:</p>\n<blockquote>\n<p>A player should use any tournament legal move available to him that maximizes his chances of winning the game.</p>\n</blockquote>\n<p>If a move is legal, then it’s fair game. Good games are written in such a way that bad moves don’t exist. Use all the tools at your disposal, and use them ruthlessly.</p>\n<p>Here’s the thing about the scrub mindset: it’s easy to do accidentally. Extra rules can be self-imposed, but they can also happen due to a poor analysis of what the rules actually are. For example, the biggest time I scrubbed out in my entire life was a choice that I made almost a decade ago: going to college. Every adult in my life told me that going to college was a good idea. They all said it was needed to get a job as a programmer. I didn’t bother to challenge them, and went along with it. Now I have a useless piece of paper and $70,000 in debt. College wasn’t all bad, but it certainly wasn’t worth the debt. I cherish the friendships I made, but not much else. This mistake was made because of my apathy towards the systems governing my life, and it’s one I try not to repeat.</p>\n<p>Anyway, these two principles are the backdrop for everything else: properly analyze your situation, maximize the outcomes you want, and minimize the ones you don’t.</p>\n<h2>Work</h2>\n<p>“So what do you want out of life, Steve?”</p>\n<p>“I want to teach, I want to work on open source, and I want to travel.”</p>\n<p>“I think we can work something out.”</p>\n<p>If you work a 9-5 job, you spend 40 hours a week at work. Let’s add a one hour commute in there too, which puts you up to 50 hours. And as Ruby will tell you…</p>\n<pre><code>1.9.3-p194 :001 > 24 * 7\n => 168 \n1.9.3-p194 :002 > 50 / 168.0\n => 0.297619047619048 \n</code></pre>\n<p>30% of your life. Let’s analyze this like a pro gamer, shall we? Here’s the situation we’re trying to maximize: “I want to live a full life.” In order to maximize this, we need to discover the constraints. The biggest one, of course, is your time on this Earth. For now, at least, we all will die someday. The primary resource that we all have to work with is time. You should guard your time more closely than any other thing you have. You can’t ever get it back. Every single thing you do consumes this most precious of resources.</p>\n<p>When you enter into an employment agreement, you are literally selling your most precious thing in order to acquire other resources. Don’t forget that.</p>\n<p>So, 9-5 job. 30% of your life. Is it worth it? Maybe! I can’t tell you. But in order to maximize your outcomes, you need proper analysis. Employment is a tricky thing, in that regard. Capitalism forces us to interact with the market in order to gain our subsistence, so we have to enter into this transaction somehow. What to do?</p>\n<p>Quick, startup crowd: what’s the first rule of pricing? If you say “do it value-based, not cost-based,” then you win a <a href=\"http://www.kalzumeus.com/blog/\">patio11</a> sticker. You’ll never get anywhere selling yourself on an hourly basis, you have to sell yourself on value. That’s another topic for another time. Point is, don’t just assume the rules: truly analyze your situation.</p>\n<p>Anyway, that’s neither here nor there. On a day-to-day basis, the only reason that I can do what I do is because <a href=\"https://twitter.com/j3\">Jeff</a> is a stand-up guy, and working with <a href=\"http://jumpstartlab.com/\">Jumpstart Lab</a> is awesome. We do the best Ruby and Rails classes in the world, and I mean that. I don’t want to get into the details of my employment, but basically, teaching classes, contributing to open source, and speaking at conferences are all part of my job. You can see how that directly maps to my statement above.</p>\n<p>So, if you appreciate</p>\n<ul>\n<li><a href=\"http://hackety.com/\">My work on Hackety Hack</a></li>\n<li><a href=\"http://contributors.rubyonrails.org/contributors/steve-klabnik/commits\">My contributions to Rails</a></li>\n<li><a href=\"https://github.com/defunkt/resque\">My contributions to Resque</a></li>\n<li><a href=\"http://vimeo.com/search?q=klabnik\">My public speaking</a></li>\n<li><a href=\"https://github.com/steveklabnik?tab=activity\">All the other Open Source work that I do</a></li>\n<li><a href=\"http://confreaks.com/videos/1089-madisonruby2012-anti-opression-101\">My advocate work for feminism and social justice</a></li>\n</ul>\n<p>then you should consider hiring us over at Jumpstart. If I had to do normal consulting to pay my bills, I would have much less time for all of this stuff.</p>\n<h3>But I don’t work at Jumpstart!</h3>\n<p>So, you’re not me. You work a 9-5 job. So you’re screwed, right?</p>\n<p>No way! Analyze your situation. Figure out how to maximize it. Before I did software development, I <a href=\"http://www.vocellipizza.com/consumer/index.php\">did this</a>. About 30 hours a week while at high school, and 40 hours a week through college. I <em>still</em> managed to put in open source time. How?</p>\n<p>The thing is, working on open source is something I truly love to do. For me, it’s not ‘work.’ That does not mean that it’s always roses. Sometimes it’s hard. Sometimes, it really, really sucks. Hard. However, I truly love it, so I stick it out through the hard times, and it ends up being good again. Once, I suffered a really bad break-up. My mom told me this maxim that her grandmother always told her:</p>\n<blockquote>\n<p>What bone you were meant to have, no dog will drag away.</p>\n</blockquote>\n<p>Grandmas. They’re so opaque! The point is, contributions to Open Source is something that’s a part of me, no matter what. So I’ll always find a way to do it. I don’t give up on it easily. When I first took over Shoes, it took me <strong>six months</strong> to figure out how to compile it. Six months. No compilation. That sucked. But I stuck it out.</p>\n<blockquote>\n<p>Eighty percent of success is showing up.Woody Allen</p>\n</blockquote>\n<h2>Relationships</h2>\n<p>Having a full life means interacting with other people. Don’t let some productivity fetish negatively affect relationships. I mean that in a broad sense, both romantic and bromantic. ;) Make time for stuff that’s not ‘ZOMG MUST GET WORK DONE’ in your schedule too. I make sure to talk to significant others at least once or twice a day, for example. Assuming they want me to. Tonight, I’m gonna go for a bike ride on the beach. I still play StarCraft from time to time. Every Saturday, my friends and I used to get together in the computer lab and hack.</p>\n<p>If you spend all of your time working, and none of your life living, you’ve failed at life. Open source is fun, but go outside, dammit. Or read a book. Or something. But if you only program computers, you’ve failed at being a human.</p>\n<h2>Music</h2>\n<p>Listen to awesome music while you work. I first discovered this while mowing my parent’s’ lawn: when I listened to Metallica, that grass got cut way quicker. So, while I work, I listen to shitty pop music. <a href=\"http://www.pandora.com/?sc=sh588283719292597139&shareImp=true\">Here’s my Pandora station</a>. I find that upbeat, happy music makes me upbeat and happy. I’m over being embarrassed about my love for Ke$ha. I just bob my head and write more code.</p>\n<h2>Parallelize</h2>\n<p>I actually cannot be productive without doing six different things at once. This is related to <a href=\"http://www.structuredprocrastination.com/\">“Structured Procrastination”</a>. I do a little bit of work on a bunch of different things, and cycle between them quickly. One day, it’s Rails, another, it’s Resque, another is training, then back to Rails, then Hackety…</p>\n<h2>lift.do</h2>\n<p>I’ve been trying out <a href=\"http://lift.do/\">Lift</a> lately, and I really like it. You can build habits with some light social interaction. You might find it useful.</p>\n<h2>Cut distractions.</h2>\n<p>That’s a bunch of things I do to maximize good outcomes, but what about minimizing bad ones? In the interest of minimalization, I won’t dwell on them too much, but here are some things that I’ve cut down or cut out in some way:</p>\n<ul>\n<li>TV</li>\n<li>Reddit/HN</li>\n<li>Cell phones</li>\n<li>Video Games</li>\n</ul>\n<p>I still indulge in some of these from time to time, but I’ve cut them out of the ‘every day’ loop. My life is better off for it. Figure out what you don’t actually need, and stop it.</p>\n<p>;)</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/seriously--numbers--use-them-/",
"title": "Seriously: numbers: use them!",
"description": null,
"url": "https://steveklabnik.com/writing/seriously--numbers--use-them-/",
"published": "2012-09-27T00:00:00.000Z",
"updated": "2012-09-27T00:00:00.000Z",
"content": "<p><a href=\"http://vimeo.com/50340416\">Turbolinks test</a> from <a href=\"http://vimeo.com/steveklabnik\">Steve Klabnik</a> on <a href=\"http://vimeo.com/\">Vimeo</a>.</p>\n<p>If you don’t feel like watching…</p>\n<p><a href=\"https://github.com/steveklabnik/turbolinks_test\">https://github.com/steveklabnik/turbolinks_test</a></p>\n<p>Results with 1000 pages:</p>\n<p>Blank Rails app:</p>\n<pre><code>$ rspec\n user system total real\n no turbolinks 11.170000 0.980000 12.460000 (138.656728)\nyes turbolinks 10.800000 0.870000 11.670000 ( 80.436286)\n</code></pre>\n<p>With Basecamp Next’s CSS file:</p>\n<pre><code>$ rspec\n user system total real\n no turbolinks 14.470000 1.540000 16.320000 (235.404727)\nyes turbolinks 10.730000 0.870000 11.600000 ( 82.176967)\n</code></pre>\n<p>With Basecamp Next’s JS file:</p>\n<pre><code>$ rspec\n user system total real\n no turbolinks 15.300000 1.700000 17.270000 (433.880904)\nyes turbolinks 10.540000 0.890000 11.430000 (170.545663)\n</code></pre>\n<p>With Both:</p>\n<pre><code>$ rspec\n user system total real\n no turbolinks 21.990000 2.890000 25.150000 (581.822206)\nyes turbolinks 10.970000 0.910000 11.880000 (196.481247)\n</code></pre>\n<h2>TL;DR</h2>\n<p>Turbolinks seems to speed up apps.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/resque--let-s-do-this/",
"title": "Resque: let's do this",
"description": null,
"url": "https://steveklabnik.com/writing/resque--let-s-do-this/",
"published": "2012-09-22T00:00:00.000Z",
"updated": "2012-09-22T00:00:00.000Z",
"content": "<p>If you’ve ever done background job stuff with Rails, you’ve probably used <a href=\"https://github.com/defunkt/resque\">Resque</a>. Resque is “Yet Another GitHub Project They’ve Open Sourced and Then Abandoned.” But Resque is super awesome: it builds a job queue on top of Redis, which is a totally awesome piece of software.</p>\n<p>Anyway, Resque has been psuedo-abandonware for a while now. In January, <a href=\"http://hone.heroku.com/\">Terence Lee</a> from Heroku got in charge of the project, but it’s a big job. Too big of a job to properly manage alone. It’s hard to sift through <a href=\"https://github.com/defunkt/resque/issues?state=open\">60 pulls and 120 open issues</a>, some of which have been open for a few <em>years</em>. And manage the 1.x line while working on new stuff for 2.0. And debug issues that basically boil down to “When I upgrade Resque and this random gem, everything breaks, but when I downgrade that other gem, it works again. But Resque throws the error.”</p>\n<p>Even outside of heroics, the bus factor on a project that’s as important as Resque should be higher than one.</p>\n<p>So Terrence gave a presentation at Frozen Rails, and in it, he outlined what needs to be done for Resque 2.0, and asked for some help getting it out the door. So myself and a few other people are gonna pitch in and help out, and we’d love to have you.</p>\n<p>Here’s the deal: I will personally be looking at (and already have read) every single open issue on GitHub. Feel ignored about something involving Resque? You can come to me. I can’t guarantee that I can get it fixed, but at least I’ll help you figure out wtf is going on, and get us a bug report that we can use to build a fix.</p>\n<p>Think of it as decoupling the front end from the back end: Terrence can focus on writing new code, and I can focus on paying attention to your problems, helping you solve them, and getting fixes applied to the 1.x branch.</p>\n<p>If you’re interested in helping out, we’d love to have you. Helping triage issues would be awesome, fixing bugs would be awesome, writing code would be awesome, updating docs would be awesome.</p>\n<p>Let’s do this.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/software-and-community/",
"title": "Software and community",
"description": null,
"url": "https://steveklabnik.com/writing/software-and-community/",
"published": "2012-09-18T00:00:00.000Z",
"updated": "2012-09-18T00:00:00.000Z",
"content": "<p>“Are you sure you’re not vegan? You look like a vegan.”</p>\n<p>Chad Fowler made me laugh pretty hard that day. We were in Austin, at a Ruby conference, and getting some very non-vegan food at the excellent Iron Works BBQ. I had only talked to Chad a few times before, and while chowing down on some excellent ribs, we talked about something that we’d discussed in a previous conversation: “the Ruby community.”</p>\n<p>I gave a talk at Lone Star Ruby last year about my involvement with Hackety Hack and Shoes, and at the end, discussed some social issues I thought Ruby has:</p>\n<p>The fun starts at 27:49. But ultimately, this whole section relies on the idea that in Ruby, we have a cohesive ‘community’ of people. After I gave my talk, Chad mentioned to me that he didn’t like the idea of “the Ruby community.” At the time, I was so starstruck that I said something like “sure, okay, I’ll think about that,” not even realizing the irony. So, while trying valiantly to eat barbecue in a not completely messy way, I asked Chad more about his thoughts on this.</p>\n<p>If you were wondering, if it’s good barbecue, it’s impossible to eat it neatly.</p>\n<p>I don’t remember his exact words, but Chad’s point was that if we talk about Ruby like a big cohesive group, we forget about all the people who use it who don’t post on blogs, or Twitter, or contribute to GitHub. Those people are people too, but they’re effectively outside ‘the community,’ either by choice or by ignorance.</p>\n<h2>Kill Your Idols</h2>\n<p>It seems to me that much of software is driven by fashion, celebrity, and superstition. This shouldn’t surprise anyone who’s not a software developer, as programmers are just people, and much of the human world is driven by fashion, celebrity, and superstition. Programmers, though, regard themselves as bastions of rationality.</p>\n<blockquote>\n<p>I don’t argue about sports, it’s stupid. I prefer to argue about important things, like whether or not to use semicolons in JavaScript.</p>\n</blockquote>\n<p>— Matthew Nielsen (@xunker) September 15, 2012</p>\n<blockquote></blockquote>\n<p>Right? This whole idea makes us, or at least me, very uncomfortable. I’m a scientist, dammit! I measure twice, cut once. I use the right tool for the job. I swear, MongoDB, CoffeeScript, and Ember.js fits my use case perfectly!</p>\n<p>Ideally, we’d make technical decisions on technical means only. However, we’re just humans. I’m not sure it’s possible to divorce our decision making from our squishy human tendencies, and after typing that sentence, I’m no longer sure that it’s a good idea, either.</p>\n<h2>Leaders vs. Bosses</h2>\n<blockquote>\n<p>Does it follow that I reject all authority? Far from me such a thought. In the matter of boots, I refer to the authority of the bootmaker; concerning houses, canals, or railroads, I consult that of the architect or the engineer. For such or such special knowledge I apply to such or such a savant. But I allow neither the bootmaker nor the architect nor the savant to impose his authority upon me. I listen to them freely and with all the respect merited by their intelligence, their character, their knowledge, reserving always my incontestable right of criticism and censure. I do not content myself with consulting a single authority in any special branch; I consult several; I compare their opinions, and choose that which seems to me the soundest. But I recognise no infallible authority, even in special questions; consequently, whatever respect I may have for the honesty and the sincerity of such or such an individual, I have no absolute faith in any person. Such a faith would be fatal to my reason, to my liberty, and even to the success of my undertakings; it would immediately transform me into a stupid slave, an instrument of the will and interests of others.Mikhail Bakunin, “What is Authority,” 1882</p>\n</blockquote>\n<p>There is a difference between a leader and a boss. There’s also a difference between a leader and a celebrity. What I’m concerned with is that we occasionally turn leaders into bosses, and we turn celebrities into leaders.</p>\n<p>One other Ruby person that straddles this line is Aaron “Tenderlove” Patterson. He is, for good reason, almost universally beloved by Rubyists. He’s known to all for two different reasons: as a leader, and as a celebrity. Aaron is the only person on both the core Ruby as well as core Rails teams. He is one of the most technically brilliant people I know.</p>\n<p>He also posts amazing pictures of his cat on Tumblr. (The caption is “My new Yoga outfit.”)</p>\n<p><img src=\"http://25.media.tumblr.com/tumblr_m3v8keZVhR1rt6kj7o1_1280.png\" alt=\"http://25.media.tumblr.com/tumblr_m3v8keZVhR1rt6kj7o1_1280.png\" /></p>\n<p>gorbypuff</p>\n<p>Another amazingly technical thing Aaron has done is <a href=\"http://confreaks.com/videos/427-rubyconf2010-zomg-why-is-this-code-so-slow\">re-writing the AST -> SQL generation stuff in ActiveRecord</a>, which sped up ActiveRecord by 5x. He deserves every single last bit of his popularity, on both sides. I happen to find the split reasons interesting.</p>\n<h2>“The community” as useful abstraction</h2>\n<p>While I think ‘the community’ may help drive a celebrity culture over a culture of leadership, there’s no denying that we do, in fact, have a community. If I had a nickel for every time someone in Ruby did something nice for me just because, I’d have a lot of nickels. I’m writing this post from the couch of <a href=\"https://twitter.com/kiiiir\">a Rubyist</a> who happened to be on the same plane as me to Helsinki, and rather than stay at the airport for 10 hours overnight (checkin isn’t for another few hours yet), I crashed here. On my last trip to Germany, Konstantin and Josh from Travis not only gave me their couches to sleep on, but bought me food when I fucked up my debit card situation. One Saturday, Luis Lavena spent 8 hours on a Saturday randomly pairing with me on some Ruby on Windows stuff, and left with an “I’m sorry, but my fiance is bugging me, I told her we’d get dinner an hour ago, I need to leave.” That’s crazy, I’m some dude from another hemisphere, hang out with your fiance! Aaron helped me debug an issue in Rails the other day, even though I kept asking him stupid questions. I have tons of stories like this, and so do many other people who participate in open source.</p>\n<p>In “Debt: The first 5,000 Years,” Graeber defines “everyday communism” as \"any community who follows the maxim ‘from each according to their abilities, to each according to their needs.\" Of course, communism and community share a root for a reason, and all of these examples above are great descriptions of ’everyday communism.’ Not every exchange must go through some sort of monetary exchange. It’s often simpler to just do each other favors, and know that it will all work out in the end. That’s the basis of a gift economy.</p>\n<p>In this way, I think community is a useful abstraction; we just need to make sure that we don’t get <a href=\"http://rubydramas.com/\">too caught up in ourselves</a>. Focus on the friendships, helping each other out, and writing great software. Try not to get too trendy. Pay attention to what matters.</p>\n<p>In the end, I wouldn’t give up the <strong>real</strong> friendships I’ve made via the Ruby community for the world.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/random-ruby-tricks--class-new/",
"title": "Random Ruby Tricks: Class.new",
"description": null,
"url": "https://steveklabnik.com/writing/random-ruby-tricks--class-new/",
"published": "2012-09-09T00:00:00.000Z",
"updated": "2012-09-09T00:00:00.000Z",
"content": "<p>If you didn’t know, classes are first-class objects in Ruby:</p>\n<pre><code>1.9.3p194 :001 > String.class\n => Class \n1.9.3p194 :002 > Class.class\n => Class\n</code></pre>\n<p>How is this useful, though?</p>\n<h2>Inheritance</h2>\n<p>You may create very simple classes at times. This often happens when subclassing an error of some sort:</p>\n<pre><code>class MyException < StandardError\nend\n\nraise MyException\n</code></pre>\n<p>That whole thing is awkward to me, though. Why bother with the end if you don’t need it? Turns out, it’s easy to make your own <code>Class</code> object: just use <code>Class.new</code>:</p>\n<pre><code>1.9.3p194 :001 > Class.new\n => #<Class:0x007fee209f42f8>\n</code></pre>\n<p>But that doesn’t help us a ton. Turns out <code>Class.new</code> takes an argument: the class this new class inherits from:</p>\n<pre><code>1.9.3p194 :002 > Class.new(StandardError)\n => #<Class:0x007fee209ee2e0>\n</code></pre>\n<p>We can then save this in our own constant:</p>\n<pre><code>1.9.3p194 :003 > MyException = Class.new(StandardError)\n => MyException \n1.9.3p194 :004 > MyException.new\n => #<MyException: MyException> \n1.9.3p194 :005 > MyException.ancestors\n => [MyException, StandardError, Exception, Object, Kernel, BasicObject]\n</code></pre>\n<p>I think I like this notation a bit better than <code>class...end</code>. But many people may not be familiar with it.</p>\n<h2>Passing a block</h2>\n<p>Of course, you can pass a block to <code>Class.new</code> and it’ll work like you’d expect:</p>\n<pre><code>1.9.3p194 :001 > Foo = Class.new do\n1.9.3p194 :002 > def bar\n1.9.3p194 :003?> puts \"bar\"\n1.9.3p194 :004?> end\n1.9.3p194 :005?> end\n => Foo \n1.9.3p194 :006 > Foo.new.bar\nbar\n => nil\n</code></pre>\n<p>The block gets <code>class_eval</code>’d. I haven’t found a good use for this one, exactly, but it’s good to know about!</p>\n<p>EDIT: <a href=\"https://twitter.com/pete_higgins/status/244903490588119040\">@pete_higgins has chimed in on Twitter.</a> I knew I’d done this before!</p>\n<h2><code>new</code> vs <code>initialize</code></h2>\n<p>Ever wonder why you call <code>Foo.new</code> but define <code>Foo.initialize</code>? It’s pretty simple. Here:</p>\n<pre><code>VALUE rb_class_new_instance(int argc, VALUE *argv, VALUE klass)\n{\n VALUE obj;\n\n obj = rb_obj_alloc(klass);\n rb_obj_call_init(obj, argc, argv);\n\n return obj;\n}\n</code></pre>\n<p>Obviously! This is the source for <code>Foo.new</code>. You might not read C, but it’s pretty simple: it first allocates the space for the object using <code>rb_obj_alloc</code>, and then calls <code>initialize</code> using <code>rb_obj_call_init</code>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/protocol-and-language/",
"title": "Protocol and language",
"description": null,
"url": "https://steveklabnik.com/writing/protocol-and-language/",
"published": "2012-09-03T00:00:00.000Z",
"updated": "2012-09-03T00:00:00.000Z",
"content": "<p>This is the second part of my series on protocol. The <a href=\"http://words.steveklabnik.com/protological-control-an-introduction\">first part</a> contained a lot of background information, but now we’re ready to get into what Protocol actually <em>is</em>.</p>\n<hr />\n<p>I live in a pretty unique place, <a href=\"http://farmhouse.la/\">the Farmhouse</a>. It’s a psuedo-public space in which we live, others work, and everybody has a good time. We do have <a href=\"http://farmhouse.la/rules\">some rules</a>, though:</p>\n<p><img src=\"http://farmhouse.s3.amazonaws.com/images/home/farmhouse-rules-chalkboard-wall-by-heather-peterson.jpg\" alt=\"http://farmhouse.s3.amazonaws.com/images/home/farmhouse-rules-chalkboard-wall-by-heather-peterson.jpg\" /></p>\n<p>They’re written up on the wall in chalk. Since we have lots of visitors to the house, and these are hard and fast rules, it’s important that we communicate them in a clear way. These rules form the protocol of our house. Phrased another way, they’re the rules that govern the way that we interact with each other and the house itself while at the house.</p>\n<p>Ultimately, this is what protocol is: a language. To Galloway, protocol is a very specific kind of language:</p>\n<blockquote>\n<p>Protocol is a language that regulates flow, directs netspace, codes relationships, and connects life-forms.</p>\n</blockquote>\n<p>Some of these properties are contradictory. Stated more accurately, there exists a tension between opposing forces within protocol itself. Let’s expand on these meanings.</p>\n<h2>‘regulates flow’</h2>\n<p>If you’ve ever read <em>1984</em>, you’re familiar with linguistic relativity, more commonly known as the Sapir-Whorf hypothesis.</p>\n<blockquote>\n<p>How could you have a slogan like “freedom is slavery” when the concept of freedom has been abolished?</p>\n</blockquote>\n<p>What’s truly interesting is that Orwell himself actually advocated for simplifying language in <a href=\"http://www.resort.com/~prime8/Orwell/patee.html\">Politics and the English Language</a>, but specifically because he felt more complicated language was <em>obfuscatory</em>, not illuminating.</p>\n<p>In any case, language enumerates the possible, and while <a href=\"http://en.wikipedia.org/wiki/Longest_words#German\">some languages can have really long words</a>, ultimately, any language will have some things that are not expressible. <a href=\"http://byfat.xxx/rien-ne-tient-en-place\">This often happens with translations</a>.</p>\n<p>If you are the proud recipient of a computer science degree, you may have run into the <a href=\"http://en.wikipedia.org/wiki/Chomsky_hierarchy\">Chomsky hierarchy</a> in your compilers class. If you weren’t required to take a compilers class, please call up your alma mater and demand a refund. Here is the hierarchy:</p>\n<ol>\n<li>unrestricted grammars</li>\n<li>context-sensitive grammars</li>\n<li>context-free grammars</li>\n<li>regular grammars</li>\n</ol>\n<p>As you go up the hierarchy, each language type is enclosed by the type above it. All context-sensitive grammars are unrestricted grammars, and all context-free grammars are context-sensitive.</p>\n<p>Regular expressions are one area where a language was forced to move up the hierarchy in order to express more things. Strictly speaking, a regular expression cannot match nested parenthesis. If you are interested in the math, the <a href=\"http://en.wikipedia.org/wiki/Pumping_lemma_for_regular_languages\">Pumping Lemma</a> demonstrates why. However, this kind of thing is useful, and so extensions were developed that move them up the chain. <a href=\"http://perldoc.perl.org/perlfaq6.html#Can-I-use-Perl-regular-expressions-to-match-balanced-text%3f\">Now you can write this</a>:</p>\n<blockquote>\n<p>qr/(<(?:[^<>]++|(?1))*>)/</p>\n</blockquote>\n<p>Maybe Orwell was on to something.</p>\n<p>Others have speculated that linguistic relativity would also apply to software. Two notable examples are Kenneth E. Iverson, who created APL, and Paul Graham, Lisp enthusiast and investor extraordinaire. Iverson wrote a paper titled <a href=\"http://www.jsoftware.com/papers/tot.htm\">“Notation as a tool of thought”</a>, which begins:</p>\n<blockquote>\n<p>The importance of nomenclature, notation, and language as tools of thought has long been recognized.</p>\n</blockquote>\n<p>Graham authored an essay called <a href=\"http://www.paulgraham.com/avg.html\">“Beating the Averages”</a> in which describes the ‘Blub paradox’. The paradox relies on linguistic relativity:</p>\n<blockquote>\n<p>You can’t trust the opinions of the others, because of the Blub paradox: they’re satisfied with whatever language they happen to use, because it dictates the way they think about programs.</p>\n</blockquote>\n<p>I don’t mean to rain on the relativity parade, but there’s one small detail that we haven’t considered: Turing completeness. At the end of the day, most programming languages are Turing complete. This means that COBOL can do everything Lisp can, and PHP can do everything that Ruby can.</p>\n<p>So are Graham and Iverson wrong?</p>\n<p>Turns out there are two variations of linguistic relativity: <em>strong</em> relativity states that language <strong>determines</strong> thought, and <em>weak</em> relativity that states language <strong>influences</strong> thought. This was coined later, as it would seem that current research has found little evidence to support strong relativity, but some to support the notion of weak relativity. Graham notes this, and confines the paradox to the realm of weak:</p>\n<blockquote>\n<p>All languages are equally powerful in the sense of being Turing equivalent, but that’s not the sense of the word programmers care about. (No one wants to program a Turing machine.)</p>\n</blockquote>\n<p>To put it another way, language features create friction that encourages or discourages certain modes of thinking.</p>\n<p>Python and Ruby are two very similar languages, yet they engender very different styles. One area in which this division is sharp is first-class functions. Both languages have them, but Python’s are simply expressions rather than fully-featured . <a href=\"http://www.artima.com/weblogs/viewpost.jsp?thread=147358\">Guido van Rossum likes them this way</a>:</p>\n<blockquote>\n<p>But such solutions often lack “Pythonicity” – that elusive trait of a good Python feature. It’s impossible to express Pythonicity as a hard constraint. Even the Zen of Python doesn’t translate into a simple test of Pythonicity.</p>\n</blockquote>\n<p>Guido specifically wants to limit the scope of language in order to encourage a particular style of thinking about problems: a Pythonic one.</p>\n<p>Galloway is very explicit about this regulation.</p>\n<blockquote>\n<p>Protocol is synonymous with possibility. From the perspective of protocol, if you can do it, it can’t be bad, because if it were bad, then it would have been outlawed years ago by protocol.</p>\n</blockquote>\n<h2>‘directs netspace’</h2>\n<p>After that massive first part, I’m basically going to punt on the second. I don’t find it very interesting or worth elaborating on: languages and protocols direct the direction of the Internet. I don’t think anyone disputes this.</p>\n<h2>‘codes relationships’</h2>\n<p>Communication has an intimate connection to relationships. The act of communicating is what creates the relationship in the first place. And language is the encoding of that particular relationship.</p>\n<p>In “A user’s guide to Capitalism and Schizophrenia: Deviations from Deleuze and Guattari,” Brian Massumi discusses an example of marriage vows:</p>\n<blockquote>\n<p>A particular man and a particular woman say “I do.” Their words undoubtedly have personal meaning for them in their heart of hearts. But their personal intention is not in itself responsible for the magical transformation that has touched their lives. What has brought them to say these words and what makes those words effectively transformative is too big to fit into a single mid. It is a complex interplay of laws, customs, social pressure, and tax law.</p>\n</blockquote>\n<p>Marriage is a social protocol that encodes a particular relationship. Once that relationship has been encoded, it’s made legible to others who weren’t privy to the details of that relationship. A wedding ring is only one form of relationship encoding; a couple holding hands is a similar example, whose bond is weaker both in symbol as well as in reality.</p>\n<p>In the software world, HTTP encodes relationships as well. The client/server relationship is integral to the protocol’s design, as Roy Fielding <a href=\"http://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch_styles.htm#sec_3_4_1\">describes in section 3 of his dissertation</a>:</p>\n<blockquote>\n<p>A client is a triggering process; a server is a reactive process. Clients make requests that trigger reactions from servers. Thus, a client initiates activity at times of its choosing; it often then delays until its request has been serviced. On the other hand, a server waits for requests to be made and then reacts to them. A server is usually a non-terminating process and often provides service to more than one client.</p>\n</blockquote>\n<p>He further goes on to describe why this relationship is important:</p>\n<blockquote>\n<p>Separation of concerns is the principle behind the client-server constraints. A proper separation of functionality should simplify the server component in order to improve scalability. This simplification usually takes the form of moving all of the user interface functionality into the client component. The separation also allows the two types of components to evolve independently, provided that the interface doesn’t change.</p>\n</blockquote>\n<p>In other words, the protocol encodes the relationship so that others know what a client should do, and what a server should do.</p>\n<p>If you remember our good friend Saussure, you’ll remember that he made this encoding explicit, and in fact central to language itself. While Saussure’s work isn’t directly useful to linguists in the present, much of his work is foundational to the ways that we approach language today.</p>\n<p>One of the most fundamental concepts in structural linguistics is the ‘sign,’ and its component parts, the ‘signifier’ and the ‘signified.’ The signified is a particular concept, and the signifier is the way the signified is expressed. A sign is a combination of a signifier and a signified. Seems pretty simple, but Saussure also posited that a sign can only gain <a href=\"http://en.wikipedia.org/wiki/Meaning_(semiotics)\">meaning</a> and <a href=\"http://en.wikipedia.org/wiki/Meaning_(semiotics)\">value</a> within their relationship to other signs, which means that you cannot consider an idea outside of the way in which the idea is expressed.</p>\n<p>The study of signs is called <a href=\"http://en.wikipedia.org/wiki/Semiotics\">semiotics</a>. In Semiotics, a particular set of conventions used to convey meaning are called <a href=\"http://en.wikipedia.org/wiki/Code_(semiotics)\">codes</a>. Hence ‘encoding’ can also refer to this kind of code, as well.</p>\n<p>Hence, if we combine Fielding, Galloway, and Saussure, the word ‘client’ is a signifier that holds a relationship to the signified concept ‘a triggering process’ (among other things), that forms the sign ‘client.’ This is encoded by the protocol HTTP.</p>\n<h2>‘connects life-forms’</h2>\n<p>From the Massumi marriage example:</p>\n<blockquote>\n<p>Say “I do,” and your life will never be the same. Your legal, social, and familial status instantly changes, along with your entire sexual, psychological, and financial economy. You have been pronounced man and wife. You may file a joint tax return.</p>\n</blockquote>\n<p>This connection is related to the previous coded relationship. In this case, ‘connects’ is an active verb rather than a passive noun. It not only <em>creates</em> a connection, but it ‘connects’. It not only relates two objects, it <em>is</em> the relation.</p>\n<p>We spent most of the last section discussing connections, though, so let’s get to the interesting part: life. Galloway is very explicit about the role protocol plays with life:</p>\n<blockquote>\n<p>…protocol is an affective, aesthetic force that has control over “life itself.”</p>\n</blockquote>\n<p>In <a href=\"http://www.amazon.com/The-Exploit-Networks-Electronic-Mediations/dp/0816650446\">the Exploit</a>, Galloway discusses the relationship between life and protocol as well:</p>\n<blockquote>\n<p>While the first type of network (Internet protocols) is silicon based and may use biological concepts (intelligent agents, artificial life, genetic algorithms), the second (DNA algorithms) is fully biological and yet recodes itself in computational terms (biology as computation, as opposed to evolution)</p>\n</blockquote>\n<p>The relationship between life and software is an interesting one.</p>\n<p><img src=\"http://upload.wikimedia.org/wikipedia/commons/e/e5/Gospers_glider_gun.gif\" alt=\"http://upload.wikimedia.org/wikipedia/commons/e/e5/Gospers_glider_gun.gif\" /></p>\n<p>In Conway’s game of life, one builds cellular automata and lets them go. There are a few simple rules that govern their reproduction, death, and life. This protocol governs all that happens within the game. Someone playing the game is an impotent god, as there is no room in the protocol for communicating with creations. Likewise, creations have no way of perceiving an ‘outside’ of the game.</p>\n<p>Galloway references this quote by Frederick Cohen from <a href=\"http://www.amazon.com/Course-Computer-Viruses-Professional-Computing/dp/0471007684\">A Short Course on Computer Viruses</a>:</p>\n<blockquote>\n<p>I personally believe that reproducing programs are living beings in the information environment.</p>\n</blockquote>\n<p>Viruses, Conway cells, <a href=\"http://thesims.com/en_us/home\">Sims</a>, and all other kinds of ‘artificial life’ exist inside of computers. They’re native to protocol. But what about actual, real life?</p>\n<p>Galloway reaches to Norbert Wiener, a mathematician who was the originator of cybernetics, for this:</p>\n<blockquote>\n<p>if one views the world in terms of information, then there is little instrumental difference between man and machine since both are able to affect dynamic systems via feedback loops. In this way the cybernetic system of man and machine are born. Its virtues are balance, self-regulation, circularity, and control. In a word, protocol.</p>\n</blockquote>\n<p>Deleuze also forms a connection between life and information in <a href=\"http://www.n5m.org/n5m2/media/texts/deleuze.htm\">“Postscript on the Societies of Control”</a>:</p>\n<blockquote>\n<p>The numerical language of control is made of codes that mark access to information, or reject it. We no longer find ourselves dealing with the mass/individual pair. Individuals have become “dividuals,” and masses, samples, data, markets, or “banks.”</p>\n</blockquote>\n<p>A <em>dividual</em> is a play on the process of <a href=\"http://en.wikipedia.org/wiki/Individuation\">individuation</a>. Galloway elaborates:</p>\n<blockquote>\n<p>Deleuze’s neologism comes from the word ‘individuate.’ Dividuation would thus be the opposite: the dissolving of the individual entity into distributed networks of information.</p>\n</blockquote>\n<p>Let’s use a personal example to demonstrate life submitting itself to dividuation in order to interface with protocol: me. <a href=\"http://steveklabnik.com/\">This is my personal website</a>. In it, my public face on the Internet, I’ve split myself up into five broad categories, and link out to all of the various sub-components. Examining the last category, you can see how if you just follow me on twitter <a href=\"https://twitter.com/steveklabnik\">;)</a>, you will not get the whole picture. You could also <a href=\"https://github.com/steveklabnik\">check out my code on GitHub</a>, but that wouldn’t be me, either. Even the sum of these two components would not tell you everything about me. Yet I can’t possibly put <em>me</em> on the network. I must break ‘me’ into a form that conforms with protocol.</p>\n<p>Finally, there’s an important relationship with Foucault’s concept of ‘biopolitics.’</p>\n<blockquote>\n<p>Foucault defines biopolitics as “the endeavor, begun in the eighteenth century, to rationalize the problems presented to governmental practice by the phenomena characteristic of a group of living human beings constituted as a population: health, sanitation, birthrate, longevity, race.”</p>\n</blockquote>\n<p>Obviously, dividuating a population into informational components would be relevant to protocol. And, in fact, life provides a path to resist protocol. Deleuze in his book <em>Foucault</em> (because that’s not confusing…):</p>\n<blockquote>\n<p>When power becomes bio-power resistance becomes the power of life, a vital power that cannot be confined within species, environment or the paths of a particular diagram.</p>\n</blockquote>\n<p>Galloway expands:</p>\n<blockquote>\n<p>Is life resistance a way of engaging with distributed forms of protological management? Part III of this book, “Protocol Futures,” answers yes. While the new networked technologies have forced an ever more reticent public to adapt to the control structures of global capital, there has emerged a new set of social practices that inflects or otherwise diverts these protological flows toward the goal of a utopian form of unalienated social life.</p>\n</blockquote>\n<h2>To conclude</h2>\n<p>Protocol has an intimate and complex relationship with language. In many ways, it is a particular language, with all of the expressive power and limiting abilities that that brings.</p>\n<p>The implications of this language, though, will have to wait for another time.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/random-ruby-tricks--struct-new/",
"title": "Random Ruby Tricks: Struct.new",
"description": null,
"url": "https://steveklabnik.com/writing/random-ruby-tricks--struct-new/",
"published": "2012-09-01T00:00:00.000Z",
"updated": "2012-09-01T00:00:00.000Z",
"content": "<p>One of my favorite classes in Ruby is <code>Struct</code>, but I feel like many Rubyists don’t know when to take advantage of it. The standard library has a lot of junk in it, but <code>Struct</code> and <code>OStruct</code> are super awesome.</p>\n<h2>Struct</h2>\n<p>If you haven’t used <code>Struct</code> before, here’s <a href=\"http://www.ruby-doc.org/core-1.9.3/Struct.html\">the documentation of Struct from the Ruby standard library</a>.</p>\n<p>Structs are used to create super simple classes with some instance variables and a simple constructor. Check it:</p>\n<pre><code>Struct.new(\"Point\", :x, :y) #=> Struct::Point\norigin = Struct::Point.new(0,0) #=> #<struct Struct::Point x=0, y=0>\n</code></pre>\n<p>Nobody uses it this way, though. Here’s the way I first saw it used:</p>\n<pre><code>class Point < Struct.new(:x, :y)\nend\n\norigin = Point.new(0,0)\n</code></pre>\n<p>Wait, what? Inherit…from an instance of something? Yep!</p>\n<pre><code>1.9.3p194 :001 > Struct.new(:x,:y)\n => #<Class:0x007f8fc38da2e8> \n</code></pre>\n<p><code>Struct.new</code> gives us a <code>Class</code>. We can inherit from this just like any other <code>Class</code>. Neat!</p>\n<p>However, if you’re gonna make an empty class like this, I prefer this way:</p>\n<pre><code>Point = Struct.new(:x, :y)\norigin = Point(0,0)\n</code></pre>\n<p>Yep. Classes are just constants, so we assign a constant to that particular <code>Class</code>.</p>\n<h2>OStruct</h2>\n<p><code>[OStruct](http://ruby-doc.org/stdlib-1.9.3/libdoc/ostruct/rdoc/OpenStruct.html)</code>s are like <code>Struct</code> on steroids. Check it:</p>\n<pre><code>require 'ostruct'\n\norigin = OpenStruct.new\norigin.x = 0\norigin.y = 0\n\norigin = OpenStruct.new(:x => 0, :y => 0)\n</code></pre>\n<p><code>OStruct</code>s are particularly good for configuration objects. Since any method works to set data in an <code>OStruct</code>, you don’t have to worry about enumerating every single option that you need:</p>\n<pre><code>require 'ostruct'\n\ndef set_options\n opts = OpenStruct.new\n yield opts\n opts\nend\n\noptions = set_options do |o|\n o.set_foo = true\n o.load_path = \"whatever:something\"\nend\n\noptions #=> #<OpenStruct set_foo=true, load_path=\"whatever:something\">\n</code></pre>\n<p>Neat, eh?</p>\n<h2>Structs for domain concepts</h2>\n<p>You can use <code>Struct</code>s to help reify domain concepts into simple little classes. For example, say we have this code, which uses a date:</p>\n<pre><code>class Person\n attr_accessor :name, :day, :month, :year\n\n def initialize(opts = {})\n @name = opts[:name]\n @day = opts[:day]\n @month = opts[:month]\n @year = opts[:year]\n end\n\n def birthday\n \"#@day/#@month/#@year\"\n end\nend\n</code></pre>\n<p>and we have this spec</p>\n<pre><code>$:.unshift(\"lib\")\nrequire 'person'\n\ndescribe Person do\n it \"compares birthdays\" do\n joe = Person.new(:name => \"Joe\", :day => 5, :month => 6, :year => 1986)\n jon = Person.new(:name => \"Jon\", :day => 7, :month => 6, :year => 1986)\n\n joe.birthday.should == jon.birthday\n end\nend\n</code></pre>\n<p>It fails, of course. Like this:</p>\n<pre><code>$ rspec\nF\n\nFailures:\n\n 1) Person compares birthdays\n Failure/Error: joe.birthday.should == jon.birthday\n expected: \"7/6/1986\"\n got: \"5/6/1986\" (using ==)\n # ./spec/person_spec.rb:9:in `block (2 levels) in <top (required)>'\n\nFinished in 0.00053 seconds\n1 example, 1 failure\n\nFailed examples:\n\nrspec ./spec/person_spec.rb:5 # Person compares birthdays\n</code></pre>\n<p>Now. We have these two birthdays. In this case, we know about why the test was failing, but imagine this failure in a real codebase. Are these month/day/year or day/month/year? You can’t tell, it could be either. If we switched our code to this:</p>\n<pre><code>class Person\n attr_accessor :name, :birthday\n\n Birthday = Struct.new(:day, :month, :year)\n\n def initialize(opts = {})\n @name = opts[:name]\n @birthday = Birthday.new(opts[:day], opts[:month], opts[:year])\n end\nend\n</code></pre>\n<p>We get this failure instead:</p>\n<pre><code>$ rspec\nF\n\nFailures:\n\n 1) Person compares birthdays\n Failure/Error: joe.birthday.should == jon.birthday\n expected: #<struct Person::Birthday day=7, month=6, year=1986>\n got: #<struct Person::Birthday day=5, month=6, year=1986> (using ==)\n Diff:\n @@ -1,2 +1,2 @@\n -#<struct Person::Birthday day=7, month=6, year=1986>\n +#<struct Person::Birthday day=5, month=6, year=1986>\n # ./spec/person_spec.rb:9:in `block (2 levels) in <top (required)>'\n\nFinished in 0.00092 seconds\n1 example, 1 failure\n\nFailed examples:\n\nrspec ./spec/person_spec.rb:5 # Person compares birthdays\n</code></pre>\n<p>We have a way, way more clear failure. We can clearly see that its our days that are off.</p>\n<p>Of course, there are other good reasons to package related instance variables into <code>Struct</code>s, too: it makes more conceptual sense. This code represents our intent better: a Person has a Birthday, they don’t have three unrelated numbers stored inside them somehow. If we need to add something to our concept of birthdays, we now have a place to put it.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/new-blog--words/",
"title": "New blog: Words",
"description": null,
"url": "https://steveklabnik.com/writing/new-blog--words/",
"published": "2012-08-31T00:00:00.000Z",
"updated": "2012-08-31T00:00:00.000Z",
"content": "<p>I’ve started a new blog: <a href=\"http://words.steveklabnik.com/\">Words</a>.</p>\n<p>You may have noticed that it’s a part of <a href=\"https://svbtle.com/\">the Svbtle blogging network</a>. I’m super excited about this, but you may be confused. Let me explain.</p>\n<h2>My relationship with writing</h2>\n<p>I’ve been involved in Internet communities for most of my life. One of the nice things about that is that it’s forced me to express myself in writing for a very, very long time.</p>\n<p>I don’t remember what site this is, but there was one that generated a PDF of your top 100 comments on Hacker News. The idea was “You’ve already written a book!” You can see this kind of thinking in many “Blog to book” books that are out there, but I found this really powerful. My top 100 comments span 19 pages; I was (at one point) number 60 on the top of Hacker News by karma. And I really was only active on HN for about a year, February 2010 to 2011. I spent much more time on /., Reddit, and a half dozen forums. I bet if you compiled a list of everything I ever wrote in a <code><textarea name=\"comment\" id=\"comment\"></code>, I would have written at least 3 novels by now. They’d be shit, of course, but the point is that if you’re active at all on the read/write web, then you’ve probably written a ton.</p>\n<p>As a side point, I really enjoyed the anonymity that posting on the web gave me. I could be 13, yet arguing with people who were 30! And they’d actually take me seriously, since they didn’t know I wasn’t an adult! Of course, the only way they wouldn’t know is if I posted proper English… grammar nazis helped me out, in a way.</p>\n<p>Anyway.</p>\n<h2>Practice makes perfect</h2>\n<p>Of course, this means that at first, I <em>wasn’t</em> practiced at writing, and that means that <a href=\"http://games.slashdot.org/comments.pl?sid=87657&cid=7604346\">it was any good…</a> That’s the only way that you get better, though: write and write and write and write. And get feedback on that writing. Originally, it was just people commenting back. Then upvotes. I’m pretty sure that I’ve averaged over two or three comments a day on the Internet over the past decade or so. If you include tweets, I bet the median is about the same, but the mean would go up to 6 or 8.</p>\n<p>Regardless, I tend to be very off-the-cuff when writing. When I’m done writing this blog post, I will type <code>:wq<enter>git commit -am \"add 'words'\"<enter>git push origin,heroku<enter><alt-tab><f5><f5><f5><f5><f5><f5><f5><f5></code> . Then tweet. I end up doing very little proofreading of the things I write, and that means that sometimes, my writing suffers. I’ve settled into a local maximum: I’m Good Enough at writing that people like what I write, and I can quickly type something out and get it up there. But I don’t think I’m getting any <strong>better</strong> at writing. And I’d like to.</p>\n<h2>Gamification and Exhibitionism</h2>\n<p>I’ve also come to realize something about myself over the years: I respond really well to gamification, and am somewhat of an exhibitionist. Or at the very least, an extrovert. These two things in combination are a powerful mixture.</p>\n<p>When I started playing StarCraft II, I knew I wanted to get better. So I set up <a href=\"http://www.twitch.tv/emmagoldmansc\">an account on Twitch</a>. I sometimes stream games there. I’m in Silver league, which means I’m not that good. But knowing that someone may be watching makes me want to be better at it. I sometimes <a href=\"https://vimeo.com/37978370\">release videos of me live coding</a>. Almost everything <a href=\"https://github.com/steveklabnik\">I code is open source</a>. Knowing that others might be watching makes me want to be at my best.</p>\n<p>Gamification can help with this, too. Monotonically increasing numbers are terribly addicting, and trigger something deep down in my brain. I’ve been known to refresh my retweet count or my upvote count every so often just to watch that number get higher. Combining this with public-ness means that I can compare myself to everyone else with a number too, and I can get to the top of the leaderboard! I’m terribly competitive at gaming, and quantification is the first step towards gamification and competition. I’ve been watching the Quantified Self movement from afar specifically because I’m afraid that I’ll do nothing but ‘self-improve’ and then not use that improvement for anything worthwhile.</p>\n<h2>Svbtle</h2>\n<p>If you haven’t heard, <a href=\"http://dcurt.is/codename-svbtle\">Svbtle</a> is Dustin Curtis’ blogging network. I remember reading that post when Dustin started it, and this was the part that stuck out to me:</p>\n<blockquote>\n<p>The goal is simple: when you see the Svbtle design, you should know that the content is guaranteed to be great.</p>\n</blockquote>\n<p>I’m always looking for good things to read on the Internet, so I kept this idea in the back of my head. When I saw a Svbtle blog, I raised my expectations. I can really enjoy bad things as long as I go into them expecting that they’re bad; I’m not the type of person that only enjoys high-brow stuff. And generally, blogs have low-brow expectations for me.</p>\n<p>Anyway, after a couple of months, I noticed that I really did think that most posts on Svbtle were pretty damn good. At the very least, they weren’t bad in the way that many blog posts are: super flamebait, shit-talking, or half-baked. I also kept looking at my blog, and realized that many of my posts are sort of trivial, or not really up to my current standards, or have those kinds of flaws. And finally, I <em>hate</em> CSS. Like, I’m no good at it. At all. And I’m sick of messing around with my own style. But most blogging software sucks…</p>\n<p>So I applied.</p>\n<p>It’s sort of funny to talk about hating startups and centralized internet companies, then go from my own Jekyll blog to a hosted solution. There are a few huge wins for me with Svbtle, though:</p>\n<p>Ultimately, it’s all posted on my own domain via the magic of CNAME. This means that in the future, I can re-host everything myself if I want to. I’ve already done this previously when I moved from Posterous to my own blog, and even have <a href=\"https://github.com/steveklabnik/blog/blob/master/redirects.rb#L32\">redirects.rb</a> already set up. Secondly, Svbtle will send me an archive of all my posts if I ask, which means that I’m never going to lose the data. I will probably keep my own backups locally as well. But in the event that Svbtle goes south, I’m not locked in.</p>\n<p>Getting kicked off of Svbtle for having bad content would make me really upset. If it’s a blog network that’s high quality, and my writing isn’t up to snuff, that would make me sad. Time to level up! Now that I’m there, I want to stay there. This gives me lots of motivation to write high quality things, and to improve. And Svbtle provides tools to help me become a better writer, like vetting and copy editing. That’s a pretty huge attraction to me.</p>\n<p>Finally, there’s light gamification with the whole Kudos thing. I know that I’ll be trying to get as many as possible with each post, too, and that it can give me reasonable feedback in some fashion. Not as good as well-written responses, but it’s easier to give someone internet points than it is to write a full rebuttal.</p>\n<h2>Two blogs</h2>\n<p>I’m still going to keep this blog going; it’s gonna shift to something that’s more about code and less about politics. Words will be more politics and theory with a de-emphasis on code. This blog will be more like a Tumblr in that I’ll post more often, do less work on each post, and make the good but quick. Words will become a place where I’ll take my time, post less frequently, but really work on the polish.</p>\n<p>I hope you’ll enjoy continuing to read the things I write about, here or there.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/protological-control-an-introduction/",
"title": "Protological Control: an introduction",
"description": null,
"url": "https://steveklabnik.com/writing/protological-control-an-introduction/",
"published": "2012-08-13T00:00:00.000Z",
"updated": "2012-08-13T00:00:00.000Z",
"content": "<blockquote>\n<p>“I finished by pointing out that, like scientists, people in the web development community had to be ethically and morally aware of what they were doing. I thought this might be construed as a bit out of line by the geek side, but the people present were the ones now creating the Web, and therefore were the only ones who could be sure that what the systems produced would be appropriate to a reasonable and fair society.”Tim Berners-Lee, “Weaving the Web,” p86.</p>\n</blockquote>\n<p>Humans are social creatures. Regardless of your position on how human society should be, we all acknowledge that it does exist. While there may be some who live a lonely life in the wilderness, they still live within the larger social organization of a state. And even though those few people do exist, they are a tiny, tiny portion of the overall population. For the vast majority of humanity, society is a reality.</p>\n<p>What do I mean when I say ‘society?’ For this, I turn to Marx:</p>\n<blockquote>\n<p>“Society does not consist of individuals, but expresses the sum of interrelations, the relations within which these individuals stand.”Karl Marx, Grundrisse der Kritik der Politischen Ökonomie</p>\n</blockquote>\n<p>Regardless of your opinions of Marx, I also don’t think this is a particularly controversial opinion: society is made up of our relationships with each other. A single person does not a society make, but our connections to each other. These connections are the foundational aspect of society; they are atomic.</p>\n<p>Society is a fractal, rhizomatic object: there is not only the broader ‘society,’ but many smaller, overlapping, societies. These societies are a set of these relations within a given context. These contexts provide us with guidelines of what sorts of qualities these relations possess. For example, when amongst friends at home, a person will act differently than when around those same set of friends in a public place. Often, “social awkwardness” is a transgression of one of these boundaries; someone assumes that they exist in a different social space than those they’re interacting with.</p>\n<blockquote>\n<p>In an extreme view, the world can be seen as only connections, nothing else. We think of a dictionary as the repository of meaning, but it defines words only in terms of other words. I liked the idea that a piece of information is really defined only by what it’s related to, and how it’s related. There really is little else to meaning. The structure is everything. There are billions of neurons in our brains, but what are neurons? Just cells. The brain has no knowledge until connections are made between neurons. All that we know, all that we are, comes from the way our neurons are connected.Tim Berners-Lee, “Weaving the Web,” p12</p>\n</blockquote>\n<p>As it turns out, Tim isn’t the only person to find this line of thinking interesting. Ferdinand de Saussure was a linguist in the early 1900s who developed the idea of “structural linguistics,” and in the 1950’s and 1960’s, a French anthropologist named Claude Lévi-Strauss took his concepts and applied them to anthropology, birthing Structuralism. Others followed his lead and used this mode to develop an analysis of psychology, sociology, and more.</p>\n<p>Societies exist naturally, but can also be man-made. Any opening of a new avenue for interpersonal relationships creates a new society amongst the multitude. The web is an example of this, and websites are a second, inner socius. The major difference between artificial and natural societies is not one of effect, but of cause. The end result is the same, but the initial conditions for the formation of the society determine the acceptable rules for the given relations that exist within it. Therefore, potential creators of said social enclosures should understand the power that they wield, and use said power to create the exact form of society they wish it make, with deliberate strokes.</p>\n<h2>Society has a very clean mirror</h2>\n<p>When people create, they end up creating something that reflects themselves and the societies that they live in. Linguistic relativity in even its weak form implies that language shapes who we are, and it is such with our creations. Many <em>ex</em> <em>nihilo</em> creation myths touch upon this property, like Adam’s receipt of God’s breath, or Atum from ancient Egypt, who not only created himself (?!?) but then other gods (first from his own shadow) and then man from his tears.</p>\n<p>Here’s a fun, terrible example of this happening:</p>\n<p>Amusingly, even though the tone is wrong, this YouTube user nails it:</p>\n<p>(image lost to time)</p>\n<p>Exactly. They didn’t test it on darker skinned folks. Of course, it may not be malicious, but this certainly meant that no dark-skinned people were involved in the production of this device, the whole way up to shipping it. If they were, it would have been caught early on before it was even beginning manufacture.</p>\n<p>Because we grow up in a society, and we create new societies through technology, it stands to reason that society and our own biases influence the societies that we create. Even beyond that, if we create societies, we should be able to use the same tools that we use to analyze naturally occurring ones on artificial ones.</p>\n<p>So how do we analyze societies?</p>\n<h2>It’s all about power and control</h2>\n<p>Remember Structuralism? Well, it has some issues, and not just the opposition that Saussure faced from Chomsky. Along comes a bunch of French philosophers, and they have beef. I won’t get into it, except to mention the name of one: Michel Foucault. He wrote this book titled “Surveiller et punir: Naissance de la Prison,” which has some awkward title translation issues, and so ends up as “Discipline and Punish: The Birth of the Prison” in English. Long time readers of my blog will remember <a href=\"http://blog.steveklabnik.com/posts/2011-01-22-a-few-notes-on-foucault\">my notes</a> on this book, but the basics are as such:</p>\n<h3>Foucault</h3>\n<p>Feudal society was predicated on divine right. The body of a sovereign is literally the body of God, and since God also created everything, the sovereign is everything. Therefore, if you commit a crime, you commit it against the body of the sovereign, and therefore, he must exact punishment against your body in kind. Hence torture.</p>\n<p>Eventually, though, torture became socially inconvenient, and people started challenging the very idea of divine right, and wanted democracy in some form. Therefore, the power to punish would need to take on a new form to survive in this new society. That mechanism is called “discipline.” Discipline (specifically of the body) is how control manifests itself in society. One of the great advances of Taylorism, for example, was studying how the bodies of assembly line workers operated and streamlining their motions.</p>\n<p>Foucault also illustrates that control and power in modern society mirror Jeremy Bentham’s concept of the ‘Panopticon,’ which is a prison in a circular shape by which the rooms can all observe each other. The observation would help to keep them all in line, since the all-seeing tower in the middle would be able to observe everyone at all times.</p>\n<p><img src=\"/img/2012-08-13/panopticon.png\" alt=\"panopticon\" /></p>\n<p>I found this concept amusing, <a href=\"http://en.wikipedia.org/wiki/Litchfield_Towers\">here</a> are the dorms I stayed in during my time at the University of Pittsburgh.</p>\n<h3>Those in charge get to make the rules?</h3>\n<p>Anyway, all of that implies this: the people who are ‘in charge’ are so because they get to define the rules. Most people would say this the other way around, but that’s wrong: You don’t get to make the rules because you’re in charge, you’re in charge because you get to make the rules. Regardless, what does it mean to make the rules?</p>\n<p>In a simple sense, discipline is pretty easy to grasp: there’s a king (or boss). He makes the rules. Done. And it’s the same with our ‘creation of societies’ from above. When you make a society, you make it, so you get to make the rules! This ‘rule-making’ property is everywhere in software: operating systems are absolutely about creating a society of software programs on your computer. APIs are all about the connections between various systems, and the web, with its hyperlinks, even more explicitly so.</p>\n<p>That’s why this stuff matters to programmers. We create societies all the time, and need to be careful of the kinds of societies that we create. We also participate in many, and need to be sure that we participate in the kinds of societies that we agree with.</p>\n<p>One problem: we’ve already moved beyond discipline society, twice.</p>\n<h2>Deleuze and Guattari</h2>\n<p>Oh these two! What a pair! If you picked up a random copy of either volume of “Capitalism and Schizophrenia” without knowing what it was, you would be oh so confused. I mean, I vaguely knew what I was getting into, and I was still confused.</p>\n<p>Deleuze specifically wrote an essay called “Postscript on the Societies of Control”. You can find it online <a href=\"http://www.n5m.org/n5m2/media/texts/deleuze.htm\">here</a>.</p>\n<blockquote>\n<p>These are the societies of control, which are in the process of replacing disciplinary societies. “Control” is the name Burroughs proposes as a term for the new monster, one that Foucault recognizes as our immediate future.</p>\n</blockquote>\n<p>He also discusses it in <a href=\"http://www.amazon.com/Negotiations-1972-1990-Gilles-Deleuze/dp/0231075812\">Negotiations</a>.</p>\n<p>Control societies are different from disciplinary societies because discipline is all about confinement. Control is a slippery beast compared to discipline:</p>\n<blockquote>\n<p>Enclosures are molds, distinct castings, but controls are a modulation, like a self-deforming cast that will continuously change from one moment to the other, or like a sieve whose mesh will transmute from point to point.</p>\n</blockquote>\n<p>Discipline is direct, while control is indirect. I won’t get into control too much more here, because we’re past that too; control’s time on this earth was fleeting. Now we live in the era of protocol.</p>\n<h2>Galloway</h2>\n<p>Alexander Galloway is an Associate Professor of Media, Culture, and Communication at NYU, and every programmer should read him. Specifically, his book <a href=\"http://www.amazon.com/Protocol-Control-Exists-Decentralization-Leonardo/dp/0262072475\">Protocol: How control exists after decentralization</a>. Galloway takes Deleuze’s work and applies it to our modern computer systems, and terms that ‘protocol.’</p>\n<blockquote>\n<p>Protocol is a system of distributed management.Protocol facilitates peer-to-peer relationships between autonomous entities.Protocol is anti-hierarchy and anti-authority.Protocol engenders localized decision making, not centralized.Protocol is robust, flexible, and universal.Protocol can accommodate massive contingency.Protocol is the outcome (not the antecedent) of distributed power.Galloway, “Protocol”, p82</p>\n</blockquote>\n<p>Sounds awesome, right? Sounds like the web.</p>\n<blockquote>\n<p>I saw one [advocate of informational organization systems] after the next shot down because the developers were forcing them to reorganize their work to fit the system. I would have to create a system with common rules that would be acceptable to everyone. This meant as close as possible to no rules at all.This notion seemed impossible until I realized that the diversity of different computer systems could be a rich resource– something to be represented, not a problem to be eradicated. The model I chose for my minimalist system was hypertext.Tim Berners-Lee, “Weaving the Web,” p15.</p>\n</blockquote>\n<p>Tim recognized that centralization was the root of exploitation:</p>\n<blockquote>\n<p>It also shows how a technical decision to make a single point of reliance can be exploited politically for power and commercially for profit, breaking the technology’s independence from those things, and weakening the web as a universal space.Tim Berners-Lee, “Weaving the Web,” p129.</p>\n</blockquote>\n<p>Tim saw that control was an issue:</p>\n<blockquote>\n<p>Whether inspired by free-market desires or humanistic ideals, we all felt that control was the wrong perspective. … Technically, if there was any centralized point of control, it would rapidly become a bottleneck that restricted the web’s growth, and the web would never scale up. Its being ‘out of control’ was very important.Tim Berners-Lee, “Weaving the Web,” p99</p>\n</blockquote>\n<p>Galloway, however, contends something else, and this is the crux of it:</p>\n<blockquote>\n<p>Thus it is an oversight for theorists like Lawrence Lessig (despite his strengths) to suggest that the origin of Internet communication was one of total freedom and lack of control. Instead, it is clear to me that the exact opposite of freedom – that is, control – has been the outcome of the last forty years of developments in networked communications. The founding principle of the Net is control, not freedom. Control has existed from the beginning.Perhaps it is a different type of control than we are used to seeing. It is control borne from high degrees of technical organization (protocol) not this or that limitation on individual freedom or decision making (fascism).To put it another way, in order for protocol to enable radically distributed communications between autonomous entities, it must employ a strategy of universalization, and of homogeneity. It must be anti-diversity. It must promote standardization in order to enable openness. It must organize peer groups into bureaucracies like the IETF in order to create free technologies.In short, control in distributed networks is not monolithic. It proceeds in multiple, parallel, contradictory, and often unpredictable ways. It is a complex of interrelated currents and counter-currents.Galloway, “Protocol”, p141-143.</p>\n</blockquote>\n<p>This is the kind of control that startups exert on the rest of the world. This is the kind of control that’s hard to see coming, because it’s inexact. This is the kind of control that lets you <em>feel</em> like you have freedom, even when you don’t. This is the kind of control that <em>likes</em> open standards, and uses them against you.</p>\n<p>Discipline is Microsoft. Protocol is Google. Protocol is Facebook. Protocol is Twitter.</p>\n<p>It’s in this tradition that I critique App.net and GitHub.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/dayz/",
"title": "DayZ",
"description": null,
"url": "https://steveklabnik.com/writing/dayz/",
"published": "2012-07-25T00:00:00.000Z",
"updated": "2012-07-25T00:00:00.000Z",
"content": "<p>I’ve recently come across one of the most addicting, interesting experiences I’ve ever had online: <a href=\"http://dayzmod.com/\">DayZ</a>. I hesitate to call it a ‘video game,’ because while it is a mod for a first person shooter game, it basically ignores all common wisdom about what makes for a good game. I think that the broader implications are also interesting across other domains: ‘common wisdom’ isn’t always correct. Sometimes communities reach local maximums, and something drastic needs to be done to break out of the rut. So what is DayZ, anyway?</p>\n<p>Essentially, DayZ is a simulation of a zombie apocalypse.</p>\n<p>You wake up on a beach with essentially nothing in your pockets. This means that you have no way to defend yourself, and no supplies. Check out the bottom right of that screenshot: See that canteen and knife and fork? Those are water and food indicators, and if you don’t eat or drink often enough, you’ll faint from fatigue, and even lose blood (which is a proxy for health, and is the middle indicator). So you have to sneak around until you can find a hatchet, food, and water. See those two things on the upper right? That’s how much audio and visual ‘noise’ you give off. Right now, this player is basically invisible, so they’re safe.</p>\n<p>The game takes realism very seriously. There’s the food and water bit, but this also isn’t some Left 4 Dead run and gun bullshit: if you take damage, you’ll start to bleed, and will need to bandage yourself or bleed out. If you get shot, you’re basically dead. You <em>might</em> be able to take two bullets from small arms. I already mentioned the food and water. You can break bones, which seriously limits mobility, and will probably cause the shakes. Better take some painkillers! There are a few vehicles, but they’re all in disrepair, so find some parts, fix them up, and then stop and put some gas in them! Voice chat only works for a 40 yard radius around where you stand, so you can’t talk to people that are far away. You don’t have a map at first, so you won’t know where the hell you are until you find one. Of course, you can read the street signs, so obviously, you’ll know when you’re in черногорск. ;). (That’s ‘Cherno’ as it’s affectionately called by players, because ‘Chernogorsk’ is too long to say.) Oh, and when you die, you die. You lose all your stuff. This also adds to the crazy tenseness of the game. There’s actually 24 hours on each server, so if you play on one with your GMT offset, it’ll be light when it’s light and dark when it’s dark outside.</p>\n<p>The biggest thing, though, is that this is a true sandbox: there are no ‘levels’ or ‘experience points’ or ‘missions’ or ‘objectives.’ You just play. Average survival time is about 45 minutes, and as you live longer, the tension goes way, way up. Because there are no missions, you develop a real attachment to the ‘emergent stories’ that come out of the game, which is the big, big draw for me. Here’s an example:</p>\n<p>I spawned just outside of Электрозаводск (‘Electro’), and one of my roommates had been playing for about twenty minutes, and was in Пута (“Puta”), just north. He had found a barn with two hatchets in it, and obviously, as a new player, that’s a big deal. Электро and Пута are about <a href=\"http://dayzdb.com/map#4.099.124\">two kilometers away from each other</a>, so he decided to wait for me. I ran up over that hill, and as I got closer, I didn’t pay close enough attention, and a zombie started chasing me. I decided to sprint to the barn, so I’d at least be able to defend myself, and Kyle decided to brace himself: my running would obviously draw more zombies, and the barn had three doors: we couldn’t cover them all. If we weren’t careful, this could be the end.</p>\n<p>As I sprint through the door, Kyle steps into it, and swings his hatchet. “Holy shit dude, more coming, hurry up!” I quickly located the other one, as Kyle took out the second undead trying to eat us. I stepped up too, and covered the other door. Thirty seconds of frantic hatchet-ing later, we were safe. But Kyle was starting to get thirsty, and Электро is really dangerous. We searched the rest of the settlement, but no luck. No water to be found. We decided that rather than risk the big costal town, it’d be best to <a href=\"http://dayzdb.com/map#4.085.109\">make the two klick journey north to Могилевка</a>. While trying to locate the road, Kyle attracted the attention of another zombie, and after dispatching it, found an empty watter bottle! Luckily, Пута has a public well, so he could fill it and drink up. Problem solved.</p>\n<p>Now, if this scenario had been a scripted part of some zombie RPG, I wouldn’t have cared about it at all. I’ve now told this story three times in person, and it doesn’t get old. This tense five minutes emerged naturally out of putting players in the world and letting them go. As a result, since they’re <em>your</em> stories, you care about them a lot more. People end up sharing these stories, too: <a href=\"http://www.youtube.com/watch?v=0oMdOPF8a8U\">Check out this Jeep theft on YouTube</a>. He spends the first 3 minutes or so telling the story.</p>\n<h2>Ignoring video game tropes</h2>\n<p>So what really makes DayZ special? There are lots of sandbox games, and there are lots of zombie games. Well, for me, it’s that DayZ ignores much about what makes for conventional games.</p>\n<p>See, I’ve been playing video games for most of my life. I started with an NES long ago, and spent many, many, many hours with various games across genres. I play fewer games these days, and actually took a pretty big break from them over the last two years or so. I would go to GameStop, look at what was offered, shrug, and walk out. It’s not that I’m growing up <a href=\"http://www.theesa.com/facts/pdfs/ESA_EF_2011.pdf\">(the average gamer is 37)</a>, it’s that I’ve played all these games before. I spent over a hundred hours with Diablo II, so why bother with Diablo III?</p>\n<p>When there’s a game that breaks out of these standard genres and franchises, I get excited. I really got into music games until Activision drove Guitar Hero into the ground, Minecraft was incredibly addicting, and now DayZ with its emergent gameplay, focus on realism, and tense atmosphere.</p>\n<h3>Games need to be accessible</h3>\n<p>DayZ does not have mass appeal. Its uncompromising approach to realism means that it will alienate many people. You will die often, and it will feel random much of the time. You will get killed by other players. You’ll break a bone miles away from the nearest hospital with no morphine, and have to crawl for an hour or die.</p>\n<h3>Games should be polished</h3>\n<p>Frankly, <a href=\"http://www.dayzwiki.com/wiki/index.php?title=Known_Bugs\">DayZ is buggy as shit</a>. And not trivial ones, either:</p>\n<blockquote>\n<p>Dropping Items can cause them to disappear</p>\n</blockquote>\n<p>Honestly, in many ways, it makes the game even more tense. Bad things can happen to you at any time. The world is scary.</p>\n<p>The interface sucks. This is more of an ARMAII issue, but one of my roommates was trying to get something out of another roommate’s backpack, and since you both click to manipulate the UI and click to shoot, accidentally shot him in the stomach. This totally could happen in a zombie apocalypse: fumble around with the safety off… the interface can be confusing, seemingly arbitrary, and complicated. Still totally worth it.</p>\n<p>In many ways, this is the <a href=\"http://www.jwz.org/doc/worse-is-better.html\">worse is better</a> principle.</p>\n<h3>More is better</h3>\n<p>Many games are ‘just like this other game, but with more stuff.’ DayZ has a really pleasant minimalism in many ways: the HUD is basically nonexistent, the core gameplay is pretty simple, and while there are a lot of details, most of them are not too complicated: break a leg, use some morphine. Hungry? Eat some food.</p>\n<p>But these simple components come together in a way that gives the game a large amount of depth. But that depth is not from some big crazy feature that took weeks to build.</p>\n<p>In fact, the author of the mod has <em>removed</em> entire features. There was a ‘humanity’ system at one time, and ‘bandits,’ which were players that killed other players and had their skin changed. That’s been taken out. I didn’t play the game with that particular feature in, but the author isn’t afraid to cut stuff out if it isn’t a good idea, even if he’s spent lots of time building the feature.</p>\n<h3>A good story matters</h3>\n<p>Most simulation games have some sort of goal. Even many ‘sandbox’ games, like Grand Theft Auto, have some sort of points, goals, or missions, and you’re free to ignore them, but they’re there. Minecraft is a great example of a true sandbox game, and I’m not suggesting that DayZ has a monopoly here. But it’s a genre that doesn’t have that many games in it, so another entry is very welcome; the genre isn’t as tired as many others.</p>\n<p>Story is often used as a reason for progression; as you go through the story, you get bigger, better, and generally more powerful. DayZ manages to have progression without a story; there’s sort of five broad stages to the game: pre-hatchet, hatchet, gun, tent, PvP. This is hard to do! A story provides nice justification. But DayZ doesn’t need it.</p>\n<p>Also, as I mentioned earlier, you sort of invent stories as you play. They emerge. And therefore, you care about <em>your</em> DayZ story more than the one that would have been concocted if this game was not a sandbox.</p>\n<h3>A player should be quite powerful</h3>\n<p>Often, the player is a total badass. Video games are often dripping in machismo, and so they are <strong>THE ULTIMATE DESTROYER OF WORLDS</strong>. Maybe, after killing hundreds of enemy soldiers, getting worn down over time, one gets a lucky shot and you go down.</p>\n<p>DayZ is different. You are weak. The environment is dangerous. Even though you can kill zombies, often, the noise attracts others, and they get dangerous in packs. And you can always run out of ammo…</p>\n<p>Also, other players are just as desperate and dangerous as you are. Sometimes, you’ll be friendly, but more often, you’re gonna end up getting shot. After all, if you’re not friendly, <em>I</em> could get shot, so maybe I’d better take you out first…</p>\n<h2>DayZ is awesome</h2>\n<p>Seriously, you should give it a try. ARMAII goes on sale on Steam often, I was able to pick it up for $17 the other day. If you’re bored with the mainstream of gaming, it just might be enough of a shake-up to be interesting to you.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/sparrow/",
"title": "Sparrow",
"description": null,
"url": "https://steveklabnik.com/writing/sparrow/",
"published": "2012-07-21T00:00:00.000Z",
"updated": "2012-07-21T00:00:00.000Z",
"content": "<p>I purchased Sparrow back when it was $9, and didn’t find it particularly amazing. I did like it, but ended up using the web interface more. I certainly got my $9 of value out of it, though, even though I used it for a short time.</p>\n<p>However.</p>\n<p>There’s been a lot of complaining about <a href=\"http://sprw.me/\">Sparrow’s acquisition</a>, and a lot of complaining about complaining. Most of the complaining about the complaining argues that those who complain should feel like me: You bought it, you probably got a ton of value out of it. You can’t (and shouldn’t) complain.</p>\n<p>But.</p>\n<p>Theoretically, a market works through feedback mechanisms. That’s the whole ‘free market’ argument, right? People that participate in the market come together, the magic happens, and everything gets allocated correctly. Vote with your dollars. Shop somewhere else, and poof! A store that’s too expensive won’t be able to keep its doors open. A triumph for ‘free commerce.’</p>\n<p>So why does everyone get all huffy when certain people provide the market with feedback about the way that business has been conducted?</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/why-i-don-t-like-factory_girl/",
"title": "Why I don't like factory_girl",
"description": null,
"url": "https://steveklabnik.com/writing/why-i-don-t-like-factory_girl/",
"published": "2012-07-14T00:00:00.000Z",
"updated": "2012-07-14T00:00:00.000Z",
"content": "<p>Once upon a time, I was building my First Serious Rails App. I was drawn to Rails in the first place because of automated testing and ActiveRecord; I felt the pain of not using an ORM and spending about a week on every deploy making sure that things were still okay in production. So of course, I tried to write a pretty reasonable suite of tests for the app.</p>\n<p>To gloss over some details to protect the innocent, this app was a marketplace: some people owned Things, and some people wanted to place Orders. Only certain Things could fulfill an Order, so of course, there was also a ThingType table that handled different types of Things. Of course, some Types came in multiple Sizes, so there also needed to be a Size Table and a ThingTypeSize table so that a User could own a Thing of a certain Type and a certain Size.</p>\n<p>Stating that creating my objects for tests was difficult would be an understatement.</p>\n<p>Then I read a blog post about FactoryGirl. Holy crap! This would basically save me. With one simple <code>Factory(:thing)</code> I could get it to automatically build a valid list of all that other crap that I needed!</p>\n<p>So of course, I had to write my spec for a thing:</p>\n<pre><code>describe Order do\n it \"generates quotes only from Things that are of the right size\" do\n order = Factory(:order)\n thing = Factory(:thing, :size => order.size)\n thing = Factory(:thing)\n order.quote!\n order.quote.thing.should == thing\n end\nend\n</code></pre>\n<p>This test worked. It also generated around 15 objects, saved them in the database, and queried them back out. I don’t have the code running anymore, but it was like 30-40 queries, and took a second or two to run.</p>\n<p>That was one test. I was trying to test a lot, even though I wasn’t good at test first yet, so my suite got to be pretty big. Also, sometimes my factories weren’t the best, so I’d spend a day wondering why certain things would start failing. Turns out I’d defined them slightly wrong, validations started to fail, etc.</p>\n<h2>How did we get here?</h2>\n<p>This story is one of Ruby groupthink gone awry, basically. Of course, we know that fixtures get complicated. They get complicated because we have these crazy ActiveRecord models, don’t use plain Ruby classes when appropriate, and validations make us make extra objects just to get tests to pass. Then fixtures get out of date. So let’s introduce a pattern!</p>\n<p>Of course, since we know that Factories are really useful when things get complicated, let’s make sure to use them from the start, so we don’t have to worry about them later. Everyone started doing this. Here’s how new Rails apps get started:</p>\n<pre><code>steve at thoth in ~/tmp\n$ rails new my_app\n create \n create README.rdoc\n<snip>\n create vendor/plugins/.gitkeep\n run bundle install\nFetching gem metadata from https://rubygems.org/.........\n<snip>\nUsing uglifier (1.2.6) \nYour bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.\n\nsteve at thoth in ~/tmp\n$ cd my_app \n\nsteve at thoth in ~/tmp/my_app\n$ cat >> Gemfile\ngem \"rspec-rails\"\ngem \"factory_girl_rails\"\n^D\n\nsteve at thoth in ~/tmp/my_app\n$ bundle\nFetching gem metadata from https://rubygems.org/........\n<snip>\nsteve at thoth in ~/tmp/my_app\nsteve at thoth in ~/tmp/my_app\n$ rails g rspec:install \n create .rspec\n exist spec\n create spec/spec_helper.rb\n\n$ rails g resource foo\n invoke active_record\n create db/migrate/20120714180554_create_foos.rb\n create app/models/foo.rb\n invoke rspec\n create spec/models/foo_spec.rb\n invoke factory_girl\n create spec/factories/foos.rb\n invoke controller\n create app/controllers/foos_controller.rb\n invoke erb\n create app/views/foos\n invoke rspec\n create spec/controllers/foos_controller_spec.rb\n invoke helper\n create app/helpers/foos_helper.rb\n invoke rspec\n create spec/helpers/foos_helper_spec.rb\n invoke assets\n invoke coffee\n create app/assets/javascripts/foos.js.coffee\n invoke scss\n create app/assets/stylesheets/foos.css.scss\n invoke resource_route\n route resources :foos\n\nsteve at thoth in ~/tmp/my_app\n$ rake db:migrate \n== CreateFoos: migrating =====================================================\n-- create_table(:foos)\n -> 0.0065s\n== CreateFoos: migrated (0.0066s) ============================================\n\n$ cat > spec/models/foo_spec.rb\nrequire 'spec_helper'\n\ndescribe Foo do\n it \"does something\" do\n foo = Factory(:foo)\n foo.something!\n end\nend\n^D\n\n$ bundle exec rake spec \n/Users/steve/.rvm/rubies/ruby-1.9.3-p194/bin/ruby -S rspec ./spec/controllers/foos_controller_spec.rb ./spec/helpers/foos_helper_spec.rb ./spec/models/foo_spec.rb\n*DEPRECATION WARNING: Factory(:name) is deprecated; use FactoryGirl.create(:name) instead. (called from block (2 levels) in <top (required)> at /Users/steve/tmp/my_app/spec/models/foo_spec.rb:5)\nF\n\nPending:\n FoosHelper add some examples to (or delete) /Users/steve/tmp/my_app/spec/helpers/foos_helper_spec.rb\n # No reason given\n # ./spec/helpers/foos_helper_spec.rb:14\n\nFailures:\n\n 1) Foo does something\n Failure/Error: foo.something!\n NoMethodError:\n undefined method `something!' for #<Foo:0x007f82c70c07a0>\n # ./spec/models/foo_spec.rb:6:in `block (2 levels) in <top (required)>'\n\nFinished in 0.01879 seconds\n2 examples, 1 failure, 1 pending\n\nFailed examples:\n\nrspec ./spec/models/foo_spec.rb:4 # Foo does something\n\nRandomized with seed 27300\n\nrake aborted!\n/Users/steve/.rvm/rubies/ruby-1.9.3-p194/bin/ruby -S rspec ./spec/controllers/foos_controller_spec.rb ./spec/helpers/foos_helper_spec.rb ./spec/models/foo_spec.rb failed\n\nTasks: TOP => spec\n(See full trace by running task with --trace)\n</code></pre>\n<p>Woo! Failing test! Super easy. But what about that test time?</p>\n<pre><code>Finished in 0.01879 seconds\n</code></pre>\n<p>Now, test time isn’t everything, but a hundredth of a second. Once we hit a hundred tests, we’ll be taking almost two full seconds to run our tests.</p>\n<p>What if we just <code>Foo.new.something!</code>?</p>\n<pre><code>Finished in 0.00862 seconds\n</code></pre>\n<p>A whole hundredth of a second faster. A hundred tests now take one second rather than two.</p>\n<p>Of course, once you add more complicated stuff to your factories, your test time goes up. Add a validation that requires an associated model? Now that test runs twice as slow. You didn’t change the test at all! But it got more expensive.</p>\n<p>Now, a few years in, we have these massive, gross, huge, long-running test suites.</p>\n<h2>What to do?</h2>\n<p>Now, I don’t think that test times are the end-all-be-all of everything. I really enjoyed <a href=\"http://gmoeck.github.com/2012/07/09/dont-make-your-code-more-testable.html\">this post</a> that floated through the web the other day. I think that the ‘fast tests movement’ or whatever (which I am/was a part of) was a branding mistake. The real point wasn’t about fast tests. Fast tests are really nice! But that’s not a strong enough argument alone.</p>\n<p>The point is that we forgot what testing is supposed to help in the first place.</p>\n<h2>Back to basics: TDD</h2>\n<p>A big feature of tests is to give you feedback on your code. Tests and code have a symbiotic relationship. Your tests inform your code. If a test is complicated, your code is complicated. Ultimately, because tests are a client of your code, you can see how easy or hard your code’s interfaces are.</p>\n<p>So, we have a pain in our interface: our objects need several other ones to exist properly. How do we fix that pain?</p>\n<p>The answer that the Rails community has taken for the last few years is ‘sweep it under the rug with factories!’ And that’s why we’re in the state we’re in. One of the reasons that this happened was that FactoryGirl is a pretty damn good implementation of Factories. I do use FactoryGirl (or sometimes Fabrication) in my request specs; once you’re spinning up the whole stack, factories can be really useful. But they’re not useful for actual unit tests. Which I guess is another way to state what I’m saying: we have abandoned unit tests, and now we’re paying the price.</p>\n<p>So that’s what it really boils down to: the convenience of factories has set Rails testing strategies and software design back two years.</p>\n<p>You live and learn.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/how-can-i-contribute-to-ruby-on-rails-/",
"title": "How can I contribute to Ruby on Rails?",
"description": null,
"url": "https://steveklabnik.com/writing/how-can-i-contribute-to-ruby-on-rails-/",
"published": "2012-07-05T00:00:00.000Z",
"updated": "2012-07-05T00:00:00.000Z",
"content": "<p>After RailsConf this year, I joined the Rails Issue Team. This means that I help triage issues that people have filed, try to reproduce errors, and point core team members at ones that are most important. Since doing that, a few people have asked me how to get started, so I decided to draw up my thoughts here.</p>\n<p>Note that there is also an <a href=\"http://edgeguides.rubyonrails.org/contributing_to_ruby_on_rails.html\">official Rails Guide on contribution</a> too.</p>\n<h2>Who helps with Rails?</h2>\n<p>First up is the Rails Core team: <a href=\"http://rubyonrails.org/core/\">http://rubyonrails.org/core/</a> Core team members essentially have the authority to merge code into the main Rails codebase. Someone from core reviews every patch that goes into Rails.</p>\n<p>Issues team: This group of people has the authority to tag, close, and edit tickets, but can’t actually merge code.</p>\n<p>Documentation team: A few awesome people contribute by writing documentation rather than just code. Docs are super important, so all these people are also awesome.</p>\n<p>Everyone else!: Rails is open source after all, and 1224 people have contributed at least one patch.</p>\n<p>You can find everyone’s commits and count here: <a href=\"http://contributors.rubyonrails.org/\">http://contributors.rubyonrails.org/</a></p>\n<h3>About payment</h3>\n<p>Nobody is paid to just work on Rails itself. Everyone is a volunteer. Rails runs on a ‘scratch your own itch’ policy. This has two big effects:</p>\n<ol>\n<li>Pull requests can sit sometimes. Pull requests must be merged by a Core member, so after your patch is submitted, you’re basically waiting on someone from a group of about 12 to decide ‘hey, I should merge some pull requests today’ and get on it. There are currently 168 pull requests open on Rails, the oldest of which is three months since its last comment.</li>\n<li>Feature requests may simply fall on deaf ears. If you have a suggestion for Rails, and you don’t want to implement it, you need to convince someone to write the code for you. But it’s not as if the core team has a list of requested features that it goes over when working. Of course, they have a list of the features that they want to add, but it’s not like it’s comprehensive from every feature requested. That said, it’s possible that your idea may spark the attention of someone, so sharing a feature request is valuable. But don’t feel entitled here.</li>\n</ol>\n<h2>I found a bug, what should I do?</h2>\n<p>File a ticket on the <a href=\"https://github.com/rails/rails/issues\">Rails issue tracker</a> and give us as much information as possible.</p>\n<p>Note that only 3.2 is getting bug fixes. 3.1 and earlier are security fix only.</p>\n<p>Good: Clear bug report, listed versions of Rails and/or Ruby and backtrace if applicable</p>\n<p>Better: Steps to reproduce, maybe with a sample app.</p>\n<p>Best: All of the above with a patch that fixes the problem.</p>\n<h2>I’ve found a security vulnerability, what should I do?</h2>\n<p>Don’t submit something to the tracker. Instead, check out <a href=\"http://rubyonrails.org/security\">this page</a> which outlines the security policy. Basically, send an email to <a href=\"mailto:[email protected]\">[email protected]</a>.</p>\n<h2>I have a feature request, what should I do?</h2>\n<p>Please don’t file an Issue; it’s a bad medium for discussing these kinds of things. It also makes it harder to keep track of what’s a bug and what’s a wish for future changes. Please post to the <a href=\"https://groups.google.com/forum/?fromgroups#!forum/rubyonrails-core\">rails-core mailing list</a> instead. Your idea is much more likely to be seen and discussed there.</p>\n<p>You may get no response from the list. People tend to say nothing if they’re not interested. It happens.</p>\n<h2>I want feedback on an idea before I work up a patch</h2>\n<p>Same as a feature request, please ping the <a href=\"https://groups.google.com/forum/?fromgroups#!forum/rubyonrails-core\">rails-core mailing list</a>.</p>\n<h2>I’ve written a patch, now what?</h2>\n<p>Submit a pull request with your patch. Someone from core will review it, and may ask you to modify what you’ve done. Eventually it will either get merged into Rails or rejected.</p>\n<h2>I want to write docs, now what?</h2>\n<p>Don’t sumbit them as issues to the main Rails repo, instead, check out <a href=\"http://weblog.rubyonrails.org/2012/3/7/what-is-docrails/\">docrails</a>.</p>\n<h2>I want to help, but don’t know what to work on!</h2>\n<p>We try to tag each issue with the part of Rails it affects. If you don’t know where to get started, pick your favorite part of Rails and sort the issues by that part. For example <a href=\"https://github.com/rails/rails/issues?labels=asset+pipeline&page=1&sort=updated&state=open\">here are all the issues relating to the asset pipeline</a>.</p>\n<p>Try to reproduce the bug locally, write a test that fails, write a patch, boom!</p>\n<h2>How do I set up my local machine to work on Rails</h2>\n<p>Check the Rails Guide <a href=\"http://edgeguides.rubyonrails.org/contributing_to_ruby_on_rails.html#running-the-test-suite\">here</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/introducing-issue2pr/",
"title": "Introducing issue2pr",
"description": null,
"url": "https://steveklabnik.com/writing/introducing-issue2pr/",
"published": "2012-06-29T00:00:00.000Z",
"updated": "2012-06-29T00:00:00.000Z",
"content": "<p>Ever had an issue on GitHub, and then you fixed the bug? Opened a brand new pull request, which becomes a new issue, and then you have to remember to link the two via a comment, or manually go back and close the issue as well?</p>\n<p>Turns out that GitHub has the power to turn Issues into Pull Requests in the API, but not in the web UI. So I wrote a tiny web app that lets you use the API call yourself: <a href=\"http://issue2pr.herokuapp.com/\">http://issue2pr.herokuapp.com/</a></p>\n<p>Of course, the code is up on GitHub, here: <a href=\"https://github.com/steveklabnik/issue2pr\">https://github.com/steveklabnik/issue2pr</a></p>\n<p>Issues (and pull requests) welcome!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/show-solidarity-to-all-who-resist/",
"title": "Show solidarity to all who resist",
"description": null,
"url": "https://steveklabnik.com/writing/show-solidarity-to-all-who-resist/",
"published": "2012-05-31T00:00:00.000Z",
"updated": "2012-05-31T00:00:00.000Z",
"content": "<p>I found this on a flyer in my neighborhood.</p>\n<hr />\n<p>On may 19th, three people associated with the Occupy movement were arrested in Chicago, just before the anti-NATO protest. Two infiltrators had been working with them for months, trying to persuade them to escalate their tactics – just as in the recent Cleveland bomb scare. Whether the infiltrators would have succeeded in pushing them further than they wanted to go, we’ll never know. Desperate to make an arrest that would intimidate the NATO protesters, divide the Occupy movement from its most radical members, and create an impressive spectacle, the police attacked. Their most substantial evidence is beer-making supplies at the house the NATO 3 were arrested in, which they ridiculously claim are bomb materials.</p>\n<p>While we are confident that the NATO 3 will beat these ludicrous charges, they are being held in solitary confinement on 1.5 million dollars bond each. It often comforts prisoners to know that people on the outside love them and are thinking about them, so a solidarity demonstration was called for May 24th, in St Louis, MO. It was noissy, vibrant, nighttime march, enjoyed by the people it passed. “Solidarity with all who resist!” was painted on a business window, and, of course, a businessman got angry, and, of course, there was an altercation; the cops showed up, hit people, were allegedly hit in return, and so on. By the end of the night, ten people were in jail – one charged with five felony counts of assault, property damage and riot, as well as two misdemeanors. Others are charged with felony resisting, interference, and misdemeanor riot. Upon their release, we learned that while being interrogated one prisoner had a knife held to their throat, that they were choked and lifted off the ground and punched in the chest for non-compliance. One comrade, who is facing the most serious charges, was brought to the interrogation room where a detective repeatedly struck him in the face, breaking his nose while he was shackled to the floor. They continued to make threats of violence if he continued to “not comply.”</p>\n<p>Fuck.</p>\n<p>This.</p>\n<p>Shit.</p>\n<p>As a wise woman once said, “If you hit me and I hit back, we still ain’t even.” There is no way in society as it exists today to have a fair fight with the police; they have vast, systemic power, and will always be better equipped and better financed than we are. Nor can we let them hurt anyone, particularly those close to us, with complete impunity, without a response from us. We must fight, but we must fight asymmetrically. If we must respond, we are forced into reaction instead of attack, let it be waves of reaction – if strange and inadequate, funny and off putting – that build off each other to create an ocean of solidarity. Solidarity actions in response to the repression of solidarity actions with people we’ve never met accused of crimes that never occurred – this is utterly bizarre, but this is our life.</p>\n<p>You should care about this. You should care that the government is making tactical strikes against its enemies. Maybe you don’t think of yourself as an enemy of the government or capitalism; maybe you would never consider it publicly, in a quiet, accepted way or otherwise. Still, think about your life: do you feel safer, happier, more fulfilled because of your job, the police, the taxes you pay? Even if you feel those things on a surface level, do you feel like something is missing, that you could be living in a way that is more glorious and beautiful? Do you see people around you who suffer, or read books about terrible things in America’s past, or hear of people in far-off lands whose lives seem impossibly hard, and wish that you could do something about it – something more meaningful than giving to a charity or voting in the election? We suggest a first step:</p>\n<p>Show solidarity to all who resist.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/finale/",
"title": "Finale",
"description": null,
"url": "https://steveklabnik.com/writing/finale/",
"published": "2012-05-30T00:00:00.000Z",
"updated": "2012-05-30T00:00:00.000Z",
"content": "<p>This morning, my mother woke up and found that my father had passed away in the night.</p>\n<p>Thanks for all the love and support you all have shown me over the last two months. It’s been hard, but you all have made it easier.</p>\n<p>Don’t be worried if you don’t see me around a lot for the next few days. Just trying to take some time off from the hustle to help my family heal. And to just play video games and be a bit mindless.</p>\n<p>No, seriously. Thank you.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/shoes-4-work-is-starting--get-involved/",
"title": "Shoes 4 work is starting: get involved!",
"description": null,
"url": "https://steveklabnik.com/writing/shoes-4-work-is-starting--get-involved/",
"published": "2012-05-25T00:00:00.000Z",
"updated": "2012-05-25T00:00:00.000Z",
"content": "<p>As you know, I’ve been doing a lot of work over the last few years to keep _why’s <a href=\"http://shoesrb.com/\">Shoes</a> project going. A few other intrepid individuals have been doing a lot as well. We’re starting to work on Shoes 4, and we’d love your help.</p>\n<h2>A little about Shoes</h2>\n<p>If you didn’t know, Shoes is a GUI toolkit for Ruby. It uses blocks heavily:</p>\n<pre><code>Shoes.app do\n para \"Push the button\"\n \n button \"Me! Me!\" do\n alert \"Good job.\"\n end\nend\n</code></pre>\n<p>Super simple. You get native widgets on all platforms.</p>\n<h2>Shoes 4</h2>\n<p>Shoes 4 is a total re-write of Shoes. It’s being done in JRuby. There are a few related projects:</p>\n<ul>\n<li>ShoesSpec: An executable specification for Shoes implementations, so that others can co-exist. A few other Shoes projects exist already, and a unified interface will be great.</li>\n<li>Shoes-mocks: Sort of the opposite, a dummy set of Shoes that you can use to test applications written in Shoes.</li>\n<li>The website: A new website is being made as well. Help here would be just as awesome as work on Shoes itself.</li>\n</ul>\n<h2>What we need</h2>\n<p>Of course, pull requests are always welcome. But if you use Shoes or want to, please get involved in <a href=\"http://github.com/shoes/shoes4/issues\">the discussions</a> about what Shoes 4 will look like on the bug tracker. Feel free to open an Issue with any topic for discussion. Thoughts from users, current and potential, are always awesome!</p>\n<p>Let’s all have fun developing Shoes 4!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/mixins--a-refactoring-anti-pattern/",
"title": "Mixins: a refactoring anti-pattern",
"description": null,
"url": "https://steveklabnik.com/writing/mixins--a-refactoring-anti-pattern/",
"published": "2012-05-07T00:00:00.000Z",
"updated": "2012-05-07T00:00:00.000Z",
"content": "<p>I spend an unusually large amount of time thinking about interactions between what I call ‘past me’ and ‘future me.’ It seems that my life changes significantly every few years, and I like to ground myself by imagining how odd it would be if ‘current me’ could tell ‘past me’ things like ‘<a href=\"http://www.oscon.com/oscon2012/public/schedule/detail/24042\">Someday, you’ll be speaking at OSCON</a>.’</p>\n<p>It’s not always general life stuff, though, it’s often a trick or technique with code. How many times have you learned something new, and then had that terrible urge to go back and re-write all of that terrible, terrible code you’ve written in the past? This happens to me constantly.</p>\n<p>So here, let me show you something I realized that I used to do wrong all the time.</p>\n<p>Let’s do it via a quiz. Of course, because I’m framing it this way, you’ll get the answer, but seriously, just play along.</p>\n<h2>Two factorizations</h2>\n<p>Which code is better factored?</p>\n<pre><code># app/models/post.rb\nclass Post\n include Behaviors::PostBehavior\nend\n\n# app/models/behaviors/post_behavior.rb\nmodule Behaviors\n module PostBehavior\n attr_accessor :blog, :title, :body\n\n def initialize(attrs={})\n attrs.each do |k,v| send(\"#{k}=\",v) end \n end\n\n def publish\n blog.add_entry(self)\n end\n\n # ... twenty more methods go here\n end\nend\n</code></pre>\n<p>or</p>\n<pre><code>class Post < ActiveRecord::Base\n def publish\n blog.add_entry(self) \n end\n\n # ... twenty more methods go here\nend\n</code></pre>\n<p>One line of reasoning asserts that the first example is better. Here’s why: all of the behavior is in one place, and the persistence is in another. A clear win.</p>\n<p>Another line of reasoning asserts that the second is better. Here’s why: it’s significantly simpler. A clear win.</p>\n<p>So which is right?</p>\n<h2>They’re both wrong</h2>\n<p>Both of these justifications are wrong. As far as I’m concerned, these two classes are equally complex, but because there is a lot more ceremony in version one, version two is preferable. Assertion one is wrong because both are still in the same place, they’re just in two different files. Assertion two is wrong because it’s not simpler than version one, it’s equivalently complex.</p>\n<h2>Measures of complexity</h2>\n<p>Whenever we refactor, we have to consider what we’re using to evaluate that our refactoring has been successful. For me, the default is complexity. That is, any refactoring I’m doing is trying to reduce complexity. So how do I measure complexity?</p>\n<p>One good way that I think about complexity on an individual object level comes from security practices: the ‘attack surface.’ We call this ‘encapsulation’ in object oriented software design. Consider these two objects:</p>\n<pre><code>class Foo\n def bar\n end\n\n def baz\n end\nend\n\nclass Bar\n def quxx\n end\nend\n</code></pre>\n<p>Bar has one less way into the object than Foo does. Hence the idea of a ‘surface’ that an attacker would have access to. A bigger surface needs more defense, more attention, and is harder to lock down.</p>\n<p>Note that I said ‘one less.’ If you said “Foo has a surface of two, and Bar has a surface of one,” you’d be wrong. Luckily, Ruby provides us with a simple way to calculate attack surface. Just use this monkey patch: (I’m not suggesting that you actually use this monkey patch.)</p>\n<pre><code>class BasicObject\n def attack_surface\n methods.count\n end\nend\n</code></pre>\n<p>This gives us</p>\n<pre><code>1.9.2p318 > Foo.new.attack_surface\n => 59 \n1.9.2p318 > Bar.new.attack_surface\n => 58\n</code></pre>\n<p>Oooooh yeah. <code>Object.new.methods.count</code> is 57… we forgot about those. This is why mixins do not really reduce the complexity of your objects. Both versions of Post above have the same number of methods. How many, you ask?</p>\n<pre><code>> Post.new.methods.count\n => 337\n > Post.new.methods.count - Object.new.methods.count\n => 236\n</code></pre>\n<p>In this case, my <code>Post</code> is an object that inherits from <code>ActiveRecord::Base</code> and defines nothing else. ActiveRecord adds in 236 methods to our object. That is a pretty huge increase.</p>\n<h2>Reducing complexity through encapsulation</h2>\n<p>Several people have been dancing around this issue for a while by saying something like ‘Consider ActiveRecord a private interface, and never use it from outside of your class.’ They feel this exact pain, but enforce it through social means rather than reducing the complexity of objects. Now, there are good reasons to do that, so I don’t think it’s categorically wrong, but it certainly is a compromise.</p>\n<p>An implementation of DataMapper would be able to solve this problem,</p>\n<pre><code>class Post\n def initialize(title, body)\n @title = title\n @body = body\n end\nend\n\nclass PostMapper\n def save(post)\n @database.insert(:posts, [post.title, post.body])\n end\nend\n\nPostMapper.new.save(Post.new(\"Hello DataMapper\", \"DM rocks\"))\n</code></pre>\n<p>or the repository pattern:</p>\n<pre><code>class PostRepository\n def first\n title, body = @database.select(:posts, \"LIMIT 1\")\n Post.new(title, body)\n end\nend\n</code></pre>\n<p>All incredibly theoretical and pretend. But note that the <code>Post</code> in both instances has no methods defined on it that deal with persistence whatsoever. We’ve used encapsulation to hide all of the details of the database in the database, the details of the post logic in the <code>Post</code>, and we use the Mapper or Repository to mediate between the two.</p>\n<h2>Using Mixins the right way</h2>\n<p>Mixins are awesome, though, and can be used to reduce duplication. They’re for cross-cutting concerns:</p>\n<pre><code>class Post\n def summary\n \"#{author}: #{body[0,200]}\"\n end\nend\n\nclass Comment\n def summary\n \"#{author}: #{body[0,200]}\"\n end\nend\n</code></pre>\n<p>moves to</p>\n<pre><code>class Post\n include Summarizer\nend\n\nclass Comment\n include Summarizer\nend\n\nmodule Summarizer\n def summary\n \"#{author}: #{body[0,200]}\"\n end\nend\n</code></pre>\n<p>This example isn’t fantastic, but whatever. We’ve eliminated some duplication, even if we didn’t reduce our method surface. This is a win along a different axis, we just shouldn’t fool ourselves that we’ve made our classes ‘simpler.’</p>\n<h2>This isn’t the only metric</h2>\n<p>Now, I don’t mean to imply that method count is the be-all, end-all metric of complexity. Real software development is too complicated to wrap up in one tiny number. However, I do think there’s a strong correlation between small, focused classes and easy to understand software systems. Aggregates can be used to make it easier to understand the use of a cluster of smaller objects. But that’s another post.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/on-the-eve-of--m1gs/",
"title": "On the eve of #m1gs",
"description": null,
"url": "https://steveklabnik.com/writing/on-the-eve-of--m1gs/",
"published": "2012-04-30T00:00:00.000Z",
"updated": "2012-04-30T00:00:00.000Z",
"content": "<p>Walking down the street, a woman cries for help. She’s blind, and the curb is large, and she’s unsure of herself. Knowing it’s not safe, she seeks the help of a stranger to guide her on her way. Three or four people pass by, but then someone stops, gives her their arm, and guides her safely to her destination.</p>\n<p><strong>HOOOONK!!!</strong> A large truck is blocked from going down the street. A Lexus with out-of-state-plates is in the way, and the driver is standing outside of the car, looking agitated. A passer-by stops to ask if they can help, and the man frantically tries to explain. His English is poor, but not that poor, but due to the stress of the situation, he’s losing his words. Luckily, the stranger speaks a little of his native tongue. The astute observer would notice that this tiny bit of familiarity causes a massive change in demeanor in this chaotic moment. 311 is called, a translator is found, and the man will be able to move his car soon.</p>\n<p>These moments, these tiny acts of selflessness and mutual aid, are what revolutions are truly made of.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/cards-for-dad--follow-up/",
"title": "Cards for dad - follow-up",
"description": null,
"url": "https://steveklabnik.com/writing/cards-for-dad--follow-up/",
"published": "2012-04-22T00:00:00.000Z",
"updated": "2012-04-22T00:00:00.000Z",
"content": "<p>Hey everyone! It’s been a few weeks, and I wanted to update you guys on the card situation.</p>\n<p>First of all, thank you so much. Every single one of you who sent cards, thank you! It was truly amazing to see so much support. I was even more surprised that I basically put my address on the internet and nothing particularly bad happened. Zero trolls. Restored some of my faith in humanity.</p>\n<p>We ended up with about 500 cards. My dad is well enough to stay at home, but the chemo is pretty rough, so he has good days and bad days. I go home to visit about once a week, but we don’t really know how well and if it’s really working yet, so no actual understanding of how long he has. Regardless, it all made a big impact, so thank you again.</p>\n<p>Here’s some pictures:</p>\n<p><img src=\"http://i.imgur.com/mcQSG.jpg\" alt=\"http://i.imgur.com/mcQSG.jpg\" /></p>\n<p><img src=\"http://i.imgur.com/GmmhJ.jpg\" alt=\"http://i.imgur.com/GmmhJ.jpg\" /></p>\n<p><img src=\"http://i.imgur.com/RV4k8.jpg\" alt=\"http://i.imgur.com/RV4k8.jpg\" /></p>\n<p><img src=\"http://i.imgur.com/QVTJK.jpg\" alt=\"http://i.imgur.com/QVTJK.jpg\" /></p>\n<p><img src=\"http://i.imgur.com/YxXqj.jpg\" alt=\"http://i.imgur.com/YxXqj.jpg\" /></p>\n<p>Oh, and I didn’t read very many of them personally, since they were for my dad, not for me. My parents read and cherished all of them, though. They said that you all said too many nice things about me, so thank you for all of that as well.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/building-storeengine/",
"title": "Building StoreEngine",
"description": null,
"url": "https://steveklabnik.com/writing/building-storeengine/",
"published": "2012-04-05T00:00:00.000Z",
"updated": "2012-04-05T00:00:00.000Z",
"content": "<p>So I said I was gonna build antisocialne.ws. But I’m changing gears instead: I’d rather do this project, which has a better spec and needs a reference implementation.</p>\n<p><a href=\"http://vimeo.com/39850969\">Building StoreEngine</a> from <a href=\"http://vimeo.com/steveklabnik\">Steve Klabnik</a> on <a href=\"http://vimeo.com/\">Vimeo</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/about-antisocialne-ws/",
"title": "About antisocialne.ws",
"description": null,
"url": "https://steveklabnik.com/writing/about-antisocialne-ws/",
"published": "2012-04-03T00:00:00.000Z",
"updated": "2012-04-03T00:00:00.000Z",
"content": "<p><a href=\"http://vimeo.com/39718634\">About Antisocialne.ws</a> from <a href=\"http://vimeo.com/steveklabnik\">Steve Klabnik</a> on <a href=\"http://vimeo.com/\">Vimeo</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/anti-capitalism-101/",
"title": "Anti-capitalism 101",
"description": null,
"url": "https://steveklabnik.com/writing/anti-capitalism-101/",
"published": "2012-03-31T00:00:00.000Z",
"updated": "2012-03-31T00:00:00.000Z",
"content": "<p>People are often surprised to find out that I identify as an anti-capitalist. At least in America, many people see capitalism as inevitable, or even as the way things always were. Unfortunately, there’s so much rhetoric and confusion around this topic that it always takes a long time to break it down. I’d been meaning to write something about this topic, but then I found this really great work called <a href=\"http://libcom.org/files/Capitalism_ex-dummies_web.pdf\">Capitalism, Class, and Class Struggle for (ex) Dummies</a>. I’ve adapted it with my own thoughts and edits below. It in turn is adapted from <a href=\"http://libcom.org/library/capitalism-introduction\">this essay</a> and <a href=\"http://libcom.org/library/class-class-struggle-introduction\">this essay</a> from LibCom.org, so if you’d like to see the history, you can read those versions as well.</p>\n<p>I plan on refining this as things go along, as it’s still a bit jargony in places. Any suggestions are very welcome!</p>\n<hr />\n<h2>Capitalism: an introduction</h2>\n<p>At its root, capitalism is an economic system based on three things: wage labour (working for a wage), private ownership of the means of production (things like factories, machinery, farms, and offices), and production for exchange and profit.</p>\n<p>While some people own means of production, or capital, most of us don’t and so to survive we need to sell our ability to work in return for a wage, or else scrape by on benefits. This first group of people is the capitalist class or “bourgeoisie” in Marxist jargon, and the second group is the working class or “proletariat”. This jargon is only included so that you can make connections to other literature, we’ll stick away from such things later in this text.</p>\n<p>Capitalism is based on a simple process—money is invested to generate more money. When money functions like this, it functions as capital. For instance, when a company uses its profits to hire more staff or open new premises, and so make more profit, the money here is functioning as capital. As capital increases (or the economy expands), this is called ‘capital accumulation’, and it’s the driving force of the economy.</p>\n<p>Those accumulating capital do so better when they can shift costs onto others. If companies can cut costs by not protecting the environment, or by paying sweatshop wages, they will. So catastrophic climate change and widespread poverty are signs of the normal functioning of the system. Furthermore, for money to make more money, more and more things have to be exchangeable for money. Thus the tendency is for everything from everyday items to DNA sequences to carbon dioxide emissions—and, crucially, our ability to work—to become commodified.</p>\n<p>And it is this last point—the commodification of our creative and productive capacities, our ability to work—which holds the secret to capital accumulation. Money does not turn into more money by magic, but by the work we do every day.</p>\n<p>In a world where everything is for sale, we all need something to sell in order to buy the things we need. Those of us with nothing to sell except our ability to work have to sell this ability to those who own the factories, offices, etc. And of course, the things we produce at work aren’t ours, they belong to our bosses.</p>\n<p>Furthermore, because of long hours, productivity improvements etc, we produce much more than necessary to keep us going as workers. The wages we get roughly match the cost of the products necessary to keep us alive and able to work each day (which is why, at the end of each month, our bank balance rarely looks that different to the month before). The difference between the wages we are paid and the value we create is how capital is accumulated, or profit is made.</p>\n<p>This difference between the wages we are paid and the value we create is called “surplus value”. The extraction of surplus value by employers is the reason we view capitalism as a system based on exploitation—the exploitation of the working class. This process is essentially the same for all wage labour, not just that in private companies. Public sector workers also face constant attacks on their wages and conditions in order to reduce costs and maximise profits across the economy as a whole.</p>\n<h3>Unwaged labour</h3>\n<p>The accumulation of capital also relies on unwaged work, such as housework or domestic labour. This includes the reproduction of labour power in the form of producing and raising children—the next generation of workers—and servicing the current workforce—physically, emotionally, and sexually. This unpaid labour is predominantely carried out by women.</p>\n<p>Servicing men and children at home serves capital: by making housework and reproduction a women’s ‘natural and feminine’ process rather than work, capitalism benefits in the form of free labour. When capital pays husbands they get two workers, not one. Denying domestic labour a wage makes this work invisible, and divides the working class into waged and unwaged at the expense of both.</p>\n<h3>Competition</h3>\n<p>In order to accumulate capital, our boss must compete in the market with bosses ofother companies. They cannot afford to ignore market forces, or they will lose ground to their rivals, lose money, go bust, get taken over, and ultimately cease to be our boss. Therefore even the bosses aren’t really in control of capitalism, capital itself is. It’s because of this that we can talk about capital as if it has agency or interests of its own, and so often talking about ‘capital’ is more precise than talking about bosses.</p>\n<p>Both bosses and workers, therefore, are alienated by this process, but in different ways. While from the workers’ perspective, our alienation is experienced through being controlled by our boss, the boss experiences it through impersonal market forces and competition with other bosses.</p>\n<p>Because of this, bosses and politicians are powerless in the face of ‘market forces,’ each needing to act in a way conducive to continued accumulation (and in any case they do quite well out of it!). They cannot act in our interests, since any concessions they grant us will help their competitors on a national or international level.</p>\n<p>So, for example, if a manufacturer develops new technology for making cars which doubles productivity it can lay off half its workers, increase its profits and reduce the price of its cars in order to undercut its competition. If another company wants to be nice to its employees and not sack people, eventually it will be driven out of business or taken over by its more ruthless competitor—so it will also have to bring in the new machinery and make the layoffs to stay competitive.</p>\n<p>Of course, if businesses were given a completely free hand to do as they please, monopolies would soon develop and stifle competition which would lead to the system grinding to a halt. The state intervenes, therefore to act on behalf of the long-term interests of capital as a whole. We observed this happen in America back in the Robber Baron days.</p>\n<h3>The State</h3>\n<p>The primary function of the state in capitalist society is to maintain the capitalist system and aid the accumulation of capital. As such, the state uses repressive laws and violence against the working class when we try to further our interests against capital. For example, bringing in anti-strike laws, or sending in police or soldiers to break up strikes and demonstrations.</p>\n<p>The “ideal” type of state under capitalism at the present time is liberal democratic, however in order to continue capital accumulation at times different political systems are used by capital to do this. State capitalism in the USSR, and fascism in Italy and Germany are two such models, which were necessary for the authorities at the time in order to co-opt and crush powerful working-class movements which threatened the very continuation of capitalism.</p>\n<p>When the excesses of bosses cause workers to fight back, alongside repression the state occasionally intervenes to make sure business as usual resumes without disruption. For this reason national and international laws protecting workers’ rights and the environment exist. Generally the strength and enforcement of these laws ebbs and flows in relation to the balance of power between employers and employees in any given time and place. For example, in France where workers are more well-organised and militant, there is a maximum working week of 35 hours. In the UK, where workers are less militant the maximum is 48 hours, and in the US where workers are even less likely to strike there is no maximum at all.</p>\n<h3>History</h3>\n<p>Capitalism is presented as a ‘natural’ system, formed a bit like mountains or land masses by forces beyond human control, that it is an economic system ultimately resulting from human nature. However it was not established by ‘natural forces’ but by intense and massive violence across the globe. First in the ‘advanced’ countries, enclosures drove self-sufficient peasants from communal land into the cities to work in factories. Any resistance was crushed. People who resisted the imposition of wage labour were subjected to vagabond laws and imprisonment, torture, deportation or execution. In England under the reign of Henry VIII alone 72,000 people were executed for vagabondage.</p>\n<p>Later capitalism was spread by invasion and conquest by Western imperialist powers around the globe. Whole civilisations were brutally destroyed with communities driven from their land into waged work. The only countries that avoided conquest were those—like Japan—which adopted capitalism on their own in order to compete with the other imperial powers. Everywhere capitalism developed, peasants and early workers resisted, but were eventually overcome by mass terror and violence.</p>\n<p>Capitalism did not arise by a set of natural laws which stem from human nature: it was spread by the organised violence of the elite. The concept of private property of land and means of production might seem now like the natural state of things, however we should remember it is a man-made concept enforced by conquest. Similarly, the existence of a class of people with nothing to sell but their labour power is not something which has always been the case—common land shared by all was seized by force, and the dispossessed forced to work for a wage under the threat of starvation or even execution. As capital expanded, it created a global working class consisting of the majority of the world’s population whom it exploits but also depends on.</p>\n<h3>The Future</h3>\n<p>Capitalism has only existed as the dominant economic system on the planet for a little over 200 years. Compared to the half a million years of human existence it is a momentary blip, and therefore it would be naive to assume that it will last for ever. It is entirely reliant on us, the working class, and our labour which it must exploit, and so it will only survive as long as we let it.</p>\n<h2>Class & Class Struggle: an introduction</h2>\n<p>The first thing to say is that there are various ways of referring to class. Often, when people talk about class, they talk in terms of cultural/sociological labels. For example, middleclass people like foreign films, working class people like football, upper-class people like top hats and so on.</p>\n<p>Another way to talk about class, however, is based on classes’ economic positions. We talk about class like this because we see it as essential for understanding how capitalist society works, and consequently how we can change it.</p>\n<p>It is important to stress that our definition of class is not for classifying individuals or putting them in boxes, but in order to understand the forces which shape our world, why our bosses and politicians act the way they do, and how we can act to improve our conditions.</p>\n<h3>Class and Capitalism</h3>\n<p>The economic system which dominates the world at present is called capitalism. As mentioned earlier, capitalism is essentially a system based on the self-expansion of capital—commodities and money making more commodities and more money.</p>\n<p>This doesn’t happen by magic, but by human labour. For the work we do, we’re paid for only a fraction of what we produce. The difference between the value we produce and the amount we’re paid in wages is the “surplus value” we’ve produced. This is kept by our boss as profit and either reinvested to make more money or used to buy swimming pools or fur coats or whatever.</p>\n<p>In order for this to take place, a class of people must be created who don’t own anything they can use to make money i.e. offices, factories, farmland or other means of production. This class must then sell their ability to work in order to purchase essential goods and services in order to survive. This class is the working class.</p>\n<p>So at one end of the spectrum is this class, with nothing to sell but their ability to work. At the other, those who do own capital to hire workers to expand their capital. Individuals in society will fall at some point between these two poles, but what is important from a political point of view is not the positions of individuals but the social relationship between classes.</p>\n<h3>The Working Class</h3>\n<p>The working class then, or ‘proletariat’ as it is sometimes called, the class who is forced to work for wages, or claim benefits if we cannot find work or are too sick or elderly to work, to survive. We sell our time and energy to a boss for their benefit.</p>\n<p>Our work is the basis of this society. And it is the fact that this society relies on the work we do, while at the same time always squeezing us to maximise profit, that makes it vulnerable.</p>\n<h3>Class Struggle</h3>\n<p>When we are at work, our time and activity is not our own. We have to obey the alarm clock, the time card, the managers, the deadlines and the targets.</p>\n<p>Work takes up the majority of our lives. We may see our managers more than we see our friends and partners. Even if we enjoy parts of our job we experience it as something alien to us, over which we have very little control. This is true whether we’re talking about the nuts and bolts of the actual work itself or the amount of hours, breaks, time off etc.</p>\n<p>Work being forced on us like this compels us to resist.</p>\n<p>Employers and bosses want to get the maximum amount of work from us, from the longest hours, for the least pay. We, on the other hand, want to be able to enjoy our lives: we don’t want to be over-worked, and we want shorter hours and more pay.</p>\n<p>This antagonism is central to capitalism. Between these two sides is a push and pull: employers cut pay, increase hours, speed up the pace of work. But we attempt to resist: either covertly and individually by taking it easy, grabbing moments to take a break and chat to colleagues, calling in sick, leaving early. Or we can resist overtly and collectively with strikes, slow-downs, occupations etc.</p>\n<p>This is class struggle. The conflict between those of us who have to work for a wage and our employers and governments, who are the capitalist class, or ‘bourgeoisie’. By resisting the imposition of work, we say that our lives are more important than our boss’s profits. This attacks the very nature of capitalism, where profit is the most important reason for doing anything, and points to the possibility of a world without classes and privately owned means of production. We are the working class resisting our own existence. We are the working class struggling against work and class.</p>\n<h3>Beyond the Workplace</h3>\n<p>Class struggle does not only take place in the workplace. Class conflict reveals itself in many aspects of life. For example, affordable housing is something that concerns all working class people. However, affordable for us means unprofitable for them. In a capitalist economy, it often makes more sense to build luxury apartment blocks, even while tens of thousands are homeless, than to build housing which we can afford to live in. So struggles to defend social housing, or occupying empty properties to live in are part of the class struggle.</p>\n<p>Similarly, healthcare provision can be a site of class conflict. Governments or companies attempt to reduce spending on healthcare by cutting budgets and introducing charges for services to shift the burden of costs onto the working class, whereas we want the best healthcare possible for as little cost as possible.</p>\n<h3>The ‘Middle Class’</h3>\n<p>While the economic interests of capitalists are directly opposed to those of workers, a minority of the working class will be better off than others, or have some level of power over others. When talking about history and social change it can be useful to refer to this part of the proletariat as a “middle class”, despite the fact that it is not a distinct economic class, in order to understand the behaviour of different groups.</p>\n<p>Class struggle can sometimes be derailed by allowing the creation or expansion of the middle class—Margaret Thatcher encouraged home ownership by cheaply selling off social housing in the UK during the big struggles of the 1980s, knowing that workers are less likely to strike if they have a mortgage, and allowing some workers to become better off on individual levels, rather than as a collective. And in South Africa the creation of a black middle class helped derail workers’ struggles when apartheid was overturned, by allowing limited social mobility and giving some black workers a stake in the system.</p>\n<p>Bosses try to find all sorts of ways to materially and psychologically divide the working class, including by salary differentials, professional status, race and by gender. It should be pointed out again that we use these class definitions in order to understand social forces at work, and not to label individuals or determine how individuals will act in given situations.</p>\n<h3>Conclusion</h3>\n<p>Talking about class in a political sense is not about which accent you have but the basic conflict which defines capitalism—those of us who must work for a living vs. those who profit from the work that we do. By fighting for our own interests and needs against the dictates of capital and the market we lay the basis for a new type of society—a society without money, class, or capitalism, organised for the direct fulfilment of our needs: a Libertarian Communist society.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/birthday-cards-for-dad/",
"title": "Birthday cards for Dad",
"description": null,
"url": "https://steveklabnik.com/writing/birthday-cards-for-dad/",
"published": "2012-03-26T00:00:00.000Z",
"updated": "2012-03-26T00:00:00.000Z",
"content": "<p>Hey everyone. If you follow me on Twitter, you’ll know that I’ve been a bit absent lately. Basically, my dad has <a href=\"http://en.wikipedia.org/wiki/Angiosarcoma\">angiosarcoma</a>. It’s a really rare cancer of the blood vessels. A city the size of Pittsburgh sees two cases every ten years, roughly. It’s certainly terminal, while he had some chemotherapy a few days ago, we won’t know if it’s helped at all for another week or so, and if it works, it’ll give him another few weeks to months at best. It came on really fast: he felt some shortness of breath the first week of February, went into the hospital in week three, by week four he almost died from a blood vessel bursting due to weakness from the disease.</p>\n<p>Anyway, you all have sent me tons of well-wishes and such over the last week, and it’s truly appreciated. It’s nice to know that I have support, as I try to support my family. I’m the oldest son, so that carries a lot of responsibilities, and you guys have helped me a lot.</p>\n<p>However, if I can ask you for one more thing…</p>\n<p>My dad turns 54 on Sunday. My aunt told my mom that for his birthday, she’s going to email everyone she knows, ask them to send him a birthday card, and she’s hoping to get 100. I think that’s a pretty awesome idea, but I also happen to know that while I’ve mentioned to my dad that ‘people on the internet I know send well-wishes,’ I think it’d be super awesome to blow my aunt out of the water and give my dad a zillion cards for his last birthday.</p>\n<p>So yeah, if you have a few minutes this week, and you’re thinking of my father, send a birthday card to my house. Here’s the address:</p>\n<p><em>note</em>: Five years later, I’ve removed the address. I haven’t lived there in years, please don’t see this now and send mail to some random person kthx.</p>\n<p>Oh yeah: My dad is also Steve. I’m not a junior because he has no middle name: his mother said they were too poor to give him one.</p>\n<p>Additionally, I mean April 1. That’s his birthday. However, if you live far away and it’ll be a bit late, that’s perfectly fine, still send one. I think that ‘oh, this card came from so far away that it took some time’ is pretty awesome in its own way.</p>\n<p>I really think this would be an awesome last gift to my dad. One of the worst things about being in the hospital is that you feel cut off from everything outside of that room, and while I’ve spent almost every minute of the last 8 days in there with him, keeping your spirits up helps the healing process quite a bit.</p>\n<p>Thanks. <3 <3 <3</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/transmuting-philosophy-into-machinery/",
"title": "Transmuting Philosophy into Machinery",
"description": null,
"url": "https://steveklabnik.com/writing/transmuting-philosophy-into-machinery/",
"published": "2012-03-08T00:00:00.000Z",
"updated": "2012-03-08T00:00:00.000Z",
"content": "<p>I’m so very close to releasing the beta of <a href=\"http://designinghypermediaapis.com/\">Get Some REST</a>. However, I’m about to get on a plane to Poland, and that’s a bad time to launch things. ;) <a href=\"http://wrocloverb.com/\">wroc_love.rb</a>, here I come!</p>\n<p>Anyway, I figured I’d give you a preview of some of the content that I’ve been working on. This particular article is short, but it’s also a public response to something, and so, to whet your appetite, I figured I’d give you a little taste of what’s to come. It’s out of context, so I’m a little light on explaining why you’d do each step of the process; there is much more on that elsewhere in the book! Rather than explaing all the <em>why</em>, I just actually do it. This is one of the last articles, and it’s a fairly simple example once you’ve grokked all the fundamentals, so it’s a bit small, lengthwise.</p>\n<p>If all goes well, I’ll be releasing at wroc_love.rb. So keep your eyes peeled.</p>\n<p>Without further ado, the article.</p>\n<hr />\n<blockquote>\n<p>Kessler’s idea was, that besides the law of mutual struggle there is in nature the law of mutual aid, which, for the success of the struggle for life, and especially for the progressive evolution of the species, is far more important than the law of mutual contest. This suggestion - which was, in reality, nothing but a further development of the ideas expressed by Darwin himself in The Descent of Man, seemed to me so correct and of so great an importance, that since I became aquainted with it I began to collect materials for further developing the idea, which Kessler had only cursorily sketched in his lecture, but had not lived to develop. He died in 1881.Peter Kropotkin, “Mutual Aid: A Factor of Evolution”, p21</p>\n</blockquote>\n<p>Rob Conery is a pretty cool guy. While I always enjoy reading <a href=\"http://blog.wekeroad.com/\">his blog</a>, he’s been at battle with the Hypermedia crowd recently. It’s always good natured, though, and he means well.</p>\n<p>He recently posed <a href=\"http://wekeroad.com/2012/03/03/moving-the-philosophy-into-machinery/\">an interesting question to his blog</a>:</p>\n<blockquote>\n<p>I would like to invite the good people who have engaged with me over the last few days to jump in and write me up an API – and by way of explanation – show how their ideas can be translated into reality.</p>\n</blockquote>\n<p>Great! After all, social exchange is one of the building blocks of society:</p>\n<blockquote>\n<p>In so far as the process of exchange transfers commodities from hands in which they are non-use-values to hands in which they are use-values, it is a process of social metabolism.Karl Marx, “Capital, Volume 1”, p198</p>\n</blockquote>\n<p>Let’s apply what we’ve learned about the basics of designing hypermedia APIs. Here are his requirements:</p>\n<h2>Use Cases</h2>\n<p>This is step one: simple authentication and then consumption of basic data. The client will be HTML, JS, and Mobile.</p>\n<h3>Logging In</h3>\n<p>Customer comes to the app and logs in with email and password. A token is returned by the server upon successful authentication and a message is also received (like “thanks for logging in”).</p>\n<h3>Productions</h3>\n<p>Joe User is logged in and wants to see what he can watch. He chooses to browse all productions and can see in[sic] the app which ones he is allowed to watch and which ones he isn’t. He then chooses to narrow his selection by category: Microsoft, Ruby, Javascript, Mobile. Once a production is selected, a list of Episodes is displayed with summary information. Joe wants to view Episode 2 of Real World ASP.NET MVC3 – so he selects it. The video starts.</p>\n<h3>Episodes.</h3>\n<p>Kelly User watches our stuff on her way to work every day, and when she gets on the train [sic] will check and see if we’ve pushed any new episodes recently. A list of 5 episodes comes up – she chooses one, and watches it on her commute.</p>\n<h2>The design process</h2>\n<h3>Step 1: Evaluate Process</h3>\n<p>Fortunately, this has been done for us, in the Use Cases above. Sweet!</p>\n<h3>Step 2: Create state machine</h3>\n<p>Taking all of this into account, I drew out this state machine:</p>\n<p><img src=\"/img/2012-03-08/state-machine.png\" alt=\"state machine\" /></p>\n<p>Basically, you start at a root. Two options: see the newest list of productions, or see them all. You can filter all of them by a category. Eventually, you end up picking one. This workflow should be enough to support all of our use cases.</p>\n<h3>Step 3: Evaluate Media Type</h3>\n<p>Okay, so, he mentions this in use cases:</p>\n<blockquote>\n<p>The client will be HTML, JS, and Mobile.</p>\n</blockquote>\n<p>I’m not 100% sure what he means here: I think it’s that we’ll be building a site on this API (html), it’ll have heavy JS usage (js), and probably a mobile version or possibly a native client (mobile).</p>\n<p>Given this information, I’m going to choose JSON as a base format. Besides, our developers tend to like it. ;)</p>\n<p>After that choice is made, we also need these things:</p>\n<ul>\n<li>Filtering things means a templated query of some kind, so we’ll need some kind of templating syntax.</li>\n<li>We need lists of things as well as singular things. I like to simply this by representing singular things as a list of one item. So, lists and individual items.</li>\n<li>We also have a few attributes we need to infer from these loose requirements. No biggie. :)</li>\n</ul>\n<h3>Step 4: Create Media Types</h3>\n<p>Based on this, I’ve made up <a href=\"/tekpub-productions.html\">the application/vnd.tekpub.productions+json media type</a>. Key features, based on our evaluation:</p>\n<ul>\n<li>Each transition in our state machine has a relation attribute</li>\n<li>Each transition that needs to be parameterized has some sort of template syntax</li>\n<li>Each attribute that we need someone to know about has a definition</li>\n<li>Everything is always a list. It may contain just one item. Our client’s interface can detect this special case and display something different if it wants.</li>\n</ul>\n<h3>Step 5: Implementation!</h3>\n<p>That’s for Rob! ahaha!</p>\n<p>However, you might want a sample implementation. I’m willing to make one if there’s confusion about it, but I figured I’d put the article out and see if that’s interesting to people before I went through the effort.</p>\n<h2>What about auth?</h2>\n<p>Oh, I didn’t handle the auth case. That’s because auth happens at the HTTP level, not at the application level. HTTP BASIC + SSL or Digest should be just fine.</p>\n<h2>But, but, but… I didn’t get any verbs! Or URLS!</h2>\n<p>I know. Fielding:</p>\n<blockquote>\n<p>A REST API should spend almost all of its descriptive effort in defining the media type(s) used for representing resources and driving application state, or in defining extended relation names and/or hypertext-enabled mark-up for existing standard media types. Any effort spent describing what methods to use on what URIs of interest should be entirely defined within the scope of the processing rules for a media type (and, in most cases, already defined by existing media types). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]</p>\n</blockquote>\n<p>As well as</p>\n<blockquote>\n<p>A REST API must not define fixed resource names or hierarchies (an obvious coupling of client and server). Servers must have the freedom to control their own namespace. Instead, allow servers to instruct clients on how to construct appropriate URIs, such as is done in HTML forms and URI templates, by defining those instructions within media types and link relations. [Failure here implies that clients are assuming a resource structure due to out-of band information, such as a domain-specific standard, which is the data-oriented equivalent to RPC’s functional coupling].</p>\n</blockquote>\n<p>And</p>\n<blockquote>\n<p>A REST API should be entered with no prior knowledge beyond the initial URI (bookmark) and set of standardized media types that are appropriate for the intended audience (i.e., expected to be understood by any client that might use the API). From that point on, all application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations. The transitions may be determined (or limited by) the client’s knowledge of media types and resource communication mechanisms, both of which may be improved on-the-fly (e.g., code-on-demand). [Failure here implies that out-of-band information is driving interaction instead of hypertext.]</p>\n</blockquote>\n<p>Soooooooooo yeah.</p>\n<h2>Further Exploration</h2>\n<h3>Sources Cited</h3>\n<ul>\n<li><a href=\"http://www.amazon.com/Mutual-Aid-Evolution-Peter-Kropotkin/dp/0875580246\">“Mutual Aid: A Factor of Evolution”, Kropotkin</a></li>\n<li><a href=\"http://www.amazon.com/Capital-Critique-Political-Economy-Classics/dp/0140445684/ref=sr_1_1?s=books&ie=UTF8&qid=1331214700&sr=1-1\">“Capital, Volume I”, Karl Marx</a></li>\n<li><a href=\"http://wekeroad.com/2012/02/28/someone-save-us-from-rest/\">“Someone save us from REST”</a></li>\n<li><a href=\"http://wekeroad.com/2012/03/03/moving-the-philosophy-into-machinery/\">“Moving the Philosophy into Machinery”</a></li>\n<li><a href=\"http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven\">“REST APIs Must be Hypertext Driven”</a></li>\n<li><a href=\"/nodes/the-design-process-an-overview\">The Design Process: An Overview</a></li>\n</ul>\n<h3>Next Articles</h3>\n<p>For more on why media types are awesome, check <a>Programming the Media Type</a>.</p>\n<h3>Terms Used</h3>\n<p>None really at the moment! This article stands alone.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/hypermedia-api-reading-list/",
"title": "Hypermedia API reading list",
"description": null,
"url": "https://steveklabnik.com/writing/hypermedia-api-reading-list/",
"published": "2012-02-27T00:00:00.000Z",
"updated": "2012-02-27T00:00:00.000Z",
"content": "<p>Originally, this post was titled “A RESTful Reading List,” but please note that <a href=\"/posts/2012-02-23-rest-is-over\">REST is over. Hypermedia API is the new nomenclature.</a></p>\n<p>I’ve been doing an intense amount of research on Hypermedia APIs over the last few months, and while I didn’t save every resource I found, I’ve made a list here of the most important.</p>\n<p>I’ll be updating this post as I get new resources, so check back!</p>\n<h2>The book list</h2>\n<p>If you want to go from ‘nothing to everything,’ you can do it by reading just a few books, actually. I’m going to make all of these links affiliate. I purchase a <em>lot</em> of books myself, maybe suggesting things to you can help defray the cost of my massive backlog. All are easily searchable on Amazon if you’re not comfortable with that.</p>\n<p>Start off with <a href=\"http://www.amazon.com/gp/product/0596529260/ref=as_li_ss_tl?ie=UTF8&tag=stesblo026-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0596529260\">Restful Web Services</a> by Leonard Richardson and Sam Ruby. This book is fantastic from getting you from zero knowledge to “I know how Rails does REST by default,” which is how most people do REST. But as you know, that’s flawed. However, understanding this stuff is crucial, not only for when you interact with REST services, but also for understanding how Hypermedia APIs work differently. This baseline of knowledge is really important.It also comes in <a href=\"http://www.amazon.com/gp/product/0596801688/ref=as_li_ss_tl?ie=UTF8&tag=stesblo026-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0596801688\">cookbook form</a>. You really only need one or the other; pick whichever format you like.</p>\n<p>Next up, read <a href=\"http://www.amazon.com/gp/product/0596805829/ref=as_li_ss_tl?ie=UTF8&tag=stesblo026-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0596805829\">REST in Practice: Hypermedia and Systems Architecture</a>. This book serves as a great <em>bridge</em> to understanding Hypermedia APIs from the RESTful world. Chapters one through four read like Richardson & Ruby; yet they start slipping in the better hypermedia terminology. Chapter five really starts to dig into how Hypermedia APIs work, and is a great introduction. Chapter six covers scaling, chapter seven is an introduction to using ATOM for more than an RSS replacement, nine is about security, and eleven is a great comparison of how Hypermedia and WS-* APIs differ. All in all, a great intermediate book.</p>\n<p>To really start to truly think in Hypermedia, though, you <em>must</em> read <a href=\"http://www.amazon.com/gp/product/1449306578/ref=as_li_ss_tl?ie=UTF8&tag=stesblo026-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1449306578\">Building Hypermedia APIs with HTML5 and Node</a>. Don’t let the title fool you, as Mike says in the introduction:</p>\n<blockquote>\n<p>[HTML5, Node.js, and CouchDB] are used as tools illustrating points about hypermedia design and implementation.</p>\n</blockquote>\n<p>This is not a Node.js book. I find Node slightly distasteful, but all the examples were easy to follow, even without more than a cursory working knowledge.</p>\n<p>Anyway, the book: Mike says something else that’s really important in the intro:</p>\n<blockquote>\n<p>While the subject of the REST architectural style comes up occasionally, this book does not explore the topic at all. It is true that REST identifies hypermedia as an important aspect of the style, but this is not the case for the inverse. Increasing attention to hypermedia designs can improve the quality and functionality of many styles of distributed network architecture including REST.</p>\n</blockquote>\n<p>And, in the afterward:</p>\n<blockquote>\n<p>However, the aim of this book was not to create a definitive work on designing hypermedia APIs. Instead, it was to identify helpful concepts, suggest useful methodologies, and provide pertinent examples that encourage architects, designers, and developers to see the value and utility of hypermedia in their own implementations.</p>\n</blockquote>\n<p>I think these two statements, taken together, describe the book perfectly. The title is “Building Hypermedia APIs,” not “Designing.” So why recommend it on an API design list? Because understanding media types, and specifically hypermedia-enabled media types, is the key to understanding Hypermedia APIs. Hence the name.</p>\n<p>Mike is a great guy who’s personally helped me learn much of the things that I know about REST, and I’m very thankful to him for that. I can’t recommend this book highly enough.</p>\n<p>However, that may leave you wondering: Where’s the definitive work on how to actually build and design a Hypermedia API? Did I mention, totally unrelated of course, that <a href=\"http://designinghypermediaapis.com/\">I’m writing a book</a>? ;)</p>\n<p>Yes, it still has REST in the title. Think about that for a while, I’m sure you can see towards my plans. I’m planning on a beta release as soon as I’m recovered from some surgery this week, but I’m not sure how long that will take, exactly. So keep your eyes peeled.</p>\n<h3>Books I don’t recommend</h3>\n<p><a href=\"http://www.amazon.com/gp/product/1449310508/ref=as_li_ss_tl?ie=UTF8&tag=stesblo026-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1449310508\">REST API Design Rulebook</a>, while I haven’t actually read it, seems quite terrible. Let me copy an <a href=\"http://www.amazon.com/review/R2F4STDF7XS7U3/ref=cm_cr_dp_perm?ie=UTF8&ASIN=1449310508&nodeID=283155&tag=&linkCode=\">Amazon review</a>:</p>\n<blockquote>\n<p>The first chapters give a good feel for the vocabulary, and some good techniques for implementing REST. A lot of the ‘rules’, especially those related to basic CRUD operations, are clean and simple with useful examples.Unfortunately, the later chapters get more and more focused on specifying something called ‘WRML’, which is a concept/language newly introduced in this book as far as I can tell.Personally I would recommend ignoring the sections dealing with WRML (or keep them in mind as a detailed example of one possible way of handling some of the REST issues).As to WRML itself: yuck. It appears to be an attempt to drag in some of the unnecessary complexity of SOAP with little added benefit. Not recommended.</p>\n</blockquote>\n<p>Looking up information about WRML, I can agree, 100%. Ugh. So nasty. So this gets a big fat downvote from me.</p>\n<h3>Books I want to read</h3>\n<p>There aren’t any in this category. Should there be? You tell me!</p>\n<h2>Web resources</h2>\n<p>There are so many, this will just be a partial list for now.</p>\n<p>Of course, <a href=\"http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm\">Fielding’s dissertation</a> is essential.</p>\n<p>Roy has also written <a href=\"http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven\">REST APIs Must be Hypertext Driven</a>. This post is central for understanding why “Hypermedia API” is a much better name than REST, and why hypermedia in general is so essential.</p>\n<p>I’ve written <a href=\"http://timelessrepo.com/haters-gonna-hateoas\">this post about HATEOAS</a>. It’s a pretty basic, simple introduction to the topic.</p>\n<p>I gave a talk called <a href=\"http://vimeo.com/30764565\">Everything you know about REST is wrong</a>.</p>\n<p><a href=\"http://vimeo.com/20781278\">This talk by Jon Moore</a> was instrumental in giving me a mental breakthrough about HATEAOS. He was kind enough to <a href=\"https://gist.github.com/1445773\">share some code with me</a> that he used in the presentation as well. This is also an earlier example of the usage of “Hypermedia APIs.”</p>\n<p>A classic: <a href=\"http://tomayko.com/writings/rest-to-my-wife\">How I explained REST to my wife</a>. A great story, simple and easy to explain.</p>\n<p>Another classic is <a href=\"http://www.infoq.com/articles/webber-rest-workflow\">How to GET a cup of coffee</a>. It does a great job of explaining how to model your business processes as state machines, and then convert them to HTTP.</p>\n<p>In which Mike Mayo has a realization that HATEOAS is not simply academic: <a href=\"http://mikemayo.org/2012/how-i-learned-to-stop-worrying-and-love-rest\">http://mikemayo.org/2012/how-i-learned-to-stop-worrying-and-love-rest</a></p>\n<p>A recent resource that’s popped up is <a href=\"http://publish.luisrei.com/rest.html\">Designing a RESTful Web API</a>. It’s a nice, basic overview of lots of things.</p>\n<h2>Related resources</h2>\n<p><a href=\"http://www.amazon.com/gp/product/1449308929/ref=as_li_ss_tl?ie=UTF8&tag=stesblo026-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=1449308929\">APIs: A Strategy Guide</a> seems really interesting. This isn’t about REST or Hypermedia APIs specifically, but more making a case for why you’d want an API in the first place. Which is a related topic for all of us API enthusiasts, for sure. I haven’t read it yet, but it’s on the related list.</p>\n<p><a href=\"http://www.amazon.com/gp/product/0262572338/ref=as_li_ss_tl?ie=UTF8&tag=stesblo026-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0262572338\">Protocol: How Control Exists after Decentralization</a> is one of my favorite books ever. This book manages to be both a hard computer science book as well as referencing a broad range of philosophy, history, and other fields as well.</p>\n<p>If sentences like</p>\n<blockquote>\n<p>A perfect exmaple of a distributed network is the rhizome described in Deleuze and Guattari’s A Thousand Plateaus. Reacting specifically to what they see as the totalitarianism inherent in centralized and even decentralized networks, Deleuze and Guattari instead describe the rhizome, a horizontal meshwork derived from botany. The rhizome links many autonomous nodes together in a manner that is neither linear nor hierarchical. Rhizomes are heterogeneous and connective, that is to say, “Any point of a rhizome can be connected to anything other.”</p>\n</blockquote>\n<p>immediately followed by a hardcore, low-level diagram of the four levels of networking: application layer, transport layer, internet layer, and link layer. With full (and accurate) descriptions of TCP, IP, DNS, the SYN-ACK/SYN-ACK handshake, and HTTP following gets you all hot and bothered, you <em>need</em> to read this book. Hell, if you don’t like literature stuff, the politics in this book are amazing. This book draws the connections between <em>why</em> the APIs we’re building matter, and for that matter, the systems that we programmers create. Seriously, I can’t recommend this book enough.</p>\n<p><a href=\"http://www.amazon.com/gp/product/0801882575/ref=as_li_ss_tl?ie=UTF8&tag=stesblo026-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0801882575\">Hypertext 3.0: Critical Theory and New Media in an Era of Globalization</a> is related, and absolutely interesting. Here’s the eight main sections:</p>\n<ol>\n<li>Hypertext: An introduction</li>\n<li>Hypertext and Critical Theory</li>\n<li>Reconfiguring the Text</li>\n<li>Reconfiguring the Author</li>\n<li>Reconfiguring Writing</li>\n<li>Reconfiguring Narrative</li>\n<li>Reconfiguring Literary Education</li>\n<li>The Politics of Hypertext: Who Controls the Text?</li>\n</ol>\n<p>While not <em>directly</em> useful for those designing APIs, those of us who like to draw broad connections between disciplines will find that this book has lots of interesting parallels. Especially around the politics angle, as well as reconfiguring narrative.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rest-is-over/",
"title": "REST is over",
"description": null,
"url": "https://steveklabnik.com/writing/rest-is-over/",
"published": "2012-02-23T00:00:00.000Z",
"updated": "2012-02-23T00:00:00.000Z",
"content": "<h1>REST is</h1>\n<p><img src=\"/img/2012-02-23/restisover.png\" alt=\"Rest is OVER\" /></p>\n<hr />\n<p>Yep. Sorry to have to inform you. REST is totally over. The cool kids are moving on. We’re building “Hypermedia APIs” now. Such is life.</p>\n<h2>A lesson from the anti-globalization movement</h2>\n<p>Way back in the day, <a href=\"http://en.wikipedia.org/wiki/COINTELPRO\">COINTELPRO</a> was at the forefront of America’s fight against “subersive” organizations and individuals. One goal of COINTELPRO was to create tension and division amongst radical groups, in order to disrupt their operations. Techniques such as Concern Trolling are really effective at this kind of thing.</p>\n<p>In 2008, for the Republican National Convention in St. Paul, a document was passed around:</p>\n<blockquote>\n<p>At an anti-RNC conference held over the weekend of February 9th and 10th, a broad spectrum of groups revealed what are being called the “St. Paul Principles” of unity for resisting the 2008 Republican National Convention (RNC).</p>\n</blockquote>\n<p>This is a departure from the sectarian squabbles that have plagued past years’ anti-convention organizing. Pitting groups of differing political beliefs against each other has been a frequent tactic of state repression since the days of COINTELPRO.</p>\n<p>By drafting the principles together, the co-signing organizations are taking historic steps to actively extinguish divisiveness from their respective groups. The principles will ensure respect for the soon-to-be-permitted march on September 1 by people planning non-permitted activities, and in turn, participants in the September 1 march will adhere to the principles and do nothing to sow division among the many activists coming to the Twin Cities to protest the RNC.</p>\n<p>The principles are:</p>\n<ol>\n<li>Our solidarity will be based on respect for a diversity of tactics and the plans of other groups.</li>\n<li>The actions and tactics used will be organized to maintain a separation of time or space.</li>\n<li>Any debates or criticisms will stay internal to the movement, avoiding any public or media denunciations of fellow activists and events.</li>\n<li>We oppose any state repression of dissent, including surveillance, infiltration, disruption and violence. We agree not to assist law enforcement actions against activists and others.</li>\n</ol>\n<blockquote></blockquote>\n<p>Please draw your attention to the third principle. The reasons behind this rule are interesting:</p>\n<ol>\n<li>‘Solidarity’ kinda goes out the window when you’re busy arguing about each other’s drama.</li>\n<li>Every second you have in the media is precious. Why waste it talking about each other when you could be talking about your issue?</li>\n<li>Media will jump on any kind of debate as a weakness. The only way to make these sort of self-reflective discussions productive is to keep them internal.</li>\n</ol>\n<p>So, keeping that all in mind, why are we arguing about what REST means all the time? Yes, it’s annoying that the common usage of REST is not actually REST. Yes, it’s really hard when someone is wrong on the Internet. Yes, words do matter. However, it’s a question of what’s most productive with our time. Every moment we waste arguing over what REST means could have been spent discussing how to properly build APIs instead. But why bother being productive when we can be critical?</p>\n<h2>‘Hypermedia API’ is more clear</h2>\n<p>The real problem is that REST is just bad branding. This isn’t Roy’s fault, he wasn’t thinking about such things when trying to write his thesis. But really, from an outside perspective, a ‘real RESTful API’ does two things:</p>\n<ol>\n<li>Uses HTTP correctly.</li>\n<li>Serves hypermedia responses.</li>\n</ol>\n<p>HTTP does most of the heavy lifting in terms of bringing us into REST compliance. So why are we talking about how to transfer representations of state?</p>\n<p>The phrase “Hypermedia API” is much more direct: it’s an API where hypermedia is at the center. Those RESTish APIs could never be called ‘Hypermedia APIs,’ as it’s quite obvious that they don’t use hypermedia. It is not quite clear that they don’t transfer state and representations, though. ;)</p>\n<h2>We just really shouldn’t fight</h2>\n<p>Ultimately, it really comes back to the first point, though. Arguing about this is just wasting everyone’s time. It’s time to take a deep breath, step back, and just let REST go. Language changes. It happens. It might be a little bit sad, but life will move on. Let’s build fantastic APIs instead. <3 <3 <3</p>\n<p>Oh, and I didn’t actually kick this off, credit for that goes to Mike Amundsen and O’Reilly with <a href=\"http://www.amazon.com/Building-Hypermedia-APIs-HTML5-Node/dp/1449306578/ref=sr_1_1?ie=UTF8&qid=1330039178&sr=8-1\">Building Hypermedia APIs with HTML5 and Node</a>. Once something has an O’Reilly book about it, it’s legit. ;) Additionally, the term has been bandied about in the past, in various presentations and talks, but I feel that now’s the time to really step forward and start calling a spade a spade.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/an-api-ontology/",
"title": "An API ontology",
"description": null,
"url": "https://steveklabnik.com/writing/an-api-ontology/",
"published": "2012-02-13T00:00:00.000Z",
"updated": "2012-02-13T00:00:00.000Z",
"content": "<p><strong>NOTE</strong>: The alpha of my book on APIs is out! Check it out at <a href=\"http://designinghypermediaapis.com/\">http://designinghypermediaapis.com</a>.</p>\n<p>As I’ve done research on APIs for <a href=\"http://designinghypermediaapis.com/\">Designing Hypermedia APIs</a>, I’ve become increasingly interested in different styles of API. I currently see most real-world deployed APIs fit into a few different categories. All have their pros and cons, and it’s important to see how they relate to one other.</p>\n<p>You may find this amusing if you’ve <a href=\"http://www.ics.uci.edu/~fielding/pubs/dissertation/net_arch_styles.htm\">read some of the literature</a> on the topic, but I’ve created this list in a top-down way: APIs as black boxes, rather than coming up with different aspects of an API and categorizing them based on that. I also decided to look at actually deployed APIs, rather than theoretical software architectures.</p>\n<p>If you have an API that doesn’t fit into one of these categories, I’d love to hear about it. I’d also like to further expand these descriptions, if you have suggestions in that regard, please drop me a line, too.</p>\n<h2>HTTP GET/POST</h2>\n<h3>Synopsis:</h3>\n<p>Provide simple data through a simple GET/POST request.</p>\n<h3>Examples:</h3>\n<ul>\n<li><a href=\"http://placekitten.com/\">http://placekitten.com/</a></li>\n<li><a href=\"http://code.google.com/apis/maps/documentation/staticmaps/\">http://code.google.com/apis/maps/documentation/staticmaps/</a></li>\n<li><a href=\"http://loripsum.net/api\">http://loripsum.net/api</a></li>\n</ul>\n<h3>Description:</h3>\n<p>Simple data is made available via an HTTP GET or POST request. The vast majority of these services seem to return images, but data is possible as well.</p>\n<p>These API are technically a sub-type of *-RPC, but I feel that their lack of business process makes them feel different. It’s basically just one specific remote procedure, available over HTTP.</p>\n<h2>*-RPC</h2>\n<h3>Synopsis:</h3>\n<p>Remote procedure call; call a function over the web.</p>\n<h3>Examples:</h3>\n<ul>\n<li><a href=\"http://codex.wordpress.org/XML-RPC_Support\">http://codex.wordpress.org/XML-RPC_Support</a></li>\n<li><a href=\"http://www.flickr.com/services/api/request.rest.html\">http://www.flickr.com/services/api/request.rest.html</a></li>\n<li><a href=\"http://services.sunlightlabs.com/docs/Sunlight_Congress_API/\">http://services.sunlightlabs.com/docs/Sunlight_Congress_API/</a></li>\n</ul>\n<h3>Description:</h3>\n<p>Similiar to how structured programming is built around functions, so is RPC. Rather than call functions from your own programs, RPC is a way to call functions over the Internet.</p>\n<p>All calls are made through some sort of API endpoint, and usually sent over HTTP POST.</p>\n<p>Major flavors include XML-RPC and JSON-RPC, depending on what format data is returned in.</p>\n<p>Note that while Flickr’s API says REST, it is very clearly RPC. Yay terminology!</p>\n<h2>WS-* (or SOAP)</h2>\n<h3>Synopsis:</h3>\n<p>Serialize and send objects over the wire.</p>\n<h3>Examples:</h3>\n<ul>\n<li><a href=\"http://www.flickr.com/services/api/request.soap.html\">http://www.flickr.com/services/api/request.soap.html</a></li>\n<li><a href=\"https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&content_ID=developer/e_howto_api_soap_PayPalSOAPAPIArchitecture\">https://cms.paypal.com/us/cgi-bin/?cmd=_render-content&content_ID=developer/e_howto_api_soap_PayPalSOAPAPIArchitecture</a></li>\n<li><a href=\"http://www.salesforce.com/us/developer/docs/api/Content/sforce_api_quickstart_intro.htm\">http://www.salesforce.com/us/developer/docs/api/Content/sforce_api_quickstart_intro.htm</a></li>\n</ul>\n<h3>Description:</h3>\n<p>SOAP stands for “Simple Object Access Protocol,” and that describes it pretty well. The idea behind these APIs is to somehow serialize your objects and then send them over the wire to someone else.</p>\n<p>This is usually accomplished by downloading a WSDL file, which your IDE can then use to generate a whole ton of objects. You can then treat these as local, and the library will know how to make the remote magic happen.</p>\n<p>These are much more common in the .NET world, and have fallen out of favor in startup land. Many larger businesses still use SOAP, though, due to tight integration with the IDE.</p>\n<h2>“REST”</h2>\n<h3>Synopsis:</h3>\n<p>Ruby on Rails brought respect for HTTP into the developer world. A blending of RPC, SOAP, and hypermedia API types.</p>\n<h3>Examples:</h3>\n<ul>\n<li><a href=\"http://developer.github.com/\">http://developer.github.com/</a></li>\n<li><a href=\"https://dev.twitter.com/\">https://dev.twitter.com/</a></li>\n<li><a href=\"http://developers.facebook.com/docs/reference/api/\">http://developers.facebook.com/docs/reference/api/</a></li>\n</ul>\n<h3>Description:</h3>\n<p>Originally, REST was synonymous with what is now called “Hypermedia APIs.” However, after large amounts of misunderstanding, REST advocates are rebranding REST to “Hypermedia APIs” and leaving REST to the RESTish folks. See ’<a href=\"/posts/2012-02-23-rest-is-over\">REST is over’</a> for more.</p>\n<p>REST is basically “RPC and/or SOAP that respects HTTP.” A large problem with RPC and SOAP APIs is that they tunnel everything through one endpoint, which means that they can’t take advantage of many features of HTTP, like auth and caching. RESTful APIs mitigate this disadvantage by adding lots of endpoints; one for each ‘resource.’ The SOAPish ones basically allow you to CRUD objects over HTTP by using tooling like ActiveResource, and the RPC ones let you perform more complicated actions, but always with different endpoints.</p>\n<h2>Hypermedia</h2>\n<h3>Synopsis:</h3>\n<p>Hypermedia is used to drive clients through various business processes. The least understood and deployed API type, with one exception: the World Wide Web.</p>\n<h3>Examples:</h3>\n<ul>\n<li><a href=\"http://www.twilio.com/docs/api/rest\">http://www.twilio.com/docs/api/rest</a></li>\n<li><a href=\"http://www.spire.io/docs/tutorials/rest-api.html\">http://www.spire.io/docs/tutorials/rest-api.html</a></li>\n<li><a href=\"http://kenai.com/projects/suncloudapis/pages/Home\">http://kenai.com/projects/suncloudapis/pages/Home</a></li>\n</ul>\n<h3>Description:</h3>\n<p>Originally called REST, Hypermedia APIs take full advantage of HTTP. They use hypermedia formats to drive business processes, providing the ultimate decoupling of clients and servers.</p>\n<p>You can tell an API is Hypermedia by providing only one API endpoint, but which accepts requests at other endpoints that are provided by discovery. You navigate through the API by letting the server’s responses guide you.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/introducing-metadown/",
"title": "Introducing metadown",
"description": null,
"url": "https://steveklabnik.com/writing/introducing-metadown/",
"published": "2012-01-23T00:00:00.000Z",
"updated": "2012-01-23T00:00:00.000Z",
"content": "<p>Because I don’t have enough gems made already, I made another one last night: <a href=\"https://rubygems.org/gems/metadown\">metadown</a>.</p>\n<h2>What’s Metadown do?</h2>\n<p>This blog originally used Jekyll. When I moved it to my own personal blog implementation, I noticed something: Markdown doesn’t actually have support for adding YAML at the top of your files, like Jekyll does. I always <em>knew</em> this, I just didn’t think about it before. And I love using Markdown, so I ended up extracting my own version of this trick into Metadown.</p>\n<p>Basically, sometimes you need metadata about your markup file. YAML is a nice format for writing key/value pairs, lists, and other things… so I’ve let you smash the two of them together in one file by adding some <code>---</code>s at the top.</p>\n<h2>Gimme an Example</h2>\n<pre><code>require 'metadown'\n\ndata = Metadown.render(\"hello world\")\ndata.output #=> \"<p>hello, world</p>\"\ndata.metadata #=> \"{}\"\n\ntext = <<-MARKDOWN\n---\nkey: \"value\"\n---\nhello world\nMARKDOWN\n\ndata = Metadown.render(text)\ndata.output #=> \"<p>hello, world</p>\\n\"\ndata.metadata #=> {\"key\" => \"value\"}\n</code></pre>\n<h2>Where’s the code?</h2>\n<p>It’s implemented using a custom parser for Redcarpet, has a test suite, and works on every Ruby that’s not JRuby. You can <a href=\"https://github.com/steveklabnik/metadown\">check out the source here</a>. Issues and Pull Requests welcome. There’s at least one thing that I know I’ll need to add to it in the near future.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/moving-from-sinatra-to-rails/",
"title": "Moving from Sinatra to Rails",
"description": null,
"url": "https://steveklabnik.com/writing/moving-from-sinatra-to-rails/",
"published": "2012-01-17T00:00:00.000Z",
"updated": "2012-01-17T00:00:00.000Z",
"content": "<p>I love both Sinatra and Rails, for different reasons. I’ve heard a few different heuristics for which framework would be better for your application, but I’m not sure the answer is all that simple, really. Regardless of which is correct for your application, I haven’t seen a lot of people discussing how to move between the two frameworks.</p>\n<h2>Step One: Evaluate your test coverage</h2>\n<p>In <a href=\"http://www.amazon.com/gp/product/0131177052/ref=as_li_ss_tl?ie=UTF8&tag=stesblo026-20&linkCode=as2&camp=1789&creative=390957&creativeASIN=0131177052\">Working Effectively with Legacy Code</a> (this is an affiliate link. Does this bother you?), Michael Feathers introduces a fantastic technique called “Lean on the Compiler.” It’s on page 315, for those of you playing along at home.</p>\n<blockquote>\n<p>The primary purpose of a compiler is to translate source code into some other form, but in statically typed languages, you can do much more with a compiler. You can take advantage of its type checking and use it to identify which changes you need to make. I call this practice Leaning on the Compiler.Lean on the Compiler is a powerful technique, but you have to know what its limits are; if you don’t, you can end up making some serious mistakes.</p>\n</blockquote>\n<p>Now, in a language like Ruby, we don’t have type checking. We do have something that double checks our code correctness: tests. Just like static type checks, tests cannot <em>ensure</em> that you’ve done your transformations, but they sure as hell can help. This becomes step one in the effort to move from one framework to another: evaluate your test coverage.</p>\n<h3>Acceptance Tests are most important</h3>\n<p>When looking at your tests, first check out your acceptance tests. These are the most important, for two reasons: they’re abstracted away from the framework itself, and their purpose in life is to make sure that major functionality is working. Do you have your happy paths covered? Is there any major functionality that’s <em>not</em> covered by acceptance tests?</p>\n<p>While the happy path is a big deal, when moving to a new framework, we’re going to introduce a high chance of people encountering issues, so the sad path is also important. I personally tend to not write very many sad path acceptance tests and leave that for tests at the lower level. This is a good time to take stock of the worst of your worst paths: is there certain functionality that’s <em>absolutely</em> important to be handled 100% correctly? Then write some new acceptance tests. As with any kind of engineering project, there’s a tradeoff here: you can’t get coverage of every possible situation. Tests are supposed to give you confidence in your code, so liberally add coverage for any situation that makes you nervous.</p>\n<h3>Integration tests are important too</h3>\n<p>These tests are a bit more tricky, as they are tied into the framework that you’re using. I’m not sure that anyone writes many of these kinds of tests with Sinatra, outside of maybe model to model integration tests.</p>\n<h3>Model and View tests: not a big deal</h3>\n<p>These kinds of tests are much more of a sanity check than anything else for the purposes of this kind of move. It’s good that they pass, but really, they shouldn’t be too reliant on the framework you’re using. They’re better for making sure you’ve put things in the correct directories and haven’t forgotten anything in the move.</p>\n<p>I’m not even sure what a ‘view test’ would be in Sinatra, and they tend to be not in favor with Rails projects anyway, so that’s more of a general ‘framework to framework’ bit of advice than anything else.</p>\n<h3>Controllers don’t exist in Sinatra…</h3>\n<p>… so you don’t really write those kinds of tests. Controller tests don’t seem to be that popular in Rails-land these days either.</p>\n<h2>Step Two: Git Can help</h2>\n<p>I’ll show you how I managed transitioning files over. However, I’m really interested in the idea of using a <a href=\"http://help.github.com/subtree-merge/\">subtree merge</a> to keep being able to update the original Sinatra project while working on the new Rails project.</p>\n<p>First step is, of course, make a new branch. This transition will probably take some time, and you may want to push hotfixes into production during that period. Using master is not advised.</p>\n<p>Next, make a copy of everything and shove it in a temp directory. Then delete everything in the git repo (except the .git directory, of course) and commit that blank slate. Back up one directory above your project directory, run <code>rails new myproject</code> with your project’s name, and you’ll get a blank rails app. <code>cd myproject</code> and <code>mkdir sinatra</code>, then copy the backup you made from the temp directory into the sinatra directory. Finally, commit this.</p>\n<p>Now you’ve got a new blank Rails app with all your old code in a directory, and you can start moving things over.</p>\n<h2>Step Three: Set up your test harness</h2>\n<p>Since we’re going to allow our tests to guide our move, it pays to get tests up and running first! Depending on what testing framework you use, get it going with Rails. In our case, we were using minitest to test our code. This took a little bit of effort to get working with Rails, but there weren’t a ton of problems.</p>\n<p>As always: red, green, refactor. I’d make a simple test that doesn’t test anything, <code>assert true</code>. Make sure that your <code>rake test</code> or <code>rspec .</code> or whatever you’ll use to run the tests works and then remove the dummy test.</p>\n<p>There are two strategies for moving tests over: you can move a chunk at a time or move one at a time. Chunks are easier, but then you get commits where the build is broken. It’s really up to you and your team’s tastes: a huge test suite with a number of failures can show you how close you are to being done with the first 90% of the work. And since you’re not planning on releasing in this half-finished state, having a broken build is not <em>that</em> terrible… As always, you’re the professional: make the call.</p>\n<h2>Step Four: Move your models</h2>\n<p>Since models are the simplest to move, I like to do them first. You <em>should</em> just be able to copy over each test file and model file, though because you were using Sinatra, your models may be all in one file. This step should be largely painless, though: you’re not really relying on any framework-specific things.</p>\n<p>Model tests can still suss out problems with your environment, though, like incorrect database settings, certain environment variables…</p>\n<p>The idea to begin with the easiest thing comes from <a href=\"http://www.daveramsey.com/home/\">Dave Ramsey</a>, oddly enough. My aunt works at a bank, and when I went to college, she bought me a few of his books to help me learn about personal finance. Dave is a big fan of the ‘get rid of all debt’ school of thought, and so a large portion of his work is strategies for getting out of debt. Dave contends that paying off loans with the highest interest, while mathematically the fastest way to pay off your obligations, is not actually the best strategy. People like to see progress and are heartened by seeing it. By paying off the smallest loan first, one is much more likely to feel more positive about the headway one is making, and so it’s a much better strategy in the end.</p>\n<p>The same logic applies with this move: why start with the hard stuff? Knock out some quick wins to keep your spirits high.</p>\n<h2>Step Five: Convert your controllers</h2>\n<p>Next up: get your controllers created. This is a pretty manual process:</p>\n<pre><code>get \"foos\" do\n @foos = Foo.all\n render :\"foos/index\"\nend\n</code></pre>\n<p>becomes</p>\n<pre><code>class FooController < ApplicationController\n def index\n @foos = Foo.all\n end\nend\n</code></pre>\n<p>The biggest issue with moving all of this stuff is that Sinatra and Rails use <code>redirect</code> and <code>redirect_to</code>, respectively. So you’ll have to convert that stuff. However, I wouldn’t recommend changing things like <code>redirect \"/foos/#{id}\"</code> to <code>redirect_to foo_path(foo)</code> just yet. When dealing with legacy code, you want to change as little as possible with each step so that you know when you have introduced an error. If you try to convert things to a Rails style as well, you run the risk of introducing errors. Therefore, in all of these moves, leave the hell enough alone as much as possible. Once your code is up and running, you can gladly refactor. Just don’t do it now.</p>\n<p>Don’t forget to generate your routes, too. Sinatra’s DSL is like a combination of Rails’ routing and controllers. Set those up in this step as well.</p>\n<p>Since we don’t have tests, this part is very error-prone. Luckily, our acceptance tests will catch these issues in step seven. So give it your best shot, but don’t worry about being 100% perfect. Focus on getting the basic structural changes in place.</p>\n<p>Having tests is so nice. :/</p>\n<h2>Step Six: Move your views</h2>\n<p>This should be as simple as the models: put your views in the correct directory, and things should be golden. If you were using inline views with Sinatra, well, you shouldn’t have that many of them, so breaking them out should be pretty easy.</p>\n<h2>Step Seven: Listen to your tests</h2>\n<p>Okay! Those last moves were really awkward since we didn’t have tests to check our work. This is where the acceptance tests come in. You can move these over in batches or one at a time, but the acceptance tests will tell you where you forgot a route, if your view is missing, or if you left some little bit of Sinatra-ness in your Rails somewhere.</p>\n<p>You’re almost home! Once you get all your acceptance tests working, you should pat yourself on the back! You’re not done yet, but you’ve made a lot of progress.</p>\n<h2>Step Eight: Track your exceptions!</h2>\n<p>You should have been doing this anyway, but if you weren’t, set up some sort of mechanism to catch and handle exceptions. <a href=\"http://airbrakeapp.com/pages/home\">Airbrake</a> is one such service, but even just recording them yourself would be fine. You <em>need</em> to have this set up in some form, as you’re likely to generate errors, and examining your exceptions is the best way to track down actual problems.</p>\n<h2>Step Nine: Plan out deployment strategies</h2>\n<p>Of course, now that you’re done developing, it’s time to get the app out to users. This’ll require a comprehensive plan for actually making the jump. A little bit of foresight can go a long way here.</p>\n<p>My strategy was to roll out the new Rails app under a new domain: http://beta.myapp.com/. I then pinged my users with a message along the lines of “We’ve made some infrastructure updates, if you’d like to help us test them out, visit the beta site.”</p>\n<p>This approach does have some sticky points, however. The first one is the database. In my case, I had a high level of confidence in my code, and this application wasn’t for anything that was considered mission-critical. We also had a decent backup strategy in place. Therefore, I connected the beta site up to the production data store. This meant there were no migration issues later. However, this may make you uncomfortable, but there are other options, too. You can treat the beta as a sandbox, and tell people that the data will not be persisted after the beta, or you can migrate people’s data back to the production data store afterwards.</p>\n<p>Another approach is to automatically migrate a portion of your users over to the new code base. Ideally, nobody notices: your code base shouldn’t have changed in looks or functionality.</p>\n<h2>Step Ten: Push to production!</h2>\n<p>Congrats! Execute on the plan you created in step nine, and get ready to answer tons of emails about why your app is broken! Just kidding, I hope! I really recommend the beta strategy for this reason. No plan survives first contact with the enemy. Your users <em>will</em> find things you didn’t, no matter how much manual testing you did along the way.</p>\n<h2>Another strategy: mounting</h2>\n<p>I haven’t actually tried this myself, but I’ve been thinking about another way to make this happen: mount your Sinatra app inside of a Rails app, and use that to move things over slowly. <a href=\"http://stackoverflow.com/a/6972706\">Here’s an example</a> of how to make this happen technically. If you had a really big Sinatra application, I could see how this might get you deploying faster; just carve out one vertical chunk of your app, move it over, and keep the rest of your app running in Sinatra.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/implementing-hateoas-with-presenters/",
"title": "Implementing HATEOS with presenters",
"description": null,
"url": "https://steveklabnik.com/writing/implementing-hateoas-with-presenters/",
"published": "2012-01-06T00:00:00.000Z",
"updated": "2012-01-06T00:00:00.000Z",
"content": "<p>I’m a big fan of using the presenter pattern to help separate logic from presentation. There’s a great gem named <a href=\"https://github.com/jcasimir/draper\">Draper</a> that can help facilitate this pattern in your Rails apps. When doing research for <a href=\"http://designinghypermediaapis.com/\">my book about REST</a>, I realized that the presenter pattern is a great way to create responses that comply with the hypermedia constraint, a.k.a. HATEOAS. I wanted to share with you a little bit about how to do this.</p>\n<p>Please note that ’<a href=\"/posts/2012-02-23-rest-is-over\">REST is over’</a>.</p>\n<p>Note: We’ll be creating HTML5 responses in this example, as HTML is a hypermedia format, and is therefore conducive to HATEOAS. JSON and XML don’t cut it.</p>\n<h2>First, some setup</h2>\n<p>I fired up a brand new Rails app by doing this:</p>\n<pre><code>$ rails new hateoas_example\n$ cd hateoas_example\n$ cat >> Gemfile\ngem \"draper\"\n^D\n$ bundle\n$ rails g resource post title:string body:text\n$ rake db:migrate\n$ rails g draper:decorator Post\n</code></pre>\n<p>Okay, now we should be all set up. We’ve got a Rails app, it’s got draper in the Gemfile, we have a Post resource, and our PostDecorator.</p>\n<h2>The View</h2>\n<p>I like to do the view first, to drive our what we need elsewhere. Here it is:</p>\n<pre><code><h2>Title</h2>\n<p><%= @post.title %></p>\n\n<h2>Body</h2>\n<p><%= @post.body %></p>\n\n<h2>Links</h2>\n<ul>\n <% @post.links.each do |link| %>\n <li><%= link_to link.text, link.href, :rel => link.rel %></li>\n <% end %>\n</ul>\n</code></pre>\n<p>We’re displaying our title and body, but we also want to spit out some links. These links should have a few attributes we need. I might even (shhhhhhh) extract this link thing out into a helper to add the rel stuff every time. It just depends. For this example, I didn’t feel like it.</p>\n<h2>The Controller</h2>\n<p>Well, we know we’re gonna need a <code>@post</code> variable set, so let’s get that going in our controller:</p>\n<pre><code>class PostsController < ApplicationController\n def show\n @post = PostDecorator.find(params[:id])\n end\nend\n</code></pre>\n<p>Super simple. Yay Draper!</p>\n<h2>The Presenter</h2>\n<p>We know we need a <code>links</code> method that returns some links, and those links need to have rel, href, and text attributes. No problem!</p>\n<pre><code>class Link < Struct.new(:rel, :href, :text)\nend\n\nclass PostDecorator < ApplicationDecorator\n decorates :post\n\n def links\n [self_link, all_posts_link]\n end\n\n def all_posts_link\n Link.new(\"index\", h.posts_url, \"All posts\")\n end\n\n def self_link\n Link.new(\"self\", h.post_url(post), \"This post\")\n end\nend\n</code></pre>\n<p>Now, we could have just returned an array of three-element arrays, but I really like to use the Struct.new trick to give us an actual class. It makes error messages quite a bit better, and reminds us that we don’t happen to have an array, we have a Link.</p>\n<p>We construct those links by taking advantage of the ‘index’ and ‘self’ rel attributes that are <a href=\"http://www.iana.org/assignments/link-relations/link-relations.xml\">defined in the registry</a>.</p>\n<h2>The output</h2>\n<p>That gives us this:</p>\n<pre><code><!DOCTYPE html>\n<html>\n<head>\n <title>HateoasSample</title>\n <link href=\"/assets/application.css?body=1\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\n<link href=\"/assets/posts.css?body=1\" media=\"screen\" rel=\"stylesheet\" type=\"text/css\" />\n <script src=\"/assets/jquery.js?body=1\" type=\"text/javascript\"></script>\n<script src=\"/assets/jquery_ujs.js?body=1\" type=\"text/javascript\"></script>\n<script src=\"/assets/posts.js?body=1\" type=\"text/javascript\"></script>\n<script src=\"/assets/application.js?body=1\" type=\"text/javascript\"></script>\n <meta content=\"authenticity_token\" name=\"csrf-param\" />\n<meta content=\"0k+SQVv6yr0d12tGWYx7KNXUWaf6f+wgUUNITsAOnHI=\" name=\"csrf-token\" />\n</head>\n<body>\n\n<h2>Title</h2>\n<p>A post, woo hoo!</p>\n\n<h2>Body</h2>\n<p>this is some text that's the body of this post</p>\n\n<h2>Links</h2>\n<ul>\n <li><a href=\"http://localhost:3000/posts/1\" rel=\"self\">This post</a></li>\n <li><a href=\"http://localhost:3000/posts\" rel=\"index\">All posts</a></li>\n</ul>\n\n</body>\n</html>\n</code></pre>\n<p>You’ll probably want to make a layout that ignores all of the JS stuff, but for this example, I just left it as-is. It’s just that easy. Happy linking!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/active-record-considered-harmful/",
"title": "ActiveRecord considered harmful",
"description": null,
"url": "https://steveklabnik.com/writing/active-record-considered-harmful/",
"published": "2011-12-30T00:00:00.000Z",
"updated": "2011-12-30T00:00:00.000Z",
"content": "<blockquote>\n<p>It is practically impossible to teach OO design to students that have had a prior exposure to Rails: as potential programmers they are mentally mutilated beyond hope of regeneration.Edsger W. Dijkstra (paraphrased)</p>\n</blockquote>\n<p>I love ActiveRecord. It was the first ORM I’d ever interacted with. My first technical employer had commissioned a DBA, and so of course, we wrote all our own queries. Which was fine; I know my way around a JOIN or two. The problem came when it was time to make a new class; time to write “SELECT * FROM ‘tableName’ WHERE ‘id’=‘%’”… for each class. “Just copy one of the other small classes, hack out everything, and change the table names,” were my instructions. Fine. Whatever. But I knew there had to be a better way…</p>\n<p>Along comes Rails. Holy crap, ActiveRecord is <em>awesome</em>! It writes the <em>exact</em> SQL I would have written myself! I don’t need to do <em>anything</em>. This is the best thing since sliced bread. But years later, not everything is rainbows and sunshine. I’ve written a <em>lot</em> of crappy code, and most of it is due to following Rails ‘best practices.’ Which is totally fine! Rails is kinda getting old these days. It’s no longer the new and shiny. But while Rails developers have gotten really good at busting out little CRUD apps, we haven’t moved web application design <em>forward</em> in a really, really long time.</p>\n<p>And that’s what Rails did, really. That fifteen minute blog video was shocking. I know several people that threw away two or three week old projects and re-built their stuff in Rails within moments of watching it. And Rails has continued to lead the way in improving how we build rich applications on the web; the (not so but kinda whatever nobody but me cares) RESTful routing was great. The asset pipeline, though it has some bugs, has been great. The obsession with DRY has been great. The test obsession has been great. But Rails has also not been great in aiding us in writing maintainable software. Many of Rails’ design decisions make it difficult to write <em>great</em> software. It helps us write good software, but that isn’t enough anymore.</p>\n<p>The real problem is that to truly move forward, Rails will have to re-invent itself, and I’m not sure that it can.</p>\n<h2>ActiveRecord is the problem</h2>\n<p>I’m not going to talk about all of the problems Rails has today, but I’d like to show you the biggest, most central one: ActiveRecord.</p>\n<p>ActiveRecord’s greatest strength is also its problem: Tying class names to table names. This means that it’s impossible to de-couple your persistence mechanism from your domain logic. You can manage it through a combination of <code>set_table_name</code>, making a bunch of <code>Repository</code> classes, and careful coding… but then you might as well be using DataMapper. Except the Ruby library of the same name doesn’t really implement the DataMapper pattern all that well either, having the same issue of tying it all together.</p>\n<p>This has another interesting effect: it led directly to the ‘fat model’ recommendation. While ‘get your stuff out of the controller’ is correct, it’s lead Rails developers to build huge, monolithic models that are hard to test, and violate SRP. It took me two and a half years to realize that Ruby classes in the models folder don’t have to inherit from <code>ActiveRecord::Base</code>. That is a problem.</p>\n<p>We’ve gotten into a situation with a local minimum: our quest for great software has lead us to DRY ourselves into a corner. Now we have code that’s incredibly tightly coupled. Our ‘single responsibility’ is ‘anything and everything that tangentially relates to a Post.’</p>\n<h2>ActionController is the problem</h2>\n<p>ActionController relies on instance variables to pass information from the controller to the view. Have you ever seen a 200 line long controller method? I have. Good luck teasing out which instance variables actually get set over the course of all those nested <code>if</code>s.</p>\n<p>The whole idea is kinda crazy: Yeah, it looks nice, but we literally just say ‘increase the scope of variables to pass data around.’ If I wrote a post saying “Don’t pass arguments to methods, just promote your data to a global” I’d be crucified. Yet we do the same thing (albeit on a smaller scale) every time we write a Rails application.</p>\n<h2>ActionView is the problem</h2>\n<p>The whole idea of logic in templates leads to all kinds of problems. They’re hard to test, they’re hard to read, and it’s not just a slippery slope, but a steep one. Things go downhill <em>rapidly</em>.</p>\n<p>What I’d really like to see is Rails adopting a ‘ViewModel + templates’ system, with logic-less templates and presenter-esque models that represent the views. The differences between Django’s idea of ‘views’ and Rails’ idea of ‘views’ are interesting here.</p>\n<h2>MVC is the problem</h2>\n<p>If you’ll notice, I basically have said that models are a problem. Controllers are a problem. Views are a problem. MVC has served the web well, even if it isn’t the GUI style MVC that named the pattern. But I think we’re reaching its limits; the impedance mismatch between HTTP and MVC, for example, is pretty huge. There are other ways to build web applications; I’m particularly excited about <a href=\"http://rubyconf-webmachine.heroku.com/\">WebMachine</a>. I don’t have a constructive alternative to offer here, I just know there’s a problem. I’m still mulling this one over.</p>\n<h2>It’s still good, even with problems</h2>\n<p>I love Rails. I build software with it daily. Even with its flaws, it’s been a massive success. But because I love Rails, I feel like I can give it straightforward criticism: it’s easier to trash something you love. The real issue is that changing these things would require some really serious changes. It’d involve re-architecting large portions of things that people classically identify with Rails, and I’m not sure that Rails wants to or can do that.</p>\n<p>This post is light on examples. I want this to be the <em>starting point</em> of a discussion, not the end of it. Expect more, in detail, from me in the future. What do you think? Are these problems pain points for you? Are they worth fixing? Are they actually problems?</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/devise-actioncontroller-routingerror-no-route-matches-get-slash-users-slash-sign-out/",
"title": "Devise: ActionController::RoutingError (No Route Matches [GET] /users/sign_out)",
"description": null,
"url": "https://steveklabnik.com/writing/devise-actioncontroller-routingerror-no-route-matches-get-slash-users-slash-sign-out/",
"published": "2011-12-22T00:00:00.000Z",
"updated": "2011-12-22T00:00:00.000Z",
"content": "<p>Just a quick note about Devise, and its RESTful implications. I ran across this error today, and thought I’d share.</p>\n<p>I was trying to log out, so I hit the normal route for such things with my browser. Here’s the error:</p>\n<pre><code>Devise: ActionController::RoutingError (No Route Matches [GET] /users/sign_out)\n</code></pre>\n<p>Uhhhhh what? I run rake routes…</p>\n<pre><code>$ rake routes | grep destroy_user\n destroy_user_session DELETE /users/sign_out(.:format) {:action=>\"destroy\", :controller=>\"devise/sessions\"}\n</code></pre>\n<p>So, wtf? Well, there is that pesky <code>DELETE</code>… Googling for this error led me to <a href=\"https://github.com/plataformatec/devise/commit/f3385e96abf50e80d2ae282e1fb9bdad87a83d3c\">this commit</a>. Looks like they changed some behavior, you can’t just go to that page anymore and log out. Bummer. But why?</p>\n<h2>HTTP verbs: transfer semantics, not state</h2>\n<p>As I’ve been doing my research for <a href=\"http://designinghypermediaapis.com/\">my book on REST</a>, I’ve been doing a lot of reading about HTTP. And I kept coming across these kinds of curious comments in the spec. For example, <a href=\"http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6\">sec 9.6: PUT</a>:</p>\n<blockquote>\n<p>HTTP/1.1 does not define how a PUT method affects the state of an origin server.</p>\n</blockquote>\n<p>Uhhh… what? Don’t we all know that PUT means that we should be updating a resource? And that we have to send the whole representation?</p>\n<p>When trying to get to the bottom of this, I came across <a href=\"http://www.imc.org/atom-protocol/mail-archive/msg05425.html\">this comment from Fielding</a>:</p>\n<blockquote>\n<p>FWIW, PUT does not mean store. I must have repeated that a million times in webdav and related lists. HTTP defines the intended semantics of the communication – the expectations of each party. The protocol does not define how either side fulfills those expectations, and it makes damn sure it doesn’t prevent a server from having absolute authority over its own resources. Also, resources are known to change over time, so if a server accepts an invalid Atom entry via PUT one second and then immediately thereafter decides to change it to a valid entry for later GETs, life is grand.</p>\n</blockquote>\n<p>Soooooooo wtf?</p>\n<p>Let’s take a look again at what <code>PUT</code> does:</p>\n<blockquote>\n<p>The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.</p>\n</blockquote>\n<p>It says “store,” Roy says “I don’t mean store.” Uhhhh…</p>\n<p>Here’s my ‘translated for the laymen’ version of that quote:</p>\n<blockquote>\n<p>PUT means ‘I’d like to later GET something at this URI.’ If something is already there, update it. If there isn’t anything there, then create it.</p>\n</blockquote>\n<p>That’s it. It’s talking about the semantics of what goes on: create or update. It doesn’t actually say anything about how this is implemented. But if you PUT something to a URI, a GET needs to 200 afterwards. So what’s the difference between PUT and POST? <a href=\"http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.5\">HTTP sec 9.5: POST</a>:</p>\n<blockquote>\n<p>The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line.The actual function performed by the POST method is determined by the server and is usually dependent on the Request-URI. The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database.The action performed by the POST method might not result in a resource that can be identified by a URI.</p>\n</blockquote>\n<p>Again, it’s really vague about what it <em>does</em>. With POST, it basically says “You have no idea what a POST does.” What you <em>do</em> know is the semantics of the action, POST ‘requests a new subordinate, but it might not create something.’</p>\n<p>The <em>main</em> difference is actually mentioned <a href=\"http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.2\">here</a>:</p>\n<blockquote>\n<p>Methods can also have the property of “idempotence” in that (aside from error or expiration issues) the side-effects of N > 0 identical requests is the same as for a single request. The methods GET, HEAD, PUT and DELETE share this property.</p>\n</blockquote>\n<p>Semantics again. PUT is <em>idempotent</em>, and POST is not. They could both be used for creation, they could both be used for updating. With POST, you don’t need a URI, and PUT specifies a specfic one. That’s it. Nowhere in those two sentences states ‘store in the database,’ nowhere does it says ‘full representation,’ nowhere does it say ‘POST is create and PUT is update.’ However you fulfill these semantics are up to you, but the semantics are what’s important.</p>\n<h2>So wtf does this have to do with Devise?</h2>\n<p>The issue is that Devise’s old semantics were wrong. A GET to <code>/users/sign_out</code> shouldn’t modify state: <a href=\"http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.1.1\">HTTP sec 9.1.1: Safe Methods</a>:</p>\n<blockquote>\n<p>In particular, the convention has been established that the GET and HEAD methods SHOULD NOT have the significance of taking an action other than retrieval. These methods ought to be considered “safe”.</p>\n</blockquote>\n<p>When Devise used GETs to log you out, that was a violation of the semantics of GET. Here’s the interesting part, though: Since POST is a super generic ‘unsafe action’ method, you could also use POST to represent a logging out. POST also has unsafe, non-idempotent semantics. DELETE specifically says delete, and POST says ‘any action,’ and ‘delete’ is a subset of ‘any action.’ So DELETE is <em>better</em>, but POST is not <em>wrong</em>, categorically.</p>\n<h2>Fixing the ‘bug’</h2>\n<p>So how do we take care of this? Personally, I did the pragmatic thing. In <code>/config/initializers/devise.rb</code></p>\n<pre><code>config.sign_out_via = :delete\nconfig.sign_out_via = :get if Rails.env.test?\n</code></pre>\n<p>Now, during normal operations, we have our usual DELETE semantics, but in our test environment, we can just hit it with GET. This way we don’t have to hit a page, use Javascript to make a link with DELETE, and hit the page. This keeps my test times down, means I can run my tests with rack-test and not Selenium, and still gives me a high level of confidence that my tests work properly, even though it’s technically not the exact same thing as production.</p>\n<h2>In conclusion</h2>\n<p>The HTTP spec defines <em>semantics</em> but not implementation details. Semantics should be obeyed. But in testing, obeying them 100% may not be worth it.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/redcarpet-is-awesome/",
"title": "Redcarpet is awesome",
"description": null,
"url": "https://steveklabnik.com/writing/redcarpet-is-awesome/",
"published": "2011-12-21T00:00:00.000Z",
"updated": "2011-12-21T00:00:00.000Z",
"content": "<p>It’s true.</p>\n<p>If you haven’t used it yet, <a href=\"https://github.com/tanoku/redcarpet\">Redcarpet</a> is the Markdown parser that <a href=\"https://github.com/blog/832-rolling-out-the-redcarpet\">GitHub uses</a> to work all that magic on their site. So of course, it’s awesome.</p>\n<h2>You can use it and abuse it</h2>\n<p>What makes it <em>really</em> awesome is the custom renderers feature. Here’s the one from the documentation:</p>\n<pre><code># create a custom renderer that allows highlighting of code blocks\nclass HTMLwithAlbino < Redcarpet::Render::HTML\n def block_code(code, language)\n Albino.safe_colorize(code, language)\n end\nend\n\nmarkdown = Redcarpet::Markdown.new(HTMLwithAlbino)\n</code></pre>\n<p>You can guess what that does: uses some library called Albino to render code blocks. There are a whole bunch of hooks you can use to make a custom renderer. They make it super easy to do something neat with markdown.</p>\n<p>So I did.</p>\n<h2>Check it: outline generation</h2>\n<p>I’m working on… this project. And it needs to render a bunch of articles that are Markdown files. There will be a sidebar. I want to have links to each section. But I’m a good programmer, which means I’m lazy. So why bother making my own sidebar? Especially when I can abuse Redcarpet to do it.</p>\n<p>Oh, and disclaimer: normally, I’m all about super clean code. Last night, that wasn’t the mood I was in. This code is probably terrible. That’s part of the fun! Please feel free to {clean up,obfuscate,golf,unit test} this code, <a href=\"mailto:[email protected]\">email me</a>, and tell me how awesome you are.</p>\n<p>With that disclaimer out of the way, round one:</p>\n<pre><code>class OutlineRenderer < Redcarpet::Render::HTML\n attr_accessor :outline\n\n def initialize\n @outline = []\n super\n end\n\n def header(text, header_level)\n text_slug = text.gsub(/\\W/, \"_\").downcase\n \n self.outline << [header_level, \"<a href='##{text_slug}'>#{text}</a>\"]\n\n \"<h#{header_level} id='#{text_slug}'>#{text}</h#{header_level}>\"\n end\nend\n</code></pre>\n<p>Every time we hit a header, Redcarpet gives us the text and the header level. We grab the text, turn it into a slug, and then append a two-element array to our outline. It keeps track of the level of this header, and makes a link from the slug. Then, we spit out a header tag, adding on an <code>id</code> element that we linked to in our link.</p>\n<p>Next up, rendering:</p>\n<pre><code>renderer = OutlineRenderer.new\nr = Redcarpet::Markdown.new(renderer)\ncontent = r.render(some_content)\noutline = renderer.outline\n</code></pre>\n<p>This instantiates our Renderer, creates our Markdown parser, and renders the content. We also grab the outline we made. Sweet.</p>\n<p>Finally, rendering the outline:</p>\n<pre><code>def format_outline(outline)\n prev_level = 2\n\n \"<ul>\" + outline.inject(\"\") do |html, data|\n level, link = data\n if prev_level < level\n html += \"<ul>\"\n elsif prev_level > level\n html += \"</ul>\"\n end\n prev_level = level\n html += \"<li>#{link}</li>\"\n html\n end + \"</ul>\"\nend\n</code></pre>\n<p>This… is amazing. And terrible. Don’t drink scotch and code, kids. Or do, whatever. This takes our <code>Array</code> of <code>Array</code>s that we made with our renderer and runs <code>inject</code> over it. If our previous level is less than the new level, we indent with a <code><ul></code>, and if it’s greater, we outdent with a <code></ul></code>. Then we render our link, and wrap the whole thing in its own set of <code><ul></code>s. Done!</p>\n<p>We can call this helper method in our layout, and bam! We get automatic outlines generated for our Markdown files. Like I said, this is quick and dirty, but for a first pass, I don’t feel too bad about it. Get it done, and then get it done right.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/write-better-cukes-with-the-rel-attribute/",
"title": "Write better cukes with the rel attribute",
"description": null,
"url": "https://steveklabnik.com/writing/write-better-cukes-with-the-rel-attribute/",
"published": "2011-12-20T00:00:00.000Z",
"updated": "2011-12-20T00:00:00.000Z",
"content": "<p>The other day, I was working on some Cucumber features for a project, and I discovered a neat technique that helps you to write better Cucumber steps.</p>\n<p>Nobody wants to be <a href=\"http://elabs.se/blog/15-you-re-cuking-it-wrong\">cuking it wrong</a>, but what does that really mean? Here’s Jonas’ prescription:</p>\n<blockquote>\n<p>A step description should never contain regexen, CSS or XPath selectors, any kind of code or data structure. It should be easily understood just by reading the description.</p>\n</blockquote>\n<p>Great. Let’s <a href=\"http://www.theregister.co.uk/2007/06/25/thoughtworks_req_manage/\">pop the why stack</a> a few times, shall we?</p>\n<p><strong>Q1</strong>: Why do we want to have descriptions not use regexen, CSS selectors, or code? <strong>A1</strong>: To give it a simpler language.</p>\n<p><strong>Q2</strong>: Why do we want it to be in a simpler language? <strong>A2</strong>: So that it’s easily understandable for stakeholders.</p>\n<p><strong>Q3</strong>: Why do we want it to be easily understandable for stakeholders? <strong>A3</strong>: Because then we can share a <a href=\"http://domaindrivendesign.org/node/132\">common language</a>.</p>\n<p><strong>Q4</strong>: Why is a common language important? <strong>A4</strong>: A shared language assists in making sure our model matches the desires of our stakeholders.</p>\n<p><strong>Q5</strong>: Why do we want to match the desires of our stakeholders? <strong>A5</strong>: That’s the whole reason we’re on this project in the first place!</p>\n<p>Anyway, that’s what it’s really all about: developing that common language. Cukes should be written in that common language so that we can make sure we’re on track. So fine: common language. Awesome. Let’s do this.</p>\n<h2>Write some cukes in common language</h2>\n<p>Time to write a cuke:</p>\n<pre><code>Scenario: Editing the home page\n Given I'm logged in as an administrator\n When I go to the home page\n And I choose to edit the article\n And I fill in some content\n And I save it\n Then I should see that content\n</code></pre>\n<p>Basic CMS style stuff. I’m not going to argue that this is the best cuke in the world, but it’s pretty good. What I want to do is examine some of these steps in more detail. How would you implement these steps?</p>\n<pre><code>When /^I choose to edit the article$/ do\n pending\nend\n\nWhen /^I fill in some content$/ do\n pending\nend\n\nWhen /^I save it$/ do\n pending\nend\n\nThen /^I should see that content$/ do\n pending\nend\n</code></pre>\n<p>Go ahead. Write them down somewhere. I’ll wait.</p>\n<p>… done yet?</p>\n<h2>Implementing a step</h2>\n<p>Done? Okay! Before I show you my implementation, let’s talk about this step:</p>\n<pre><code>When /^I choose to edit the article$/\n</code></pre>\n<p>When writing this step, I realized something. When trying to write steps like these, there’s a danger in tying them too closely to your specific HTML. It’s why many people don’t write view tests: they’re brittle. I actually like view tests, but that’s another blog post. Point is this: we know we’re going to follow a link, and we know that we want that link to go somewhere that will let us edit the article. We don’t really care <em>where</em> it is in the DOM, just that somewhere, we’ve got an ‘edit article’ link. How to pull this off?</p>\n<h3>First idea: id attribute</h3>\n<p>You might be thinking “I’ll give it an id attribute!” Here’s the problem with that: ids have to be unique, per page. With article editing, that might not be a problem, but it’s certainly not a general solution. So that’s out.</p>\n<h3>Second time: class attribute</h3>\n<p>“Okay, then just use a class. Your blog sucks.” Well, let’s check out what <a href=\"http://www.w3.org/TR/html5/elements.html#classes\">the HTML5 spec says about classes</a>.</p>\n<blockquote>\n<p>Basically nothing about semantics.</p>\n</blockquote>\n<p>Okay, so that’s paraphrased. But still, the spec basically says some stuff about the details of implementing classes, but absolutely nothing about the semantics of a class. In practice, classes are largely used for styling purposes. We don’t want to conflate our styling with our data, so overloading class for this purpose might work, but feels kinda wrong.</p>\n<h3>What about the text?</h3>\n<p>We could match on the text of the link. After all, that’s what people use to determine what links to click on. The link with the text “Edit this article” lets us know that that link will let us edit a article.</p>\n<p>Matching on the text is brittle, though. What happens when marketing comes through and changes the text to read “Edit my article”? Our tests break. Ugh.</p>\n<p>There’s got to be a better way. Otherwise, I wouldn’t be writing this blog post.</p>\n<h3>The best way: the rel attribute</h3>\n<p>When doing research for <a href=\"http://designinghypermediaapis.com/\">my book on REST</a>, I’ve been doing a lot of digging into various standards documents. And one of the most important attributes from a REST perspective is one that nobody ever talks about or uses: the <code>rel</code> attribute. From <a href=\"http://www.w3.org/TR/html5/links.html#attr-hyperlink-rel\">the HTML5 spec</a>:</p>\n<blockquote>\n<p>The rel attribute on a and area elements controls what kinds of links the elements create. The attribue’s value must be a set of space-separated tokens. The allowed keywords and their meanings are defined below.</p>\n</blockquote>\n<p>Below? That’s <a href=\"http://www.w3.org/TR/html5/links.html#linkTypes\">here</a>:</p>\n<blockquote>\n<p>The following table summarizes the link types that are defined by this specification. This table is non-normative; the actual definitions for the link types are given in the next few sections.alternate: Gives alternate representations of the current document. author: Gives a link to the current document’s author. bookmark: Gives the permalink for the nearest ancestor section.</p>\n</blockquote>\n<p>Hey now! Seems like we’re on to something. There’s also <a href=\"http://tools.ietf.org/html/rfc5988\">RFC 5988</a>, “Web Linking”. <a href=\"http://tools.ietf.org/html/rfc5988#section-4\">Section four</a> talks about Link Relation Types:</p>\n<blockquote>\n<p>In the simplest case, a link relation type identifies the semantics of a link. For example, a link with the relation type “copyright” indicates that the resource identified by the target IRI is a statement of the copyright terms applying to the current context IRI.Link relation types can also be used to indicate that the target resource has particular attributes, or exhibits particular behaviours; for example, a “service” link implies that the identified resource is part of a defined protocol (in this case, a service description).</p>\n</blockquote>\n<p>Bam! Awesome! This is exactly what we want!</p>\n<h2>So how do we use rel attributes?</h2>\n<p>I’ll be going into more depth about these kinds of topics in <a href=\"http://designinghypermediaapis.com/\">my book</a>, but here’s the TL;DR:</p>\n<ol>\n<li>There are a set of official types. Try to use those if they’re applicable, but they’re quite general, so that’s often not the case.</li>\n<li>You can put whatever else you want. Space delineated. *</li>\n<li>The best way is to use a URI and then make a resource at that URI that documents the relation’s semantics.</li>\n</ol>\n<p>We’ll go with option two for now, for simplicity. In a real application, make it a URI.</p>\n<p>(*) Technically, this isn’t true. Extension relations are <em>required</em> to be URIs, or something that can be serialized to a URI. Again, details are outside of the scope of this post.</p>\n<h2>Making our link, with semantics.</h2>\n<p>Here’s what a link with our newly minted relation looks like:</p>\n<pre><code><a href=\"/articles/1/edit\" rel=\"edit-article\">Edit this article</a>\n</code></pre>\n<p>Super simple. Just that one little attribute. Now we can write a step to match:</p>\n<pre><code>When /^I choose to edit the article$/ do\n find(\"//a[@rel='edit-article']\").click\nend\n</code></pre>\n<p>This code matches what we’d do as a person really well. “Find the link that edits an article, and click on it.” We’ve not only made the title of our step match our idea of what a person would do, but the code has followed suit. Awesome. We can move this link anywhere on the page, our test doesn’t break. We can change the text of the link, and our test doesn’t break. So cool.</p>\n<h2>What about stuff that’s not links? What about data attributes?</h2>\n<p>That’s what <a href=\"http://designinghypermediaapis.com/\">my book</a> is going to talk about, sorry. These kinds of practical examples are one of the reasons I decided to write it in the first place, and I don’t want to publish all the content on my blog…</p>\n<h2>Better tests through web standards</h2>\n<p>Turns out that diving around in standards has some practical benefits after all, eh? Think about the relationship between your cukes, your tests, and your API clients: Cucumber, through Selenium, is an automated agent that interacts with your web service. API clients are automated agents that interact with your web service. Hmmmm…</p>\n<p>If you want to know more about this, that’s what <a href=\"http://designinghypermediaapis.com/\">my book</a> is for. I’ll be covering topics like this in depth, and explaining standards in simple language.</p>\n<p>Seriously. Did you sign up for <a href=\"http://designinghypermediaapis.com/\">my book</a> yet? ;)</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/marx-anarchism-and-web-standards/",
"title": "Marx, anarchism, and web standards",
"description": null,
"url": "https://steveklabnik.com/writing/marx-anarchism-and-web-standards/",
"published": "2011-12-15T00:00:00.000Z",
"updated": "2011-12-15T00:00:00.000Z",
"content": "<h3>An aside for software devs</h3>\n<p>You might not care about anarchism, and I can almost guarantee that you don’t care about Marx, but please bear with me. I think my point is best made by putting the web stuff at the end, so please, just read it. ;) It’ll be good for you, I swear. I’ll explain better at the end.</p>\n<h2>Domain Specific Languages and information density</h2>\n<p>I’ve been noticing an effect lately that’s certainly not new; it’s just come up frequently. When working within a domain that’s new to them, most people tend to not respect the density of the information being given to them, and it causes them to draw incorrect inferences.</p>\n<p>For example, this tweet earlier:</p>\n<blockquote>\n<p>Does anybody else’s head explode when they read an “unless” statement? What good is readability if comprehension goes out the window?</p>\n</blockquote>\n<p>— Christopher Deutsch (@cdeutsch) December 15, 2011</p>\n<blockquote></blockquote>\n<p>And this response:</p>\n<blockquote>\n<p>@cdeutsch There are weirder idioms. @users.collect{|u| u.email} vs @users.collect(&:email) :) That should weird you out more.</p>\n</blockquote>\n<p>— Brian P. Hogan (@bphogan) December 15, 2011</p>\n<blockquote></blockquote>\n<p>Now, I’m not saying that Christopher is in the wrong here, in fact, he agrees with what I’m about to say:</p>\n<blockquote>\n<p>@philcrissman @bphogan I’m sure I’ll get used to the “unless” pattern someday. But from a Ruby outsiders perspective it’s hard to comprehend</p>\n</blockquote>\n<p>— Christopher Deutsch (@cdeutsch) December 15, 2011</p>\n<blockquote></blockquote>\n<p>From someone new to Ruby, <code>Symbol#to_proc</code> or <code>unless</code> are hard to understand initially, and that’s because they’ve increased the density of information being conveyed. <code>unless</code> is the same as <code>if not</code> and <code>&:foo</code> is the same as <code>{|a| a.foo }</code>. Both of these constructs condense something more complicated into something that is simpler, but denser.</p>\n<p>You’ll note that I said ‘simpler,’ but by a certain measure, <code>&:foo</code> is actually more complex. When I say <code>&:foo</code> is simpler, I’m meaning for someone who’s well versed in Ruby, functional programming, or first-class functions. I have this exact background, and so for me, <code>collection.map &:foo</code> is simpler and more readable than <code>collection.map {|a| a.foo }</code>. When I read the first example, I say in my head “Map foo over the collection.” You have to grok what map really is to get that sentence or the corresponding code. Whereas what (I imagine) someone who does not have this kind of background thinks when they see the second example is “Okay, so map is like an each, and for each thing in the collection, we’re going to call foo on it.” This is a totally valid interpretation, but notice how much longer it is, and how much more involved in the details it is. That’s cool, but to someone who groks map, it has a much larger amount of mental overhead, in the same way that my concise explanation causes much more thinking for someone who doesn’t grok it.</p>\n<p>This happens often in education. DHH made <a href=\"http://news.ycombinator.com/item?id=3328427\">a comment</a> about this recently that illustrates this principle, and he couches it in terms of “learnability” vs. “readability”:</p>\n<blockquote>\n<p>If you optimize a framework for beginners, you’re optimizing for learnability. That’s great for the first few days or even weeks of learning. But once you’re past that, it’s not so great. What you care more about is the usability of the thing once you know it. There are plenty of cases where learnability and usability are in conflict. Letting learnability win is a short-term relief.If you on the other hand optimize for usability, for making things simple and easy to use for someone who understands the basics, you’ll often end up with something that has great learnability as well. Maybe not as much as could be achieved as if that was your only goal, but plenty still.</p>\n</blockquote>\n<p>I think that ‘learnability’ is a pretty good shortening for ‘light density’ and ‘usability’ is decent for ‘heavy density.’ It’s the same effect, though. For the rest of this essay, I’ll be using ‘learnable’ and ‘usable’ to mean this particular usage.</p>\n<p>Any time you encounter experts in a particular domain, they’ll often have fairly specific language that corresponds to that domain. This language is usually designed to be usable, not learnable. This is because they’ve already done the learning; learnability holds no utility for them. However, usability is incredibly… useful. To them. They’re working at a higher level of abstraction, and don’t want to get bogged down in details they already know well. Using learnable language would cause them to take twice as long to say things; to return to the density analogy, dense information is transferred from one person to another more quickly. If you can read a sentence a second, but that sentence is dense, you acquire much more information per second than if it was lighter.</p>\n<p>This tendency means that most language that’s specific to a domain will generally trend towards the usable at the expense of the learnable. The impact this has on individuals new to the domain, however, is that of a wall. An impediment. Overcoming this obstacle requires a bit of good faith on the part of the beginner; to cross quickly over the chasm between beginner and expert, they must recognize and respect this aspect of the conversations they will invariably become a part of. When faced with a term that is used in a strange way, beginners should ask for clarification, and not start arguments over semantics they don’t yet even understand. Experts will recognize these arguments as coming from a place where concepts are not yet fully understood, and while they may recognize the need to help educate, if the newbie is being belligerent, they may just ignore them instead. Nobody wins; the signal/noise ratio has been decreased, the beginner doesn’t learn, and everyone’s time is wasted.</p>\n<p>Here’s three other situations where I’ve seen this happen lately:</p>\n<h2>Marx and the labor theory of value</h2>\n<p>Philosophy writing is generally a great example of text that is very much usable, and not at all learnable. Some writers can still be learnable, but most are not, in my experience. One of the reasons this happens is that they introduce concepts early in a text and then utilize them later without referring back to the earlier definition. This isn’t a problem for anyone who’s thinking about taking off the training wheels or anyone who reads the entire text. The danger comes in when someone <em>not</em> versed in the entire text attempts to take portions of it out of its context.</p>\n<p>Consider Marx, and <em>Capital</em>. The meaning of ‘value’ is a central concern of his writing, and indeed, entire critique of the political economy. It’s so important that the first few chapters are devoted to an (excruciatingly, frankly) detailed explanation of his thoughts on the true meaning of value. The rest of <em>Capital</em> is built on top of this: at least in my understanding of Marx, it all boils back down to that one question. And when having discussions between people who are devotees of Marx and those who come from other schools of economics, this kind of language gets in the way.</p>\n<p>It also causes ideas to be misrepresented: the first time I was ever introduced to the labor theory of value, it was by a close friend who’s very libertarian. This was a few years ago, so it’s an obvious paraphrase, but he summarized it thusly: “Yeah, I mean, the labor theory of value basically says that people should be paid for however much work they put into things, so if I take six hours to make a widget and you take four, the price of a widget from you should be six yet mine should only be four, and ideas like ‘market value’ should be disregarded. It’s totally irrelevant if I suck at making widgets, I should get paid more than you anyway.” Which, to put it lightly, is a misrepresentation. While explaining the labor theory of value is outside of the scope of this post, what I will say is that to Marx, ‘value’ is something intrinsic to an object; it’s the ‘socially necessary abstract labor’ inherent to it. Talk about dense language! What a capitalist would call ‘value,’ a Marxist would call ‘price.’</p>\n<p>As you can see, even just one little word, ‘value,’ can be quite dense! Can you imagine a discussion intended to be ‘learnable’ to outsiders about what’s meant? Imagine the expansion: ‘value’ -> ‘socially necessary abstract labor’ -> … Marx is already long enough; Capital would be thousands of pages! Yet to a beginner who flips to Chapter 12, they’ll read a sentence that contains ‘value’ and draw poor conclusions! They wouldn’t even realize they’re making a mistake, I mean, how could five letters be misinterpreted?</p>\n<p>Furthermore, people who haven’t read Marx don’t generally draw distinctions between his critique of capitalism and his solution: communism. This is annoying when trying to explain to people that I love his critique, but am critical of his answers to its problems; they perceive this weakness in his answer as a weakness in his description of the problem. Furthermore, they then say “but I thought you call yourself a communist?” and I respond with “sure; the issues I have are with the dictatorship of the proletariat, not with the general idea of communism” and then their eyes glaze over and they change the subject. Information density claims another victim…</p>\n<p>Oh, and a great example of Marxist economics in a usable form <a href=\"http://people.wku.edu/jan.garrett/303/marxecon.htm\">is here</a>.</p>\n<h2>Anarchism and ‘anarcho’-capitalists</h2>\n<p>Arguments often boil down to these kinds of questions of definition, but one place where I see it happen almost <em>constantly</em> is amongst anarchists. I mean, from the outset, anarchists have to battle against the general definition of ‘chaos and disorder’ versus the domain-specific ‘without rulers.’ Within that, ‘rulers’ in anarchism is fraught with the same sort of questions that ‘value’ has for Marx. The prime example of this are the terribly misguided ‘anarcho’-capitalists, better described as ‘voluntaryists.’</p>\n<p>Here’s the deal: ancaps lack an understanding of the vast majority of historical anarchist thought, and so try to appropriate the term ‘anarchism’ to describe their philosophy which is decidedly not anarchist. The ones who do have started using ‘voluntaryist’ to describe themselves, which is a great example of using information density to mislead, but that’s a whole separate rant. Here’s the 411, from <a href=\"http://infoshop.org/page/AnarchistFAQSectionF1\">the Anarchist FAQ</a>, which has its own things to say about density when it comes to the language specific to political theory discussions:</p>\n<blockquote>\n<p>“Anarcho”-capitalists claim to be anarchists because they say that they oppose government. As noted in the last section, they use a dictionary definition of anarchism. However, this fails to appreciate that anarchism is a political theory. As dictionaries are rarely politically sophisticated things, this means that they fail to recognise that anarchism is more than just opposition to government, it is also marked a opposition to capitalism (i.e. exploitation and private property). Thus, opposition to government is a necessary but not sufficient condition for being an anarchist – you also need to be opposed to exploitation and capitalist private property. As “anarcho”-capitalists do not consider interest, rent and profits (i.e. capitalism) to be exploitative nor oppose capitalist property rights, they are not anarchists.Part of the problem is that Marxists, like many academics, also tend to assert that anarchists are simply against the state. It is significant that both Marxists and “anarcho”-capitalists tend to define anarchism as purely opposition to government. This is no co-incidence, as both seek to exclude anarchism from its place in the wider socialist movement. This makes perfect sense from the Marxist perspective as it allows them to present their ideology as the only serious anti-capitalist one around (not to mention associating anarchism with “anarcho”-capitalism is an excellent way of discrediting our ideas in the wider radical movement). It should go without saying that this is an obvious and serious misrepresentation of the anarchist position as even a superficial glance at anarchist theory and history shows that no anarchist limited their critique of society simply at the state. So while academics and Marxists seem aware of the anarchist opposition to the state, they usually fail to grasp the anarchist critique applies to all other authoritarian social institutions and how it fits into the overall anarchist analysis and struggle. They seem to think the anarchist condemnation of capitalist private property, patriarchy and so forth are somehow superfluous additions rather than a logical position which reflects the core of anarchism.</p>\n</blockquote>\n<p>Part of the problem with the second half of this quote is that I’m such an ‘expert’ on this kind of language that I don’t even know if it’ll make sense to you; without the kind of background reading in socialist political philosophies, it might just be gibberish. At least, the first paragraph should be pretty straightforward, and you can take the second as an example of this kind of language.</p>\n<p>Giving you a short explanation of why anarchists are against Capitalism is a great example in and of itself of domain specific language and density. Here:</p>\n<blockquote>\n<p>Property is theft.</p>\n</blockquote>\n<p>This is a quote by <a href=\"http://en.wikipedia.org/wiki/Pierre-Joseph_Proudhon\">Proudhon</a>, the first person to call himself an anarchist. Let’s unpack the first few layers of this statement:</p>\n<p>Property. Here’s <a href=\"http://en.wikipedia.org/wiki/Private_property\">Wikipedia’s explanation</a>:</p>\n<blockquote>\n<p>Private property is the right of persons and firms to obtain, own, control, employ, dispose of, and bequeath land, capital, and other forms of property. Private property is distinguishable from public property, which refers to assets owned by a state, community or government rather than by individuals or a business entity. Private property emerged as the dominant form of property in the means of production and land during the Industrial Revolution in the early 18th century, displacing feudal property, guilds, cottage industry and craft production, which were based on ownership of the tools for production by individual laborers or guilds of craftspeople.Marxists and socialists distinguish between “private property” and “personal property”, defining the former as the means of production in reference to private enterprise based on socialized production and wage labor; and the latter as consumer goods or goods produced by an individual.</p>\n</blockquote>\n<p>Whew! There’s a few things to note here: Captialists like to pretend that capitalism is synonymous with ‘trade,’ and not something that started in the 1800s. Likewise, that private property rights are something that has always existed. However, as this alludes to, there are many different kinds of property rights that have existed at different places and times.</p>\n<p>So in ‘property is theft,’ Proudhon is referring to private ownership of the ‘means of production.’ Let’s expand that. Again, <a href=\"http://en.wikipedia.org/wiki/Means_of_production\">Wikipedia</a>:</p>\n<blockquote>\n<p>Means of production refers to physical, non-human inputs used in production—the factories, machines, and tools used to produce wealth — along with both infrastructural capital and natural capital. This includes the classical factors of production minus financial capital and minus human capital. They include two broad categories of objects: instruments of labour (tools, factories, infrastructure, etc.) and subjects of labour (natural resources and raw materials). People operate on the subjects of labour, using the instruments of labour, to create a product; or, stated another way, labour acting on the means of production creates a product. When used in the broad sense, the “means of production” includes the “means of distribution” which includes stores, banks, and railroads. The term can be simply and picturesquely described in an agrarian society as the soil and the shovel; in an industrial society, the mines and the factories.</p>\n</blockquote>\n<p>We could continue to discuss how to distinguish between this ‘private property’ and ‘possessions,’ which anarchists are <em>not</em> against, but I’m just trying to demonstrate that this word ‘property’ is incredibly complex.</p>\n<p>Okay, so Proudhon claims that ‘owning the physical inputs used in factories used to produce wealth is theft.’ I could expand on ‘theft,’ but really, I think my point about density is made. For more on this, see <a href=\"http://anarchism.pageabode.com/afaq/secB3.html\">Why are anarchists against private property?</a>.</p>\n<h2>Web standards</h2>\n<p>Web standards are another great example of a domain that has a lot of very specific language. And one that people often think they can grab random chunks out of and quote without fully understanding the rest of the context.</p>\n<p>This is going on right now <a href=\"https://github.com/rails/rails/pull/505\">with Rails</a>. There’s a discussion about if the <code>PATCH</code> HTTP verb should get support, and if it should be the verb that matches to the <code>update</code> action or not. It’s ended up in a discussion about the semantics of <code>PUT</code>, which has resulted in a lot of random quoting of standards documents by myself and others. Here’s some running commentary on some of the comments. It’s impossible to do this without it becoming semi-personal, so let me just say upfront that I think everyone is participating honestly in this discussion, but I think it’s a great example of people who aren’t familiar with a domain jumping in and drawing incorrect conclusions.</p>\n<p>First up, benatkin comments:</p>\n<blockquote>\n<p>I googled for HTTP verbs and clicked the first result and PATCH isn’t listed.http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.htmlWhere is it?</p>\n</blockquote>\n<p>Anyone participating in a discussion about <code>PATCH</code> should be reasonably familiar with <code>PATCH</code>. Learning where <code>PATCH</code> is defined is as simple as <a href=\"https://www.google.com/#hl=en&q=http+patch\">Googling HTTP PATCH</a>, which shows it being defined in <a href=\"http://tools.ietf.org/html/rfc5789\">RFC 5879</a>. With that said, this is a good example of asking for clarification, and not immediately progressing into an argumentatitive “It’s not in HTTP 1.1, so it’s bullshit!” style of learning where <code>PATCH</code> is being defined.</p>\n<p>Of course, the thread starts to dissolve later, when @stevegraham mentions:</p>\n<blockquote>\n<p>i must be the only person in the world that disagrees with “PUT requires a complete replacement”, as per RFC2616 “HTTP/1.1 does not define how a PUT method affects the state of an origin server”</p>\n</blockquote>\n<p>Now, Steve is an awesome guy, but he’s a bit misguided in this case. This is a great example of drawing an incorrect conclusion based on one sentence out of context. He’s not <em>wrong</em> in a strict sense, <a href=\"http://www.ietf.org/rfc/rfc2616.txt\">RFC2616</a> does contain that line. However, this is because <a href=\"http://www.imc.org/atom-protocol/mail-archive/msg05425.html\">HTTP defines the semantics of communication</a>, and the semantics are ‘idempotent creation or replacement.’ The fact that HTTP does not define how PUT affects state is irrelevant, an entire representation is needed for idempotency reasons. <a href=\"http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.6\">PUT’s definition</a> also says pretty clearly:</p>\n<blockquote>\n<p>The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource, and that URI is capable of being defined as a new resource by the requesting user agent, the origin server can create the resource with that URI.</p>\n</blockquote>\n<p>“The enclosed entity be stored” is pretty straightforward: it needs an entity, not a portion of an entity. Furthermore, how is a server going to create a resource without a complete representation if the resource didn’t already exist?</p>\n<p>In this case, the standard’s organization also doesn’t help: if you just read the section titled <code>PUT</code>, you wouldn’t get the full understanding, since the fact that it’s safe and idempotent is mentioned above in the section regarding those two things. I’m not sure why those aspects aren’t in each definition, and are in a different section above, but the point is that you need to consider the full document in its entirety to understand the semantics of <code>PUT</code>. Steve is only reading one section and then extrapolating from there.</p>\n<p>There’s a lot more in that pull request, but one more example: Konstantin points out that Rails supports a lot of things that aren’t standards:</p>\n<blockquote>\n<p>You mean Rails should not support proposed standards like, say, Cookies?</p>\n</blockquote>\n<p>Yep. Did you know that? Cookies aren’t actually a standard, just a proposed one.</p>\n<p>Anyway, I also want to say this: I am not a perfect interpreter of these things either. I often change my opinions after learning more things, and I think this is a good thing. There’s nothing the matter with being wrong; it’s how you handle it that matters. The discussion in this thread continues. RFCs are also not perfect, and do have wiggle-room; but it’s important that agreements are followed. Amongst people who discuss REST and HTTP, the fact that PUT requires a full representation is not controversial; it’s simply understood as true.</p>\n<h2>It’s good for you!</h2>\n<p>I don’t want to turn anyone off from learning new things; exploring new domains is a great path towards personal growth. I’ve said a few times that I think more programmers should read Marx, and it’s because I think this experience of jumping into the deep end of a new set of language that you don’t fully understand is a tremendous educational experience. But to truly learn, an open mind and open ears are crucial. Making arguments on semantics doesn’t work if you don’t understand the context and semantics of words, as words change significantly when placed in different surroundings.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/fast-rails-tests-with-cancan/",
"title": "Fast Rails tests with CanCan",
"description": null,
"url": "https://steveklabnik.com/writing/fast-rails-tests-with-cancan/",
"published": "2011-12-12T00:00:00.000Z",
"updated": "2011-12-12T00:00:00.000Z",
"content": "<p>If you haven’t used it, <a href=\"https://github.com/ryanb/cancan\">CanCan</a> is a great library for Rails that handles authorization for you. Its calling card is simplicity; just do this:</p>\n<pre><code>class Ability\n include CanCan::Ability\n\n def initialize(user)\n user ||= User.new\n\n if user.is? :paying_customer\n can :show, Article\n else\n can :show, Article, :free => true\n cannot :show, Article, :free => false\n end\n end\nend\n</code></pre>\n<p>And then in a controller:</p>\n<pre><code>def show\n @article = Article.find(params[:id])\n authorize! :read, @article\nend\n</code></pre>\n<p>Super simple! However, as simple as CanCan is, if you want to keep your test times super low, there’s a few things that you should know about.</p>\n<h2>Incidental coupling: the silent killer</h2>\n<p>The biggest problem with CanCan from the Fast Tests perspective is that we’ve got coupling with class names. In order to test our Ability class, we also have to load User and Article. Let’s try it without concern for how isolated our tests are:</p>\n<pre><code>require \"cancan/matchers\"\nrequire \"spec_helper\"\n\ndescribe Ability do\n let(:subject) { Ability.new(user) }\n let(:free_article) { Article.new(:free => true) }\n let(:paid_article) { Article.new(:free => false) }\n\n context \"random people\" do\n let(:user) { nil }\n\n it \"can see free Articles\" do\n subject.can?(:show, free_article).should be_true\n end\n\n it \"cannot see non-free Articles\" do\n subject.can?(:show, paid_article).should be_false\n end\n end\n\n context \"paying users\" do\n let(:user) do \n User.new.tap do |u|\n u.roles << :paying_customer\n end\n end\n\n it \"can see free Articles\" do\n subject.can?(:show, free_article).should be_true\n end\n\n it \"can see non-free Articles\" do\n subject.can?(:show, paid_article).should be_true\n end\n end\nend\n</code></pre>\n<p>A few notes about testing with CanCan: I like to organize them with contexts for the different kinds of users, and then what their abilities are below. It gives you a really nice way of setting up those <code>let</code> blocks, and our tests read real nicely.</p>\n<p>Anyway, so yeah. This loads up all of our models, connects to the DB, etc. This is unfortunate. It’d be nice if we didn’t have to load up <code>User</code> and <code>Article</code>. Unfortunately, there’s no real way around it in our <code>Ability</code>, I mean, if you want them to read all Articles, you have to pass in <code>Article</code>… hmm. What about this?</p>\n<pre><code>let(:free_article) { double(:class => Article, :free => true) }\nlet(:paid_article) { double(:class => Article, :free => false) }\n</code></pre>\n<p>Turns out this works just as well. CanCan reflects on the class that you pass in, so if we just give our double the right class, it’s all gravy. Now we’re not loading <code>Article</code>, but what about <code>User</code>?</p>\n<p>The key is in the <code>||=</code>. If we pass in a <code>User</code>, we won’t call <code>User.new</code>. So let’s stub that:</p>\n<pre><code>let(:user) { double(:is? => false) }\nlet(:user) do \n double.tap do |u| #tap not strictly needed in this example, but if you want multiple roles...\n u.stub(:is?).with(:paying_customer).and_return(true)\n end\nend\n</code></pre>\n<p>Sweet! The first one is a random user: all their <code>is?</code> calls should be false. The second is our user who’s paid up; we need to stub out their role properly, but that’s about it.</p>\n<h2>It’s that easy</h2>\n<p>With some dillegence, isolating your tests isn’t hard, and works really well. I have a few more tests in the example I took this from, and 14 examples still take about a third of a second. Not too shabby!</p>\n<p>Happy authorizing!</p>\n<p>Oh, and the final spec, for reference:</p>\n<pre><code>require \"cancan/matchers\"\nrequire \"app/models/ability\"\n\ndescribe Ability do\n let(:subject) { Ability.new(user) }\n let(:free_article) { double(:class => Article, :free => true ) }\n let(:paid_article) { double(:class => Article, :free => false) }\n\n context \"random people\" do\n let(:user) { double(:is? => false) }\n\n it \"can see free Articles\" do\n subject.can?(:show, free_article).should be_true\n end\n\n it \"cannot see non-free Articles\" do\n subject.can?(:show, paid_article).should be_false\n end\n end\n\n context \"users\" do\n let(:user) do \n double.tap do |u|\n u.stub(:is?).with(:user).and_return(true)\n end\n end\n\n it \"can see free Articles\" do\n subject.can?(:show, free_article).should be_true\n end\n\n it \"can see non-free Articles\" do\n subject.can?(:show, paid_article).should be_true\n end\n end\nend\n</code></pre>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/book-review-new-programmers-survival-manual/",
"title": "Book review: New Programmer's Survival Manual",
"description": null,
"url": "https://steveklabnik.com/writing/book-review-new-programmers-survival-manual/",
"published": "2011-11-28T00:00:00.000Z",
"updated": "2011-11-28T00:00:00.000Z",
"content": "<p>Disclaimer: I do have a small credit in this book. A long time ago, the author mentioned that he was looking for people to help out, so I stepped up. It was only participating in discussions like “What do you think about version control’s use in industry?”; I didn’t help with any actual writing, and yesterday was my first time reading virtually all the final content. No monetary interest, of course.</p>\n<h3>Lots of new books lately</h3>\n<p>I’ve been doing a lot of reading over the last few weeks, some of it technical, some of it non. I’ve been meaning to type up a few book reviews, and yesterday I finished reading my copy of Josh Carter’s <a href=\"http://pragprog.com/book/jcdeg/new-programmer-s-survival-manual\">“New Programmer’s Survival Manual,”</a> published by PragProg. Since that’s a bit lighter material than Spinoza, I figured I’d write it up first.</p>\n<p>The <em>Survival Manual</em> is intended to be the book you should buy after you graduate from college, but before you get a job. It’s a bunch of tips, arranged by topic, that give you a taste of what ‘real world’ programming is like, as well as what working life is about. I think that it accomplishes this goal wonderfully, but if you’d like to hear specifics, read on.</p>\n<h2>The Good</h2>\n<p>One of my roommates is not particularly technical. She picked the book up, and read the introduction and the first tip. She then said something that, after reading all of it, I agree with 100%. “I like this guy’s writing style. It’s very… readable.” Josh’s prose feels… comfortable. Like you’re talking to a down-to-earth friend, or co-worker. Which is great, given the topic. I really enjoyed the actual <em>reading</em> of this book. Especially in technical books, this is a great quality to have, but I guess this book isn’t exactly technical; it’s much more about soft skills than hard.</p>\n<p>I also really enjoyed the cookbook-style organization. Basically, the book is one big ball of tips that you can read individually. They’re grouped together with three or four others that are on the same topic, but each stands alone, and they cross-reference each other when appropriate. While I sat down and read it front to back, I’d imagine it’s pretty easy to cherry-pick certain things and read it in a more non-linear fashion.</p>\n<p>I also feel that ‘pragmatic’ is a good description of most of the content. In particular, as much as we all love DVCS, the book frankly states that most places still use centralized version control, which is totally true. He makes sure to carefully not be particularly dogmatic about the specifics of anything, focusing on strategy more than tactics. For example, “Get good with your editor, make sure it can work effectively without using the mouse, for speed,” not “zomg emacs r teh suck, use vim.”</p>\n<p>Finally, Josh links to copious other works, and provides references for all of it. Some stuff is just general internet culture, but the bibliography alone would assist any programmer in finding some useful primary materials to read. Here’s the first six: <em>SICP</em>, <em>Getting Things Done</em>, <em>Extreme Programming Explained</em>, <em>TDD By Example</em>, <em>The Mythical Man-Month</em>, and <em>Foundations of Object Oriented Languages: Types and Semantics</em>. Oh, and just because there’s some XP/Agile stuff in those particular books, don’t think it’s slanted towards that: waterfall is explained as well. Just so happens those books were first.</p>\n<h2>The Bad</h2>\n<p>There’s not that much bad that I can say about this book, really. There are a few small things, though:</p>\n<p>There’s a ‘white belt brown belt black belt’ metaphor that’s supposed to indicate a difficulty of each tip. Given that there’s only two black belt tips, I didn’t feel that three levels were really needed. I also just thought that there wasn’t a huge difference between white and brown, either, so I’m not sure that it really adds any utility.</p>\n<p>Because this book is fairly high-level, it might be hard to actually <em>apply</em> these lessons. This might be more of a problem with readers than with the actual book, but I can’t count how many times I (and others) have said “Of course, write descriptive variable names!!1” and then typed <code>x = 5</code>. It’s just the way things are. And since this book is essentially a collection of advice at that high level, I can see someone reading this book, and then just simply not following the advice. I’m not sure if this is something Josh could have fixed, or if it’s inherent in this kind of book. Also, it won’t affect a diligent reader. It is something to think about, though.</p>\n<h2>Summary: If you just graduated, buy this book</h2>\n<p>This book is totally worth it. The style reminds me of the seminal <em>Pragmatic Programmers</em>. If you’re a freshly minted graduate, put this book at the top of your reading list, then also grab a copy of all the books in the bibliography to read later.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/burnout/",
"title": "Burnout",
"description": null,
"url": "https://steveklabnik.com/writing/burnout/",
"published": "2011-11-17T00:00:00.000Z",
"updated": "2011-11-17T00:00:00.000Z",
"content": "<p>I have a confession to make: Other than one little stint at RubyC, I haven’t really written code in almost a month. And I haven’t really written a ‘real’ blog post in almost six weeks.</p>\n<p>It happens. I was hitting it really hard there for a while, and the combination of stresses from travel, speaking, writing, conference parties, and everything else was really getting to me. I don’t think that I was burned out <em>yet</em>, but I decided to explicitly take the last two weeks off from doing almost anything to prevent myself from getting to that point.</p>\n<p>This doesn’t mean that I’m going to <em>stop</em> doing any of these things. I’d be lying if I tried to say that I’d be reintroducing long-term balance into my life. That balance never really existed. I’ve always been one to throw myself at things 100%. Balance is something I don’t do particularly well.</p>\n<p>One thing I am going to be working on, though, is this: Hackety and Shoes stress me out. Like a lot. This causes me to avoid working on them, which makes them not improve so quickly, which makes me stress about it worse. I’m not really sure how to fix that one… but at some point, I’m going to have to confront that.</p>\n<p>Anyway, hopefully I’ll be back at it soon. Those of you waiting on some of my projects, my bad. Things will still be slow for a bit. I’m going to D.C. this weekend to teach kids with Hackety and Lego Mindstorms, so maybe that’ll get me out of this funk. Or maybe it’ll delay it another couple of days.</p>\n<p>But I’ll be around again soon.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/your-startup-is-not-a-platform/",
"title": "Your startup is not a platform",
"description": null,
"url": "https://steveklabnik.com/writing/your-startup-is-not-a-platform/",
"published": "2011-10-25T00:00:00.000Z",
"updated": "2011-10-25T00:00:00.000Z",
"content": "<p>No, seriously. This is a public service announcment to all MBAs who want to do startups: ‘platform’ does not mean ‘web site.’</p>\n<p>I know that ‘platform’ is a hot thing right now. But seriously, the term has become basically useless. It’s almost to the point that I just pretend you don’t even say it.</p>\n<p>A platform is primarily a B2B play. It can be B2C too, maybe, but largely, to have a platform, you need to enable other people to build something. Usually, that’s another business. But if your website does not enable others to build something, or you’re not white labeling it in some way, you’re not a platform.</p>\n<p>A platform basically needs an API. APIs are the primary way that two pieces of software connect to one another. If your web service doesn’t have an API, it’s not a platform. White labeling being an exception.</p>\n<p>I’d make this rant longer, but I really need a good platform that addresses my market segment. Ideally, it’ll enable some win-win synergies with other channel partners and grow the pie for both of us. Until then, I really won’t find a good solution for this real pain I feel as a content producer.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/for-science-im-killing-my-cell-phone/",
"title": "For science: I'm killing my cell phone",
"description": null,
"url": "https://steveklabnik.com/writing/for-science-im-killing-my-cell-phone/",
"published": "2011-10-19T00:00:00.000Z",
"updated": "2011-10-19T00:00:00.000Z",
"content": "<p>Yep. Doin’ it. This morning I swung by the Sprint store and picked up a 4G hotspot. Gonna transfer my number to Google Voice this evening.</p>\n<h2>lolwhut?</h2>\n<p>Yep. I’m terribly addicted to my phone right now. This is fine, but I’m always interested in improving myself. I have a few friends who’ve done similar things, and really enjoyed it. So I’m giving it a solid 6 months. Sometimes, experiments work, and sometimes they don’t. I’m doing this for fun more than anything, and so it might not work out. Google Maps is damn handy.</p>\n<h2>So I can’t call you?</h2>\n<p>Naw, it just means there might be a tad more latency. I’m on my computer all the effing time anyway, so I doubt that you’ll really notice. If someone calls when I’m not at my computer, Google will just take a recording and email me the transcript. No biggie.</p>\n<h2>Yeah, well, have fun with that!</h2>\n<p>I will. And I’ll keep you all posted on how it goes. I plan to do a retrospective monthly.</p>\n<p>EDIT: I had to change my number because Google Voice wouldn’t transfer my old one. So get a hold of me some other way, and I’ll give you the new one.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/matz-is-nice-so-we-are-nice/",
"title": "Matz is nice so we are nice",
"description": null,
"url": "https://steveklabnik.com/writing/matz-is-nice-so-we-are-nice/",
"published": "2011-10-19T00:00:00.000Z",
"updated": "2011-10-19T00:00:00.000Z",
"content": "<p>Rubyists have a saying, or at least, we used to. “Matz is nice so we are nice.” This has sort of been lost over the years… some people who have been around Ruby for a long time still say this, but it’s something that’s gone in and out of popularity.</p>\n<h2>Nice should be the default</h2>\n<p>Most of being nice, in a software context, is about being considerate. Not unnecessarily trash talking other people’s stuff. Not going out of your way to be a jerk. Try to be helpful. Treat others how you want to be treated.</p>\n<h2>Sometimes, the truth hurts</h2>\n<p>Being considerate of others, however, can only go so far. While a lot of what we do is subjective, a lot of is objective, too. If you’re building a new project, not saying “this is why we’re awesome” is silly. Old code is often bad. Other people’s code is often bad. The key in “not unnecessarily trash talking’ is the <em>neccesary</em> part. There’s a difference between”My library is better\" and “My library is better and their library sucks.”</p>\n<h2>The last thing I’ll say about RVM Drama</h2>\n<p>After talking to a few people, I guess I committed a bit of the sin I was trying to rail against: I got some feedback that it was thought that I was villainizing Sam, and a little easy on Wayne. This is natural, because I’m friends with Wayne and I don’t really know Sam, but in the interest of being nice myself, I just want to say this: I think that all that drama last week was largely a perfect storm of a few different factors, and I don’t think that Sam set out to hurt anyone specifically. If and when I ever have an issue with RVM, I’m totally going to try out rbenv. I like having options. I like there being multiple options. TMTOWTDI. And Sam builds a ton of other software that I really enjoy, like Sprockets, and pow. There’s a big difference between “that guy is an asshole” and “I think that that situation could have been handled better.”</p>\n<p>Let’s all build lots of cool software together.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/rubinius-is-awesome/",
"title": "Rubinius is awesome",
"description": null,
"url": "https://steveklabnik.com/writing/rubinius-is-awesome/",
"published": "2011-10-04T00:00:00.000Z",
"updated": "2011-10-04T00:00:00.000Z",
"content": "<p>You walk into work tomorrow morning, and your boss says this:</p>\n<blockquote>\n<p>Boss: Hey, we’re gonna need to go ahead and have you implement require_relative in rubinius. We have some new servers coming in, and we’ll be running it in production, but we use some 1.9.2 features of MRI that haven’t been implemented yet. So if you could just go ahead and get to implementing that, that would be terrific, OK?</p>\n</blockquote>\n<p>Wat do?</p>\n<p>(Disregard that your boss would never say this. I kinda wish some of mine would…)</p>\n<blockquote>\n<p>You: Excuse me, I believe you have my stapler…</p>\n</blockquote>\n<p>Err, sorry.</p>\n<blockquote>\n<p>You: No problem, boss. Rubinius is mostly just Ruby, and it has a really helpful core team. I’m sure it won’t be a big deal.</p>\n</blockquote>\n<p>Would you say this? Well, you should. I’m not gonna lie and say that <em>every</em> feature is super simple to implement, or rbx-2.0 would be out already, but for a language interpreter, Rubinius is one of the easiest I’ve ever tried to get into. I actually did implement require_relative last night, and it took me about an hour and a half.</p>\n<p>This is my story.</p>\n<p>(note: dramatazation. Actual wording made unneccesarily silly to protect the innocent. please do try this at home)</p>\n<pre><code>$ ssh stevessecretserver\n$ tmux attach\n<brings up irssi, I'm already in #rubinius on Freenode>\n#rubinius: steveklabnik: hey, I need require\\_relative to run my app. How hard\nwould you think that is to implement?\n#rubinius: evan: Naw, man, it'd take like fifteen minutes. Go check out\nhttp://rubygems.org/gems/rbx-require-relative, it's alredy basically got what\nyou want\n#rubinius: steveklabnik: sweet.\n#rubinius: brixen: just grep for 'def require'\n#rubinius: brixen: and run bin/mspec -tx19 core/kernel/require\\_relative to\nrun the RubySpecs for it.\n</code></pre>\n<p>Armed with this knowledge, I found where require is defined in rbx:</p>\n<pre><code>def require(name)\n Rubinius::CodeLoader.require name\nend\nmodule_function :require\n</code></pre>\n<p>Of course, I had to look at CodeLoader:</p>\n<pre><code># Searches for and loads Ruby source files and shared library extension\n# files. See CodeLoader.require for the rest of Kernel#require\n# functionality.\ndef require\n Rubinius.synchronize(self) do\n return false unless resolve_require_path\n return false if CodeLoader.loading? @load_path\n\n if @type == :ruby\n CodeLoader.loading @load_path\n end\n end\n\n if @type == :ruby\n load_file\n else\n load_library\n end\n\n return @type\nend\n</code></pre>\n<p>Hmm, see <code>CodeLoader.require</code>. There are codeloader18 and codeloader19 files…</p>\n<pre><code>module Rubinius\n class CodeLoader\n\n # Searches $LOAD_PATH for a file named +name+. Does not append any file\n # extension to +name+ while searching. Used by #load to resolve the name\n # to a full path to load. Also used by #require when the file extension is\n # provided.\n def search_load_path(name, loading)\n $LOAD_PATH.each do |dir|\n path = \"#{dir}/#{name}\"\n return path if loadable? path\n end\n\n return name if loading and loadable? \"./#{name}\"\n\n return nil\n end\n end\nend\n</code></pre>\n<p>Okay. So we define specific stuff into the 19 file. There’s also a kernel19. So I worked this up:</p>\n<pre><code># Attempt to load the given file, returning true if successful. Works just\n# like Kernel#require, except that it searches relative to the current\n# directory.\n#\ndef require_relative(name)\n Rubinius::CodeLoader.require_relative(name)\nend\nmodule_function :require_relative\n</code></pre>\n<pre><code># requires files relative to the current directory. We do one interesting\n# check to make sure it's not called inside of an eval.\ndef self.require_relative(name)\n scope = Rubinius::StaticScope.of_sender\n script = scope.current_script\n if script\n require File.join(File.dirname(script.data_path), name)\n else\n raise LoadError.new \"Something is wrong in trying to get relative path\"\n end\nend\n</code></pre>\n<p>But then… running the specs gives me bunches of errors. What gives?</p>\n<pre><code><back in irc>\n#rubinius: steveklabnik: yo, my specs are failing, and I'm a n00b. Halp\npl0x?\n#rubinius: brixen: oh, yeah, well, the StaticScope is the current scope,\nand you're down another method, so that's _totally_ gonna screw up\nyour paths. Just get it above and pass it down. Oh, and File.join is\ndumb, File.expand_path that shit!\n#rubinius: steveklabnik: words.\n</code></pre>\n<p>So then I made this one little change:</p>\n<pre><code># Attempt to load the given file, returning true if successful. Works just\n# like Kernel#require, except that it searches relative to the current\n# directory.\n#\ndef require_relative(name)\n scope = Rubinius::StaticScope.of_sender\n Rubinius::CodeLoader.require_relative(name, scope)\nend\nmodule_function :require_relative\n</code></pre>\n<pre><code># requires files relative to the current directory. We do one interesting\n# check to make sure it's not called inside of an eval.\ndef self.require_relative(name, scope)\n script = scope.current_script\n if script\n require File.expand_path(name, File.dirname(script.data_path))\n else\n raise LoadError.new \"Something is wrong in trying to get relative path\"\n end\nend\n</code></pre>\n<pre><code>$ bin/mspec -tx19 core/kernel/require_relativerubinius 2.0.0dev (1.9.2 cade3517 yyyy-mm-dd JI) [x86_64-apple-darwin10.8.0].....................Finished in 0.191995 seconds1 file, 20 examples, 47 expectations, 0 failures, 0 errors\n</code></pre>\n<p>One more pull request, and I’ve got a patch in on rbx. I had one a few months ago, but this one was way cooler.</p>\n<h2>In conclusion</h2>\n<p>Even if I didn’t have some code to look at, because Rubinius is just Ruby, it was really easy to dive in and check out the code, because it’s all Ruby. Seriously. Rubinius is just Ruby. Just Ruby. Ruby.</p>\n<p>So dive in and add some features today! If having an awesome alternate Ruby implementation isn’t enough for you, the rbx team will bribe you with special stickers for committers, and sweet black-on-black tshirts for people with 10 commits or more!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/github-is-anarchy-for-programmers/",
"title": "GitHub is anarchy for programmers",
"description": null,
"url": "https://steveklabnik.com/writing/github-is-anarchy-for-programmers/",
"published": "2011-10-01T00:00:00.000Z",
"updated": "2011-10-01T00:00:00.000Z",
"content": "<p>I finally got to see Zach Holman give his talk, <a href=\"http://speakerdeck.com/u/holman/p/how-github-uses-github-to-build-github\">How GitHub Uses GitHub to Build GitHub</a>. It was awesome. Zach is a great speaker, and the slides don’t do it justice. I highly recommend catching it on video or in person sometime.</p>\n<p>Anyway, there’s something that I found really interesting about GitHub’s ‘process’: it’s basically anarchism, applied to software development.</p>\n<p>If you’re not familliar with anarchism, check out <a href=\"http://truth-reason-liberty.blogspot.com/p/what-i-believe-in.html\">“What I believe in”</a>. It’s a pretty decent introduction to a complicated topic. Ultimately, anarchism is fairly simple, but it requires dismantling a <em>lot</em> of beliefs that Americans in particular hold about society, power, and control. We’re quite happy in this country to be slaves to the rich, but that’s another blog post. ;)</p>\n<p>Another excellent post on that blog is <a href=\"http://truth-reason-liberty.blogspot.com/2010/11/digression-on-what-it-means-to-be.html\">\"A Digression On What It Means to Be An Anarchist</a>. I’m going to use it as a framework to point out my comparison to GitHub. The first sentence is stolen from TFA, but the second sentence is my GitHub comparison.</p>\n<h2>What anarchists oppose</h2>\n<ul>\n<li>Hierarchy – anarchists oppose domination of one person or group of people by another as detrimental to human society. GitHub has built a very flat process, where “some days the CEO ships more code than I do.”</li>\n<li>Authority – all forms of authority must bear a heavy burden of proof in order to demonstrate their legitimacy and necessity. GitHub has hired people that have built awesome things, and have shown themselves to be a trusted authority in whatever kind of software that they build. Nobody needs to question if anyone is in the position to deploy or review some code.</li>\n<li>The state – centralised rule of a set geographical area (country) or people (nation) by a government of elites is inherently illegitimate. Okay, this has nothing to do with GitHub all together. All abstractions are leaky here. :) The same goes for Capitalism and State Socialism too.</li>\n<li>Nationalism and fascism – these are but the worst forms of the state, gaining the loyalty of the people with strong, often brutal discipline and by developing an almost religious, fevered love of the state and the rulers in the form of patriotism. GitHub actively uses multiple programming languages in production, and while it came out of the Ruby world, they don’t really run around pitching “Ruby is awesome and the only thing you should ever use.”</li>\n<li>Discrimination – nobody should be excluded or discriminated against based on nothing more than their gender, ethnicity, sexuality, background, or beliefs. I don’t know everyone at GitHub, but it’s a pretty diverse crew overall. This is hard for any tech company, but they’re doing pretty well on the race/sex thing.</li>\n</ul>\n<h2>What anarchists stand for</h2>\n<ul>\n<li>Liberty – all people should be free to live their life as they see fit, without rules and laws passed from above that serve no purpose other than control and domination, as long as they are not infringing the right of anybody else to the same. See Homan’s talk. That’s pretty much how they work.</li>\n<li>Equality – as stated above, nobody should face discrimination because of their gender, ethnicity, sexuality, background, or beliefs. See above comment about GitHub’s varied devs.</li>\n<li>Community – human civilisation evolved, from its primitive roots, through the priciple of Mutual Aid. GitHub has a lot of elements of mutal aid in their process, where they mention some people in some pull requests, work together on features, and generally high five until the code is running in production.</li>\n<li>Solidarity – humanity is divided only between the rulers and the ruled. Other divisions, those which bring about sexism, racism, heterosexism, and other bigotries, are promoted by the ruling class in order to divide their subjects and keep them under control. GitHub holds tons of events, buys everyone lots of beer, and just generally encourages lots of friendly interactions between everyone. CodeConf was a great group of diverse people hanging out and having a good time.</li>\n</ul>\n<h2>How anarchy would work</h2>\n<ul>\n<li>Self-management – groups, such as workforces or local communities, would be free to operate and govern themselves free of any higher authority. GitHub has no deadlines, no prioritization meetings, and 100% “do what you want” time.</li>\n<li>Free association – all individuals would be free to live where they wanted and associate with who they chose. Zach mentioned in his talk that different people work on different parts of the site, and it’s really based on what people are interested in and what they’ve done in the past.</li>\n<li>Voluntary federation – instead of the state, where indivudal communities and groups of people are bound together by the coercive force of a central authority, local communities and workers collectives can choose for themselves which other communities or collectives to associate with. Sort of like the free association comment, people get together and work on what they want to, and these groups are organic.</li>\n<li>Direct democracy – unlike in parliamentary democracy, these spokespeople would be just that, elected not to a position of authority but to voice decisions that remain in the hands of the people, as in trade union and workers council structures. Anybody can comment on a pull request, add some commits, put in their two cents, and be heard.</li>\n<li>Mutual Aid – in participatory communities and workers collectives, Mutual Aid is a central principle. Easily summed up with the phrase “from each according to his ability, to each according to his need,” this boils down to voluntary cooperation, fair distribution of resources, and help and support to those who need it within a local community. As mentioned before, everybody helps each other out at GitHub to get things done.</li>\n<li>Free, fair trade – for the sharing of resources between different communities and individuals who opt out of Mutual Aid, anarchy would see the emergence of a truly free market. This doesn’t really fit with software, I think. Same with ‘individual liberty.’</li>\n<li>Collective defence – this is not to say that anarchist society will contain “perfect people,” and there will certainly be acts of aggression, oppression, and violence – albeit on a lesser scale than is commonplace in today’s world. I’d liken this to things like “the site is down.” Everyone at GitHub cares about GitHub being up, and so they don’t need someone who’s responsible for ‘defense,’ as if shit hits the fan, everyone chips in and takes care of it.</li>\n<li>Justice, not vengeance – courts would be elected for each individual case, rather than appointed and given unnecessary authority, with the aim to establish guilt or innocence, negotiate reparations, and organise rehabilitation rather than to support the oppressive prison systems which only make matters worse by serving as little more than universities of crime. I’m not sure this fits well here.</li>\n</ul>\n<h2>All abstractions are leaky</h2>\n<p>Just like when ‘lean’ was a concept that applied manufacturing techniques to software development, I don’t think that ‘anarchism’ is a 100% fit with GitHub’s process. I do think that it’s close enough to hold a lot of value, however, even to the point that Zach had to write a blog post about <a href=\"http://zachholman.com/posts/scaling-github-employees/\">scaling</a>, which is something that <em>always</em> gets leveled at people talking about anarchism. I see GitHub in the same way that I see <a href=\"http://en.wikipedia.org/wiki/Anarchism_in_Spain\">anarchist Catalonia</a> or the <a href=\"http://en.wikipedia.org/wiki/Free_Territory\">Free Territory</a> in the Ukraine. It’s a place where ideals are actually working in practice. Let’s just hope that the <a href=\"http://en.wikipedia.org/wiki/Anarchist_Catalonia#Clashes_with_the_Communists\">communists don’t betray them when the fascists show up</a>.</p>\n<p>Damn leaky abstractions.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/real-modern-ruby-development/",
"title": "Real, modern Ruby development",
"description": null,
"url": "https://steveklabnik.com/writing/real-modern-ruby-development/",
"published": "2011-09-28T00:00:00.000Z",
"updated": "2011-09-28T00:00:00.000Z",
"content": "<p>I came across a blog post the other day titled <a href=\"http://ascarter.net/2011/09/25/modern-ruby-development.html\">Modern Ruby Development</a>. While it’s a perfectly fine blog post (other than the digs at rvm…) it really should have been titled something more along the lines of “My default tooling to build a Rails application.” I thought of this yesterday, as I was invited to show up at <a href=\"http://www.vertigo.com/\">Vertigo</a> and show them how we do things in the Open Source world. Vertigo builds lots of cool things, but they tend to be very .NET centric, and it was neat for both of us to see how the other half lives.</p>\n<p>Shortly before my ‘presentation,’ we decided that I’d basicaly just build a little web app in front of them. Live coding to the max! We decided for maximum awesome, and since I was in town for the Twilio conference, that I’d make a Twilio app that they could text and it’d text you ’hello world: you said #{msg}\" back. I thought I’d share that process with you, too. I ended up throwing away the code, though, because I committed my account credentials into the git repo to simplify things, so I’ll just be describing it to you. Check my other posts for code stuff. ;)</p>\n<p>Oh, and one more thing: lots of this stuff is about getting ‘iteration zero’ out of the way: it’s setup work that you have to do to set the tone for everything else. In its own way, it’s also a tooling overview, I’ve just focused on the tools that help the process rather than the language stuff. I’ve already written a more ‘dev proccess’ style post over at <a href=\"http://timelessrepo.com/bdd-with-rspec-and-steak\">Timeless</a>, about BDD.</p>\n<h2>rvm –rvmrc –create 1.9.2@hello_text</h2>\n<p>First up: decide what Ruby you’re using. I talked breifly about MRI, Rubinius, and JRuby, and some pros and cons for each. I still tend to use MRI for my production code at the moment, but soon, rbx will become my default.</p>\n<p>I use rvm because it does exactly what I want.</p>\n<h2>mvim README.md</h2>\n<p>I really enjoy <a href=\"http://tom.preston-werner.com/2010/08/23/readme-driven-development.html\">README driven development</a>. I didn’t actually demo this due to time constraints, but I generally write at least a minimal README.md before doing anything else. During Windy City Rails, Tom shared something that he likes doing: <code>mv README.md SPEC.md</code>. That way, you can keep your actual README in sync with what the code does right now, but have the final one that you want to work towards too.</p>\n<h2>git push origin</h2>\n<p>Now that we have a file, we can push it up to GitHub. A few clicks on the web, a <code>git add remote</code> and <code>git push -u origin master</code> and we’re off to the races.</p>\n<h2>mvim Gemfile && bundle</h2>\n<p>Next up we talked about Bundler, Gemfiles, and managing gems. I talked about the ability to run your own private gem server for internal gems, and how easy it is to build and publish a gem.</p>\n<h2>mkdir spec</h2>\n<p>I then generally type this. I would be lying to you if I told you that I do TDD 100% of the time. Sometimes, I get lazy. But I try to. And since testing is such a huge part of Ruby culture, and we have world-class testing tools, this is where I started.</p>\n<p>I wrote specs for a model class called SmsGateway. This class would handle sending a message to a phone via Twilio. I was able to show off rspec, talk about DSLs, mocking, and TDD.</p>\n<h2>mvim something.rb</h2>\n<p>In this case it’s sms_gateway.rb, but you know what I mean. Write some code, <code>rspec spec/sms_gateway_spec.rb</code>, repeat. Red, Green, Refactor. Wheee!</p>\n<h2>mvim .travis.yml</h2>\n<p>I wanted to show Travis off, so in my demo I did this before implementing the gateway class, but normally I’d do this after the first spec passes. Continuous Integration and even Continuous Deployment are awesome. Travis makes it super easy.</p>\n<h2>Sign up for external services</h2>\n<p>I went over to Twilio’s site, registered a new account, and got my credentials. I used their docs to figure out where to put them, and how to get it all going inside my gateway class (I just mocked it out before), and talked about how needing to Google for docs is a weakness of Ruby.</p>\n<h2>heroku create && git push heroku master</h2>\n<p>Then I showed them how to get going with Heroku, and how easy it was to start giving them hundreds of dollars per month when you need to scale. .NET people are used to paying for everything. ;)</p>\n<p>I also showed off one of my latest projects, which is an IRC bot that lets me deploy rstat.us into production. I have some serious Hubot envy. :D</p>\n<h2>… repeat!</h2>\n<p>I’d be lying if I said it worked the first time, I accidentally made a loop where the app texted itself. Whoops! But that was cool, and showing off my debugging process (I’m a <code>puts/throw params</code> debugger) was fun, as well. After realizing what I did wrong, we had the app working.</p>\n<h2>It’s not quite perfect</h2>\n<p>This dev process isn’t perfect yet. Nothing ever is. I’m always trying to improve upon it, but I’m pretty happy with how quickly I’m able to turn out new projects with a reasonable amount of quality. What’s your workflow? How does mine suck? <a href=\"mailto:[email protected]\">Email me</a>, I’d love to hear about it.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/im-deleting-my-facebook-tonight/",
"title": "I'm deleting my Facebook tonight",
"description": null,
"url": "https://steveklabnik.com/writing/im-deleting-my-facebook-tonight/",
"published": "2011-09-26T00:00:00.000Z",
"updated": "2011-09-26T00:00:00.000Z",
"content": "<p>Well, <a href=\"http://adrianshort.co.uk/2011/09/25/its-the-end-of-the-web-as-we-know-it/\">it’s the end of the web as we know it</a>. I’d already been thinking about this all weekend, and last night, I decided. I just can’t deal with Facebook’s crazy privacy stuff anymore. I mean, I already basically <a href=\"http://rstat.us/\">forked Twitter</a> over their ToS, and Facebook has been a lot more evil for a lot longer. Frictionless sharing is pretty much the last straw.</p>\n<p>I’ll be using this as an excuse to keep helping out with the federated web. There’s a lot of cool stuff in this area, but it needs help. If you’re a developer, consider helping out with the <a href=\"https://github.com/LockerProject/Locker\">Locker Project</a> or <a href=\"https://github.com/homesteading\">the Homesteading Project</a>. The only way that we can prevent people from becoming digial sharecroppers is by helping them to ‘own their own land.’</p>\n<p>If we’re Facebook friends, you can always find out how to reach me at <a href=\"http://steveklabnik.com/\">http://steveklabnik.com</a>. Please be careful, and consider if Facebook’s privacy settings are really worth it to you. They may be, and they may not. I’m under no illusions that the vast majority of people will give up Facebook, but I figured I’d warn you anyways.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/more-rstat-dot-us-refactoring/",
"title": "More rstat.us refactoring",
"description": null,
"url": "https://steveklabnik.com/writing/more-rstat-dot-us-refactoring/",
"published": "2011-09-23T00:00:00.000Z",
"updated": "2011-09-23T00:00:00.000Z",
"content": "<p>Hey everyone! I just wanted to share One More Thing with you about this rstat.us refactoring.</p>\n<p>The main thrust of the last article I posted was to show you a technique for extracting a class, getting it under some kind of test, and then refactoring it a bit. Refactoring is always an iterative process, and Ryan Bates from the always awesome <a href=\"http://railscasts.com/\">Railscasts</a> asked me why I made the Salmon class into a module, especially given my recent <a href=\"/2011/09/06/the-secret-to-rails-oo-design.html\">rant</a> against modules. The answer was, ‘because it’s simpler, and the first thing I thought of.’ He shared with me an alternate implementation that I like too, and I wanted to share with you. Check it:</p>\n<pre><code>class SalmonNotifier\n\tdef initialize(user, feed)\n\t\t@user = user \n\t\t@feed = feed \n\tend\n\n\tdef mention(update) \n\t\tdeliver OStatus::Salmon.new(update.to_atom(\"http://#{@user.author.domain}/\")) \n\tend\n\n\tdef follow \n\t\tdeliver OStatus::Salmon.from_follow(@user.author.to_atom, @feed.author.to_atom) \n\tend\n\n\tdef unfollow \n\t\tdeliver OStatus::Salmon.from_unfollow(@user.author.to_atom, @feed.author.to_atom) \n\tend\n\n\tprotected \n\n\tdef deliver(salmon) \n\t\tsend_envelope_to_salmon_endpoint salmon.to_xml(@user.to_rsa_keypair) \n\tend\n\n\tdef send_envelope_to_salmon_endpoint(envelope)\n\t\turi = URI.parse(@feed.author.salmon_url)\n\t\thttp = Net::HTTP.new(uri.host, uri.port) \n\t\thttp.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"}) \n\tend \nend\n</code></pre>\n<p>This follows a more OO style, and is a bit more well-factored. Here’s his description of what he did:</p>\n<blockquote>\n<ol>\n<li>Switched to a class and gave it a name that is a noun (maybe name should be different though).</li>\n</ol>\n</blockquote>\n<ol>\n<li>\n<p>Moved common method params into instance variables.</p>\n</li>\n<li>\n<p>This simplified the mention/follow/unfollow methods enough to be on one line.</p>\n</li>\n<li>\n<p>Renamed \"<code>send_to_salmon_endpoint</code>\" to \"<code>deliver</code>\" because it feels nicer, the \"salmon endpoint\" can be assumed due to the name of the class. I generally don't like to put the class name in the method name.</p>\n</li>\n<li>\n<p>Extracted commented out into its own method with the same name. I don't know if this is really necessary though (same reason as above).</p>\n</li>\n</ol>\n<p>The only thing that bothers me now is the constant access of <code>@user.author</code> and <code>@feed.author</code>. This makes me think it should be interacting directly with authors. However you still need access to <code>@user.to_rsa_keypair</code> but maybe that method could be moved elsewhere more accessible.</p>\n<blockquote></blockquote>\n<p>All of these changes are pretty good. Ryan’s suggestions for moving forward are pretty good, as well, but I’d add one more thing: we’re starting to collect a bunch of methods all related to delivering more generic HTTP requests in the protected section. This also might be worth pulling out, too. Protected/private in general is a minor smell that indicates there’s code being used multiple times that’s not part of the main objective of the class. It might not be worth it yet, but then again, it might. Speaking of protected methods, you might wonder why I ended up mocking out my protected method in the last post, as well. There are no more tests that actually test the method is working properly. This is true, and it’s because I was leading into thinking of something like this. Mocked tests are almost always unit tests, and two methods are two separate units, so I had just ended up doing it out of habit. This further shows that maybe an extraction of another class is worthwhile.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/extracting-domain-models-a-practical-example/",
"title": "Extracting domain models: a practical example",
"description": null,
"url": "https://steveklabnik.com/writing/extracting-domain-models-a-practical-example/",
"published": "2011-09-22T00:00:00.000Z",
"updated": "2011-09-22T00:00:00.000Z",
"content": "<p>Hey everyone! We’ve been doing a lot of refactoring on rstat.us lately, and I wanted to share with you a refactoring that I did. It’s a real-world example of doing the domain models concept that I’ve been talking about lately.</p>\n<h2>Step one: check the tests</h2>\n<blockquote>\n<p>I don’t know how much more emphasized step 1 of refactoring could be: don’t touch anything that doesn’t have coverage. Otherwise, you’re not refactoring; you’re just changing shit. - Hamlet D’Arcy</p>\n</blockquote>\n<p>One of the best reasons to extract extra domain models is that they’re often much easier to test. They have less dependencies, less code, and are much simpler than the Mega Models that happen if you have a 1-1 model to table ratio.</p>\n<p>Let’s check out the code. It’s in our <a href=\"https://github.com/hotsh/rstat.us/blob/362cb38031/app/models/user.rb#L209\">User model</a>:</p>\n<pre><code># Send an update to a remote user as a Salmon notification\ndef send_mention_notification update_id, to_feed_id\n f = Feed.first :id => to_feed_id\n u = Update.first :id => update_id\n\n base_uri = \"http://#{author.domain}/\"\n salmon = OStatus::Salmon.new(u.to_atom(base_uri))\n\n envelope = salmon.to_xml self.to_rsa_keypair\n\n # Send envelope to Author's Salmon endpoint\n uri = URI.parse(f.author.salmon_url)\n http = Net::HTTP.new(uri.host, uri.port)\n res = http.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"})\nend\n</code></pre>\n<p>You’re probably saying “this is pretty reasonable.” Prepare to get shocked…</p>\n<p>This code has very little to do with the user, other than using some of the User’s attributes. This really could be factored out into a domain object whose job it is to push Salmon notifications. But first, let’s see if we can get some tests in place so we know <em>everything</em> isn’t broken. Examining this method, we need two things: an <code>update_id</code> and a <code>feed_id</code>. Here’s my first stab at an integration test:</p>\n<pre><code>describe \"salmon update\" do\n it \"integration regression test\" do\n feed = Feed.create\n update = Update.create\n\n salmon = double(:to_xml => \"<xml></xml>\")\n uri = double(:host => \"localhost\", :port => \"9001\", :path => \"/\")\n Ostatus::Salmon.should_receive(:new).and_return(salmon)\n Uri.should_receive(:parse).and_return(uri)\n Net::HTTP.should_receive(:new).and_return(mock(:post => true))\n\n user = User.create\n user.send_mention_notification(update.id, feed.id)\n end\nend\n</code></pre>\n<p>This is an integration test, we’re obviously testing stuff with a ton of models. After running <code>ruby test/models/notify_via_salmon.rb</code> about a zillion times, I ended up with this running test:</p>\n<pre><code>require 'minitest/autorun'\nrequire 'rspec/mocks'\n\nrequire 'mongo_mapper'\nrequire 'whatlanguage'\nMongoMapper.connection = Mongo::Connection.new(\"localhost\")\nMongoMapper.database = 'rstatus-test'\n\nrequire_relative '../../app/models/feed'\nrequire_relative '../../app/models/update'\nrequire_relative '../../app/models/author'\nrequire_relative '../../app/models/authorization'\n\n$:.unshift(\"lib\")\nrequire_relative '../../app/models/user'\n\nrequire 'ostatus'\n\ndescribe \"salmon update\" do\n before :each do\n User.all.each(&:destroy)\n RSpec::Mocks::setup(self)\n end\n\n it \"integration regression test\" do\n author = Author.create\n user = User.create!(:username => \"steve\", :author => author)\n\n feed = user.feed\n update = Update.create!(:feed_id => feed.id, :author_id => author.id, :text => \"hello world\")\n\n salmon = double(:to_xml => \"<xml></xml>\")\n uri = double(:host => \"localhost\", :port => \"9001\", :path => \"/\")\n OStatus::Salmon.should_receive(:new).and_return(salmon)\n URI.should_receive(:parse).and_return(uri)\n Net::HTTP.should_receive(:new).and_return(mock(:post => true))\n\n user.send_mention_notification(update.id, feed.id)\n end\nend\n</code></pre>\n<p>Holy. Fuck. Seriously. Let me just quote <a href=\"http://avdi.org/devblog/2011/04/07/rspec-is-for-the-literate/\">Avdi Grimm</a>:</p>\n<blockquote>\n<p>Mocks, and to some degree RSpec, are like the Hydrogen Peroxide of programming: they fizz up where they encounter subtle technical debt.</p>\n</blockquote>\n<p>This test class is insane. It runs faster than many people’s integration tests, clocking in at about 0.4 seconds. But that’s still an order of magnitude slower than I’d like. A suite with 200 tests would still take over a minute. Also, look at all this junk that we have to do for setup.</p>\n<p>All of this pain and effort doesn’t mean that testing sucks: it means that our implementation is terrible. So let’s use that to guide us. We’ll try to fix this code by eliminating all of this junk.</p>\n<h2>Step two: simple extraction</h2>\n<p>First, let’s extract this out to a domain model. Here’s the new code:</p>\n<pre><code># Send an update to a remote user as a Salmon notification\ndef send_mention_notification update_id, to_feed_id\n NotifyViaSalmon.mention(update_id, to_feed_id)\nend\n</code></pre>\n<pre><code>module NotifyViaSalmon\n extend self\n\n def mention(update_id, to_feed_id)\n f = Feed.first :id => to_feed_id\n u = Update.first :id => update_id\n\n base_uri = \"http://#{author.domain}/\"\n salmon = OStatus::Salmon.new(u.to_atom(base_uri))\n\n envelope = salmon.to_xml self.to_rsa_keypair\n\n # Send envelope to Author's Salmon endpoint\n uri = URI.parse(f.author.salmon_url)\n http = Net::HTTP.new(uri.host, uri.port)\n res = http.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"})\n end\nend\n</code></pre>\n<p>Now, when we run the test (via <code>ruby test/models/notify_via_salmon_test.rb</code>) we see an error:</p>\n<pre><code>1) Error:\ntest_0001_integration_regression_test(SalmonUpdateSpec):\nNameError: undefined local variable or method `author' for NotifyViaSalmon:Module\n /Users/steveklabnik/src/rstat.us/app/models/notify_via_salmon.rb:8:in `mention'\n</code></pre>\n<p>This is a form of Lean On The Compiler. This test is now failing because it’s relying on stuff that used to be internal to the User, and we don’t have that stuff now. After doing this a few times, we’re left with this:</p>\n<pre><code># Send an update to a remote user as a Salmon notification\ndef send_mention_notification update_id, to_feed_id\n NotifyViaSalmon.mention(self, update_id, to_feed_id)\nend\n</code></pre>\n<pre><code>module NotifyViaSalmon\n extend self\n\n def mention(user, update_id, to_feed_id)\n f = Feed.first :id => to_feed_id\n u = Update.first :id => update_id\n\n base_uri = \"http://#{user.author.domain}/\"\n salmon = OStatus::Salmon.new(u.to_atom(base_uri))\n\n envelope = salmon.to_xml user.to_rsa_keypair\n\n # Send envelope to Author's Salmon endpoint\n uri = URI.parse(f.author.salmon_url)\n http = Net::HTTP.new(uri.host, uri.port)\n res = http.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"})\n end\nend\n</code></pre>\n<p>Okay. Now we’re cooking. We have a test that’s isolated, yet still does what we thought it did before.</p>\n<h2>Step three: break dependencies</h2>\n<p>Next step: let’s break the hard dependency this method has on both Feed and Update. We can do this by moving the finds up into the User method instead of keeping them in the mention method:</p>\n<pre><code># Send an update to a remote user as a Salmon notification\ndef send_mention_notification update_id, to_feed_id\n feed = Feed.first :id => to_feed_id\n update = Update.first :id => update_id\n\n NotifyViaSalmon.mention(self, update, feed)\nend\n</code></pre>\n<pre><code>module NotifyViaSalmon\n extend self\n\n def mention(user, update, feed)\n base_uri = \"http://#{user.author.domain}/\"\n salmon = OStatus::Salmon.new(update.to_atom(base_uri))\n\n envelope = salmon.to_xml user.to_rsa_keypair\n\n # Send envelope to Author's Salmon endpoint\n uri = URI.parse(feed.author.salmon_url)\n http = Net::HTTP.new(uri.host, uri.port)\n res = http.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"})\n end\nend\n</code></pre>\n<p>Okay. Tests passing. Sweet. Now we can try testing <em>just</em> the mention method. Let’s try it by killing most of that crap that was in our test:</p>\n<pre><code>require 'minitest/autorun'\nrequire 'rspec/mocks'\n\nrequire_relative '../../app/models/notify_via_salmon'\n\ndescribe \"salmon update\" do\n before :each do\n RSpec::Mocks::setup(self)\n end\n\n it \"integration regression test\" do\n NotifyViaSalmon.mention(stub, stub, stub)\n end\nend\n</code></pre>\n<p>Let’s try running this and seeing what happens:</p>\n<pre><code>$ ruby test/models/notify_via_salmon_test.rb\nLoaded suite test/models/notify_via_salmon_test\nStarted\nE\nFinished in 0.000992 seconds.\n\n 1) Error:\ntest_0001_integration_regression_test(SalmonUpdateSpec):\nRSpec::Mocks::MockExpectationError: Stub received unexpected message :author with (no args)\n</code></pre>\n<p>First of all, daaaaamn. 0.001 seconds. Nice. Secondly, okay, we are getting some messages sent to our stubs. Let’s flesh them out to make things pass:</p>\n<pre><code>require 'minitest/autorun'\nrequire 'rspec/mocks'\nrequire 'ostatus'\n\nrequire_relative '../../app/models/notify_via_salmon'\n\ndescribe \"salmon update\" do\n before :each do\n RSpec::Mocks::setup(self)\n end\n\n it \"integration regression test\" do\n user = stub(:author => stub(:domain => \"foo\"), :to_rsa_keypair => stub)\n update = stub(:to_atom => \"\")\n feed = stub(:author => stub(:salmon_url => \"\"))\n\n salmon = stub(:to_xml => \"\")\n OStatus::Salmon.should_receive(:new).and_return(salmon)\n\n uri = stub(:host => \"\", :port => \"\", :path => \"\")\n URI.should_receive(:parse).and_return(uri)\n Net::HTTP.should_receive(:new).and_return(stub(:post => \"\"))\n\n NotifyViaSalmon.mention(user, update, feed)\n end\nend\n</code></pre>\n<p>Okay! Note how similar this looks to the previous test. We’re still mocking a bunch of ‘external’ stuff. But other than that, our test is pretty simple: we make three stubs, and we call our mention method.</p>\n<h3>Step four: repeat</h3>\n<p>We could do some more work on this method. There are a few things that are a bit smelly: the nested stub of User is not great. The fact that we’re stubbing out three external dependencies isn’t great. But before we get to that, let’s check out another method that looks similar: <code>send_{follow,unfollow}_notification</code>. Here’s <a href=\"https://github.com/hotsh/rstat.us/blob/362cb38031/app/models/user.rb#L167\">some code</a>:</p>\n<pre><code># Send Salmon notification so that the remote user\n# knows this user is following them\ndef send_follow_notification to_feed_id\n f = Feed.first :id => to_feed_id\n\n salmon = OStatus::Salmon.from_follow(author.to_atom, f.author.to_atom)\n\n envelope = salmon.to_xml self.to_rsa_keypair\n\n # Send envelope to Author's Salmon endpoint\n uri = URI.parse(f.author.salmon_url)\n http = Net::HTTP.new(uri.host, uri.port)\n res = http.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"})\nend\n\n# Send Salmon notification so that the remote user\n# knows this user has stopped following them\ndef send_unfollow_notification to_feed_id\n f = Feed.first :id => to_feed_id\n\n salmon = OStatus::Salmon.from_unfollow(author.to_atom, f.author.to_atom)\n\n envelope = salmon.to_xml self.to_rsa_keypair\n\n # Send envelope to Author's Salmon endpoint\n uri = URI.parse(f.author.salmon_url)\n http = Net::HTTP.new(uri.host, uri.port)\n res = http.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"})\nend\n</code></pre>\n<p>Look familliar? Yep! 90% the same code! We’ll do the same process to these methods, just like the other one. Check out this code and test:</p>\n<pre><code># Send Salmon notification so that the remote user\n# knows this user is following them\ndef send_follow_notification to_feed_id\n feed = Feed.first :id => to_feed_id\n\n NotifyViaSalmon.follow(self, feed)\nend\n\n# Send Salmon notification so that the remote user\n# knows this user has stopped following them\ndef send_unfollow_notification to_feed_id\n feed = Feed.first :id => to_feed_id\n\n NotifyViaSalmon.unfollow(self, feed)\nend\n\n# Send an update to a remote user as a Salmon notification\ndef send_mention_notification update_id, to_feed_id\n feed = Feed.first :id => to_feed_id\n update = Update.first :id => update_id\n\n NotifyViaSalmon.mention(self, update, feed)\nend\n</code></pre>\n<pre><code>module NotifyViaSalmon\n extend self\n\n def mention(user, update, feed)\n base_uri = \"http://#{user.author.domain}/\"\n salmon = OStatus::Salmon.new(update.to_atom(base_uri))\n\n envelope = salmon.to_xml user.to_rsa_keypair\n\n # Send envelope to Author's Salmon endpoint\n uri = URI.parse(feed.author.salmon_url)\n http = Net::HTTP.new(uri.host, uri.port)\n res = http.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"})\n end\n\n def follow(user, feed)\n salmon = OStatus::Salmon.from_follow(user.author.to_atom, feed.author.to_atom)\n\n envelope = salmon.to_xml user.to_rsa_keypair\n\n # Send envelope to Author's Salmon endpoint\n uri = URI.parse(feed.author.salmon_url)\n http = Net::HTTP.new(uri.host, uri.port)\n res = http.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"})\n end\n\n def unfollow(user, feed)\n salmon = OStatus::Salmon.from_unfollow(user.author.to_atom, feed.author.to_atom)\n\n envelope = salmon.to_xml user.to_rsa_keypair\n\n # Send envelope to Author's Salmon endpoint\n uri = URI.parse(feed.author.salmon_url)\n http = Net::HTTP.new(uri.host, uri.port)\n res = http.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"})\n end\nend\n</code></pre>\n<pre><code>require 'minitest/autorun'\nrequire 'rspec/mocks'\nrequire 'ostatus'\n\nrequire_relative '../../app/models/notify_via_salmon'\n\ndescribe NotifyViaSalmon do\n before :each do\n RSpec::Mocks::setup(self)\n end\n\n it \".mention\" do\n user = stub(:author => stub(:domain => \"foo\"), :to_rsa_keypair => stub)\n update = stub(:to_atom => \"\")\n feed = stub(:author => stub(:salmon_url => \"\"))\n\n salmon = stub(:to_xml => \"\")\n OStatus::Salmon.should_receive(:new).and_return(salmon)\n\n uri = stub(:host => \"\", :port => \"\", :path => \"\")\n URI.should_receive(:parse).and_return(uri)\n Net::HTTP.should_receive(:new).and_return(stub(:post => \"\"))\n\n NotifyViaSalmon.mention(user, update, feed)\n end\n\n it \".follow\" do\n user = stub(:author => stub(:domain => \"foo\", :to_atom => \"\"), :to_rsa_keypair => stub)\n feed = stub(:author => stub(:salmon_url => \"\", :to_atom => \"\"))\n\n salmon = stub(:to_xml => \"\")\n OStatus::Salmon.should_receive(:from_follow).and_return(salmon)\n\n uri = stub(:host => \"\", :port => \"\", :path => \"\")\n URI.should_receive(:parse).and_return(uri)\n Net::HTTP.should_receive(:new).and_return(stub(:post => \"\"))\n\n NotifyViaSalmon.follow(user, feed)\n end\n\n it \".unfollow\" do\n user = stub(:author => stub(:domain => \"foo\", :to_atom => \"\"), :to_rsa_keypair => stub)\n feed = stub(:author => stub(:salmon_url => \"\", :to_atom => \"\"))\n\n salmon = stub(:to_xml => \"\")\n OStatus::Salmon.should_receive(:from_unfollow).and_return(salmon)\n\n uri = stub(:host => \"\", :port => \"\", :path => \"\")\n URI.should_receive(:parse).and_return(uri)\n Net::HTTP.should_receive(:new).and_return(stub(:post => \"\"))\n\n NotifyViaSalmon.unfollow(user, feed)\n end\nend\n</code></pre>\n<p>Now we can see some of these patterns start to emerge, eh? All of this stuff is starting to come together. Let’s get rid of the URI and Net::HTTP stuff, as it’s just straight up identical in all three. Pretty basic Extract Method:</p>\n<pre><code>module NotifyViaSalmon\n extend self\n\n def mention(user, update, feed)\n base_uri = \"http://#{user.author.domain}/\"\n salmon = OStatus::Salmon.new(update.to_atom(base_uri))\n\n send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)\n end\n\n def follow(user, feed)\n salmon = OStatus::Salmon.from_follow(user.author.to_atom, feed.author.to_atom)\n\n send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)\n end\n\n def unfollow(user, feed)\n salmon = OStatus::Salmon.from_unfollow(user.author.to_atom, feed.author.to_atom)\n\n send_to_salmon_endpoint(salmon, user.to_rsa_keypair, feed.author.salmon_url)\n end\n\n protected\n\n def send_to_salmon_endpoint(salmon, keypair, uri)\n envelope = salmon.to_xml keypair\n\n # Send envelope to Author's Salmon endpoint\n uri = URI.parse(uri)\n http = Net::HTTP.new(uri.host, uri.port)\n res = http.post(uri.path, envelope, {\"Content-Type\" => \"application/magic-envelope+xml\"})\n end\nend\n</code></pre>\n<p>We run our tests, it still all works. Now we can just mock out the endpoint method, and all of our tests on the individual methods become much, much simpler:</p>\n<pre><code>require 'minitest/autorun'\nrequire 'rspec/mocks'\nrequire 'ostatus'\n\nrequire_relative '../../app/models/notify_via_salmon'\n\ndescribe NotifyViaSalmon do\n before :each do\n RSpec::Mocks::setup(self)\n end\n\n it \".mention\" do\n user = stub(:author => stub(:domain => \"foo\"), :to_rsa_keypair => stub)\n update = stub(:to_atom => \"\")\n feed = stub(:author => stub(:salmon_url => \"\"))\n\n NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)\n\n NotifyViaSalmon.mention(user, update, feed)\n end\n\n it \".follow\" do\n user = stub(:author => stub(:domain => \"foo\", :to_atom => \"\"), :to_rsa_keypair => stub)\n feed = stub(:author => stub(:salmon_url => \"\", :to_atom => \"\"))\n\n salmon = stub(:to_xml => \"\")\n OStatus::Salmon.should_receive(:from_follow).and_return(salmon)\n\n NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)\n\n NotifyViaSalmon.follow(user, feed)\n end\n\n it \".unfollow\" do\n user = stub(:author => stub(:domain => \"foo\", :to_atom => \"\"), :to_rsa_keypair => stub)\n feed = stub(:author => stub(:salmon_url => \"\", :to_atom => \"\"))\n\n salmon = stub(:to_xml => \"\")\n OStatus::Salmon.should_receive(:from_unfollow).and_return(salmon)\n\n NotifyViaSalmon.should_receive(:send_to_salmon_endpoint)\n\n NotifyViaSalmon.unfollow(user, feed)\n end\nend\n</code></pre>\n<p>We’ve managed to get rid of all of the mocking of the URI stuff. It’s all details now. We also have great information about what is actually needed for this test: we know exactly what our objects expect the caller to pass. We’re only depending on ourselves and some external libraries. Our tests are fast. Overall, I’m feeling much better about this code than I was before.</p>\n<h2>In conclusion</h2>\n<p>I hope you’ll take a few things away from this post.</p>\n<p><strong>Start with tests first!</strong> If you don’t do this, it’s very easy to introduce simple errors. Don’t be that guy. You know, the one that has to make <a href=\"https://github.com/hotsh/rstat.us/pull/398/files\">pull requests like this…</a> I suck.</p>\n<p><strong>Don’t be afraid to change the tests!</strong> As soon as you’ve verified that you’ve transcribed the code correctly, don’t be afraid to just nuke things and start again. Especially if you have integration level tests that confirm that your features actually <em>work</em>, your unit tests are expendable. If they’re not useful, kill them!</p>\n<p><strong>Mock like a motherfucker.</strong> When done correctly, mocks allow you to help design your code better, make your tests fast, and truly isolate the unit under test. It’s pretty much all upside, unless you suck at them.</p>\n<p><strong>Extract, extract, extract!</strong> Adele Goldberg once said, “In Smalltalk, everything happens somewhere else.” Make tons of little methods. They help to provide seams that are useful for testing. Smaller methods are also easier to understand.</p>\n<p><strong>Refactoring is a marathon.</strong> Even with all this work, this code isn’t the best. The tests could even use a little TLC. But it’s still better than it was before. Those who ship, win. Make things a little better, <code>git commit && git push</code>, repeat.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/we-forget-that-open-source-is-made-of-people/",
"title": "We forget that open source is made of people",
"description": null,
"url": "https://steveklabnik.com/writing/we-forget-that-open-source-is-made-of-people/",
"published": "2011-09-12T00:00:00.000Z",
"updated": "2011-09-12T00:00:00.000Z",
"content": "<p>Programmers like to think that software is the purest form of meritocracy. We like to consider ourselves scientists; what we do is entirely impersonal. I’m at Lone Star Ruby right now, and the first two talks are entirely about software as a science.</p>\n<p>But this is a fallacy. There’s a human element to open source that’s often totally forgotten, and it drives people away.</p>\n<h2>Software Celebrity and Hero Worship</h2>\n<p>We sort of acknowledge this, in a way. There are most certainly software celebrities, even if their celebrity is localized to the community in which they operate. I heard someone say this today:</p>\n<blockquote>\n<p>If I can go to one Ruby conference and not hear anyone mention Zed Shaw or _why, I can die happy.</p>\n</blockquote>\n<p>If I say “jnicklas” you say “Capybara.” If I say “DHH” you say “Rails.” If I say “matz” you say “Ruby.” We have our heroes, villains, and foils, just like anybody else. But we forget that in every celebrity, there is a real person in there. Actual people, who have real thoughts, feelings, ideas, emotions, and that spend all of their time working for free just to make your lives better out of the goodness of their heart. Often <em>after</em> working at day jobs, often <em>at the expense</em> of a personal life, often <em>straining relationships</em> with people they care about.</p>\n<h2>Competition Means Somebody Loses</h2>\n<p>I used to think that Git was great because it made forks trivial. It changed what we mean by ‘fork.’ This is largely a good thing. But ‘the eff word’ used to be a last resort. The nuclear option. You’re fed up, can’t take it anymore, and so this is what they’ve driven you to do. Forking was Serious Business.</p>\n<p>Now, GitHub has made it so easy to fork a project and distribute it to tons of people, people just fork things willy-nilly. And, from the outside, competition is good: in theory, it means that the good ideas work and the bad ideas don’t.</p>\n<p>(A small aside: forking in git is different than ‘forking’ in svn. This overloading of terminology is unfortunate. The fork button is actually one of my favorite features of GitHub, because it makes contribution easier. But as it’s easier to publish new software, it’s easier to ‘fork’ in the svn sense. It used to be a pain to start a new project, and GitHub’s made that so easy that it happens more often than I think it should.)</p>\n<p>But guess what: competition sucks. Cooperation is much better. One of the biggest problems with competition is that it means that somebody has to lose. For every winner, someone is a loser. That person is going to be hurting. In the case of open source, someone has spent (possibly) years of their life working on something, and now, all of that is gone. That shit stings.</p>\n<p>Sometimes, forks are necessary. Sometimes, there are approaches that are fundamentally different. Forking is still an option, even if it’s a last resort. I’m also not saying that feelings should totally trump technical merit. All I’m saying is this: consider the blood, sweat, and tears of others before you act. Just in general.</p>\n<p>It’s also not a zero-sum game, either. The pie can be grown. But that also doesn’t mean that it’s grown in a way that makes anyone feel good about it. There’s a difference between “We represent two sides of some coin” and “Those fuckers make shitty software.” Both approaches can grow the pie. One is far better than the other.</p>\n<p>What’s the human cost of rapid competition between software projects? Is this cost worth it?</p>\n<h2>Open Source: All take, no give</h2>\n<p>The Pareto Principle: 80% of the effects come from 20% of the causes. This is absolutely true in OSS, and it might be more imbalanced. A few people contribute, and lots of people use. Sometimes, this is a good thing: it’s actually what draws me to open source. I can help people in a scalable way. My code is currently being used by tens of thousands of people. This is awesome. Some documentation that I’ve written is being read by millions of people. This is way more awesome.</p>\n<p>But I rarely get any help with my projects. It’s really easy to let that get to a person.</p>\n<p>Do you know how many people have told me that what I do is important? I got given a Ruby Hero award for my work on Hackety and Shoes. Do you know how many commits I have in on them? Probably less than 200. Do you know why? I’ve spent weeks of my life trying to track down bugs, but some of them are too hard for me, at my current stage of programming-fu. And it gets depressing. And everyone that’s ever offered to help has flaked on me. And all the recognition for something that was largely done by someone else gets to me, really deeply. And I get tons of people screaming about Lion support, and ‘this doesn’t work,’ and all sorts of things… and no patches. It’s why the ‘patches accepted’ saying was coined: this shit grinds on you after a while.</p>\n<h2>All of my Heroes are Dying</h2>\n<p>Today, this situation with rbenv and rvm made me realize something: everyone that I truly, deeply respect in open source has either burned out or left. I’ve said once before that there are four people that I truly respect in software: Zed Shaw, _why, Yehuda Katz, and (more recently) Wayne Seguin (and also Aaron Patterson. Dammit. This list is getting long). All of these developers are brilliant, work hard, and are tirelessly supporting their communities. That is, until they were driven away. This isn’t totally applicable to Yehuda, but it happened to Zed. It happened to _why. And I can see it happening, right now, to Aaron and Wayne. And Yehuda isn’t really doing Ruby…</p>\n<p>But seeing this happen makes me deeply sad. For them personally, for my own selfish reasons, and that that’s the state of our communities.</p>\n<h2>Where Does This Leave Us?</h2>\n<p>I don’t have any answers. I only have a bunch of questions:</p>\n<ul>\n<li>Why do we drive people away from Open Source?</li>\n<li>Can we learn to work together, instead of competing?</li>\n<li>How can we better recognize this human element?</li>\n</ul>\n<p>If you don’t contribute to Open Source: please consider thanking someone that does open source, in a project that you love. Consider helping out, in even the smallest of ways.</p>\n<p>Thanks.</p>\n<p><a href=\"/2011/08/19/matz-is-nice-so-we-are-nice.html\">My next post is a bit of a follow-up to this</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/better-ruby-presenters/",
"title": "Better Ruby Presenters",
"description": null,
"url": "https://steveklabnik.com/writing/better-ruby-presenters/",
"published": "2011-09-09T00:00:00.000Z",
"updated": "2011-09-09T00:00:00.000Z",
"content": "<p>My <a href=\"/2011/09/06/the-secret-to-rails-oo-design.html\">last blog post</a> caused a bit of a stir in some circles. I got a bunch of emails. Apparently, I need to expand on a few things. So here we go. Let’s rap about the Presenter pattern, shall we?</p>\n<h2>No seriously, helpers suck</h2>\n<blockquote>\n<p>In Ruby, everything is an object. Every bit of information and code can be given their own properties and actions. - ruby-lang.org/about</p>\n</blockquote>\n<p>… except helpers. Why is it in Ruby that everything is an object, even integers, yet as soon as we need to format a date, we all turn into Dijkstra and bust out structured programming? Actually, come to think of it, Dijkstra wouldn’t even write any code, because it’s beneath him, but you get the idea. (if you don’t get this, it’s a joke. About proofs, telescopes, and CS…) Helpers are like a compliment that you don’t want to give, but feel obligated to: “Yeah, I mean, well, you tried, and that’s what counts, right?”</p>\n<p>This is the topic of a future post, but when programming in a language, you want to work with its primary methods of abstraction. In Ruby, that’s objects, and there’s a good reason for that: functions don’t provide a sufficient amount of power to tackle hard problems. I don’t want to get into this either, but there’s a reason that objects exist, and that nobody’s making procedural languages anymore. No biggie. C has a special place in my heart.</p>\n<p>But I digress: objects > functions. At least in the context of getting stuff done in Ruby. This pretty much applies to most of the rest of the points in this post, so just keep that in the back of your brain.</p>\n<h2>Why not a class method?</h2>\n<p>Well, first of all, <a href=\"http://nicksda.apotomo.de/2011/07/are-class-methods-evil/\">class methods also suck</a>. Here’s the issue: Can you tell me the difference between these two methods?</p>\n<pre><code>def foo\n \"hello\"\nend\n\nclass Bar\n def self.foo\n \"hello\"\n end\nend\n\nfoo\nBar.foo\n</code></pre>\n<p>Yep, that’s right. You can see it clearly from the last two lines: class methods are functions that happen to be namespaced. That means they’re slightly better, but really, <code>DictionaryPresenter.as_dictionary(something)</code> might as well just be a helper. At least <code>as_dictionary(something)</code> is shorter.</p>\n<p>Here’s another reason: I didn’t make that method be a class method, even though there was no state, because a really real presenter (I mean, that was a real world example, but…) generally works a little different. Usually, I make presenters that actually stand in for the objects they’re presenting. Check this out. Here’s the presenter I showed you:</p>\n<pre><code>class DictionaryPresenter\n def initialize(collection)\n @collection = collection\n end\n\n def as_dictionary\n dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}\n\n @collection.each do |p|\n dictionary[p.title[0]] << p\n end\n\n dictionary\n end\nend\n</code></pre>\n<p>The real-world presenter that I used this for looked like this:</p>\n<pre><code>class DictionaryPresenter\n include Enumerable\n\n def initialize(collection)\n @dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}\n\n collection.each do |p|\n @dictionary[p.title[0]] << p\n end\n end\n\n def each &blk\n @dictionary.each &blk\n end\nend\n</code></pre>\n<p>… or close to this. There was an ‘other’ category, and a few other things… but you get the idea. Now, instead of this:</p>\n<pre><code>@posts = DictionaryPresenter.new(Post.all).as_dictionary\n</code></pre>\n<p>You do this:</p>\n<pre><code>@posts = DictionaryPresenter.new(Post.all)\n</code></pre>\n<p>And the presenter actually stands in for the hash. A subtle but important difference. This gives you more options, because you have a real, live object instead of just some random processing. With this simplistic presenter, you might not see a huge benefit, but often, you’ll want something a bit more complicated. And since this was what I had in my head, I didn’t make <code>as_dictionary</code> be a static method.</p>\n<p>Oh, and presenters that decorate a single object rather than a collection often implement <code>method_missing</code>, so that you only <em>add</em> methods rather than hide most of them. Some don’t. It depends.</p>\n<h2>Blocks vs. Policy</h2>\n<p>I also showed off a Ruby version of a design pattern known as the Policy pattern, or sometimes the Strategy pattern. Turns out that Policy is often a bit heavyweight for Ruby, especially in an example like this, so I wanted to show you an alternate version of it, taking advantage of a Ruby feature: blocks.</p>\n<p>For reference, the old code, put into the new code:</p>\n<pre><code>class DictionaryPresenter\n include Enumerable\n\n def initialize(policy, collection)\n @dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}\n\n collection.each do |p|\n @dictionary[policy.category_for(p)] << p\n end\n end\n\n def each &blk\n @dictionary.each &blk\n end\nend\n\nclass UserCategorizationPolicy\n def self.category_for(user)\n user.username[0]\n end\nend\n</code></pre>\n<p>We could use blocks instead:</p>\n<pre><code>class DictionaryPresenter\n include Enumerable\n\n def initialize(collection, &blk)\n @dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}\n\n collection.each do |p|\n @dictionary[blk.call(p)] << p\n end\n end\n\n def each &blk\n @dictionary.each &blk\n end\nend\n\nDictionaryPresenter.new(Post.all) do |item|\n item.title[0]\nend\n</code></pre>\n<p>While this is shorter, and a bit more Rubyish, it also means that we lose the <a href=\"http://en.wikipedia.org/wiki/Reification_(computer_science)\">reification</a> of our concept of a policy. While this is <em>shorter</em>, I find it more confusing. Why am I giving this one line block to the presenter? It’s not really clear to me. And I can’t give a better name to item… and if I want to change the policy, it has to be done in every place we use the presenter…</p>\n<p>This does get to the root of another thing that will end up being a follow-up: What’s the difference between closures and objects? If you don’t know, maybe this will get you thinking:</p>\n<pre><code>#ugh, this is slightly odd, thanks Ruby!\ndef apply(proc=nil, &blk)\n if block_given?\n blk.call\n else\n proc.call\n end\nend\n\nproc = Proc.new do\n puts \"a proc!\"\nend\n\nclass MyProc\n def call\n puts \"an object!\"\n end\nend\n\nmy_proc = MyProc.new\n\napply do\n puts \"a block!\"\nend\n\napply proc\n\napply my_proc\n</code></pre>\n<p>Chew on that a while.</p>\n<h2>This is too verbose</h2>\n<p>Absolutely. <code>DictionaryPresenter.new(PostCategorizationPolicy, Post.all).as_dictionary</code> is too long. 100% there. Good thing that I’d <em>actually</em> write this:</p>\n<pre><code>Dictionary.new(Post.all, ByTitle)\n</code></pre>\n<p>I switched the arguments around, because it reads better. When writing blog posts, you have to balance writing code samples that explain what you’re trying to say, and ones that are ‘real.’ I tend to be a bit more verbose with things like this, because it’s easier for someone who just jumps in to remember what all the parts are… then again, by <em>not</em> showing the ‘real code,’ people might end up implementing the crappy, verbose version. It’s always a trade-off, and I may or may not make the right choice. It happens.</p>\n<p>And yes, even earlier in this blog post: I said <code>DictionaryPresenter</code> and I’d really just say <code>Dictionary</code>.</p>\n<h2>Where does all of this go?</h2>\n<p>You’ve got two options, and only one of them is good: You should make a directory in app for your presenters. In this case, I’d be running <code>touch app/presenters/dictionary.rb</code> pretty quick. The other case is to put things in lib. This whole debacle uncovers something that @avdi went on a rant about over the last few days: models don’t have to inherit from anything. Here’s <a href=\"http://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller#Concepts\">what Wikipedia says about models</a>:</p>\n<blockquote>\n<p>The model manages the behaviour and data of the application domain, responds to requests for information about its state (usually from the view), and responds to instructions to change state (usually from the controller). In event-driven systems, the model notifies observers (usually views) when the information changes so that they can react.</p>\n</blockquote>\n<p>See anything about “saves stuff to the database?” Okay, ‘manages… data’ could be, but the point is, it’s not <em>required</em>. Models are the reification of a concept that exists in your application domain. Persistance is not required for every concept.</p>\n<p>Therefore, if it relates to your problem domain, it goes in app. If it doesn’t, it goes in lib. Something like <code>Dictionary</code>? Depends on how important you think it is. It’s probably pretty important; that’s why you made it a class.</p>\n<h2>Design Patterns? Are you serious?</h2>\n<p>Yes. Design patterns don’t suck, Design patterns <em>in Java</em> suck. I’m not even going to start this one here, expect more in this space soon.</p>\n<p>Design patterns are great, and they’re totally useful in Ruby.</p>\n<h2>In conclusion: A shout-out</h2>\n<p>I hope you’ve enjoyed this small digression on Presenters. If you’re gonna use presenters in your application, I encourage you to write simple ones as simple Ruby classes, but if they get slightly more complicated, you need to check out <a href=\"https://github.com/jcasimir/draper\">Draper</a> by my buddy Jeff Casimir. It lets you do stuff like this:</p>\n<pre><code>class ArticleDecorator < ApplicationDecorator\n decorates :article\n\n def author_name\n model.author.first_name + \" \" + model.author.last_name\n end\nend\n\narticle = ArticleDecorator.find(1)\narticle.author_name\n</code></pre>\n<p>There’s a lot of features to it, so check it out on GitHub. Totally worthwhile.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-secret-to-rails-oo-design/",
"title": "The secret to Rails OO design",
"description": null,
"url": "https://steveklabnik.com/writing/the-secret-to-rails-oo-design/",
"published": "2011-09-06T00:00:00.000Z",
"updated": "2011-09-06T00:00:00.000Z",
"content": "<p>UPDATE: I also have a follow-up <a href=\"/2011/09/09/better-ruby-presenters.html\">here</a>.</p>\n<p>I often tell people that I learned Ruby via Rails. This is pretty much the worst way to do it, but I’d learned so many programming languages by then that it didn’t hinder me too much. The one thing that it did do, however, was give me a slightly twisted sense of how to properly design the classes needed in a Rails app. Luckily, I obsessively read other people’s code, and I’ve noticed that there’s one big thing that is common in most of the code that’s written by people whose design chops I respect.</p>\n<p>This particular thing is also seemingly unique to those people. It’s not something that people who don’t write good code attempt, but do badly. It’s like a flag, or signal. Now, when I see someone employ this, I instantly think “they get it.” Maybe I’m giving it too much credit, but this advanced design technique offers a ton of interconnected benefits throughout your Rails app, is easy to imploy, and speeds up your tests by an order of magnitude or more. Unfortunately, to many beginning Rails devs, it’s non-obvious, but I want <em>you</em> to write better code, and so I’m here to ‘break the secret,’ if you will, and share this awesome, powerful technique with you.</p>\n<p>It’s called the ‘Plain Old Ruby Domain Object.’</p>\n<p>Yep, that’s right. A Ruby class that inherets from nothing. It’s so simple that it hides in plain sight. Loved by those who’ve mastered Rails, Plain Old Ruby Objects, or “POROs” as some like to call them, are a hidden weapon against complexity. Here’s what I mean. Examine this ‘simple’ model:</p>\n<pre><code>class Post < ActiveRecord::Base\n def self.as_dictionary\n dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}\n\n Post.all.each do |p|\n dictionary[p.title[0]] << p\n end\n\n dictionary\n end\nend\n</code></pre>\n<p>We want to display an index page of all our posts, and do it by first letter. So we build up a dictionary, and then put our posts in it. I’m assuming we’re not paginating this, so don’t get caught up in querying for all Posts. The important thing is the idea: we can now display our posts by title:</p>\n<pre><code>- Post.as_dictionary do |letter, list|\n %p= letter\n %ul\n - list.each do |post|\n %li= link_to post\n</code></pre>\n<p>Sure. And in one way, this code isn’t <em>bad</em>. It’s also not good: We’ve mixed a presentational concern into our model, which is supposed to represent buisness logic. So let’s fix that, via a Presenter:</p>\n<pre><code>class DictionaryPresenter\n def initialize(collection)\n @collection = collection\n end\n\n def as_dictionary\n dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}\n\n @collection.each do |p|\n dictionary[p.title[0]] << p\n end\n\n dictionary\n end\nend\n</code></pre>\n<p>We can use it via <code>DictionaryPresenter.new(Post.all).as_dictionary</code>. This has tons of benefits: we’ve moved presentation logic out of the model. We’ve <em>already</em> added a new feature: any collection can now be displayed as a dictionary. We can easily write isolated tests for this presenter, and they will be <em>fast</em>.</p>\n<p>This post isn’t about the Presenter pattern, though, as much as I love it. This sort of concept appears in other places, too, “this domain concept deserves its own class.” Before we move to a different example, let’s expand on this further: if we want to sort our Posts by title, this class will work, but if we want to display, say, a User, it won’t, because Users don’t have titles. Furthermore, we’ll end up with a lot of Posts under “A,” because the word “a” is pretty common at the beginning of Posts, so we really want to take the second word in that case. We can make two kinds of presenters, but now we lose that generality, and the concept of ‘display by dictionary’ has two representations in our system again. You guessed it: POROs to the rescue!</p>\n<p>Let’s change our presenter slightly, to also accept an organizational policy object:</p>\n<pre><code>class DictionaryPresenter\n def initialize(policy, collection)\n @policy = policy\n @collection = collection\n end\n\n def as_dictionary\n dictionary = ('A'..'Z').inject({}) {|h, l| h[l] = []; h}\n\n @collection.each do |p|\n dictionary[@policy.category_for(p)] << p\n end\n\n dictionary\n end\nend\n</code></pre>\n<p>Now, we can inject a policy, and have them be different:</p>\n<pre><code>class UserCategorizationPolicy\n def self.category_for(user)\n user.username[0]\n end\nend\n\nclass PostCategorizationPolicy\n def self.category_for(post)\n if post.starts_with?(\"A \")\n post.title.split[1][0]\n else\n post.title[0]\n end\n end\nend\n</code></pre>\n<p>Bam!</p>\n<pre><code>DictionaryPresenter.new(PostCategorizationPolicy, Post.all).as_dictionary\n</code></pre>\n<p>Yeah, so that’s getting a bit long. It happens. :) You can see that now each concept has one representation in our system. The presenter doesn’t care how things are organized, and the policies only dictate how things are organized. In fact, my names sorta suck, maybe it should be “UsernamePolicy” and “TitlePolicy”, actually. We don’t even care what class they are!</p>\n<p>It goes further than that in other directions, too. Combining the flexibility of Ruby with one of my favorite patterns from “Working Effectively with Legacy Code,” we can take complex computations and turn them into objects. Look at this code:</p>\n<pre><code>class Quote < ActiveRecord::Base\n #<snip>\n def pretty_turnaround\n return \"\" if turnaround.nil?\n if purchased_at\n offset = purchased_at\n days_from_today = ((Time.now - purchased_at.to_time) / 60 / 60 / 24).floor + 1\n else\n offset = Time.now\n days_from_today = turnaround + 1\n end\n time = offset + (turnaround * 60 * 60 * 24)\n if(time.strftime(\"%a\") == \"Sat\")\n time += 2 * 60 * 60 * 24\n elsif(time.strftime(\"%a\") == \"Sun\")\n time += 1 * 60 * 60 * 24\n end\n\n \"#{time.strftime(\"%A %d %B\")} (#{days_from_today} business days from today)\"\n end\nend\n</code></pre>\n<p>Yikes! This method prints a turnaround time, but as you can see, it’s a complex calculation. We’d be able to understand this much more easily of we used Extract Method a few times to break it up, but then we risk polluting our Quote class with more stuff that’s only relevant to pretty turnaround calculation. Also, please ignore that this is also presentation on the model; we just care that it’s a complex bit of code for this example.</p>\n<p>Okay, so here’s the first step of this refactoring, which Feathers calls “Break Out Method Object.” You can open your copy of “Working Effectively With Legacy Code” and turn to page 330 to read more. If you don’t have a copy, get one. Anyway, I digress. Here’s the plan of attack:</p>\n<ol>\n<li>Create a new class for the computation</li>\n<li>Define a method on that class to do the new work.</li>\n<li>Copy the body of the old method over, and change variable references to instance variables.</li>\n<li>Give it an initialize method that takes arguments to set the instance variables used in step 3.</li>\n<li>Make the old method delegate to the new class and method.</li>\n</ol>\n<p>I’ve changed this slightly for Ruby, since we can’t Lean On The Compiler, and a few of Feathers’ steps are about doing this. Anyway, let’s try this on that code. Step 1:</p>\n<pre><code>class Quote < ActiveRecord::Base\n def pretty_turnaround\n #snip\n end\n\n class TurnaroundCalculator\n end\nend\n</code></pre>\n<p>Two:</p>\n<pre><code>class TurnaroundCalculator\n def calculate\n end\nend\n</code></pre>\n<p>Three:</p>\n<pre><code>class TurnaroundCalculator\n def calculate\n return \"\" if @turnaround.nil?\n if @purchased_at\n offset = @purchased_at\n days_from_today = ((Time.now - purchased_at.to_time) / 60 / 60 / 24).floor + 1\n else\n offset = Time.now\n days_from_today = @turnaround + 1\n end\n time = offset + (@turnaround * 60 * 60 * 24)\n if(time.strftime(\"%a\") == \"Sat\")\n time += 2 * 60 * 60 * 24\n elsif(time.strftime(\"%a\") == \"Sun\")\n time += 1 * 60 * 60 * 24\n end\n\n \"#{time.strftime(\"%A %d %B\")} (#{days_from_today} business days from today)\"\n end\nend\n</code></pre>\n<p>I like to give it a generic name at first, and then give it a better one in step 5, after we see what it really does. often our code will inform us of a good name.</p>\n<p>Four:</p>\n<pre><code>class TurnaroundCalculator\n def initialize(purchased_at, turnaround)\n @purchased_at = purchased_at\n @turnaround = turnaround\n end\n\n def calculate\n #snip\n end\nend\n</code></pre>\n<p>Five:</p>\n<pre><code>class Quote < ActiveRecord::Base\n def pretty_turnaround\n TurnaroundCalculator.new(purchased_at, turnaround).calculate\n end\nend\n</code></pre>\n<p>Done! We should be able to run our tests and see them pass. Even if ‘run our tests’ consists of manually checking it out…</p>\n<p>So what’s the advantage here? Well, we now can start the refactoring process, but we’re in our own little clean room. We can extract methods into our TurnaroundCalcuator class without polluting Quote, we can write speedy tests for just the Calculator, and we’ve split out the idea of calculation into one place, where it can easily be changed later. Here’s our class, a few refactorings later:</p>\n<pre><code>class TurnaroundCalculator\n def calculate\n return \"\" if @turnaround.nil?\n\n \"#{arrival_date} (#{days_from_today} business days from today)\"\n end\n\n protected\n\n def arrival_date\n real_turnaround_time.strftime(\"%A %d %B\")\n end\n\n def real_turnaround_time\n adjust_time_for_weekends(start_time + turnaround_in_seconds)\n end\n\n def adjust_time_for_weekends(time)\n if saturday?(time)\n time + 2 * 60 * 60 * 24\n elsif sunday?(time)\n time + 1 * 60 * 60 * 24\n else\n time\n end\n end\n\n def saturday?(time)\n time.strftime(\"%a\") == \"Sat\"\n end\n\n def sunday?(time)\n time.strftime(\"%a\") == \"Sun\"\n end\n\n def turnaround_in_seconds\n @turnaround * 60 * 60 * 24\n end\n\n def start_time\n @purchased_at or Time.now\n end\n\n def days_from_today\n if @purchased_at\n ((Time.now - @purchased_at.to_time) / 60 / 60 / 24).floor + 1\n else\n @turnaround + 1\n end\n end\nend\n</code></pre>\n<p>Wow. This code I wrote three years ago isn’t perfect, but it’s almost understandable now. And each of the bits makes sense. This is after two or three waves of refactoring, which maybe I’ll cover in a separate post, becuase this was more illustrative than I thought… anyway, you get the idea. This is what I mean when I say that I shoot for roughly five-line methods in Ruby; if your code is well-factored, you can often get there.</p>\n<p>This idea of extracting domain objects that are pure Ruby is even in Rails itself. Check out this route:</p>\n<pre><code>root :to => 'dashboard#index', :constraints => LoggedInConstraint\n</code></pre>\n<p>Huh? LoggedInConstraint?</p>\n<pre><code>class LoggedInConstraint\n def self.matches?(request)\n current_user\n end\nend\n</code></pre>\n<p>Whoah. Yep. A domain object that describes our routing policy. Awesome. Also, validations, blatantly stolen from <a href=\"http://omgbloglol.com/post/392895742/improved-validations-in-rails-3\">omgbloglol</a>:</p>\n<pre><code>def SomeClass < ActiveRecord::Base\n validate :category_id, :proper_category => true\nend\n\nclass ProperCategoryValidator < ActiveModel::EachValidator\n def validate_each(record, attribute, value)\n unless record.user.category_ids.include?(value)\n record.errors.add attribute, 'has bad category.'\n end\n end\nend\n</code></pre>\n<p>This isn’t a plain Ruby class, but you get the idea.</p>\n<p>Now, you might be thinking, “Steve: This isn’t just for Rails! You’ve lied!” Why yes, actually, you’ve caught me: this isn’t the secret to Rails OO, it’s more of a general OO design guideline. But there’s something special about Rails which seems to lure you into the trap of never breaking classes down. Maybe it’s that <code>lib/</code> feels like such a junk drawer. Maybe it’s that the fifteen minute examples only ever include ActiveRecord models. Maybe it’s that more Rails apps than not are (WARNING: UNSUBSTANTIATED CLAIM ALERT) closed source than open, so we don’t have as many good examples to draw upon. (I have this hunch since Rails is often used to build sites for companies. Gems? Sure? My web app? Not so much. I have no stats to back this up, though.)</p>\n<p>In summary: Extracting domain objects is good. They keep your tests fast, your code small, and make it easier to change things later. I have some more to say about this, specifically the “keeps test fast” part, but I’m already pushing it for length here. Until next time!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-self-improvement-pomodoro/",
"title": "The self improvement pomodoro",
"description": null,
"url": "https://steveklabnik.com/writing/the-self-improvement-pomodoro/",
"published": "2011-09-04T00:00:00.000Z",
"updated": "2011-09-04T00:00:00.000Z",
"content": "<p>By now, the <a href=\"http://www.pomodorotechnique.com/\">pomodoro technique</a> is pretty well-known amongst programmers. I forget where I heard of it first, but if you haven’t, here it is: Break your day up into 30 minute chunks. Work for 25 minutes, break for 5. Repeat. It’s easy to stay concentrated when you know a break is coming up soon, the fact that you can fit two in an hour fits with my natural sense of order in the world, and once you get in the rhythm, you Get Shit Done. Heck, I liked the idea so much that I <a href=\"http://rubygems.org/gems/pomodoro\">made a Ruby gem</a>.</p>\n<p>That said, I hadn’t been following the technique for a while, because it’s easy to lose discipline. Happens to the best of us. Recently, though, it’s crept back into my workflow, largely thanks to Jeff Casimir. I was even lazy enough to not use my gem, and try something else. So I grabbed the software he uses from the App Store, and fired it up. It presented me with an interesting option to give my pomodoro a name before starting it. Pretty much without thinking, I typed ‘rstat.us rails3 port,’ since that’s what I was working on, and away I went.</p>\n<p>The more I think about it, however, the more interesting the naming aspect becomes. I can see how long and how often I’m working on various things, and over the past few weeks, it’s been neat to examine. When doing some reading this morning, my groggy brain combined a few different things together:</p>\n<ul>\n<li>Programmers often don’t have the time to practice their craft</li>\n<li>I work on a lot of random software projects</li>\n<li>These projects are often of varying quality, especially regarding things like documentation and tests</li>\n<li>Almost none of the gems that have been released are at a 1.0 release (can anyone find this blog post for me? I can’t. It’s something about “Just Release It”…)</li>\n</ul>\n<p>So, with an aim of combining all these things, I’m going to give something a try, and you should too: I’m going to name at least one pomodoro a day “self-improvement,” and use that time to go back over my random projects on GitHub and clean them up. I’m going to use that time to start programming in a totally new language. I’m going to use that time to do that refactoring I’ve been avoiding. I’m going to use that time to read a book.</p>\n<p>Basically, I’m going to force myself to invest in myself. Make sure to find time to invest in you, too.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/im-making-it-dead-simple-to-contribute-to-ruby-s-documentation/",
"title": "I'm making it dead simple to contribute to Ruby's documentation",
"description": null,
"url": "https://steveklabnik.com/writing/im-making-it-dead-simple-to-contribute-to-ruby-s-documentation/",
"published": "2011-08-22T00:00:00.000Z",
"updated": "2011-08-22T00:00:00.000Z",
"content": "<p>Okay! So, if you’d read <a href=\"/2011/05/10/contributing-to-ruby-s-documentation.html\">my previous article on this</a>, you’d know how easy it is to contribute to Ruby’s Documentaiton.</p>\n<blockquote>\n<p>But Steve, I’m still kinda scared.</p>\n</blockquote>\n<p>Okay, so here we go: I’m making it even easier on you.</p>\n<p>Send me what you want changed, and how, and I’ll make a patch and submit it on your behalf.</p>\n<p>No, seriously, I will. I already did once. <a href=\"http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/38873\">This guy</a> posted something about how it was too hard, <a href=\"http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/38875\">so I made a patch for him</a>. And now Ruby is better because of it.</p>\n<h2>Patches don’t have to be patches</h2>\n<p>Seriously, I don’t even mean diff outputs. I just got this email:</p>\n<pre><code>> I'm eager to contribute docs but I don't know where to start: I don't\n> mean pulling down the repo and all that, I mean, I see this link\n> here:http://segment7.net/projects/ruby/documentation\\_coverage.txt but\n> all I see is a ton of \"# is documented\" comments. Where does a\n> relative n00b begin? Poking around in the source code is, well,\n> intimidating. I'm completely new to this but I take direction well.\n> Just point me in the right direction. Just give me something to\n> document, please!\n</code></pre>\n<p>I sent this back:</p>\n<pre><code>> No code diving needed! That's part of it. Let's do this: Find the documentation for one of your favorite methods. Here, I'll pick one: http://ruby-doc.org/core/classes/Array.html#M000278 okay, that looks okay, but look at compact!:\n> \n> > Removes nil elements from the array. Returns nil if no changes were made, otherwise returns </i>ary</i>.\n> \n> Why's that </i> there? Total screw-up. For example. So, send me this email:\n> \n> \"Hey Steve-\n> \n> Check out the docs for Array#compact!, here: http://ruby-doc.org/core/classes/Array.html#M000279 . There's an extra </i> that's screwing things up.\n> \n> -Jonathan\"\n> \n> Done! I'll go from there. How about this one: \n> \n> \"Hey Steve-\n> \n> I checked out the docs for Time._load, here: http://www.ruby-doc.org/core/classes/Time.html#M000394\n> \n> These docs kind of suck. What if I don't know what Marshal is? There should at _least_ be a link to Marshall, here: http://ruby-doc.org/core/classes/Marshal.html And it should probably say something like \"You can get a dumped Time object by using _dump: http://www.ruby-doc.org/core/classes/Time.html#M000393 \n> \n> - Jonathan\"\n> \n> I'm sure you can find something that's formatted wrong, or worded incorrectly, or anything else.\n> \n> -Steve\n</code></pre>\n<p>Now, the <em>closer</em> it is to an actual patch, the faster I’ll be able to do it. A vague “this documentation is confusing, but I’m not sure why” is helpful, but will take longer. I’d rather get that email than not. If you’re not sure, hit send.</p>\n<h2>Just Do It</h2>\n<p>And so I’ll do it again. Scared to deal with doc patches? I’ll make them up. I’m serious. I’ll do it. Send them to me.</p>\n<p>That’s all.</p>\n<p>Send me patches. <a href=\"mailto:[email protected]\">[email protected]</a>. Do it.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/some-people-understand-rest-and-http/",
"title": "Some people understand REST and HTTP",
"description": null,
"url": "https://steveklabnik.com/writing/some-people-understand-rest-and-http/",
"published": "2011-08-07T00:00:00.000Z",
"updated": "2011-08-07T00:00:00.000Z",
"content": "<p>This is a follow-up post to my post <a href=\"/2011/07/03/nobody-understands-rest-or-http.html\">here</a>. You probably want to read that first.</p>\n<p>UPDATE: Please note that ‘<a href=\"/posts/2012-02-23-rest-is-over\">REST is over’</a>. ’Hypermedia API’ is the proper term now.</p>\n<h2>A few words on standards versus pragmatism</h2>\n<p>When I wrote my first post on this topic, I tried to take a stance that would be somewhat soft, yet forceful. Engineering is the art of making the proper trade-offs, and there are times when following specifications is simply not the correct decision. With that said, my motivation for both of these posts is to eradicate some of the ignorance that some developers have about certain areas of the HTTP spec and Fielding’s REST paper. If you understand the correct way, yet choose to do something else for an informed reason, that’s absolutely, 100% okay. There’s no use throwing out the baby with the bathwater. But ignorance is never a good thing, and most developers are ignorant when it comes to the details of REST.</p>\n<p>Secondly, while I think that REST is the best way to develop APIs, there are other valid architectural patterns, too. Yet calling non-REST APIs ‘RESTful’ continues to confuse developers as to what “RESTful” means. I’m not sure what exactly we should call “RESTish” APIs (hey, there we go, hmmm…) but I’m also not under the illusion that I personally will be able to make a huge dent in this. Hopefully you, humble reader, will remember this when dealing with APIs in the future, and I’ll have made a tiny dent, though.</p>\n<h2>So who <em>does</em> understand REST?</h2>\n<p>As it turns out, there are two companies that you’ve probably heard of who have APIs that are much more RESTful than many others: <a href=\"http://www.twilio.com/docs/api/rest/\">Twilio</a> and <a href=\"http://developer.github.com/\">GitHub</a>. Let’s take a look at GitHub first.</p>\n<h3>GitHub: logically awesome</h3>\n<p>GitHub’s developer resources are not only beautiful, but thorough. In addition, they make use of lots more of REST.</p>\n<h3>The good</h3>\n<p>GitHub uses <a href=\"http://developer.github.com/v3/mime/\">custom MIME</a> types for all of their responses. They’re using the vendor extensions that I talked about in my post, too. For example:</p>\n<p>application/vnd.github-issue.text+json</p>\n<p>Super cool.</p>\n<p>Their <a href=\"http://developer.github.com/v3/#authentication\">authentication</a> works in three ways: HTTP Basic, OAuth via an Authentication Header, or via a parameter. This allows for a maximum amount of compatibility across user agents, and gives the user some amount of choice.</p>\n<p>Their <a href=\"http://developer.github.com/v3/#pagination\">Pagination</a> uses a header I didn’t discuss in part I: the Link header. <a href=\"http://tools.ietf.org/html/rfc5988\">Here</a>’s a link to the reference. Basically, Link headers enable HATEOAS for media types which aren’t hypertext. This is important, especially regarding JSON, since JSON isn’t hypermedia. More on this at the end of the post. Anyway, so pagination on GitHub:</p>\n<pre><code>$ curl -I \"https://api.github.com/users/steveklabnik/gists\"\nHTTP/1.1 200 OK\nServer: nginx/1.0.4\nDate: Sun, 07 Aug 2011 16:34:48 GMT\nContent-Type: application/json\nConnection: keep-alive\nStatus: 200 OK\nX-RateLimit-Limit: 5000\nX-RateLimit-Remaining: 4994\nLink: <https://api.github.com/users/steveklabnik/gists?page=2>; rel=\"next\", <https://api.github.com/users/steveklabnik/gists?page=33333>; rel=\"last\"\nContent-Length: 29841\n</code></pre>\n<p>The Link header there shows you how to get to the next page of results. You don’t need to know how to construct the URL, you just have to parse the header and follow it. This, for example, is a great way to connect a resource that’s not text-based, such as a PNG, to other resources.</p>\n<h3>The bad</h3>\n<p>There’s really only one place that GitHub doesn’t knock it out of the park with their new API, and that’s HATEOAS. GitHub’s API isn’t discoverable, because there’s no information at the root:</p>\n<pre><code>$ curl -I https://api.github.com/\nHTTP/1.1 302 Found\nServer: nginx/1.0.4\nDate: Sun, 07 Aug 2011 16:44:02 GMT\nContent-Type: text/html;charset=utf-8\nConnection: keep-alive\nStatus: 302 Found\nX-RateLimit-Limit: 5000\nLocation: http://developer.github.com\nX-RateLimit-Remaining: 4993\nContent-Length: 0\n</code></pre>\n<p>Well, at least, this is how they present it. If you ask for JSON:</p>\n<pre><code>$ curl -I https://api.github.com/ -H \"Accept: application/json\"\nHTTP/1.1 204 No Content\nServer: nginx/1.0.4\nDate: Sun, 07 Aug 2011 16:45:32 GMT\nConnection: keep-alive\nStatus: 204 No Content\nX-RateLimit-Limit: 5000\nX-RateLimit-Remaining: 4991\nLink: <users/{user}>; rel=\"user\", <repos/{user}/{repo}>; rel=\"repo\", <gists>; rel=\"gists\"\n</code></pre>\n<p>You do get Links, but you have to construct things yourself. As a user, you get the same thing. It doesn’t change the links to point to your repos, it doesn’t give you links to anything else that you can do with the API.</p>\n<p>Instead, the root should give you a link to the particular resources that you can actually view. Maybe something like this:</p>\n<pre><code>$ curl -I https://api.github.com/ -H \"Accept: application/json\" -u \"username:password\"\nHTTP/1.1 204 No Content\nServer: nginx/1.0.4\nDate: Sun, 07 Aug 2011 16:45:32 GMT\nConnection: keep-alive\nStatus: 204 No Content\nX-RateLimit-Limit: 5000\nX-RateLimit-Remaining: 4991\nLink: </gists/public>; rel=\"public_gists\", </user/repos>; rel=\"repos\", <gists>; rel=\"gists\"\n</code></pre>\n<p>And a bunch more, for all of the other resources that are available. This would make the API truly discoverable, and you wouldn’t be forced to read their gorgeous documentation. :)</p>\n<h3>Twilio</h3>\n<p>I’ve always really enjoyed Twilio. Their API is incredibly simple to use. I once hooked up a little “Text me when someone orders something from my site” script, and it took me about fifteen minutes. Good stuff.</p>\n<h3>The good</h3>\n<p>Twilio has got the HATEOAS thing down. Check it out, their home page says that the base URL is “https://api.twilio.com/2010-04-01”. Without looking at any of the rest of their docs, (I glanced at a page or two, but I didn’t really read them fully yet), I did this:</p>\n<pre><code>$ curl https://api.twilio.com/2010-04-01\n<?xml version=\"1.0\"?>\n<TwilioResponse>\n <Version>\n <Name>2010-04-01</Name>\n <Uri>/2010-04-01</Uri>\n <SubresourceUris>\n <Accounts>/2010-04-01/Accounts</Accounts>\n </SubresourceUris>\n </Version>\n</TwilioResponse>\n</code></pre>\n<p>I introduced some formatting. Hmm, okay, Accounts. Let’s check this out:</p>\n<pre><code>$ curl https://api.twilio.com/2010-04-01/Accounts<?xml version=\"1.0\"?>\n<TwilioResponse><RestException><Status>401</Status><Message>Authenticate</Message><Code>20003</Code><MoreInfo>http://www.twilio.com/docs/errors/20003</MoreInfo></RestException></TwilioResponse>\n</code></pre>\n<p>Okay, so I have to be authenticated. If I was, I’d get something like this:</p>\n<pre><code><TwilioResponse>\n <Account>\n <Sid>ACba8bc05eacf94afdae398e642c9cc32d</Sid>\n <FriendlyName>Do you like my friendly name?</FriendlyName>\n <Type>Full</Type>\n <Status>active</Status>\n <DateCreated>Wed, 04 Aug 2010 21:37:41 +0000</DateCreated>\n <DateUpdated>Fri, 06 Aug 2010 01:15:02 +0000</DateUpdated>\n <AuthToken>redacted</AuthToken>\n <Uri>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d</Uri>\n <SubresourceUris>\n <AvailablePhoneNumbers>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/AvailablePhoneNumbers</AvailablePhoneNumbers>\n <Calls>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Calls</Calls>\n <Conferences>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Conferences</Conferences>\n <IncomingPhoneNumbers>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/IncomingPhoneNumbers</IncomingPhoneNumbers>\n <Notifications>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Notifications</Notifications>\n <OutgoingCallerIds>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/OutgoingCallerIds</OutgoingCallerIds>\n <Recordings>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Recordings</Recordings>\n <Sandbox>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Sandbox</Sandbox>\n <SMSMessages>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/SMS/Messages</SMSMessages>\n <Transcriptions>/2010-04-01/Accounts/ACba8bc05eacf94afdae398e642c9cc32d/Transcriptions</Transcriptions>\n </SubresourceUris>\n </Account>\n</TwilioResponse>\n</code></pre>\n<p>Awesome. I can see my all of the other resources that I can interact with. Other than knowing how to authenticate, I can follow the links from the endpoint, and discover their entire API. Rock. This is the way things are supposed to be.</p>\n<h3>The bad</h3>\n<p>This:</p>\n<pre><code>$ curl https://api.twilio.com/2010-04-01/Accounts -H \"Accept: application/json\"\n<?xml version=\"1.0\"?>\n<TwilioResponse><RestException><Status>401</Status><Message>Authenticate</Message><Code>20003</Code><MoreInfo>http://www.twilio.com/docs/errors/20003</MoreInfo></RestException></TwilioResponse>\n</code></pre>\n<p>Versus this:</p>\n<pre><code>$ curl https://api.twilio.com/2010-04-01/Accounts.json\n{\"status\":401,\"message\":\"Authenticate\",\"code\":20003,\"more_info\":\"http:\\/\\/www.twilio.com\\/docs\\/errors\\/20003\"}\n</code></pre>\n<p>:/ Returning JSON when your resource ends with ‘.json’ isn’t bad, but not respecting the Accept header, even when you return the right MIME type, is just unfortunate.</p>\n<h2>… and a little Announcement</h2>\n<p>It seems that this is a topic that people are really interested in. Part I of this article was pretty well-received, and I got lots of great email and feedback from people. It was also made pretty clear by <a href=\"http://twitter.com/#!/wayneeseguin/status/97733413611638784\">a few</a> people that they want more content from me on this topic.</p>\n<p>So I decided to write a book about it. You can check out the site for “<a href=\"http://designinghypermediaapis.com/\">Get Some REST</a>”, and put in your email address. Then you’ll get updated when I start pre-release sales.</p>\n<p>So what’s in “Get Some REST”? It’s going to be a full description of how to build RESTful web applications, from the ground up. Designing your resources, laying out an API, all the details. I’m going to try to keep most of the content language-agnostic, but provide code samples in Rails 3.1, as well.</p>\n<p>I plan on writing a bunch of content, and then releasing the book at half-price in beta. Early adopters will be able to get their two cents in, and I’ll cover things they still have questions on. It’ll be available under a liberal license, in PDF, ePub, all that good stuff.</p>\n<p>I’ve also set up a Twitter account at <a href=\"http://twitter.com/hypermediaapis\">@hypermediaapis</a>. I’ll be tweeting updates about the book, and also other good content related to RESTful design.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/twitter-is-to-important-to-be-owned-by-twitter/",
"title": "Twitter is too important to be owned by Twitter",
"description": null,
"url": "https://steveklabnik.com/writing/twitter-is-to-important-to-be-owned-by-twitter/",
"published": "2011-07-24T00:00:00.000Z",
"updated": "2011-07-24T00:00:00.000Z",
"content": "<p>When I spoke on a panel at <a href=\"http://therealtimereport.com/\">RTLM NY ’11</a>, I got a little rise out of the crowd when I said “Twitter is too important to be owned by Twitter.” This is still absolutely true, and yesterday was a great example of that: the #fuckyouwashington debacle.</p>\n<p>If you haven’t heard, <a href=\"http://twitter.com/#!/jeffjarvis\">Jeff Jarvis</a> started something on Twitter last night: <code>#fuckyouwashington</code>.</p>\n<p>As of right this second, when I <a href=\"http://twitter.com/#!/search/realtime/%23fuckyouwashington\">search for #fuckyouwashington on Twitter</a>, they’re coming in at two or three a second. It’s crazy. And it’s really interesting to read what people have to say, pro and con, picking their own issues, all of that stuff. Yet, it’s been removed from the trending topics.</p>\n<p>Now, maybe this particular trend was simply blocked because of the profanity. Sure, whatever. I don’t like that, but some might. And Twitter certainly allows all kinds of trends to stick around that would also be ‘offensive,’ for example, #babymakingsong, #turnoff, and #itsalwaystheuglypeople are currently trending. Don’t even get me started on trends like #whatblackgirlslike #whatwhitegirls like and the other racially charged topics. Sure, fine, it’s their site, they can do whatever they want.</p>\n<p>That, however, is the root of the issue: they can do whatever they want. And we’re all beholden to them. Nothing we can do. Maybe I’m not sensitive to four letter words, and I <em>do</em> want to hear about things like this. Too bad, Twitter says no. Maybe I don’t want <a href=\"http://venturebeat.com/2011/07/08/twitter-adding-promoted-tweets-to-your-stream-this-summer/\">promoted tweets added to my stream</a>. Too bad, Twitter says yes.</p>\n<p>And that’s why I’m feeling worse and worse about Twitter every day. It sucks, because I really like Twiter, the website. But Twitter, the company, really bums me out. They promised all of us nerds long ago that they wouldn’t do this kind of thing, and they’re not keeping that promise. Which means I’m not really getting emotionally invested in Twitter any more. It’s still valuable enough to me to keep using, but I can’t care about it the way I used to.</p>\n<p>This is why we’re still hacking away at <a href=\"http://rstat.us/\">rstat.us</a>. If you haven’t heard about it before, you can <a href=\"http://blog.steveklabnik.com/2011/03/23/announcing-rstat-us.html\">read my release announcement</a>. If you’re lazier than that, here’s the summary: rstat.us is a distributed, open-source Twitter. You can use our public node at the website, or get your own copy of the code and run your own, make whatever customizations you’d like, and you’re still on equal footing with everyone else in the network. We’re on the cusp of a 1.0 announcement, and I’d love to have been able to make that in this post, but we’re still working out some last kinks. If you’ve seen it before, you won’t find much differences right now, but we have 100% standards compliance, a complete visual refresh, and some other fun goodies waiting in the wings, ready to come out shortly.</p>\n<p>Just remember: if you use a service like Twitter that’s controlled by a company like Twitter, you have to play by their rules. And sometimes, they’ll change the rules, and it’ll be time to move on.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/nobody-understands-rest-or-http/",
"title": "Nobody understands REST or HTTP",
"description": null,
"url": "https://steveklabnik.com/writing/nobody-understands-rest-or-http/",
"published": "2011-07-03T00:00:00.000Z",
"updated": "2011-07-03T00:00:00.000Z",
"content": "<h2><strong>HI <a href=\"http://news.ycombinator.com/item?id=3635085\">HN</a>, PLEASE READ THIS!!!</strong></h2>\n<p>Since I’ve posted this, I’ve refined a few of my positions on things. Everyone learns and grows, and while I still stand by most of what I said, I specifically don’t agree that versioning the media type is how to properly version APIs. Hypermedia APIs should not actually use explicit versioning, but I’d rather see a version in the URI with HATEOAS than no HATEOAS and versioned media types. I’ve been meaning to update this post and write more, but alas, my work on <a href=\"http://designinghypermediaapis.com/\">Get some REST</a> has taken priority. I don’t have a HN account, so feel free to <a href=\"mailto:[email protected]\">email me</a> with any thoughts or questions!</p>\n<p>Furthermore, everything in engineering is ALWAYS a trade-off. I primarily wish that more people understood the tools that HTTP provides them with, and made an informed choice, rather than cargo-culting what they’ve seen others do.</p>\n<p>Update: Part II of this post is <a href=\"/2011/08/07/some-people-understand-rest-and-http.html\">here</a>. Check it out, and there’s an announcement at the end!</p>\n<p>Update: Please note that <a href=\"/posts/2012-02-23-rest-is-over\">REST is over. Hypermedia API is the new nomenclature.</a></p>\n<p>The more that I’ve learned about web development, the more that I’ve come to appreciate the thoroughness and thoughtfulness of the authors of the HTTP RFC and Roy Fielding’s dissertation. It seems like the answers to most problems come down to “There’s a section of the spec for that.” Now, obviously, they’re not infallible, and I’m not saying that there’s zero room for improvement. But it really disappoints me when people don’t understand the way that a given issue is supposed to be solved, and so they make up a partial solution that solves their given case but doesn’t jive well with the way that everything else works. There are valid criticisms of the specs, but they have to come from an informed place about what the spec says in the first place.</p>\n<p>Let’s talk about a few cases where either REST or HTTP (which is clearly RESTful in its design) solves a common web development problem.</p>\n<h3>I need to design my API</h3>\n<p>This one is a bit more general, but the others build off of it, so bear with me.</p>\n<p>The core idea of REST is right there in the name: “Representational State Transfer” It’s about transferring representations of the state… of resources. Okay, so one part isn’t in the name. But still, let’s break this down.</p>\n<h3>Resources</h3>\n<p>From <a href=\"http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_2_1_1\">Fielding’s dissertation</a>:</p>\n<blockquote>\n<p>The key abstraction of information in REST is a resource. Any information that can be named can be a resource: a document or image, a temporal service (e.g. “today’s weather in Los Angeles”), a collection of other resources, a non-virtual object (e.g. a person), and so on. In other words, any concept that might be the target of an author’s hypertext reference must fit within the definition of a resource. A resource is a conceptual mapping to a set of entities, not the entity that corresponds to the mapping at any particular point in time.</p>\n</blockquote>\n<p>When we interact with a RESTful system, we’re interacting with a set of resources. Clients request resources from the server in a variety of ways. But the key thing here is that resources are <em>nouns</em>. So a RESTful API consists of a set of URIs that map entities in your system to endpoints, and then you use HTTP itself for the verbs. If your URLs have action words in them, you’re doing it wrong. Let’s look at an example of this, from the early days of Rails. When Rails first started messing around with REST, the URLs looked like this:</p>\n<pre><code>/posts/show/1\n</code></pre>\n<p>If you use Rails today, you’ll note that the corresponding URL is this:</p>\n<pre><code>/posts/1\n</code></pre>\n<p>Why? Well, it’s because the ‘show’ is unnecessary; you’re performing a GET request, and that demonstrates that you want to show that resource. It doesn’t need to be in the URL.</p>\n<h3>A digression about actions</h3>\n<p>Sometimes, you need to perform some sort of action, though. Verbs are useful. So how’s this fit in? Let’s consider the example of transferring money from one Account to another. You might decided to build a URI like this:</p>\n<pre><code>POST /accounts/1/transfer/500.00/to/2\n</code></pre>\n<p>to transfer $500 from Account 1 to Account 2. But this is wrong! What you really need to do is consider the nouns. You’re not transferring money, you’re creating a Transaction resource:</p>\n<pre><code>POST /transactions HTTP/1.1\nHost: <snip, and all other headers>\n\nfrom=1&to=2&amount=500.00\n</code></pre>\n<p>Got it? So then, it returns the URI for your new Transaction:</p>\n<pre><code>HTTP/1.1 201 OK\nDate: Sun, 3 Jul 2011 23:59:59 GMT\nContent-Type: application/json\nContent-Length: 12345\nLocation: http://foo.com/transactions/1\n\n{\"transaction\":{\"id\":1,\"uri\":\"/transactions/1\",\"type\":\"transfer\"}}\n</code></pre>\n<p>Whoah, <a href=\"http://timelessrepo.com/haters-gonna-hateoas\">HATEOS</a>! Also, it may or may not be a good idea to return this JSON as the body; the important thing is that we have the Location header which tells us where our new resource is. If we give a client the ID, they might try to construct their own URL, and the URI is a little redundant, since we have one in the Location. Regardless, I’m leaving that JSON there, because that’s the way I typed it first. I’d love to <a href=\"mailto:[email protected]\">hear your thoughts on this</a> if you feel strongly one way or the other.</p>\n<p>EDIT: I’ve since decided that yes, including the URI is a bad idea. The Location header makes much more sense. More on this in Part ii, yet to come.</p>\n<p>Anyway, so now we can GET our Transaction:</p>\n<pre><code>GET /transactions/1 HTTP/1.1\nAccept: application/json\n</code></pre>\n<p>and the response:</p>\n<pre><code>HTTP/1.1 blah blah blah\n\n{\"id\":1,\"type\":\"transfer\",\"status\":\"in-progress\"}\n</code></pre>\n<p>So we know it’s working. We can continue to poll the URI and see when our transaction is finished, or if it failed, or whatever. Easy! But it’s about manipulating those nouns.</p>\n<h3>Representations</h3>\n<p>You’ll notice a pair of headers in the above HTTP requests and responses: Accept and Content-Type. These describe the different ‘representation’ of any given resource. From <a href=\"http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_2_1_2\">Fielding</a>:</p>\n<blockquote>\n<p>REST components perform actions on a resource by using a representation to capture the current or intended state of that resource and transferring that representation between components. A representation is a sequence of bytes, plus representation metadata to describe those bytes. Other commonly used but less precise names for a representation include: document, file, and HTTP message entity, instance, or variant.A representation consists of data, metadata describing the data, and, on occasion, metadata to describe the metadata (usually for the purpose of verifying message integrity).</p>\n</blockquote>\n<p>So <code>/accounts/1</code> represents a resource. But it doesn’t include the form that the resource takes. That’s what these two headers are for.</p>\n<p>This is also why adding <code>.html</code> to the end of your URLs is kinda silly. If I request <code>/accounts/1.html</code> with an <code>Accept</code> header of <code>application/json</code>, then I’ll get JSON. The <code>Content-Type</code> header is the server telling us what kind of representation it’s sending back as well. The important thing, though, is that a given resource can have many different representations. Ideally, there should be one unambiguous source of information in a system, and you can get different representations using <code>Accept</code>.</p>\n<h3>State and Transfer</h3>\n<p>This is more about the way HTTP is designed, so I’ll just keep this short: Requests are designed to be stateless, and the server holds all of the state for its resources. This is important for caching and a few other things, but it’s sort of out of the scope of this post.</p>\n<p>Okay. With all of that out of the way, let’s talk about some more specific problems that REST/HTTP solve.</p>\n<h3>I want my API to be versioned</h3>\n<p>The first thing that people do when they want a versioned API is to shove a /v1/ in the URL. <em>THIS IS BAD!!!!!1</em>. <code>Accept</code> solves this problem. What you’re really asking for is “I’d like the version two representation of this resource.” So use accept!</p>\n<p>Here’s an example:</p>\n<pre><code>GET /accounts/1 HTTP/1.1\nAccept: application/vnd.steveklabnik-v2+json\n</code></pre>\n<p>You’ll notice a few things: we have a + in our MIME type, and before it is a bunch of junk that wasn’t there before. It breaks down into three things: <code>vnd</code>, my name, and <code>v2</code>. You can guess what v2 means, but what about <code>vnd</code>. It’s a <a href=\"http://tools.ietf.org/html/rfc4288#section-3.2\">Vendor MIME Type</a>. After all, we don’t really want just any old JSON, we want my specific form of JSON. This lets us still have our one URL to represent our resource, yet version everything appropriately.</p>\n<p>I got a comment from <a href=\"http://avdi.org/\">Avdi Grimm</a> about this, too:</p>\n<blockquote>\n<p>Here’s an article you might find interesting: http://www.informit.com/articles/article.aspx?p=1566460The author points out that MIMETypes can have parameters, which means you can actually have a mimetype that looks like this:vnd.example-com.foo+json; version=1.0Sadly, Rails does not (yet) understand this format.</p>\n</blockquote>\n<h3>I’d like my content to be displayed in multiple languages</h3>\n<p>This is related, but a little different. What about pages in different languages? Again, we have a question of representation, not one of content. /en/whatever is not appropriate here. Turns out, <a href=\"http://tools.ietf.org/html/rfc2616#section-14.4\">there’s a header for that: Accept-Language</a>. Respect the headers, and everything works out.</p>\n<p>Oh, and I should say this, too: this doesn’t solve the problem of “I’d like to read this article in Spanish, even though I usually browse in English.” Giving your users the option to view your content in different ways is a good thing. Personally, I’d consider this to fall out in two ways:</p>\n<ul>\n<li>It’s temporary. Stick this option in the session, and if they have the option set, it trumps the header. You’re still respecting their usual preferences, but allowing them to override it.</li>\n<li>It’s more permanent. Make it some aspect of their account, and it trumps a specific header. Same deal.</li>\n</ul>\n<h3>I’d like my content to have a mobile view</h3>\n<p>Sounds like I’m beating a dead horse, but again: it’s a representation question. In this case, you’d like to vary the response by the User-Agent: give one that’s mobile-friendly. There’s a whole list of <a href=\"http://www.w3.org/TR/mobile-bp/\">mobile best practices</a> that the w3c recommends, but the short of it is this: the User-Agent should let you know that you’re dealing with a mobile device. For example, here’s the first iPhone UA:</p>\n<pre><code>Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A543a Safari/419.3\n</code></pre>\n<p>Then, once detecting you have a mobile User-Agent, you’d give back a mobile version of the site. Hosting it on a subdomain is a minor sin, but really, like I said above, this is really a question of representation, and so having two URLs that point to the same resource is kinda awkward.</p>\n<p>Whatever you do, for the love of users, please don’t detect these headers, then redirect your users to m.whatever.com, at the root. One of my local news websites does this, and it means that every time I try to follow a link from Twitter in my mobile browser, I don’t see their article, I see their homepage. It’s infuriating.</p>\n<h3>I’d like to hide some of my content</h3>\n<p>Every once in a while, you see a story like this: <a href=\"http://www.boingboing.net/2010/10/25/local-newspaper-boas.html\">Local paper boasts ultimate passive-agressive paywall policy</a>. Now, I find paywalls distasteful, but this is not the way to do it. There are technological means to limit content on the web: making users be logged-in to read things, for example.</p>\n<p>When this was discussed on Hacker News, <a href=\"http://news.ycombinator.com/item?id=1834075\">here’s</a> what I had to say:</p>\n<p>nkurz:</p>\n<blockquote>\n<p>I presume if I had an unattended roadside vegetable stand with a cash-box, that I’d be able to prosecute someone who took vegetables without paying, certainly if they also made off with the cash-box. Why is this different on the web? And if a written prohibition has no legal standing, why do so many companies pay lawyers to write click-through “terms of service” agreements?</p>\n</blockquote>\n<p>me:</p>\n<blockquote>\n<p>Why is this different on the web?Let’s go through what happens when I visit a web site. I type a URL in my bar, and hit enter. My web browser makes a request via http to a server, and the server inspects the request, determines if I should see the content or not, and returns either a 200 if I am allowed, and a 403 if I’m not. So, by viewing their pages, I’m literally asking permission, and being allowed.It sounds to me like a misconfiguration of their server; it’s not doing what they want it to.</p>\n</blockquote>\n<h3>I’d like to do some crazy ajax, yet have permalinks</h3>\n<p>This is an example of where the spec is obviously deficient, and so something had to be done.</p>\n<p>As the web grew, AJAXy ‘web applications’ started to become more and more the norm. And so applications wanted to provide deep-linking capabilities to users, but there’s a problem: they couldn’t manipulate the URL with Javascript without causing a redirect. They <em>could</em> manipulate the anchor, though. You know, that part after the #. So, Google came up with a convention: <a href=\"http://code.google.com/web/ajaxcrawling/docs/getting-started.html\">Ajax Fragments</a>. This fixed the problem in the short term, but then the spec got fixed in the long term: <a href=\"http://dev.w3.org/html5/spec-author-view/history.html\">pushState</a>. This lets you still provide a nice deep URL to your users, but not have that awkward #!.</p>\n<p>In this case, there was a legitimate technical issue with the spec, and so it’s valid to invent something. But then the standard improved, and so people should stop using #! as HTML5 gains browser support.</p>\n<h3>In conclusion</h3>\n<p>Seriously, most of the problems that you’re solving are social, not technical. The web is decades old at this point, most people have considered these kinds of problems in the past. That doesn’t mean that they always have the right answer, but they usually do have an answer, and it’d behoove you to know what it is before you invent something on your own.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/single-text-many-masters/",
"title": "Single text, many masters",
"description": null,
"url": "https://steveklabnik.com/writing/single-text-many-masters/",
"published": "2011-05-21T00:00:00.000Z",
"updated": "2011-05-21T00:00:00.000Z",
"content": "<p>Hey everyone. Here’s a draft of an essay I’ve been working on. I’d love to hear your feedback.</p>\n<p>The word ‘engineering’ has a deep connection to the word ‘trade-offs’ in my mind. Most engineering decisions come down to evaluating a few differing alternatives, and often multiple factors end up being negatively correlated. You can make something stronger, but then it will be heavier. It can be made faster, but then it’s significantly more expensive. A good engineer is able to take all of these factors into account, and design a system such that it maximizes its effectiveness across the sum of all of the relevant constraints. No matter if you consider the act of writing software an art, science, or engineering, its indisputable that designing complex software systems is identical in this respect. There are dozens of different metrics that system architects take into consideration while crafting a plan of attack, but but there’s a deeper question of balance here that’s significantly different than these more traditional engineering issues.</p>\n<p>Text, in the form of source code, presents unique challenges of composition. These difficulties all stem from the same root: source code is a singular text, but must be intelligible to multiple, simultaneous audiences. More traditional forms of authorship still take audience into consideration, of course, but the decision to engage a diverse group of people is the choice of the writer. While mass appeal may be something that certain authors strive to attain, it’s not an inherent property of their chosen form of expression. Source code, while text, inhabits a multiplicity of forms, and software developers are confronted with this inherent multi-faceted complexity when composing any particular software work. Some of these forms suit certain audiences better than others, and so it falls to the programmer to manage which form they are currently working in, consider which audience they are attempting to write for, and arrange all of these forms amongst one another in such a way that any particular audience is able to navigate and extract the proper information from the source without confusion.</p>\n<p>In this post, I’ll expand on the concept of audiences for code, and in a future post, I’ll explore the simultaneous forms that code takes.</p>\n<h2>The multitude of audiences</h2>\n<h3>The default audience: the computer</h3>\n<blockquote>\n<p>Science is what we understand well enough to explain to a computer. Art is everything else we do.Don Knuth</p>\n</blockquote>\n<p>This may seem obvious, but the when considering the question of “Who are programs written for?”, many would say “The computer. Duh.” In many ways, a computer is the primary reader of a particular piece of source code. The code that’s given to a computer will be executed billions of times per second, repeated, recalculated, and re-interpreted over and over and over again.</p>\n<p>The computer’s native understanding of software comes from machine code. Machine code are the binary numbers that the CPU loads into memory and processes directly. For example, here’s a line of machine code for an x86 machine that puts the value ‘97’ into the AL register, graciously stolen <a href=\"http://en.wikipedia.org/wiki/Assembly_language#Assembly_language\">from Wikipedia</a>:</p>\n<pre><code>10110000 01100001\n</code></pre>\n<p>While most would consider this unintelligible, computers were actually programmed this way at one time. My uncle actually did this by flipping switches to set the binary and pushed a button to store it in memory. Unfortunately, what’s good for the computer isn’t good for the human programmer. This is why assembly language was created. Assembly language has a 1 to 1 mapping to machine code, but is much easier for humans to understand. Here’s that same line in assembly:</p>\n<pre><code>MOV AL, 61h ; Load AL with 97 decimal (61 hex)\n</code></pre>\n<p>The <code>MOV</code> corresponds with <code>10110</code>, <code>AL</code> maps to <code>000</code>, and 61 hex is <code>01100001</code>. <code>MOV</code> is short for ‘move,’ though, and this mnemonic is just a bit easier to understand than <code>10110</code>. This is the most basic example of a compositional trade-off. It’s not a true trade-off, because they map perfectly to one another, but it illustrates the difference between composing in a language that the computer understands and one that’s more natural for the programmer. Another important concept comes into play, that of <em>compilation</em>. Virtually every work of composition created in software is automatically translated to another form before it is executed. We’ll address this concept more fully when we discuss form later.</p>\n<p>If that’s where assembly stopped, it would remain a 1 to 1 mapping. However, virtually every assembly language also offers macros, and this moves the code further away from the machine and destroys the synchrony of the two forms. Here’s an example:</p>\n<pre><code>MOV EAX, [EBX]\n</code></pre>\n<p>The <code>[]</code> characters change the meaning of <code>EBX</code>, rather than be the value stored in that particular register, they imply that the value is a memory address, and we want to move the contents of that address to <code>EAX</code>. The generated machine code could now be processed into multiple valid assembly forms, and so the transformation is only perfect in one direction, even if it’s possible to ‘decompile’ it into one of those possible encodings. This is considered to be an acceptable trade-off for human readability; we very rarely want to turn machine code back into assembly.</p>\n<p>There’s also a jump between higher level languages, as well. Here’s the assembly statements that adds 2 and 3 together:</p>\n<pre><code>MOV EAX, 2\nADD EAX, 3\n</code></pre>\n<p><code>ADD</code>, of course, is the statement that adds a number to the register that’s given. Now <code>EAX</code> has the value 5. Here’s the same code, but in C:</p>\n<pre><code>int x = 2;\nx = x + 3;\n</code></pre>\n<p>Pretty simple. You can see how the C is much easier to understand; we say what type <code>x</code> is (an integer), and it’s a bit more explicit. <code>x</code> is equal to <code>x</code> + three. However, since the C is divorced from the machine, and is written for the person, we can change our compiler to make assembly code that works on a different kind of processor architecture. If we were to compile the above C for x86_64, a 64 bit version of x86, we might get some assembly that’d look like this:</p>\n<pre><code>MOVL RAX, 2\nADDL RAX, 3\n</code></pre>\n<p>While this code looks similar to the above, it is quite different. This uses the native 64 bit types, rather than the 32 bit types above. The other important thing is that by writing code that’s divorced from the machine, and written for people, we’re able to translate it into the code for multiple machines. If we had written the assembly above, when moving to another architecture, it would have required a total re-write. And while this particular sample looks very similar, a more complex piece of code will be significantly divergent, but I don’t want to go into the details of two kinds of assembly code. Because we can define the languages for humans, and the language of computers is somewhat beholden to the physical machine itself, it’s significantly easier to do the translation from C to the two kinds of machines, rather than trying to translate from one machine to another. What we lose in this kind of translation, though, is efficiency. Code that was hand-crafted for each machine would be more performant, and better represent each individual platform.</p>\n<p>Even though we may choose to use a language that’s more understandable to people, it still has to be understood by the computer in some form. This translation will introduce some amount of penalty, and so it’s important that this gets taken into consideration. Sometimes, code must be written in a way that’s not easy for a person to read, because it’s easier for the computer to be efficient with a more opaque implementation.</p>\n<h3>The reflexive audience: the programmer himself</h3>\n<blockquote>\n<p>Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.Brian Kernighan</p>\n</blockquote>\n<p>Everyone who writes code has experienced this at some time or another. You write a whole bunch of code, and then a few months goes by, and you take a look at it, and it’s absolutely unintelligible. This happens because at the time of inception, the author of a particular piece of code has an incredibly close relationship to it. As it was just written, the code is obvious to the author. They’re in the proper mindset to understand the intention that was drawn upon to necessitate bringing those lines into the world, and so no extra explanation is necessary. As time goes on, however, the author becomes more close to the third audience, other programmers. It’s important for coders to recognize this fact, and take preventative steps to ameliorate this confusion.</p>\n<p>Even though the author will approach the position of the other audience eventually, this audience is distinct because there is a certain level of explanation that sits between undocumented, inexpressive code and code that’s well explained, and this is the position most code is in. An explanation that’s helpful to those who understand the code, but not to those who don’t is better than nothing. This sort of code may be overly contextual, and could use some added information to improve its clarity.</p>\n<h3>The other audience: colleagues and coworkers</h3>\n<blockquote>\n<p>Always code as if the guy who ends up maintaining your code is a violent psychopath who knows where you live.Martin Golding</p>\n</blockquote>\n<p>As I touched on earlier, there’s a similarity between the ‘other’ audience and the reflexive. The primary distinction is drawn around the proximity to the source. The other does not have the advantage of having authored the code, and therefore doesn’t have the native’s understanding of the underlying logical organization of the source. This disadvantage can be overcome via composing in such a manner that the meaning is emergent from the design. Even if it’s too complex to be obvious, good documentation can address this particular deficiency.</p>\n<p>Ultimately, much of software design is about modeling. Software that solves a particular problem should emulate the nature of the challenge it’s attempting to address. If this can be achieved, it’s significantly easier for those who understand the problem to figure out how the software works. Therefore, good design can help improve the effectiveness of a given piece of source to communicate its intent. Along a similar vector, if the design is similar to code that solves a particular issue, it’s easier to understand. As an example, a friend recently asked for feedback about an interface that he’d designed. It loaded a save game file for StarCraft 2. This is what he came up with:</p>\n<pre><code>replay_file = File.new(\"spec/fixtures/1v1-game1.sc2replay\")\n@replay = SC2Refinery::Parser.parse(replay_file)\n</code></pre>\n<p>However, Ruby already has several kinds of code in its standard library that loads some information from disk and parses it into some kind of data structure that you can use in your program. The JSON, YAML, and Marshall classes already use a set of methods to import and export data, and they’re named <code>load</code> and <code>dump</code>, and they’re part of the class directly. Also, in this case, the user of the code shouldn’t need to deal with the creation of a file, since it’s unreasonable to assume that a game replay would come from any other source. Therefore, after some discussion, he adopted the following interface instead:</p>\n<pre><code>@replay = SC2Refinery.load(\"spec/fixtures/1v1-game1.sc2replay\")\n</code></pre>\n<p>This is much nicer to use, and is much simpler. While it may not seem like a whole lot, when rules like this are applied across an entire codebase, they can significantly increase understanding. Multiple reductions of mental overhead add up quickly.</p>\n<p>My new favorite trick for adding a little bit of modeling that significantly reduces overhead for the user is the Presenter Pattern. Jeff Casimir demonstrated this very clearly in his presentation at RailsConf 2011, “<a href=\"http://dl.dropbox.com/u/69001/Fat%20Models%20Aren%27t%20Enough%20-%20RailsConf.pdf\">Fat Models Aren’t Enough</a>”. Here’s a slightly modified example. Imagine that we have a system that manages students, and we’d like to display a report card for them. We might start with some code that looks like this:</p>\n<pre><code>student = Student.find(options[:student_id])\nterm = Term.find(options[:term_id])\nreport_type = ReportType.find(options[:report_type])\n\nputs \"#{student.last_name}, #{student.first_name}\"\nputs \"#{report_type.report_title} for #{term.start_date} to #{term.end_date}\"\nstudent.courses.each do |course|\n course_report = student.report_data_for_course(course)\n puts course_report.to_s\nend\n</code></pre>\n<p>Honestly, until this past week, this is the exact code that I would have written. But it turns out that we can do better. Basically, we’re displaying some information that comes from a combination of three different objects. If we think about it some more, we’re really trying to display a report card. So let’s make an object that represents the card, and delegates to its sub-objects. It will then know how to display itself.</p>\n<pre><code>class ReportCard\n delegate :start_date, :end_date, :to => :term\n delegate :first_name, :last_name, :courses, :report_data_for_course, :to => :student\n delegate :report_title, :to => :report_type\n\n def initialize(params)\n @student = Student.find params[:student_id]\n @term = Term.find params[:term_id]\n @report_type = ReportType.find params[:report_type_id]\n end\n\n def student_name\n [last_name, first_name].join(\", \")\n end\n\n def title\n \"#{report_title} for #{start_date} to #{end_date}\"\n end\n\n def course_reports\n out = \"\"\n courses.each do |course|\n out += report_data_for_course(course)\n end\n out\n end\nend\n</code></pre>\n<p>Now, this is a lot of code. However, as you can see, it’s all focused on composing the information and exposing an interface that makes sense for a report card. Using it is super easy:</p>\n<pre><code>report = ReportCard.new(options)\nputs report.student_name\nputs report.title\nputs report.course_reports\n</code></pre>\n<p>Bam! It’s incredibly obvious. This code is much more clear than before. We’ll see if I’m still as hot on this pattern as I am now in a few months, but I feel the extra object adds a significant amount of clarity.</p>\n<p>If the model is too hard to create, or if additional clarity is needed, documentation in the form of comments can also help to improve the understanding of the ‘other’ audience. Comments can be a difficult form of prose to write, because they need to be written at the correct level of abstraction. If they simply repeat what the code does, they’re useless, and if they’re too high-level, certain details and semantics may not be made clear.</p>\n<p>Individual bits of code can also be made more clear by developing a narrative within any particular method that’s being written. Telling a story with code may not be something you’ve considered before, but it’s really about maintaining a proper flow in the actions your code is taking. For example, if there’s a bunch of error handling strewn about inside of a method, it’s less clear than bunching all of the error handling near the end. Most code should be an act in three parts: input, processing, and output. If these three parts are mixed together, it can appear much more complicated.</p>\n<h3>The forgotten audience: end-users</h3>\n<blockquote>\n<p>If I asked my customers what they wanted, they’d have told me, “A faster horse.”Henry Ford</p>\n</blockquote>\n<p>In the end, all software is used by someone. Use-value is the driving force of virtually all code. Code that doesn’t do anything may be making some kind of important philosophical statement, but isn’t really the sort that I’m talking about.</p>\n<p>The introduction of a user imposes significant restrictions upon the way that code is composed. End-users do not need to understand the code itself, but they do need to be able to understand its external interfaces. These needs place an imposition on the way that the code needs to be written, because it <em>must</em> address this issue of interface. Sometimes, interface requirements can create a burden on the internal implementation. Needing to support certain behaviors and forms can create complexity for an implementor.</p>\n<p>Documentation created for end users must be completely different than that which is created for those inspecting the code itself. Most end-users will not be literate in the arts of software development, and so approach the software object in an entirely different way than those who write code do. Yet, the same semantics must be passed on to them, but at a higher level. There’s a strong movement within the community to start designing software with this kind of end-user documentation in mind, called <a href=\"http://tom.preston-werner.com/2010/08/23/readme-driven-development.html\">README driven development</a>. There are advantages to thinking on this level when beginning, but a nice effect of doing it first is that you can ensure it gets done. A surprising amount of software has poor documentation for its users, because it’s created after the software is finished, and at that time there’s intense pressure to ship it out the door. Writing down information for the end user first ensures that it’s done properly, that all development works in accordance with the documentation, and that all of the use-cases for an end-user have been thought of and are being addressed.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/contributing-to-ruby-s-documentation/",
"title": "Contributing to Ruby's documentation",
"description": null,
"url": "https://steveklabnik.com/writing/contributing-to-ruby-s-documentation/",
"published": "2011-05-20T00:00:00.000Z",
"updated": "2011-05-20T00:00:00.000Z",
"content": "<h3>Update!</h3>\n<p>I’m now making it even easier than this: <a href=\"/2011/08/22/im-making-it-dead-simple-to-contribute-to-ruby-s-documentation.html\">Read my new post about how I’ll do this for you</a>. That said, if you want to do it yourself, the following will tell you how.</p>\n<h3>Original Article</h3>\n<p>Ruby 1.9.3 is coming out soon! <a href=\"http://blog.segment7.net/2011/05/09/ruby-1-9-3-documentation-challenge\">drbrain has challenged the Ruby community to improve its documentation</a>, but some people were asking about how to do so. So I made a video!</p>\n<p>Some small errata: drbrain has informed me that he should edit the Changelog, not me. So don’t do that. :)</p>\n<p><a href=\"http://vimeo.com/23522731\">How to contribute to Ruby’s documentation.</a> from <a href=\"http://vimeo.com/steveklabnik\">Steve Klabnik</a> on <a href=\"http://vimeo.com/\">Vimeo</a>.</p>\n<p>If you don’t want to watch me talk about it, here’s the same info, in text:</p>\n<p>Getting the Ruby source is pretty easy. You can find it on GitHub, here: <a href=\"http://github.com/ruby/ruby\">http://github.com/ruby/ruby</a> . Click the “fork” button and clone down your own fork:</p>\n<pre><code>$ git clone [email protected]:YOURUSERNAME/ruby.git\n</code></pre>\n<p>After that’s done, type <code>cd ruby</code> and add the main project as an upstream. This will let you keep up-to-date with the latest changes:</p>\n<pre><code>$ git remote add upstream https://github.com/ruby/ruby.git\n\n$ git fetch upstream\n</code></pre>\n<p>Okay! Now that you’re all set up, poke around and find something that needs documented. I like to just look through the source, but you can also look <a href=\"http://segment7.net/projects/ruby/documentation_coverage.txt\">here</a> for a list of things that have no docs. Documentation is written in rdoc, and I’d check the recent commits that drbrain has been making to guide you in style. <a href=\"https://github.com/ruby/ruby/commit/071a678a156dde974d8e470b659c89cb02b07b3b\">This commit</a>, for example, is a pretty good template. You can also check out the formatting guides <a href=\"http://rdoc.rubyforge.org/RDoc/Markup.html\">here</a>. There’s also <a href=\"http://rdoc.rubyforge.org/RDoc/Parser/Ruby.html\">this</a> which explains some directives for .rb files and <a href=\"http://rdoc.rubyforge.org/RDoc/Parser/C.html\">this</a> which handles directives for .c files.</p>\n<p>Now that you’ve made a change to the documentation, you can regenerate the docs by using rdoc. First, grab the latest version from rubygems:</p>\n<p>$ gem install rdoc</p>\n<p>Always best to have the latest tools. Now do this to generate the docs:</p>\n<pre><code>$ rdoc --o tmpdoc lib/rss*\n</code></pre>\n<p>I’m passing it in an output directory with op, since the doc directory is not an rdoc directory. rdoc will complain and refuse to overwrite those files, which is a good thing. I’m also passing in a pattern of what to compile documentation for, compiling all of it takes a few minutes! In this case, I chose to document the rss library.</p>\n<p>Now you have a website in rdocout. Open up its index.html, and poke around for what you’ve changed. If it all looks good, you’re ready to make a patch!</p>\n<pre><code>$ rm -r tmpdoc\n\n$ git add .\n\n$ git commit -m \"adding documentation for $SOMETHING\"\n</code></pre>\n<p>Now, you have two options here. One is to simply push the change up to GitHub, and make a pull request.</p>\n<pre><code>$ git push origin\n</code></pre>\n<p>… aaand pull request. The core Ruby development doesn’t really happen on GitHub though, and so your patch may take a while to get included. If you really want to do it right, submit a patch to RedMine. We’ll use git to make this patch:</p>\n<pre><code>$ git format-patch HEAD~1\n</code></pre>\n<p>This says “make a patch out of the last commit.” It’ll tell you a file name, it should start with 000.</p>\n<p>Now, sign up for the Ruby RedMine <a href=\"http://bugs.ruby-lang.org/account/register\">here</a>. Once you’ve clicked the confirmation email, <a href=\"http://bugs.ruby-lang.org/projects/ruby-trunk/issues/new\">open a new ticket</a>, and assign it to Eric Hodel, category DOC, and give it your Ruby version, even though it’s not a big deal here. Click ‘choose file’ and pick your patch, then ‘create and continue’ and BAM! You’re done!</p>\n<p>Let’s all pitch in and make this the best documented Ruby release ever! In writing documentation, you might even find some things that you’d like to help improve. ;)</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-next-iteration-of-my-blog/",
"title": "The next iteration of my blog",
"description": null,
"url": "https://steveklabnik.com/writing/the-next-iteration-of-my-blog/",
"published": "2011-05-12T00:00:00.000Z",
"updated": "2011-05-12T00:00:00.000Z",
"content": "<p>Well well well, some things never change. Long ago, I hosted my own blog, and here we are again. I’ve moved everything over to Jekyll, if that means anything to you. You can now <a href=\"https://github.com/steveklabnik/blog\">look at my blog on GitHub</a>. Big thanks to <a href=\"http://www.jstorimer.com/2009/12/29/jekyll-on-heroku.html\">this article</a> for lots of help and pointers with final deployment stuff, and <a href=\"http://mikeferrier.ca/2011/04/29/blogging-with-jekyll-haml-sass-and-jammit/\">this one</a> for haml and sass and stuff. Yay!</p>\n<h2>New-ish beginnings</h2>\n<p>So why’d I do this? Well, this is just part of moving more towards hosting all of my own services, and not relying on services that others provide. This isn’t some sort of argument over individualism vs. collectivism, it’s more about <a href=\"http://nomoresharecropping.org/\">No More Sharecropping</a>.</p>\n<p>I’m still relying on people for hosting, of course. Nobody is 100% removed, but with my own backup of everything, I’m in total control here. It’s really minimal work to keep this going, and only a few hours of setup to move everything over.</p>\n<h2>A small change in focus, too</h2>\n<p>I’m also going to stop bitching about politics so often, and try to be more constructive in general. I have a lot to say about this new humanities angle that my life has taken as of late. Hence, I’ve changed the title of my blog. And you can expect more posts along those lines. More about books, more about art and code, less whining. One may think I’m maturing or something…</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-first-week-of-rstat-us-users-press-scaling-oh-my/",
"title": "The first week of rstat.us: users, press, and scaling, oh my!",
"description": null,
"url": "https://steveklabnik.com/writing/the-first-week-of-rstat-us-users-press-scaling-oh-my/",
"published": "2011-03-30T00:00:00.000Z",
"updated": "2011-03-30T00:00:00.000Z",
"content": "<p>Hey everyone. A lot of people have been asking me about <a href=\"http://rstat.us/\">rstat.us</a> lately, so I figured I’d tell a little bit of the story as it’s gone down so far.</p>\n<h2>Stats</h2>\n<p>First, here’s some numbers:</p>\n<ul>\n<li>Users: 4553</li>\n<li>Uniques: 25,000</li>\n<li>Pageviews: 119,385</li>\n<li>Pages/visit: 4.77</li>\n<li>Statuses posted: 9387</li>\n</ul>\n<p>Here’s some fun info: stats on statuses by language:</p>\n<ul>\n<li>german 118</li>\n<li>russian 1355</li>\n<li>none 97 <- ha!</li>\n<li>english 4836</li>\n<li>spanish 1412</li>\n<li>dutch 98</li>\n<li>french 1155</li>\n<li>portuguese 272</li>\n<li>farsi 66</li>\n<li>pinyin 1</li>\n</ul>\n<h2>Code</h2>\n<p>Lines, by committer:</p>\n<ul>\n<li>28 AndrewVos</li>\n<li>124 BRIMIL01</li>\n<li>6 Blaine Cook</li>\n<li>107 Brendan Taylor</li>\n<li>924 Brian Miller</li>\n<li>45 Caley Woods</li>\n<li>162 Carol Nichols</li>\n<li>20 Claudio Perez Gamayo</li>\n<li>1347 Dominic Dagradi</li>\n<li>4 James Larkby-Lahet</li>\n<li>99 Jorge H. Cuadrado</li>\n<li>258 Kat Hagan</li>\n<li>10 Lauren Voswinkel</li>\n<li>143 LindseyB</li>\n<li>16 MenTaLguY</li>\n<li>3 Michael Stevens</li>\n<li>3 Murilo Santana</li>\n<li>10 Nate Good</li>\n<li>1 Peter Aronoff</li>\n<li>24 Shebanian</li>\n<li>44 Stephen Paul Weber</li>\n<li>1409 Steve Klabnik</li>\n<li>8 Tony Arcieri</li>\n<li>478 Zachary Scott</li>\n<li>104 burningTyger</li>\n<li>28 james cook</li>\n<li>56 kat</li>\n<li>200 wilkie</li>\n<li>10 wolfwood</li>\n</ul>\n<p>Also, wilkie has a lot more code in the 3 gems that we build off of, for our ostatus implementation.</p>\n<p>That makes for 29 unique committers. These stats were generated by git-blame.</p>\n<p>Just over 200 pull requests</p>\n<p>872 commits</p>\n<h2>The deal with Heroku</h2>\n<p>Crazy! Such good stuff. I’m really excited as to how it’s going, but there were a few rocky points along the way as well. Some people saw me tweeting in frustration with Heroku, so here’s the skinny:</p>\n<p>On Sunday, for some reason, our 3 line app.js file, as well as our favicon.ico, started to take 60 seconds to respond. This meant that a dyno was getting tied up, and since Heroku changed the way their mesh works, they kept routing requests to these screwed up dynos. This caused a rash of 500 errors, and I frantically filed a support ticket. This was the first time I’d had a paid app on Heroku, and they’ve always been fantastic. The Hackety Hack site was on the free account, and performed beautifully, with similar traffic.</p>\n<p>What it boils down to is this, though: Heroku took about 22 hours to respond to my support ticket, in any way. Not like “we’re looking at it, but it’s Sunday, so it’ll take a while.” Crickets. Meanwhile, the errors kept happening. Eventually, I heard something from them, where they pretty much said ‘I see other errors, try fixing those.’ with no response on the actual root cause. I got an email from a Level2 person who promised to investigate and escalate, but by now, I still haven’t heard.</p>\n<p>I also had a smaller issue with MongoHQ. To their credit, I heard back Sunday night, and the issue with them was an annoyance, not a showstopper. I ended up hearing from the CEO, who ended up following up with me multiple times, and in a pretty timely fashion.</p>\n<p>So, we moved our hosting away from Heroku and onto Duostack. Now, full disclosure: I’m friends with the guys from DuoStack. That said, they took care of me. I found a small bug in their platform when deploying the app, but everything’s gone really well. Bunches of users have commented on how fast the site is now, and I’ve been really happy with it so far. We’ll see how it goes.</p>\n<h2>The Future</h2>\n<p>Anyway, here’s what’s next for <a href=\"http://rstat.us/\">rstat.us</a>: The biggest thing is getting to 100% ostatus compliance. We spent lots of time this week getting screwed over by a bug in Google’s hub implementation, but now that it’s all taken care of now. While we work on that, we’re also getting ready to deploy a statusnet compliant API, so people can use statusnet clients on mobile and desktop. This is huge for me, as I’m really missing notifications.</p>\n<p>Other than that, just keep using it! Tell your friends! If you code, come join us on the IRC (#rstatus on freenode), and help file bugs and patch them!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/announcing-rstat-us/",
"title": "Announcing rstat.us",
"description": null,
"url": "https://steveklabnik.com/writing/announcing-rstat-us/",
"published": "2011-03-23T00:00:00.000Z",
"updated": "2011-03-23T00:00:00.000Z",
"content": "<p>Whew.</p>\n<p>If you’ve been <a href=\"http://twitter.com/steveklabnik\">following me on Twitter</a> at all lately, you’ll know that I’ve been working hard on a new project lately. Tonight, even though it’s late, I’m finally getting it out there. Please welcome <a href=\"http://rstat.us/\">http://rstat.us/</a> to the world! (as soon as the DNS updates. ;) )</p>\n<p><a href=\"http://rstat.us/\">rstat.us</a> is a micro-blogging service, just like Twitter. There’s one big difference, though: it is distributed. Oh, and it’s 100% open source. So two things. It’s also integrated with Facebook and Twitter, so you can just log in with either of those accounts, and it’ll syndicate your posts. Well, just to Twitter right now, but we’re getting there.</p>\n<p>Here’s the other cool thing: Want to own your data? You can actually <a href=\"http://github.com/hotsh/rstat.us\">get the code</a> and run a copy yourself! But you won’t get left out: you can follow anyone else that uses the <a href=\"http://ostatus.org/\">ostatus</a> protocol, like <a href=\"http://identi.ca/\">http://identi.ca/</a>, and it’ll Just Work. You just have to grab the URL to their feed, put it in, and it’s all good! But you probably shouldn’t do that yet. We’re still working on some features.</p>\n<p><a href=\"http://rstat.us/\">rstat.us</a> is still very much alpha quality. 7 or 8 of my best friends and I have been fighting off sleep, food, and tests for the last two weeks to put this together. Please <a href=\"http://rstat.us/\">give it a try</a> and let me know what you think, tell me if you can break it, or anything else that’s particularly interesting. If you’re the coding type, please check out the <a href=\"http://github.com/hotsh/rstat.us/wiki\">wiki</a> for a big old list of stuff we’d like to do.</p>\n<p>Oh, and of course, you should <a href=\"http://rstat.us/users/steveklabnik\">follow me on rstat.us</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/id-you-hear-i-m-ruby-mendicant-university-s-first-visiting-teacher/",
"title": "Did you hear? I'm Ruby Mendicant University's first visiting teacher",
"description": null,
"url": "https://steveklabnik.com/writing/id-you-hear-i-m-ruby-mendicant-university-s-first-visiting-teacher/",
"published": "2011-03-18T00:00:00.000Z",
"updated": "2011-03-18T00:00:00.000Z",
"content": "<p>I’ve heard <a href=\"http://university.rubymendicant.com/\">Ruby Mendicant University</a> (also known as RMU, not to confuse you Robert Morris alums!) described as “the best kept secret of the Ruby community.” High praise, indeed. If you haven’t heard about RMU before, check out this presentation by Andrew Cox:</p>\n<p>TL;DR: RMU is a free, online ‘unversity’ from Gregory Brown, author of ‘Ruby Best Practices.’ You enter an intermediate Rubyist, and you exit an expert.</p>\n<p>Anyway, RMU recently announced that <a href=\"http://university.rubymendicant.com/changelog/visiting-teacher-2011-t2-steve-klabnik\">I’ll be their first visiting teacher</a>. Basically, I’ll be hanging out, giving advice, and just generally being even more accessible to RMU students than I already am. Greg and I have talked a lot about education, Ruby, and a lot of other things… between Hackety’s ‘nothing to intermediate’ and RMU’s ‘intermediate to expert’ approaches to learning, anyone wanting to get up to speed with Ruby should have no shortage of resources at their disposal.</p>\n<p>If you’re feeling like you’ve got the hang of Ruby, but want to take it to the next level, <a href=\"http://university.rubymendicant.com/admissions\">applications are open</a>! Hopefully I’ll see you there.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/i-m-quitting-hacker-news/",
"title": "I'm quitting Hacker News",
"description": null,
"url": "https://steveklabnik.com/writing/i-m-quitting-hacker-news/",
"published": "2011-03-03T00:00:00.000Z",
"updated": "2011-03-03T00:00:00.000Z",
"content": "<p>I’ve been thinking about this post for a while.</p>\n<p>I freaking love Hacker News. It’s my favorite place on the Internet. It’s incredibly valuable. I have learned so much from my time there. With that said, I feel like it’s taken up too much of my time.</p>\n<p>I’m trying to accomplish things. I got accepted to grad school. I’m trying to rebuild Shoes. I’m working on improving Hackety. I’m finding that I don’t have enough hours in the day to accomplish the things I want to accomplish. And arguing for the hundredth time about how artists will survive in a post-IP world, or how to get your first couple of users, or whatever it is that Google or Apple did this week hasn’t gotten boring yet, and that’s a problem.</p>\n<p>The only respectable solution is to go cold-turkey for a while.</p>\n<p>I’ve been doing good on some of my new years’ resolutions, and bad on others. I haven’t spent a lot of time writing lately, and I have only been doing okay at my writing. Hacker News is cutting into that. I feel like I’m using it in the same way some use TV. Yeah, it’s significantly deeper, but I really should be reading more books and less HN. At least, that’s what’s right for me right now.</p>\n<p>I won’t be gone forever. I just need to spend more time elsewhere.</p>\n<p>With that said, I’ve been spending a <em>lot</em> of time on Shoes. I’ll just leave this here: <a href=\"https://github.com/shoes/shoes/tree/feature/gemify\">https://github.com/shoes/shoes/tree/feature/gemify</a></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/living-in-the-cloud/",
"title": "Living in the cloud",
"description": null,
"url": "https://steveklabnik.com/writing/living-in-the-cloud/",
"published": "2011-02-24T00:00:00.000Z",
"updated": "2011-02-24T00:00:00.000Z",
"content": "<p>I’ve been joking for a while that I’d like to “move to the Internet.” Generally I cite my familiarity and love with online culture as opposed to American culture. I don’t really care what’s going on in “the real world,” as I feel it’s kind of played out. The whole cycle of “fight wars, reduce freedoms, sit in front of your TV” has gotten really boring quickly. I find Internet culture to be more complex, entertaining, and useful. But I’m getting way off topic, here.</p>\n<p>As a programmer, one of the key skills I rely on daily is the ability to recognize abstractions. Writing great code requires the ability to see similarities in things, even when they may not be immediately apparent. Lately, I’ve been giving a lot of thought to the idea of “living in the cloud.” Primarily, cloud computing is about providing services on-demand, and in the amount you need. Dynamic allocation. It’s generally great stuff, and even though “the cloud” is a bit buzzwordy, it’s an awesome concept.</p>\n<p>So what happens when we apply that to meatspace? What does “cloud living” look like? “Cloud working”?</p>\n<p>Living in the cloud would look a lot like the world Tim Ferriss encourages with his whole lifestyle design concept, or at least the lifestyle he’s designed for himself. Move around from place to place, live where you find things interesting, reallocate yourself to a better place when the feeling dictates. The concept of being a “digital nomad” certainly makes me think of “living in the cloud.” Doing this right means giving up most material possessions, as they impede freedom of movement.</p>\n<p>Another important aspect of cloud living would be finding a way to make a living while being location independent. There was a great article making the rounds a while back called “<a href=\"http://romansnitko.posterous.com/jobs-dont-scale\">Jobs don’t scale</a>.” It’s only tangentially related, but his point about working for other people certainly relates to freedom of movement. If you have to convince your boss to let you work remotely, it’s much harder than just doing it yourself.</p>\n<p>In any case, I’m very attracted to all of this as an idea. I’d love to travel the world, working from a different place every day, seeing new sights and exploring new places. It requires re-thinking a lot of things about what it means to work, how relationships work, friendships. Can you have kids if you keep moving around from place to place? How can you stay connected to friends if you never see them, because you’re halfway across the globe? Can your significant other do the same thing? Can you get month to month leases easily?</p>\n<p>I’m still thinking about this as a concept, but I thought I’d share my thoughts so far. I’m not moving around the world yet, but I am trying to increase my own mobility as much as possible. I think I’m finally at a place where I don’t care much for most of my possessions anymore…</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/a-few-notes-on-foucault/",
"title": "A few notes on Foucault",
"description": null,
"url": "https://steveklabnik.com/writing/a-few-notes-on-foucault/",
"published": "2011-01-22T00:00:00.000Z",
"updated": "2011-01-22T00:00:00.000Z",
"content": "<p>I’ve almost always been in a habit of reading. When I was younger, I spent a lot of time in the library. I’d just grab an entire series of novels, and read them all, right in a row. Sometimes I’d end up reading non-fiction too. I ended up with a lot of books on software.</p>\n<p>Then my local library got Internet access. Slowly, I started to spend less time in the stacks and more in front of a terminal. This wasn’t a bad thing; I still kept reading. What’s more, what was previously a one way street turned into two: I didn’t just read the Internet, I wrote it. I spent hours and hours discussing the things I’d read with others.</p>\n<p>In any case, as the years flew by, the things that I’ve been reading have become less and less substantial. Current events are fine and all, and pictures of cute cats are nice, but I feel like the volume of what I’ve been reading has gone up, but the quality has gone down. It happens. I can’t really be snide about not owning a TV while being subscribed to <a href=\"http://www.reddit.com/r/fffffffuuuuuuuuuuuu\">/r/fffffffuuuuuuuuuuuu</a>. Reddit somehow has morphed into a place where you can feel superior, but it’s really just the same exact thing in different clothing. Combine this general unease with my aspirations for grad school in the fall, as well as my renewed interest in political philosophy, and my need to hit the books has become readily apparent.</p>\n<p>Luckily, I have an ally in this quest, and Jamie has given me a reading list. Since reading and writing are two sides of a single coin, I’ll be writing about the things that I read here. Like many other things I’ve written about in the past, I’m sure that putting my thoughts down on paper (?) will help to gel my understanding and thoughts. First up: Michel Foucault’s “Discipline and Punish.”</p>\n<p>I’ll get right to it: I really enjoyed reading this book. It’s partially because it took a lot of random things that I kind of knew and tied them to some experiences that I’ve had in a meaningful way. It’s also partially because I have a substantial infatuation with the conspiratorial; I re-read “1984” every year, and I often think about its all-knowing, all-seeing telescreen imagery when discussing anything vaguely political. “Discipline and Punish” gave me the same sort of images, but they were fewer, and more firmly rooted in history and reality. The book opens with a section named “Torture,” and the scene of Robert-François Damiens’ punishments for attempted regicide. I’m sure that before the release of “Saw,” these images were even more shocking:</p>\n<p>… he was to be ‘taken and conveyed in a cart, wearing nothing but a shirt, holding a torch of burningwax weighing two pounds’; then, ’in the said cart, to the Place de Greve, where, on a scaffold that will be erected there, the flesh will be torn from his breasts, arms, thighs, and calves with red-hot pincers, his right hand, holding the knife with which he committed the said parricide, burnt with sulphur, and, on those places where the flesh will be torn away, poured molten lead, boiling oil, burning resin, wax, and sulphur melted together and then his body drawn and quartered by four horses and his limbs and body consumed by fire, reduced to ashes, and his ashes thrown to the winds.</p>\n<p>Let it never be said that the people of 1757 were not thorough. Regardless, by January of 1840, we’d transitioned to a prison system that looks pretty much the same as it does now. The book’s primary theme is discussing how we got from point A to point B, and then examining the ‘why’ to explain how modern society has a new sovereign institution: the ‘carceral system.’</p>\n<p>Before we can examine that question, though, we need to ask why we used torture as a form of punishment in the first place. The reasoning is actually straightforward: during the period of monarchy, everything revolves around the monarch. He is sovereign in a more absolute way than we even initially think of; the pop-culture image of a king has more to do with something of a popularity contest or that he’s simply the guy on the top of the pyramid, but the nature of a monarch’s power runs more deeply than that. It was called ‘divine right’ for a reason, the physical body of the sovereign was the representation of God himself, and since the entire world belongs to God, thus it belongs to and is a part of the monarch. It reminds me of the kind of doublethink necessary to grasp the Catholic conception of the Holy Trinity, in this case God the Father, the king his son, and the world rather than a Holy Ghost. All one, yet three at the same time. In any case, if the land itself is the literal body of the king, then any transgression is an act of defiance not only of the rule of the monarch, but is making war upon God himself. And since damage has been done to the body of God, so must an equivalent exchange be made with the body of the aggressor. Torture also has an element of the theatrical to it, and therefore demonstrates to all of those watching that they must also comply with the rule of law or face the consequences.</p>\n<p>However, eventually, torture became socially inconvenient. Basically, it was a case of the Streisand Effect: when you place that much attention on someone, you create a forum for sympathizers to create romantic images of their fallen hero. There’s a great historical example of this in the Christian mythos: consider the polarizing effect that Christ’s torture on the cross maintains to this day. History is littered with the songs of fallen heros, and a call to follow in their stead. Eventually, whenever a new convict was to be strung up at a gallows, there’d be a state of disarray. Foucault describes several images of rioters throwing stones and even in some cases killing the executioner.</p>\n<p>As a result of this, the nature of punishment slowly changed. Reformists argued that punishment was metered out unevenly, and inconsistently. Thus in the same way that monarchy gave way to democracy, the absolute right of the king to punish became distributed as well. However, centralized and distributed systems are quite different, and require different constructs to operate properly. Therefore, a distributed form of the right to punish would need some mechanism by which to operate. This mechanism is termed “discipline” by Foucault. Discipline creates a certain order all by itself, and he uses a great example of monks and monasteries to illustrate the concept of discipline. Think about all of these things that we consider virtuous:</p>\n<ul>\n<li>Studying is a disciplined form of reading</li>\n<li>Dieting is discipline applied to eating</li>\n<li>The image of a soldier is almost entirely one of discipline</li>\n<li>Morality is discipline applied to all of life</li>\n<li>Exercise is disciplined form of the body</li>\n</ul>\n<p>But discipline has even greater roots in our society. Think about Taylor’s Scientific Management, for example: it’s a means of imposing discipline on workers to maximize their production. Schooling is a way of giving children a ‘structured environment’ (structure seems to be synonymous with discipline in many cases) to develop in. Churches are places for the soul to become disciplined.</p>\n<p>Submitting to discipline has deeper psychological effects as well. It creates the idea of a division: there’s those who follow the rules, and those that disregard them. And since we’ve established that those who follow the rules are virtuous, those who don’t must not be. Since those that don’t follow the rules are doing bad things, they should be subject to punishment, so that they can remain disciplined. And thus the system of rules can be used as the distributed form of this right to punish, replacing the absolute right of the monarch. Submitting to this mentality makes people into ‘docile bodies’ that perfectly fit into this worldview.</p>\n<p>As an example of how far this disciplinary approach has gone, Foucault presents the Panopticon, which was a prison shaped like a pie, with a tower in the middle. Prisoners would be able to observe others directly across from themselves, and the guard who may or may not be in the tower would be able to watch all of the prisoners at once. Since you couldn’t tell if the guard was in the tower or not, discipline would become internalized, since you always had to assume that Big Brother is watching you… This way of thinking about society ends up creating an all-encompassing ‘carceral system’ that we now live in.</p>\n<p>It’s also important to note that this carceral system is total and absolutely permeating every aspect of society, yet they aren’t presented as such. Foucault specifically mentions that it’s important to consider that the delinquent are still a part of the system, and not outside of it. Yet we’re constantly presented with images that serve to present the idea that there’s a ‘criminal underworld,’ that those who lead a life of crime are part of a shadowy alternate universe. Foucault refers to this idea as ‘enclosure’: “Discipline sometimes requires enclosure, the specification of a place heterogeneous to all others and closed in upon itself. It is the protected space of disciplinary monotony.” The enclosure embodies this separation, since there’s a space both within and outside of the enclosure. A self and an Other.</p>\n<p>… so yeah. That’s my summary. I’m still digesting a lot of this stuff, and so I may have more to say about it later.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/goals-for-2011/",
"title": "Goals for 2011",
"description": null,
"url": "https://steveklabnik.com/writing/goals-for-2011/",
"published": "2011-01-02T00:00:00.000Z",
"updated": "2011-01-02T00:00:00.000Z",
"content": "<p>This year I’m happy to report that I have much more upbeat plans than last year’s kind of sad post. With that said, here’s my 2011 goals:</p>\n<ul>\n<li>I want to work for other people 2 days per week on average for the year.</li>\n<li>I want to write something every day</li>\n<li>I want to keep building on the initial success that Hackety Hack’s 1.0 launch has brought. I’d like to get a new release out every month or so, with a 2.0 for Christmas 2011.</li>\n<li>I want to start exercising again. My current lifestyle is still too sedentary for my liking.</li>\n<li>I want to eat better.</li>\n<li>I want to start logging my activities every day.</li>\n</ul>\n<p>Most of these are kinda standard. I used to go to the gym all the time, but I’ve never eaten healthily. Both of these things are going to become more important as I age, however, and so I’m starting to actually give them serious consideration.</p>\n<p>Hackety Hack had an impressive 1.0 launch, but I have lots of work to do. Please check out the <a href=\"http://blog.hackety-hack.com/\">Hackety Hack blog</a> tomorrow for a report on that. Basically, I now have a sizable userbase to attend to, and they’ve found a bunch of bugs that I need to iron out before starting to build more stuff.</p>\n<p>The logging is an idea that I got from <a href=\"http://www.sebastianmarshall.com/\">Sebastian Marshall</a>. We’ve had some good conversation recently, and we talked briefly about the importance of measuring things. You can only improve things if you know what’s actually wrong… I’m going to be tracking the time that I do various activities, and seeing if I can improve. If I feel this is working, I’ll write about it in a month or so.</p>\n<p>I’ve already made some progress on all of these, actually. Obviously, most of them can only truly be realized in a year’s time, but here’s what I’ve got going:</p>\n<ul>\n<li>I’ve got my logging set up going. I’m trying to figure out how to make it as unobtrusive as possible, but I think I have a good first run at it.</li>\n<li>I’ve got a set up for the work situation, and if all goes well, that should be accomplished. I originally thought I’d have to front-load a bit, but it seems like I’ll be doing this most of the year.</li>\n<li>Working that little will enable me to really accomplish my Hackety goals.</li>\n<li>Now that Maria is graduated, she’s got some time to cook things, and she’s also interested in eating better, and her habits are better than mine already. Hopefully this will lead to success.</li>\n<li>Logging should help keep my eyes on the prize for my exercise and writing goals.</li>\n</ul>\n<p>So, that’s it! Good luck with your goals in the new year!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/hackety-hack-v1-0/",
"title": "Hackety Hack v1.0",
"description": null,
"url": "https://steveklabnik.com/writing/hackety-hack-v1-0/",
"published": "2010-12-25T00:00:00.000Z",
"updated": "2010-12-25T00:00:00.000Z",
"content": "<p>It’s been a long journey. I have a lot to say about this, but it’ll have to wait until I’m not in the middle of nowhere spending some time with Maria’s family. I just wanted to tell you all Merry Christmas, and point you at the <a href=\"http://blog.hackety-hack.com/post/2457961910/merry-christmas-and-a-1-0-announcement\">Hackety Hack v1.0 release announcement</a> and the new <a href=\"http://hackety-hack.com/\">Hackety Hack website</a>.</p>\n<p>Merry Christmas!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-hardest-decision-i-ve-ever-made/",
"title": "The hardest decision I've ever made",
"description": null,
"url": "https://steveklabnik.com/writing/the-hardest-decision-i-ve-ever-made/",
"published": "2010-11-17T00:00:00.000Z",
"updated": "2010-11-17T00:00:00.000Z",
"content": "<p>I’ll cut to the chase: I’ve stepped down as CTO of CloudFab. I don’t want to discuss details, but life is too short to not be doing what you want with it. And, after a period of careful thought and consideration, it’s no longer the right thing for me. It was an amicable parting, and my replacements are already in place, and doing cool things. I’m happy with what I’ve accomplished, but it’s time for me to move on.</p>\n<p>This raises the question: what am I moving on to? Here’s the answer: Hackety Hack. What I truly care about is teaching, writing, and discussing software. If I could do anything, I’d hack on Hackety, share what I know with others, and keep learning and playing with new things all day long.</p>\n<p>However, there’s a certain question of sustainability here: I’m not made of money, and I have lots of bills to pay. However, doing the ‘obvious’ thing would be really, truly wrong: Hackety Hack is not and will never be a startup. It’s a social venture, if anything: it would be absolutely unquestioningly wrong for me to ever ask anyone for a single cent for it. It’d also be counterproductive: how could I share programming with everyone if I excluded those without disposable income?</p>\n<p>So to this end, Hackety cannot be all that I do. And while I really want to teach, I also have a lot to learn. Luckily, I have some great allies in this area: I’ve been working alongside two teachers at the University of Pittsburgh over the past month or so: one in the English department, and one in the Computer Science department. In doing so, I’ve come to realize that my place in life is actually in the middle: while I love the theoretical side of CS, I’m actually much more interested in the application. And teachers need to be effective communicators, so investing in writing is absolutely worthwhile for me. At the same time, I’ve really enjoyed every chance I’ve had to write: there’s a reason I keep three blogs now! I’m also incredibly interested in the parallels that exist between computer and human languages. I like the comparison between refactoring and composition. I think the two fields have a lot to learn from one another. It’s something I’d like to explore more.</p>\n<p>To do so, I’ll have to finish off my undergrad degree. Luckily, I’m very close, so I’ll be doing that in the spring. Afterward, it’s my intention to go to grad school in the English department, so I’m furiously working on my application, with the help of the aforementioned teacher. At the same time, I’ll possibly help teach or TA classes in the CS department as well. I’m hoping to combine my solid foundations in the hard sciences with the new perspective and improved analysis that the humanities can offer.</p>\n<p>I don’t think that academia is where I want to stay forever, but I think it’s a good place for me to grow for now. It’s time for me to invest in myself a little. I think I’ll end up a much better person in the end.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/trolling-is-a-art/",
"title": "Trolling is a art",
"description": null,
"url": "https://steveklabnik.com/writing/trolling-is-a-art/",
"published": "2010-09-24T00:00:00.000Z",
"updated": "2010-09-24T00:00:00.000Z",
"content": "<p>I find it incredibly hard not to be judgmental.</p>\n<p>I’m not sure what part of my personality makes this happen, but even when I try to curb tearing down other people, I end up doing it anyway. I’m not sure if it’s just me, but I think part of the problem is that these kinds of things are rewarded.</p>\n<p>One of the reasons that I use my real name everywhere is to try to improve the civility of my discourse. When you’re anonymous, it’s really easy to be callous. And when I used a handle, I was even worse. But even now, I can think of three times when I accidentally was mean for basically no reason: That tweet, the Diaspora incident, and the only Hacker News comment I’ve ever deleted.</p>\n<p>In all three of these instances, it’s not so much that I was wrong. It’s that I ended up presenting my criticisms poorly. I think I did a decent job with the Diaspora one, but people still took it the wrong way, so I failed to communicate. I’m certainly not infallible. I guarantee you there are n00b errors and security holes in my own codebases. So I really shouldn’t be so quick to judge. That’s all I have to say about that. I’ll just have to do better going forward.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/80-of-success-is-showing-up/",
"title": "80% of success is showing up",
"description": null,
"url": "https://steveklabnik.com/writing/80-of-success-is-showing-up/",
"published": "2010-09-21T00:00:00.000Z",
"updated": "2010-09-21T00:00:00.000Z",
"content": "<p>It’s funny how true this quote is.</p>\n<p>There are three people that I really admire: _why, Zed Shaw, and Yehuda Katz. And to simplify my tenses a bit, I’ll speak of _why in the present tense. What do these three guys have in common? They all churn out badass software like nobody’s business. _why started so many projects that it just adds to his mythical qualities; Zed has been cranking out software and blogs left and right, and Yehuda’s “<a href=\"http://yehudakatz.com/2010/09/14/heres-to-the-next-3-years/\">I’m quitting Engine Yard</a>” post basically said “I’m pretty much done rebuilding the server side of software in my own image, now it’s time for me to focus on the client side of the equation.” Not with that much hubris, of course, but it’s pretty much true.</p>\n<p>Now, I don’t mean to imply that they’ve ‘just shown up,’ but it seems to me that productivity is directly related to just getting started on things. I’ve had people tell me that they think I’m superhumanly productive, which is funny to me: I’m almost constantly fretting about how much time I waste every day. And while I do have a ton of projects, unlike all those guys, I tend to not fully finish them. It’s something that I’ve been trying to work on lately, because I have so many things that I want to accomplish.</p>\n<p>For me, the biggest hold on cranking out even more stuff is just simply starting to work. Often, if i slack off for half an hour, I slack off all evening. Now that there’s a TV with Netflix in my house again, it’s difficult to not watch all of the amazing documentaries that are on there. But if I manage to start working on stuff, I find that I keep doing it. It’s like I get to pick one or the other: projects or slacking. I can’t quite seem to do both. And so I’ve been starting some minor experimentation to see if I can’t work on projects more often than not.</p>\n<p>One thing that I’ve sort of half-tried is the <a href=\"http://en.wikipedia.org/wiki/Pomodoro_Technique\">Pomodoro Technique</a>. It’s pretty sweet: basically, you set a timer for 25 minutes, and start working. You can take a break when it goes off. Surely you can wait to refresh Hacker News for 25 minutes, right? That’s something else I’ve done: on Friday, I put Reddit and HN in my /etc/hosts file. They’re totally blocked, for now. After <a href=\"http://blog.steveklabnik.com/trouble-with-diaspora\">last week’s situation</a>, I spent waaaay too much freaking time on HN, even for me. I love it, and I’m learning a ton every day, but it’s becoming an issue for my productivity. So it’s gone for a bit. It’s been rough, but I’ve been getting lots done, and I think when I start reading it again, I’ll be able to keep it at reasonable levels.</p>\n<p>Anyway, so that’s what I’m doing to keep productive. Anyone else have any thoughts on this? How do you keep working on side projects? What works for you?</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/trouble-with-diaspora/",
"title": "Trouble with Diaspora",
"description": null,
"url": "https://steveklabnik.com/writing/trouble-with-diaspora/",
"published": "2010-09-17T00:00:00.000Z",
"updated": "2010-09-17T00:00:00.000Z",
"content": "<p>So, Wednesday, Diaspora was released.</p>\n<p>If you’re not familiar, a few months ago everyone was up in arms about the latest Facebook privacy change. So four college kids started <a href=\"http://www.kickstarter.com/projects/196017994/diaspora-the-personally-controlled-do-it-all-distr\">a Kickstarter project</a> with a dream: let’s make a distributed, private Facebook. They asked for $10,000. They got about $200,000.</p>\n<p>They worked on the code all summer, in secret. Wednesday, a ‘developer preview’ came out. They’d promised something by the end of the summer, and the 15th of September is the last day. So <a href=\"http://github.com/diaspora/diaspora\">they put it up on GitHub</a>.</p>\n<p>Oh boy.</p>\n<p>Basically, the code is really, really bad. I don’t mean to rain on anyone’s parade, but <a href=\"http://www.theregister.co.uk/2010/09/16/diaspora_pre_alpha_landmines/\">there are really, really bad security holes</a>. And they’re there due to things that any professional programmer would never dream of leaving out of their code. I don’t want to disclose too many details, but you can see the code yourself on GitHub.</p>\n<p>At first, I found one. So I tried to patch it. And I did, and it got accepted into master. Awesome. But then, the more I read, the more bad things I found. They’re going to need a complete overhaul to fix this. Go over every last piece of code. And don’t even get me started on their encryption code.</p>\n<p>But basically, please heed their own warning from <a href=\"http://www.joindiaspora.com/2010/09/15/developer-release.html\">the announcement</a>:</p>\n<blockquote>\n<p>Feel free to try to get it running on your machines and use it, but we give no guarantees. We know there are security holes and bugs, and your data is not yet fully exportable. If you do find something, be sure to log it in our bugtracker, and we would love screenshots and browser info.</p>\n</blockquote>\n<p>If you find one of the many, many nodes that people are publicly posting, please don’t use your real passwords, or information.</p>\n<p>And if you’re a developer, consider helping out. It’s going to be rough going: the mailing list and issue trackers are full of people that have no idea what they’re doing. Be prepared to wade through tons of crap. But they could really, really use the help. I’ll be submitting another patch or two, but it needs much, much more than I can give.</p>\n<p>EDIT: Please check out <a href=\"http://blog.steveklabnik.com/this-is-why-new-media-hates-old-media\">my follow up post</a>, where I talk about being misquoted regarding this post.</p>\n<p>EDIT 2: Patrick has a post up describing the exploits, now that there have been some patches applied. If you’re a technically inclined individual, you might be interested. You can find it on <a href=\"http://www.kalzumeus.com/2010/09/22/security-lessons-learned-from-the-diaspora-launch/\">his blog</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/a-word-about-why-whyday-and-hackety-hack/",
"title": "A word about _why, Whyday, and Hackety Hack",
"description": null,
"url": "https://steveklabnik.com/writing/a-word-about-why-whyday-and-hackety-hack/",
"published": "2010-08-19T00:00:00.000Z",
"updated": "2010-08-19T00:00:00.000Z",
"content": "<p>Today is Whyday. A programmer’s holiday. Today is the day that we don’t forget that programming isn’t all Agile and driven by behavior and Serious Business. Inside each cubicle, the hacker spirit is trying to burst free. Today, it will.</p>\n<p>Today, I’m also releasing a prerelease version of Hackety Hack 1.0. It’s been a long time coming. The announcement of Whyday got me thinking about the past year of a world without _why, and the ‘your domain is about to expire’ notices got me thinking about how I’ve now been the head of the Hackety Hack project for almost 12 months, too. I don’t like thinking about this, because I almost let _why down.</p>\n<p>I never met _why. I can’t really claim to know him, or know what his intentions were, but I think about him a lot. ‘_why’ the persona, really embodies everything that I love about software. The world owes _why a great debt, and I’d like to think that I’m doing what I can to repay him.</p>\n<p>At least, I am now. I don’t like to think about roughly 6 or 8 months of the last year of Hackety Hack; I did an exceptionally poor job of being a steward. Being alone, having no help, and having a bit of an identity crisis is an excuse; you can decide if it’s a good or poor one. But things have been looking up, and we’ve got some momentum, I owe a great debt to Fela for helping make this summer happen.</p>\n<p>I really do feel that the Little Coder still has a predicament, and I’m almost to the point where they can find some solace. I still feel that Hackety Hack could become an absolutely invauable project, I’ve just got to keep going. I can, I will, and I am. But I wouldn’t be able to if so many people didn’t care, say a nice world, or lend a hand.</p>\n<p>So thanks. And as ashbb would say, “Let’s all have fun with Hackety Hack!”</p>\n<p>Happy Whyday.</p>\n<p>EDIT: The release is now out. See the announcement <a href=\"http://hackety-hack.com/posts/happy_whyday\">here</a>, and get it <a href=\"http://hackety-hack.com/\">here</a>.</p>\n<p>Thanks!</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/a-blip-in-time/",
"title": "A blip in time",
"description": null,
"url": "https://steveklabnik.com/writing/a-blip-in-time/",
"published": "2010-07-24T00:00:00.000Z",
"updated": "2010-07-24T00:00:00.000Z",
"content": "<p>Sometimes, insight comes from very unlikely places.</p>\n<p>I’m keenly interested in the music industry, and how it will evolve, or possibly die. I care deeply about music; it’s had a profound effect on my life over the years. Even though the entire industry has been screaming that they’re dying, I truly believe that music will never die. It’s bigger than money. I’m not exactly sure about the details, even though I have some ideas.</p>\n<p>In any case, I was reading about this the other day, and I came across <a href=\"http://sometimesright.com/2010/05/sir-mick-on-music-profits/\">this quote from Mick Jaggar</a>:</p>\n<blockquote>\n<p>But I have a take on that – people only made money out of records for a very, very small time. When The Rolling Stones started out, we didn’t make any money out of records because record companies wouldn’t pay you! They didn’t pay anyone!Then, there was a small period from 1970 to 1997, where people did get paid, and they got paid very handsomely and everyone made money. But now that period has gone.So if you look at the history of recorded music from 1900 to now, there was a 25 year period where artists did very well, but the rest of the time they didn’t.</p>\n</blockquote>\n<p>An interesting bit of history, and an interesting perspective from someone who has some skin in the game, to say the least. But it’s the last sentence of that blog post that gives the true insight:</p>\n<blockquote>\n<p>Don’t stop at 1900, though. If you think of the entire history of the world, the notion that you could make an outsized return on making music is a complete aberration.</p>\n</blockquote>\n<p>When I first read this sentence, something clicked inside my head, and I had a realization: does this apply to everything?</p>\n<p>We always think that we’re moving forward. It’s called ‘progress’ for a reason. But it’s kind of obvious why we’d think this: we’re the ones living it! Of course we’d do know wrong!</p>\n<p>But what if we are?</p>\n<p>What if we’re not progressing, but regressing? What if the latest and greatest isn’t where things are going… it’s just a small blip in the graph?</p>\n<p>Maybe we’re actually going down the wrong path, building towards deflation, and need to turn around and revise our position. Maybe we’ve misstepped. What if we’re doing what we think is right, but it’s actually wrong? What if we’re just on the upswing of some local minima?</p>\n<p>I don’t necessarily mean that, for example, the music industry will cause the total collapse of everything. But maybe it was a temporary peak. Things will return back to some even level again, and over the grand sum of history, such a small deviation will be forgotten.</p>\n<p>Then I start to apply this line of reasoning to everything. How can we really know if we’re doing better than yesterday?</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/case-study-in-being-excellent-divvy/",
"title": "A case study in being excellent: Divvy",
"description": null,
"url": "https://steveklabnik.com/writing/case-study-in-being-excellent-divvy/",
"published": "2010-06-30T00:00:00.000Z",
"updated": "2010-06-30T00:00:00.000Z",
"content": "<p>The images in this post have been lost to time. Oh well. Original post below, images removed.</p>\n<hr />\n<p>The other day I wrote a post about <a href=\"http://blog.steveklabnik.com/being-excellent-to-each-other\">being excellent to each other</a>. Today, I’ve got a great example of that for you: Divvy.</p>\n<p>What would you do if you received an email like this?</p>\n<p>Here’s what happened:</p>\n<p>Here’s a <a href=\"http://www.reddit.com/r/programming/comments/ckcbr/mizage_a_small_development_company_that_made/\">link to the thread</a>. Check out some of these comments:</p>\n<p>… okay, maybe that last one is a bit much. But still, they’ve obviously been repaid many times over for something that <em>didn’t even cost them money</em>. This <a href=\"http://news.ycombinator.com/item?id=1473770\">comment on HN</a> really says it all:</p>\n<p>If you’d also like to support a company for being awesome, here’s a link to <a href=\"http://www.mizage.com/divvy/\">Divvy</a>.</p>\n<p>EDIT: They’ve posted the sales afterward on Reddit <a href=\"http://www.reddit.com/r/programming/comments/cmmfg/hey_reddit_a_week_ago_someone_posted_about_divvy/\">here</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/productivity-waves/",
"title": "Productivity Waves",
"description": null,
"url": "https://steveklabnik.com/writing/productivity-waves/",
"published": "2010-06-08T00:00:00.000Z",
"updated": "2010-06-08T00:00:00.000Z",
"content": "<p>Right now, I’m having a bit of a ‘darkness before the dawn’ kind of moment. I feel like I might soon start to become productive again.</p>\n<p>It’s sort of strange, how these things travel in waves. Just a month or two ago, I was super-ultra-crazy productive. I was working on Hackety, coding Bindlr, writing three days a week on the blog, and more. This doesn’t even count the more-than-a-full-time CloudFab. But lately, all I’ve wanted to do was hang out and play video games.</p>\n<p>It seems like this is a cyclic kind of thing. I think 80 hour weeks, even of things that you love, is kind of unsustainable. It was lots of fun, and I was feeling pretty good while I was doing it, but now that I’m off that horse, it’s really hard to get back on.</p>\n<p>In any case, here’s the status of some of the stuff I’ve been up to, if you’ve been wondering:</p>\n<ul>\n<li>Blog: I’ve got a bunch of stuff half-written. I’ll be starting to finish those off and getting them up here. I’ve had lots to say, just no motivation to say it.</li>\n<li>Watch.steve: I want a better design, so I’m not doing anything with the site ’till that’s done. I don’t want to mess with Ryan’s look and feel. But I’ve got episodes written up, they just need to be recorded.</li>\n<li>Hackety Hack: Fela is making great progress, and I’m going to start reaching out to Mac development communities to see if I can get some help on the issue that’s ruining my ability to fix things.</li>\n<li>Bindlr: Up, running, and working. Needs more marketing/outreach.</li>\n<li>Twitter: started using it again, you read that post.</li>\n<li>Selling my stuff: most of it is in boxes or thrown away. Time to start selling the things that were worth money.</li>\n<li>Maria: currently missing her.</li>\n<li>Friends: started actually making it back to XOmB meetings on Saturdays, and actually going out with some people I haven’t talked to in a while. I was really really bad at this in some cases.</li>\n</ul>\n<p>This post is kind of random, and I apologize. I’ve got a good mix of technical things, book reviews, and social commentary waiting in the wings.</p>\n<p>I’m excited for what June will bring.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/you-are-terrified-of-your-own-children-since-they-are-natives-in-a-world-where-you-will-always-be-immigrants/",
"title": "A Declaration of the Independence of Cyberspace",
"description": null,
"url": "https://steveklabnik.com/writing/you-are-terrified-of-your-own-children-since-they-are-natives-in-a-world-where-you-will-always-be-immigrants/",
"published": "2010-05-01T00:00:00.000Z",
"updated": "2010-05-01T00:00:00.000Z",
"content": "<p>I just saw this today. It kind of goes along with my “Move to the Internet” post from a few days ago.</p>\n<p>Enjoy.</p>\n<p><a href=\"https://www.eff.org/cyberspace-independence\">A declaration of the Independence of Cyberspace</a></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/a-break-with-the-past/",
"title": "A break with the past",
"description": null,
"url": "https://steveklabnik.com/writing/a-break-with-the-past/",
"published": "2010-04-27T00:00:00.000Z",
"updated": "2010-04-27T00:00:00.000Z",
"content": "<p><a href=\"http://www.countdowntooauth.com/\">Pretty soon</a>, Twitter is going to turn off Basic Authentication and switch entirely to OAuth. <a href=\"http://www.scripting.com/stories/2010/04/26/theToxicCoralReef.html\">People are upset</a>. It’s natural. If apps aren’t updated, they’ll stop working, entirely. This could be bad.</p>\n<p>But in the long run, it won’t be.</p>\n<p>In any sort of long-term endeavor, things can go sour, due to accumulating cruft over time. You see this in married couples that argue every day, you see this in old-guard companies that have six layers of management, you see this in software of all kinds. You don’t want to break old stuff, so you support it. Rocking the boat may cause some problems. It could bring up old wounds. You might have to ask someone to improve. Things won’t ever be the same, and you’re gambling with the outcome, hoping that in the end, everything will be better.</p>\n<p>Sometimes, this gamble pays off big.</p>\n<p>Apple’s managed to do this three times in its history, in a huge way. They’ve reinvented themselves a few times, but there were three huge, backward- compatibility breaking changes that could have killed the company. The first was the transition from the 68k architecture to the PowerPC architecture, the eventual change from PowerPC to x86, and the transition from OS9 to OSX.</p>\n<p>Apple managed to make all of these transitions successfully, and it’s been one of the greatest benefits to the company. A huge part of why Windows is so horrible is that it goes above and beyond the call of duty with backwards compatibility; Microsoft never wants to rock the boat. When they try to, they get punished. Look at how poorly the transition to Vista went. So what made it go so right for Apple, and so wrong for Microsoft?</p>\n<p>The first thing that Apple did right with all of these transitions was being open about them. People were given ample time to move over to the new platform. I was young when the 68k change happened, so my remembering of that time might be fuzzy, but the OSX announcement certainly was made with ample time. An entire year elapsed between the public beta and the release of 10.1, which was widely considered the first version of OSX that was worth using. I remember buying a copy of 10.0, and Apple gave you an upgrade to 10.1 for $30, similar to the Snow Leopard upgrade. 10.0 was just too buggy to be considered a proper replacement. Anyway, that was a year, and that’s not counting the time between the announcement and the beta release. For the transition to Intel, the initial announcement was made in 2005, and Steve Jobs said the transition would happen over the next two years. That gave everyone plenty of time to get their ducks in a row.</p>\n<p>The second thing that Apple did correctly was provide ample tools for dealing with the transition. For the first switch, they provided a 68k emulator, and kept it going all the way up until the Intel transition. This meant that people didn’t have to re-write their apps from scratch, and gave them a big window for re-writing apps. Rosetta fulfills the same role for the PowerPC/Intel change. And during the OS9/OSX changeover, Apple not only let you emulate OS9, but also created the Carbon framework to bridge the gap between System 9 and Cocoa.</p>\n<p>Finally, Apple made these changes boldly, and didn’t back down. With any kind of massive change like this, unless someone forces the change through, people will stay with the old system, and the transition fails. You can see this happen with both the Python move to Python 3000, the move from Ruby 1.8 to 1.9, and the move from XP to Vista. There’s a fine line between giving people leeway to make the transition and actually forcing them to do it. Apple did a poor job with this when it came to Adobe, who’s being an incredible laggard with Photoshop. It’s a massive application, sure, but Apple announced Carbon’s end of life ages ago, they really should have gotten their shit together. Generally, Apple’s not afraid to end-of-life products after the transition time is over, though.</p>\n<p>In any case, those seem to be the three things that make a transitional period work: transparency, tools, and finality. If Twitter wants to survive the Oauthcalypse (as if they won’t), they need to follow the same path. And they have been. They announced the move to Oauth over a year ago (I think), they’ve had libraries and examples out there for ages, and they’ve picked a date and they’re sticking to it.</p>\n<p>I’m glad they’re doing it. OAuth is an important part of Twitter’s future, and it’s good that they’re embracing it fully.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/book-review-rework/",
"title": "Book review: Rework",
"description": null,
"url": "https://steveklabnik.com/writing/book-review-rework/",
"published": "2010-03-10T00:00:00.000Z",
"updated": "2010-03-10T00:00:00.000Z",
"content": "<p>I’ve been waiting for this book for a while. “Rework” is the new book by Jason Fried and David Heinemeier Hansson from 37signals. It hit stores on Tuesday. Here’s a (non-affiliate) link to <a href=\"http://www.amazon.com/gp/product/0307463745/ref=s9_simh_gw_p14_t1?pf_rd_m=ATVPDKIKX0DER&pf_rd_s=center-2&pf_rd_r=1RRQJN39HSJ4SMVA2EM4&pf_rd_t=101&pf_rd_p=470938631&pf_rd_i=507846\">Rework on Amazon</a>.</p>\n<p>For those of you who don’t know, I’m down in Houston, Texas. I’m here for a few days before heading over to Austin for SXSW. There’s a lot of really cool start up stuff, as well as 3D people, and a friend of mine. It’s been a good trip so far. But even with waking up at 3am, connecting flights, and a few hours of driving, I still managed to find a spare moment to head over to a Borders and grab a copy of Rework. And even though I’m running all over town, Nick is driving, so I’ve been able to read all of Rework in between lunches and networking events.</p>\n<p>Rework is interesting. I described it earlier today <a href=\"http://twitter.com/steveklabnik/status/10281523422\">as a philosophy text</a>, and I feel that description is absolutely apt. It’s 37signals in its purest, most potent form. If you’ve read <a href=\"http://gettingreal.37signals.com/\">Getting Real</a>, this territory will be familiar. In fact, a lot of it is basically the same. I’d be lying if I told you otherwise. The real truth is that it doesn’t matter. People who don’t already know and love 37signals won’t have read Getting Real, and so this stuff will be novel to them. People who do won’t mind re-reading this information again, as they’ve bought into the philosophy. And an update isn’t a bad thing, either. What makes Rework interesting is how it’s different from Getting Real, not what’s the same.</p>\n<p>I thought it’d be most interesting to talk about Rework in the context of it’s own philosophy. I think there are three points in particular in the book itself that point out just why this book is so good. The first is an old 37signals standby, the other two are new.</p>\n<h3>Build half a product, not a half-assed product</h3>\n<p>This got <a href=\"http://gettingreal.37signals.com/ch05_Half_Not_Half_Assed.php\">a section devoted to it in Getting Real</a>. Here’s the core idea:</p>\n<blockquote>\n<p>Throw in every decent idea that comes along and you’ll just wind up with a half-assed version of your product. What you really want to do is build half a product that kicks ass.</p>\n</blockquote>\n<p>They mention that Rework used to be twice as big. The next to last draft was 57,000 words, and the final draft was 27,000. This is the biggest difference between the two books. It’s the most pure dosage of Kool-Aid I’ve ever read. Each section feels finely honed. They’re all either one, two, or three pages, and an accompanying picture. This book is about what’s worked for the company so far over its lifetime, and this refinement process is clearly demonstrated here.</p>\n<p>It’s always easy to ask for more. I’m really curious about the things that were cut. Were there more sections? Was each section twice as long? A little of both? At the same time, this exactly exemplifies the thinking this section is railing against. If the book was twice as long, would I have learned twice as much? Probably not. YAGNI.</p>\n<h3>Decommoditize your product</h3>\n<blockquote>\n<p>Make you part of your product or service. Inject what’s unique about the way you think into what you sell.</p>\n</blockquote>\n<p>This is something that these guys do really well. It’s part of having such strong opinions, and sharing them with the world. Everyone knows who 37signals is and what they stand for. If I wrote each chapter of Rework into a blog post, you’d still recognize it as their philosophy. It also comes through in the writing. They mention in the notes that Matthew Linderman helped them pull their distinct styles of writing into a more cohesive whole. He did a good job, and didn’t let the voice get lost in the editing.</p>\n<h3>Out-teach your competition</h3>\n<blockquote>\n<p>Teaching is something individuals and small companies can do that big companies can’t.</p>\n</blockquote>\n<p>Teaching is a topic I’ve been getting more and more into lately. Hackety Hack is about teaching; when I speak, I’m teaching; this blog is about teaching. Rework is about teaching the lessons 37signals have learned about business to the world. A lot of Signal vs. Noise is about teaching. It’s a great way to get people to recognize you, and a great way to give back. The world can always use more great teachers.</p>\n<h2>Wrapping up</h2>\n<p>There’s a reason I’m up late, writing this review. I couldn’t put Rework down. I enjoyed revisiting the old topics; the new ones are still tumbling around in my brain. I’m sure this book is going to spawn a bunch of posts on this blog in the future, as I add my own thoughts to the stuff I’ve learned. I’d recommend this book to anyone who’s working in the business world or doing a startup, even if they hate that term. It’ll give you a lot of interesting thoughts to chew on.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/create-a-more-compelling-experience-for-your-users-through-game-mechanics/",
"title": "Create a more compelling experience for your users through game mechanics",
"description": null,
"url": "https://steveklabnik.com/writing/create-a-more-compelling-experience-for-your-users-through-game-mechanics/",
"published": "2010-03-08T00:00:00.000Z",
"updated": "2010-03-08T00:00:00.000Z",
"content": "<p>Ever wonder why some websites are so addictive? Certain sites always keep you going back, time after time after time. Well, I can’t speak for all of them, but there’s a subtle reason that some sites draw your attention on such a repeated basis: They’re actually games.</p>\n<h2>Wait, games?</h2>\n<p>Try a little thought experiment: If I say, “Yeah, he’s a ______ addict,” what are the first few things that pop into your mind? For me, top two are “heroin” and “World of Warcraft.” I’m not sure what that says about me as a person, but ignore that for now. What makes these two things so addicting? Why are they basically synonymous with the word “addict”? Lots of people smoke pot. Lots of people play Call of Duty. Lots do both, and in copious amounts. So why don’t they get the same label?</p>\n<h2>Heroin: it’s a hell of a drug</h2>\n<p>Yeah, that reference is to cocaine, another famously addictive substance. Oh well.</p>\n<p>Heroin is the poster child for addiction because it’s got a built-in viral loop. That sentence sounds ridiculous, but it’s true. It’s very easy to start out with, as it’s snorted. No scary needles or anything. You get high really quickly, due to its chemical properties combined with the fact that your nose is already close to your brain. It gives a really intense high that is also fairly short. As you do it, you develop both a psychological addiction as well as a tolerance. You simultaneously develop a deep desire for more of the drug as you need a larger quantity of the drug to get the same high. Eventually, it becomes more and more difficult, but you’re so addicted that you get over your fear of needles and start mainlining.</p>\n<p>World of Warcraft works the same way. It’s easy to try, as there are mechanisms to invite your friends, and the system requirements are fairly low for a video game. The first few quests are super easy, and so you hit that quick reward. You get addicted to “Ding!” but it takes longer and longer every time you do it. Eventually, you max out on levels and have to start doing other things to get your fix. It may sound funny, but it’s absolutely true. People talk about “relapsing.” They speak of “craving.” That’s why WoW has so many subscribers.</p>\n<h2>How to replicate this success</h2>\n<p>I can’t guarantee that you’ll be able to make your site as addictive as heroin is, but many sites use the same basic psychology to keep you coming back. Game mechanics are one of the tools they use to develop that psychological addiction. This is something we’ve been seeing more and more of lately, but it isn’t really being talked about explicitly as a major trend. I really think that this stuff is really important and useful.</p>\n<p>There are a couple of different mechanisms that web sites can incorporate that fall under the realm of “game mechanics:”</p>\n<ul>\n<li>Collectibles: <em>Any sort of item you can accumulate. Sometimes comes in “sets,” which are finite lists.</em></li>\n<li>Points: A concrete number that lets you compare two people.</li>\n<li>Levels: A target number of points, you gain the “level” when you go over that number.</li>\n<li>Trophies: A special kind of level that’s unrelated to points. You get it for some other arbitrary reason.</li>\n<li>Rankings: A place where you can go to see how many points, levels, and trophies others have</li>\n<li>Tournaments: A competition between people.</li>\n</ul>\n<p>We’ve all heard these terms used in games. But in web sites? Okay, let’s try those things again:</p>\n<ul>\n<li>Collectibles: Gowalla items. Facebook “Gifts”</li>\n<li>Points: Twitter followers. Facebook friends. Number of feedbacks. Reddit Karma.</li>\n<li>Levels: eBay “Power Sellers.” Foursquare “Super Users.”</li>\n<li>Trophies: Badges, of any kind. “Achievements”</li>\n<li>Rankings: FourSquare’s Leaderboard. Klout. Listorious. Hacker News’ top list.</li>\n<li>Tournaments: I actually can’t come up with a good example of this. Thoughts?</li>\n</ul>\n<p>The same feedback loop happens on these websites. You say something interesting on Twitter, you gain another follower or two. You say something else, another follower. You check in, oh look, you’re the mayor! You sell an extra hundred things and get your Power Seller discount.</p>\n<p>That’s the hard stuff. It’ll get you hooked, and coming back for more.</p>\n<h2>Where’s all of this going?</h2>\n<p>This is the current stuff that’s being done with game mechanics. But where could we go, in the future?</p>\n<p>A while back, there was a huge debacle over ReadWriteWeb and Facebook connect. To give you the basic idea, <a href=\"http://www.readwriteweb.com/\">ReadWriteWeb</a> is a blog that talks about everything Web2.0. They wrote an article entitled “<a href=\"http://www.readwriteweb.com/archives/facebook_wants_to_be_your_one_true_login.php\">Facebook Wants to be your One True Login</a>.” Read the comments. Notice something funny? Due to some Google magic, if you were to Google “Facebook login” the day that was posted, that article would appear at the top under the “Google News” results. Now, RWW uses Facebook Connect for their commenting system, and a ton of people apparently don’t know how to use the Internet. So when they said, “Hey, I think I’ll go to Facebook today,” they Googled “facebook login,” clicked the news story, and went to RWW. They then ignored that RWW is a blog completely covered in red that looks nothing like Facebook, scrolled until they found the Facebook icon, clicked it, logged in, and then said “wtf, this isn’t my facebook? Why’d they change the interface again???” This happened a week after a middle-sized interface upgrade on Facebook, for extra hilarity.</p>\n<p>Now, I won’t comment on those people or that situation directly. But one of my favorite Hacker News posters, <a href=\"http://news.ycombinator.com/user?id=patio11\">patio11</a>, posted <a href=\"http://news.ycombinator.com/item?id=1119186\">a really interesting comment</a> about the situation. I’m linking to the person he’s responding to, for context:</p>\n<blockquote>\n<p>Pyre: Facebook can’t improve their interface to make users not type “facebook login” into Google as a way of accessing their site.</p>\n</blockquote>\n<blockquote>\n<p>patio11: That is a failure of the imagination. They certainly could – whether it is worth doing or not is another question, but hey, that is what God gave us A/B testing to figure out.</p>\n</blockquote>\n<blockquote>\n<p>“Hey user, it looks like you came to us today from Google searching for [Facebook login]. Did you know that there is a better way? Type facebook.com into [blah blah blah]. Try it now and we’ll give you 5 free credits for [without loss of generality: FarmVille]!”</p>\n</blockquote>\n<blockquote>\n<p>Great job! You should do that every time. If you do that to log into Facebook the next five days you use the service, we’ll award you a Facebook Diploma and give you another 10 free credits for [without loss of generality: FarmVille]!\"</p>\n</blockquote>\n<blockquote>\n<p>On the back end, you show the above prompts to N% of your users who you detect coming to the login page from Google search results (this is trivial – check the referer). You then compare any user metric you want for the “Was Shown Facebook Login Course” population and “Complete Facebook Login Course” population with the population at large. Kill the test if it hurts your metrics, deploy it sitewide if it helps them.</p>\n</blockquote>\n<p>How cool would that be? Now the game mechanics aren’t being used just to increase engagement, but to actually teach people how to use your site or service. It’s classical conditioning; reward people for doing the right thing, and they’ll keep doing the right thing.</p>\n<h2>Game mechanics are your MVP</h2>\n<p>So how’s this stuff relevant to your startup? Well, I think this idea ties in really well with the concept of a Minimum Viable Product. Here’s the idea: Build your MVP, and then build game mechanics in. Unlock new features based on game mechanics. This gives you a few advantages:</p>\n<ol>\n<li>Your new users only get the most basic experience, which is still useful. It’s a simplified, streamlined experience.</li>\n<li>Users only get the new features added that are relevant to how they use the site itself.</li>\n<li>You can “fake it till you make it” by implementing the features that are most useful to your users. Is everyone getting Badge A and not Badge B? Implement Feature A Level 2 first!</li>\n</ol>\n<p>I think that this makes for a really strong experience, if done right. Foursquare kind of does this already in a crude way with their Super User features. But I think it could be taken to a whole new level.</p>\n<p>Think about this: Facebook, where you can only friend people, update your profile, and send messages at first. Soon you unlock the ability to use applications. Then the ability to create pages and groups. The interface slowly unfolds in front of you. What about Reddit, where posting comments is all you can do at first? A hundred upvotes gives you the ability to downvote. Ten comments lets you post stories. (Hacker News sort of does this already, with a minimum karma before downvoting is enabled.)</p>\n<p>If you could pull it off, I think it’d make for a really compelling user experience. It does bring one extra design skill that many people may not have, though: balance. Game designers are used to this already, but your potential “Power Users” might not like having to wait to get more advanced features. Then again, this might also solve some issues, like spam. If you had to have 100 positively moderated comments before posting a story on Digg, it’d be much harder to just sign up for spam accounts to submit bogus stories.</p>\n<p>This idea can be taken in a lot of different directions. I’m sure I’m only barely scratching the surface with this idea, but I think it’ll go really far. What do you think? Any interesting mechanics I’ve missed? Any really interesting thoughts for how services can incorporate game mechanics? I’ve decided to re-open comments, but if nobody uses them, I’ll just shut them off again. Let me know what you think.</p>\n<p>This post has been featured on the Startup Pittsburgh blog, <a href=\"http://startuppittsburgh.com/2010/04/create-a-more-compelling-experience-for-your-users-through-game-mechanics/\">here</a>.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/writing-a-su-feature-with-authlogic/",
"title": "Writing a su feature with Authlogic",
"description": null,
"url": "https://steveklabnik.com/writing/writing-a-su-feature-with-authlogic/",
"published": "2010-03-05T00:00:00.000Z",
"updated": "2010-03-05T00:00:00.000Z",
"content": "<p>Sometimes, when responding to a support request, it’s nice to see what your users see. At the same time, you don’t want to ask your users for their passwords, out of respect for their privacy. So what do you do?</p>\n<p>Well, *NIX systems have a program called su. Here’s what man su has to say:</p>\n<pre><code>NAME\n su - run a shell with substitute user and group IDs\n\nSYNOPSIS\n su [OPTION]... [-] [USER [ARG]...]\n\nDESCRIPTION\n Change the effective user id and group id to that of USER.\n</code></pre>\n<p>su can be thought of as “substitute user” or “switch user.” It’s a command system administrators use to assume the identity of one of their users, or a way for someone with the root password on the system to switch to the root account itself. So how can we incorporate this into a web application?</p>\n<p>Well, we want to first log ourselves out, and then log in as the user we’re su-ing to. That’s it. The tricky part, however, comes in when we’re logging in: as we said before, we don’t want to ask for their password. Luckily, Authlogic provides a way to create our UserSession object directly from a User object by just passing it to create.</p>\n<p>This lets us write a controller method to do this pretty easily:</p>\n<pre><code>def su\n @user = User.find params[:id]\n current_user_session.destroy\n UserSession.create!(@user)\n flash[:notice] = \"You've been su-d to that user.\"\n redirect_to dashboard_path\nend\n</code></pre>\n<p>Add in a route:</p>\n<pre><code>map.admin_su \"/admin/su/:id\", :controller => \"admin\", :action => \"su\"\n</code></pre>\n<p>And to a view somewhere in your administrative tools:</p>\n<pre><code><%= link_to \"log in as this user\", admin_su_path(@user) %>\n</code></pre>\n<p>And we’re good to go!</p>\n<p>One last thing about this, though: You don’t want to let anyone who’s not an administrator do this, for obvious reasons. My administrative controllers always include a block like this:</p>\n<pre><code>access_control do\n allow :admin\nend\n</code></pre>\n<p>acl9 makes this really easy, but it’s really important.</p>\n<p>So there you have it. Easy as pie.</p>\n<p>EDIT: This post made the Rails subreddit, and <a href=\"http://www.reddit.com/r/rails/comments/cb0da/writing_a_su_feature_with_authlogic/c0rf26w\">brettbender posted his code</a> to get you back to admin.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/reddit-drama-an-interesting-look-at-community-values/",
"title": "Reddit drama: an interesting look at community values",
"description": null,
"url": "https://steveklabnik.com/writing/reddit-drama-an-interesting-look-at-community-values/",
"published": "2010-03-01T00:00:00.000Z",
"updated": "2010-03-01T00:00:00.000Z",
"content": "<p>I bet social psychologists are having a field day with the Internet. I know nothing about the field, so I don’t know if there are thousands of papers about it already written or not, but there should be. I can’t image that we’ve ever had a period in history when more communities are being formed; they’re also being formed out in the open.</p>\n<p>I’ve been involved in many different online communities over the years. I’ve already written before about the seeming decline of Reddit’s content… but this past week, something really interesting has happened. Reddit has had two major events occur that have caused it to be increasingly self-reflexive on the values and norms that the community desires for itself. This is particularly interesting because normally, cultural norms have to be observed, not codified. But here, we’re seeing completely open discussion about “The community says this is not okay.” It’s really interesting stuff.</p>\n<h2>Speed Issues</h2>\n<p>I won’t comment on the technical details involved, but Reddit has gotten significantly slower over the past few months. This is normal, as the community is growing. But it’s caused quite a bit of a stir lately. Many users are seeing increased loading times, error messages, missing posts, and various other problems.</p>\n<p>What’s a user to do in this situation? Post about it. See the <a href=\"http://www.reddit.com/search?q=fix+search&sort=hot&t=month\">posts about the search feature</a>, or posts about the slowdown.</p>\n<p>For example, as I look for the links to these posts, I get this:</p>\n<blockquote>\n<p>Our search machines are under too much load to handle your request right now. :( Sorry for the inconvenience.</p>\n</blockquote>\n<p>Amusing. But unfortunate. The interesting part, though, is the backlash against the complaints. See this thread, “<a href=\"http://www.reddit.com/r/reddit.com/comments/b7jnr/reddit_fix_your_fucking_users_they_are_unbearably/\">Reddit, fix your fucking users. They are unbearably bitchy.</a>” (The cursing is emulating the posts made complaining about Reddit’s lack of speed.)</p>\n<p>There’s a huge discussion about what it means to be part of a free online community. There’s a contingent of people who say that people aren’t entitled to complain, because Reddit is free. Another points out that complaining is better than silently leaving the site, and that while people don’t pay for Reddit accounts, the eyeballs Reddit users provide enable the site to make money.</p>\n<p>Some choice comments:</p>\n<blockquote>\n<p>Hey, my incredibly expensive Reddit subscription fees entitle me to 100% perfect service! LEAVE ME ALONE!</p>\n</blockquote>\n<blockquote>\n<p>Man, I’ve been loving this particular strawman for over a decade now. Thanks for breaking out the classics.</p>\n</blockquote>\n<blockquote>\n<p>The reality is that Reddit is a business. The owners get paid because the users keep coming back to the site. Having users bitch and moan is not a problem for the admins - this isn’t their personal blog, they’re not mopy teenagers who are gonna cut themselves when they see how mean people are being to them on the interwebs.</p>\n</blockquote>\n<blockquote>\n<p>The problem for the admins is when users stop visiting Reddit altogether because of constant technical problems, or fishy administrative decisions about deleting threads, or too many spammers, or etc. They would much rather have us bitch about these things on the front page, create giant threads about them that keep thousands of posters and tens of thousands of readers engaged and on the site, and provide catharsis and a public forum to address the issues.</p>\n</blockquote>\n<blockquote>\n<p>tl;dr: Reddit is a business, they’d rather have us complain than have us leave.</p>\n</blockquote>\n<blockquote>\n<p>Meta-bitching about bitching time</p>\n</blockquote>\n<blockquote>\n<p>The OP posted a well thought out comment simply suggesting that Reddit’s users should be polite and respectful when addressing each other and the people that make using this site possible, and 90% of the responses here just prove his point. It seems like people are more concerned with gaining comment karma by posting bad one-liners than actually participating in intelligent conversation.</p>\n</blockquote>\n<blockquote>\n<p>Is anyone else absolutely done with the reddit user base? I mean… I was told that reddit was this great intellectual news aggregator that through the use of ‘karma’ was able to bad submissions off the front page.</p>\n</blockquote>\n<blockquote>\n<p>Over the past few months I’ve been here all that has been up on the front page has been Glen Beck, Conan O’Brian, Tiger Woods, weekly “Reddit is slow” posts and now this Saydrah BS.</p>\n</blockquote>\n<blockquote>\n<p>I’ll say what I said when Conan O’Brian was on the front page for a week longer then Haiti ever was: I DON’T CARE. The Reddit admins don’t think Saydrah has abused her power as a moderator. At the very least (I personally don’t think she is a spammer) go flame some of the other 1000000000000000 spammers on reddit FFS. This is boring uneducated dribble.</p>\n</blockquote>\n<blockquote>\n<p>KEEP THIS TRASH OFF THE FRONT PAGE.</p>\n</blockquote>\n<p>And more. Interesting stuff, overall. There’s still quite a few trash comments, though.</p>\n<h2>The Saydrah Situation</h2>\n<p>The other event involves a user named Saydrah. She’s been accused of promoting links on Reddit for money, and abusing her moderator powers in conjunction with those offenses. This has spawned a huge amount of discussion on if these actions were inappropriate or not. Here’s <a href=\"http://www.reddit.com/r/reddit.com/comments/b7e25/today_i_learned_that_one_of_reddits_most_active/\">the first big thread</a>. Then <a href=\"http://www.reddit.com/r/IAmA/comments/b7hpb/re_the_alleged_conflict_of_interest_on_reddit/\">an AMA with moderators about their opinions</a>. Lastly, <a href=\"http://www.reddit.com/r/IAmA/comments/b7tew/fine_here_saydrah_ama_it_couldnt_get_much_worse/\">Saydrah does an AMA</a> herself and explains her side of the story.</p>\n<p>I won’t show you a bunch of comments, only one. And it echoes my opinion on the matter:</p>\n<blockquote>\n<p>Two things: Frankly, I don’t care if people are paid to submit links…if they’re interesting, I upvote. If not, I ignore them.</p>\n</blockquote>\n<blockquote>\n<p>Second, I have lurked around AR and RA a fair amount, and consistently find myself thinking, “That’s a helpful and well-reasoned response! Oh, it’s Saydrah again!” Whatever else people may say, I feel that you at least do try to contribute positively to this site.</p>\n</blockquote>\n<blockquote>\n<p>…I guess I don’t have a question, so I’ll just leave my un-asked-for opinion here.</p>\n</blockquote>\n<p>There’s a lot more, on both sides of the conflict. Regardless, people are hashing it out.</p>\n<h2>Growth Hurts</h2>\n<p>There’s a reason they’re called “growing pains.” Change is always a painful time in any organism’s life, and even though there’s a lot of drama, the current Reddit staff can take pride that they’ve created something that’s so important to people that they feel the need to scream about it for hours. It’s unfortunate that their baby is being embroiled in a flamewar, but these things happen.</p>\n<p>We’ll see what the community ends up deciding is acceptable. I’ve managed to not get involved in these particular conflicts, but it sure is interesting to watch!</p>\n<p>Edit: Jen added some interesting links on my facebook feed: <a href=\"http://j.mp/baRqdy\">http://j.mp/baRqdy</a></p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-hackety-mainfesto/",
"title": "The Hackety Manifesto",
"description": null,
"url": "https://steveklabnik.com/writing/the-hackety-mainfesto/",
"published": "2009-12-31T00:00:00.000Z",
"updated": "2009-12-31T00:00:00.000Z",
"content": "<p>For those of you that don’t know, I’ve taken over a project by someone named _why. He dropped off of the face of the Internet back in March, and the community has picked up his works. Because I’ll be writing about the project in the future, and because I feel these words are still relevant, I’m reprinting the two of his essays that led up to the creation of the project, Hackety Hack, here on my blog. Here’s the second one.</p>\n<p>Enjoy.</p>\n<h1>The Hackety Manifesto</h1>\n<p>Nearly four years ago, I wrote an essay called The Little Coder’s Predicament. It’s not too unusual. Lots of others like it have been written. The point is: programming just isn’t available to people like it was with the Commodore 64. I also outlined my requirements for a new cross-platform programming environment for beginners.</p>\n<p>The essay was widely linked on Slashdot, Reddit, Lambda, etc. I got lots of mail from people who both agreed and disagreed. Great. Nice. Good people all of them! And yet, nothing has changed. Not really!</p>\n<p>I’ve been told of the Xbox development kit and possible programming of Wii channels. The Playstation actually had a language. But none if it has met my criteria for a proper coding platform.</p>\n<h2>An Aside: We Care, But Not Enough</h2>\n<p>So, why has nothing been done about this? Maybe our interpreters and IDEs (ewww…) are good enough? Lots of people wrote in to say that HTML and JavaScript are the new BASIC. NO!!! You can’t be serious!!! So people have to write two languages now, which are intertwined in an almost inexplicable and unfathomable way? This doesn’t do it.</p>\n<p>Hello world should be one line.</p>\n<p>In fact, downloading an MP3 should be one line!!</p>\n<p>We just don’t care right now, do we? Programmers have a paid gig. So business is happily slurping them up. Look at our books. Look at the programming sites. Programming is tightly coupled to business. Often the first example is an e-commerce site! Our books are like FIFTY DOLLARS!! For crying out loud.</p>\n<p>This diatribe isn’t about business being bad. Of course you need to feed your family and drive an Audi.</p>\n<p>This diatribe is about adding some balance to the world of programming. Okay, so, let’s take things into our own hands and bring hacking to the young folks.</p>\n<h2>The Bylaws of Hackety</h2>\n<p>Here are the rules by which Hackety Hack was established:</p>\n<ul>\n<li>Beginners should be greeted to Hackety Hack by a cartoon character. (For the sake of argument, let’s call this character: Hacky Mouse.)</li>\n<li>Also, helpful sentences. Preferably short and with a period.</li>\n<li>Hackety Hack is fundamentally a browser and a programming language. For now, Gecko and Ruby.</li>\n<li>I’m only using Ruby because I know it. Hopefully, more languages can be added!</li>\n<li>Again, this isn’t about Ruby, it’s about simply offering a place for plainspeople to tinker with code.</li>\n<li>IDEs are a disaster. Newbs should see only one non-scary window free of tree controls and pinned windows and toolbars.</li>\n<li>As such, we want to stay away from project files and makefiles, the trappings of an IDE.</li>\n<li>Hackety Hack also adds simple libraries for common things.</li>\n<li>Common things are one-liners.</li>\n<li>Keep args and options to a minimum.</li>\n<li>In Ruby, blocks should be used to open up a method to more advanced possibilities.</li>\n<li>Help files are clean, short, simple. Lots of short examples. No frames.</li>\n<li>While all bug tickets are helpful and great, I just value tickets from beginners to a greater degree.</li>\n<li>Hackety Hack is free and will remain free henceforth.</li>\n</ul>\n<p>Beyond that… anything which makes life easy and fun for Hackety Hackers is definitely encouraged, even demanded.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/the-little-coders-predicament/",
"title": "The Little Coder's Predicament",
"description": null,
"url": "https://steveklabnik.com/writing/the-little-coders-predicament/",
"published": "2009-12-28T00:00:00.000Z",
"updated": "2009-12-28T00:00:00.000Z",
"content": "<p>For those of you that don’t know, I’ve taken over a project by someone named _why. He dropped off of the face of the Internet back in March, and the community has picked up his works. Because I’ll be writing about the project in the future, and because I feel these words are still relevant, I’m reprinting the two of his essays that led up to the creation of the project, Hackety Hack, here on my blog.</p>\n<p>Enjoy.</p>\n<h1>The Little Coder’s Predicament</h1>\n<p>Okay, then, children of the modern age (where we live in a world so tied together with wires that Pangaea ain’t goin’ nowhere!), you tell me if this is a predicament or not.</p>\n<p>In the 1980s, you could look up from your <a href=\"http://www.lemon64.com/\">Commodore 64</a>, hours after purchasing it, with a glossy feeling of empowerment, achieved by the pattern of notes spewing from the speaker grille in an endless loop. You were part of the movement to help machines sing! You were a programmer! The <a href=\"http://kl.net/atari/\">Atari 800</a> people had BASIC. They know what I’m talking about. And the <a href=\"http://web.archive.org/web/20070612155724/www.guidry.org/ti994a/systempics/4asystem-2.jpg\">TI-994A</a> guys don’t need to say a word, because the TI could <a href=\"http://www.obsoletecomputermuseum.org/ti99/ti99_syn.jpg\">say it</a> for them!</p>\n<p>The old machines don’t compare to the desktops of today, or to the consoles of today. But, sadly, current versions of Windows have no immediately accessible programming languages. And what’s a kid going to do with Visual Basic? Build a modal dialog? Forget coding for XBox. Requires registration in the XBox Developer Program. Otherwise, you gotta crack the sucker open. GameCube? GameBoy? Playstation 2?</p>\n<h2>Coding Just Isn’t Accessible</h2>\n<p>Yes, there are burgeoning <a href=\"http://openxdk.sourceforge.net/\">free</a> <a href=\"http://gbdk.sourceforge.net/\">SDKs</a> for many of these platforms. But they are obscure and most children have no means of actually deploying or executing the code on their own hardware! This is obvious to us all and likely doesn’t seem such a big deal. But ask yourself what might have happened had you not had access to a programming language on an Atari 800 or a Commodore. You tell me if this is a predicament.</p>\n<p>It turns out, most of the kids in my neighborhood are exposed to coding through the <a href=\"http://www.ticalc.org/\">TI calculator</a>. A handful of <a href=\"http://www.ticalc.org/programming/columns/\">languages</a> are available on the TI and its processor is interesting enough to evoke some curiousity. But this hasn’t spread to its PDA big brothers, where young people could have more exposure to programming. And undoubtedly the utility of a language on the Palm, Pocket PC and others would be useful to many.</p>\n<p>So what’s the problem here? We have no shortage of new languages, but they become increasingly distanced from the populace. Are the companies behind these platforms weary of placing the power of a programming language in the hands of users? Is there not a demand any longer? It’s got to be some kind of greed, power, money thing, right?</p>\n<p>Perhaps this is just another reason to push Linux and BSD on consumer systems. Still, are scripting languages easily accessible to beginners on those systems? OSX has made several scripting languages available (including Ruby and Python), but most users are unaware of their presence.</p>\n<p>I should mention that Windows is equipped with its own scripting host for developing in JScript and VBScript. But the use of the scripting host is (I believe) under-documented and limited for beginners. Try doing something useful in a script without using Server.CreateObject. Let’s not let kids touch the COM objects, please!</p>\n<h2>The Christmas List</h2>\n<p>I’m thinking a toy language for consoles and desktops alike could be monumental. I’m ot saying it needs to be cross-platform. A language for GameCube that took advantage of platform-specific features could be more appealing to GameCube users than a language that used a reduced featureset, but could execute on a handheld. Really, we live in a world where both choices should be available.</p>\n<p>As for essential features:</p>\n<h3>1. Transportable code.</h3>\n<p>On my TI-994A, I could make a little, animated Optimus Prime from pixels. Insert cassette. Record. Pass around to friends. Receive high fives from friends. Put on wraparound shades. Thank you, TI! Thank you, Optimus Prime!</p>\n<p>A little language for the consoles could be wildly popular if combined with the good ature of sharing code. This could be done by trading memory cards, but would be more effective if code could be easily obtained and posted on the Web. Learning would accelerate and collaborative development could take place.</p>\n<p>A suitable language should give coders access to I/O devices, to allow experimentation with network devices and the ability to enhance one’s connectivity with others. For the consoles, games could provide hooks for user mods. This has long proven a successful staple of the desktop gaming world.</p>\n<h3>2. Simplicity.</h3>\n<p>You’ve got to be able to write a single line of code and see a result. We need some instant results to give absolute beginners confidence. Simple methods for sending an e-mail, reading a web page, playing music. Demonstrable in a one- liner.</p>\n<p>Admittedly, as our systems have grown complex, it is difficult to balance simplicity and capability. Most users will be unimpressed by code that emits beeps and bloops from a PlayStation 2. If Ruby were available on the PS2, then I would hope that I could hear rich symphonic sounds from a wee bit of code.</p>\n<p><code>Orchestra.play( \"A:2\", \"C:4\", \"E:1\", \"G:1\" )</code></p>\n<p>Access to the graphic engine might require more complex code. But simple drawing methods could be provided for beginners. Or images could be stored alongside code and accessed programmatically.</p>\n<p><code>ImageLibrary.load( \"GolfingOldMan\" ).drawAt( 12, 10 )</code></p>\n<p>The trick would be to uncover what small applications might entice novices and still provide the ability to write large applications that would drive developers to master the language and not limit their growth.</p>\n<h3>3. Sensible environment.</h3>\n<p>Considering that many won’t want to purchase a keyboard for their gaming unit, let’s make sure that a reasonable environment is provided for entry of text. Controllers could be worked like the <a href=\"http://www.handykey.com/\">Twiddler</a>. Or code could be transferred via IR, TCP/IP. (Dare I say cassette? :D)</p>\n<h3>4. Give it away!</h3>\n<p>It used to be that programming was practically an inalienable right for users. Include a language with the system, situated in a friendly spot. Each of the game consoles I’ve mentioned has launchers. (With the exception of Game Boy and its successors.) Provide a development prompt from the launcher. From desktop software, provide shortcuts for both the command prompt and a development prompt.</p>\n<p>Remember, we’re looking for a language that requires no system hacks. No obscure links. No warranty violation. We’ve become so used to these techniques that it seems to be an essential part of getting our way.</p>\n<p>And in many ways it is essential. Tinkering with hardware is learning. Lobotomizing and renovating is meaningful, magical. On behalf of those who prefer to code, I make these wishes. Not to take away jobs from the Phillips screwdriver.</p>\n<h2>The Ultimatum</h2>\n<p>My challenge is to Sony, Nintendo, Microsoft, Apple, and to those who manufacture and develop our interactive technology. Let us interact with these machines more deeply. Provide us a channel for having a dialogue with the entertainment boxes we nurture and care for. I swear to you, the relationship between the public and your product will assuredly blossom. That box will become more of a chest for our personal works.</p>\n<p>In addition, if your developers start putting out crap, then you have a whole world of people to pick up the slack.</p>\n<p>My challenge is for you to bundle a useful programming language with your product. Ruby, Squeak, REBOL, Python. Take your pick. It will be inexpensive to add any of these languages to your systems. And people will seriously pray to you. You know how geeks get when they pledge allegiance to something. But, yes, Ruby is preferable.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
},
{
"id": "https://steveklabnik.com/writing/about-crows/",
"title": "About Crows",
"description": null,
"url": "https://steveklabnik.com/writing/about-crows/",
"published": "2009-12-02T00:00:00.000Z",
"updated": "2009-12-02T00:00:00.000Z",
"content": "<p>A short time ago, the G20 came right here to Pittsburgh. With an event of such worldwide importance happening in our backyards, several people decided to document it via social media. The Mattress Factory did a site called “MyG20” and it did pretty well. Some smaller groups did their own thing. But IndyMedia did one better: <a href=\"http://indypgh.org/g20/\">http://indypgh.org/g20/</a></p>\n<p>The <a href=\"http://thecaucus.blogs.nytimes.com/2009/09/24/getting-out-the-anti-globalization-message/\">New York Times</a> called it “One of the best all-purpose sites with updates from the streets.” It got quite a lot of attention and hits from all over. And it was put together by a good friend of mine, <a href=\"http://www.twitter.com/edfilo\">@edfilo</a> I helped a teeny tiny bit, but he (with the help of LibSyn) really did an awesome job.</p>\n<p>So why should this matter to you? The G20 is old news by now.</p>\n<p>He’s releasing the code for the site, making it available to anyone who wants to cover events. It’s called “Crows,” and it lives at <a href=\"http://crowsne.st/\">http://crowsne.st/</a> . With Crows, you can quickly and easily build a site, pulling in content from Flickr, Twitter, and elsewhere, and posting it all on a geotagged Google Map.</p>\n<p>It’s still in its rough infancy, but after the awesome role that it played during the G20, several other organizations had contacted IndyMedia about getting access to the code, and so it’s already gaining traction in the space. It’s undergoing rapid development, with new features being added quite fast. Ed really wants to make this useful for people, and I’m going to be contributing to the project in what little spare time I can as well. If you’re a coder too, you can <a href=\"http://github.com/edfilo/crows\">fork crows on GitHub</a> to contribute back to the project.</p>\n<p>Check it out, and you can follow <a href=\"http://twitter.com/crowsource\">@crowsource</a> on Twitter for updates, too.</p>",
"image": null,
"media": [],
"authors": [],
"categories": []
}
]
}