Blog

Notes on GAMs in R with a binary dependent variables

A few weeks ago I was working on a project using GAMs to estimate the gradients of the marginal effects of the likelihood of a customer to convert. This was a very cool problem, and I learned a lot, but it took me quite a bit of time to figure out how to get the built-in functions to generate outputs that are interpretable in a regression with a binary dependent variable (e.g., logistic-style regression). Below I’ve included a slightly-cleaned-up version of my notes from my research in case they’re helpful to any of y’all!

If you’re not sure what GAMs are, I’d check out these resources for a good overview:


 

Model Validation

Use gam.check() There’s two big things to check:

  • Does the model converge at all? If not, your results aren’t valid.
  • Check the p-values for the smooths at the bottom — if any of these are significant you should increase the smoothe values (k= in the model specification) — remember the k is a ceiling on the value so erring too high won’t impact your results, but erring too low will!
  • If you’re using a binary model, the QQ plot and other automatically generated plots aren’t useful. However, uou can use DHARMa to get interpretable fit checks

Logistic GAMs

  • Use the `family = “binomial”` option to estimate a logistic-style GAM model
  • Like a logistic regression, the output is on the log scale — we can convert to the probability scale using the plogis() function
    • When plotting, we can convert our output to the probability scale by using the trans argument. The trans argument takes a function that transforms the output. So we can pass the plogis() logistic function to this argument, and all values in plots will be transformed from log-odds to probabilities. plot(binom_mod, pages = 1, trans = plogis)(Source)
  • To get appropriately-scaled marginal effects plots, we can use the following specification: plot(binom_mod, pages = 1, trans = plogis, shift = coef(binom_mod)[1], seWithMean = TRUE)
    • Now our partial effect plots have much more intuitive interpretation. Each partial effect plot can be interpreted as showing the probability of the outcome if all other variables were at their average value. At their own average value, you get only the effect of the intercept. (Source)

The different GAM smoothing options

You can select s(), ti(), or te() for a smooth. ti() and te() work like interaction terms, but they’re crucially different:

  • ti() only works with variables that are on the same scale. It’s commonly used for something like latitude and longitude. (Sometimes the documentation refers to these variables as being “isotropic”)
  • te() works with variables are not on the same scale (they’re “anisotropic”)
  • You have to choose the number of basis functions to use for each smooth, using the k argument of s or te. (Source])
    • Given the penalization process, the exact choice of k isn’t too big of a deal, and can be seen as an upper limit to the flexibility of the term. The actual flexibility is determined via the penalization process. (Source)

Plotting interaction effects

  • In mgcv’s plot() command, interactions are represented with contour plots. A single plot represents both variables and their interaction. In this plot the axes represent values of our predictor variables, x1 and x2. The interior is a topographic map of predicted values. The contour lines represent points of equal predicted values, and they are labeled. The dotted lines show uncertainty in prediction; they represent how contour lines would move if predictions were one standard error higher or lower. (Source)
  • The plots generated by mgcv’s plot() function are partial effect plots. That is, they show the component effect of each of the smooth or linear terms in the model, which add up to the overall prediction. (Source)
  • It’s often useful to plot the standard errors of a partial effect term combined with the standard errors of the model intercept. This is because confidence intervals at the mean value of a variable can be very tiny, and don’t reflect overall uncertainty in our model. Using the seWithMean argument adds in this uncertainty. (Source)
    • To make the plots even more interpretable, it’s useful to shift the scale so that the intercept is included. Using the shift argument, we can shift the scale by value of the intercept, which is the first coefficient of the model. Note how the y-axis has changed. Now, the partial effect plot has a more natural interpretation – it shows us the prediction of the output, assuming other variables are at their average value. For instance, this plot shows that the miles per gallon of a 2000 pound car is about 30, all else being equal. (Source)

References

Distancia de Rescate / Fever Dream

I just finished Distancia de Rescate (Fever Dream) by Samantha Schweblin and, after a bit of a slow start, I really, really loved it. The reviews call it a “horror” novel, which it is, I suppose, but in a very different way than any horror novel I’m familiar with. I was afraid that it would be something like a pulpy Stephen King novel — something that keeps you up at night but doesn’t provide a whole ton of literary engagement (pretty much the opposite of what I’m generally looking for).

What I got really blew me away — once I got about 1/3 of the way into the novel (it’s very short!) I couldn’t put it down and finished the back two thirds in one sitting.

The novel follows the story of a mother and a young daughter who go for a vacation in a small farming town in rural Argentina (presumably). Very strange things start to happen and the story is told from the perspective of the mother after the events occurred as she is struggling to remember and understand what happened.

The horror builds slowly, and Schweblin uses the device of the retrospective “fever dream” brilliantly to keep the reader in the dark as much as the narrator even as she’s recounting events from the past. Her interlocutor, David, is also just the right amount of creepy and ominous to keep the story moving and the reader vaguely apprehensive.

** Spoilers start here **


 

Besides the technical craft of the novel (which is very, very good) I was really floored by the way that the author made use of toxic pesticides as the specter that torments the character — I had never thought of this before, but toxic pollution really does have many of the characteristics of what make for a terrifying monster in a work of horror:

  • You can’t see it
  • You don’t know when it’s close or far away
  • You can’t protect yourself with any normal methods (bullets, knives, running away)
  • Because they can’t see it, other people won’t heed your warnings until it’s too late

Schweblin makes the fear and the damage caused by toxic pollution visceral and terrifying — I had always thought of pollution as being generally “very bad” but not something I spent much time thinking about — as with any good work of fiction, Fever Dreams makes the unseen and unseeable feel real, oppressive, and truly terrifying.

Most compelling to me is how Schweblin shows us different characters with differing abilities or desires to believe in what is happening — from Carla, who prefers to believe that her son is possessed rather than poisoned to the nurse in the emergency medical center who seems to be on the payroll of the industrial agriculture company that props up the town, these characters blindness to and complicity with the toxins that are killing their town and their children only adds to the horror — however, unlike in a Stephen King novel, this horror feels incredibly real and surely a reflection of the real world today, not a horrifying imaginary world that we can take comfort in not living in.

Schweblin shows us a nightmare, and then reminds us that the nightmare is real and the characters are real and we aren’t doing anything about it.

Setting up a new note-taking system

This weekend I invested a bunch of time in researching note-taking systems and applications — I want to get in the habit of both writing more and maintaining a repository of things I’ve thought about and learned and the quarantine seems like a good idea to really sink some time into these sorts of organizational tasks (I’m also experimenting with a new email client Airmail and have been working hard to get better code linting set up in my main text editor).

I took a few notes on some note-taking philosophies and the applications that are purpose-built for note-taking (focusing on macOS) which I’ll share here.

Note-Taking

I’ve been greatly influenced by the idea of the memex developed by Vannevar Bush and of the better-phrased “bicycle of the mind” (RIP Steve Jobs) and have wanted for a long time to implement such a system for myself — while I’m a compulsive highlighter in my Kindle, Pocket, and Zotero apps, I practically never actually go back to review those highlights. Occasionally they come in handy when I know that I want to review some article and I can quickly check the highlights to see what’s most important, but I don’t have a good organic way of reviewing things I’ve found interesting over time or, for example, reviewing highlights by subject.

Stephen Johnson writes about the tools he uses for writing in the article Tool for Thought from 2005 in the NYTimes:

The raw material the software relies on is an archive of my writings and notes, plus a few thousand choice quotes from books I have read over the past decade: an archive, in other words, of all my old ideas, and the ideas that have influenced me.

I would write a paragraph that addressed the human brain’s remarkable facility for interpreting facial expressions. I’d then plug that paragraph into the software, and ask it to find other, similar passages in my archive. Instantly, a list of quotes would be returned: some on the neural architecture that triggers facial expressions, others on the evolutionary history of the smile, still others that dealt with the expressiveness of our near relatives, the chimpanzees. Invariably, one or two of these would trigger a new association in my head — I’d forgotten about the chimpanzee connection — and I’d select that quote, and ask the software to find a new batch of documents similar to it. Before long a larger idea had taken shape in my head, built out of the trail of associations the machine had assembled for me.

Similar in objective if not in implementation is the Zettelkasten method for collecting and cross-linking notes on things you’ve read or ideas you’re working on. In general the ideas can be boiled down to:

  • Be deliberate about capturing your thoughts and ideas, and don’t be too precious about what’s considered a complete thought
  • Store them in a cross-referenced and searchable system that allows for both browsing and discovery

So I want to start moving toward a system more similar to this — where I can compile and store my notes from different projects and (hopefully) promote better recollection of these thoughts and ideas as well as additional spontaneous connections.

Tools

The tools I considered are:

I was looking for an application to check a few important boxes:

  • macOS and iOS native support
  • Markdown support
  • Beautiful interface I enjoy using
  • A tagging / cross-linking system (as opposed to foldering)
  • Support for multiple media types (images, pdfs, etc)

Here are my notes on each (I ended up going with Bear)

Evernote (free / $8 per month)

Evernote is the oldest of these apps and probably the most popular — Ezra Klein is a big proponent and the key feature seems to be the clip-from-browser extension and the oceans of integration for sending data to Evernote. This didn’t feel write for me because I want an app that promotes writing rather than being a passive receptacle for things I read on the internet — that being said I’m still considering setting up Evernote to be the one-true-receptacle and funneling all of my highlights from Kindle, Pocket, and Zotero there. But that’s a project for a future weekend.

Notion ($4 per month)

Notion was a strong contender — they’ve made a really cool product and the flexibility of the tool is very impressive. However, the PMs at notion definitely seem to be headed in a collaboration and project-management direction which doesn’t really fit my use-case at all. I would definitely consider Notion for my startup if I was doing a lot of collaboration and complex project-oriented tasks.

Bear ($1.50 per month)

I landed on Bear for three big reasons: great markdown support, a beautiful editor, and a sophisticated tagging system that allows for the idea of linking posts together in a more fluid way. So far I’m happy with the decision and have set up a global keyboard shortcut to open a new Bear note to (hopefully) encourage me to more fluidly drop into writing-mode more frequently rather than just passively reading. I’ve also pro-actively created notes for the books I’m currently reading so that (again, extremely hopefully) I’ll have reduced the friction enough to find it easy to jot down notes as I’m reading.

Ulysses ($5.00 per month)

Ulysses seems like the most beautiful of the options, and the one dedicated most to supporting writers. If I wrote professionally for a living I think Ulysses might have been a stronger contender, but for my needs (focused more on note-taking than long-form writing) Ulysses doesn’t have much on Bear and is more than 3x the price.

Roam (??? $15 – $30 per month)

Roam research is definitely the most interesting of these options. Conor White-Sullivan, the founder of Roam very explicitly lists the note-taking applications and methods I mentioned above as inspiration for Roam. The UI is focused explicitly on building a web of connections between different ideas in ways that utilize bi-directional linking and some keyboard shortcuts that encourage the user to quickly and easily create new notes and concepts.

Honestly, I would have loved to give Roam a try but at that price point it’s just not worth it for me (looking at you too, Superhuman email). My hope right now is that if Roam is successful some of the other writing apps will steal the best ideas around the UI and linking and I’ll be able to take advantage of them in a more cost-effective editor.

Ask the people what they want

I’ve been reading James C. Scott’s Seeing Like a State and as Scott chronicles the myriad ways that governments totally screw things up for their constituents through (often) well-meaning schemes of modernization and improvement, I’ve been thinking a lot about how this concept applies to the work I do in software engineering and product management. (Note: if you want to read a more general review of the book, go check out Scott Alexander’s review from 2017.)

The fundamental thesis of Seeing Like a State is that governments, in order to be able to be governments, necessarily have to abstract from the truth on the ground — a census can capture certain data points about the population, but necessarily cannot reflect all of the wonderful human variety that makes us who we are and that we observe and rely on in ourselves and in our neighbors. Like in Borges’s story fragment On Exactitude in Science we know that a map is only useful to the extent that it abstracts (strategically) from the true detail of our world — a map that actually includes all of the detail that might be relevant to any living being will necessarily be as large as the territory itself!

Where this is a problem is when so-called “High Modernist” governments believe that they can effectively use their birds-eye (and necessarily abstracted) view of the world to impose “improvements” on their constituents. Scott points out that these improvement schemes consistently fail because the governments fail to recognize that the differences they have abstracted over are actually incredibly meaningful to the human beings actually operating on the ground. Scott gives lots of examples from agriculture where government bureaucrats insist on the use of certain farming methods that have been “scientifically proven” to work in the lab (or in another country).

Unfortunately, when put into place in a different context, the methods fail abysmally and, though well intentioned, cause great suffering among the populations they were supposed to help. In general, the farmers on the ground tend to know better than the bureaucrats in the capital what will or won’t work well in their particular farm — they actually tend to have lots of expert knowledge about their particular soil, how it holds or releases moisture in different seasons of the year and how well different crops may or may not perform (and they recognize that this might vary substantially within their own property and from their neighbor’s land). When bureaucrats impose rules from the capital, they elide over these “minute” details that are actually critically important to maximizing the productivity of the land.

As I’ve been reading these accounts, I keep thinking that what the government should do is not force the farmers to “modernize”, but rather simply … ask the farmers what they need. If the farmers need tractors, deliver tractors! If they need more consistent water for their crops, the government can help installing water pipelines and irrigation systems. If they need seeds, the government can deliver seeds. Where the government consistently fails in these schemes is in assuming that the farmers are ignorant and that they do not know what is best for themselves and their families.

While it’s easy to mock these schemes as clearly destined for failure, I often see the same thing happening in software! Especially for tools- or services-oriented internal teams (like the Analytics and Data Teams that I tend to work with) it’s very easy to assume the hubris of the misguided bureaucrats and believe that all your users need is your particular solution to fix their problem (that they might not even be aware of). When this happens, the best case scenario is that the software gets built and goes unused — in the worst case scenario the software builders break a solution that used to work and replace it with a less-flexible system that actually increases the work load of the users it was intended to help. Here’s how I’ve seen this in the real world:

  • The software engineer observers a workflow that seems cumbersome and error-prone. Often involving complicated spreadsheets.
  • The software engineer thinks to himself: I know, I can code this process up in R or Python, get the benefits of version control and it will save these excel jockeys a boatload of time
  • Software engineer starts coding. Realizes this excel spreadsheet is more complicate than they thought. Keeps coding, eventually gets to feature parity.
  • The software engineer attempts to role out their shiny new tool, only to learn that 1) the excel model they were attempting to re-create has already changed substantially since they started this project and 2) the users don’t want to use the new tool because they crucially rely on the flexibility of Excel to do their jobs

I’ve seen this happen a number of times (and have been guilty of it myself!) — while this happens very frequently with data scientists and analytics engineers who look down on excel workflows, I know it also happens in other realms of software engineering as well.

Where this process goes wrong is that the all-too-helpful software engineer failed to ask their users what they actually needed help with — they jumped from their assessment of the problem to a proposed solution that missed out on crucial details of how the users actually do their jobs and the features that they need to continue doing that job well.

What we as software engineers and product people must always remember is that you have to actually talk to your users. In the example above, the software engineer might have learned that the most pressing problem was actually a version-control and file-naming problem — in which case the software engineer could have focused on that problem in particular and saved the user (and the rest of the organization a lot of headaches) while also wasting less time (==money) on the project.

That user probably does recognize that their workflow isn’t ideal, but in order to figure out how to help them, you actually have to talk to them to understand their true pain points. As internal tool-builders, we are often at greater risk of building tools that our users don’t want or need because there are no market forces that drive our product adoption. With external user-facing features, our business will observe if users like them or not (or, if our business isn’t paying attention to these things, probably won’t be around much longer anyway). However, with internal tools the danger of being the bureaucrat imposing changes from a too-abstracted view of the wold is much more present, and we should remain vigilant that we aren’t building useless tools or, even worse, imposing our broken new tools on users just trying to get their jobs done.

Price and Value

Last night on a boozy zoom call with some friends I did a very poor job of trying to explain how I think about the difference between price and value (and why that difference is important). I’m going to try to use this space to clarify my thoughts on the subject in a way which will hopefully be informative to y’all as well.

I believe that the term “value” can apply to a few distinct concepts:

  • Usage value — this is something like the “utility” that I get from using  a product or a service
  • Financial value — this is the purely monetary value of an asset and generally refers to future expected income streams from owning an asset

The consumption of consumer goods often only have usage value — for example, buying a ticket to go see a movie has no financial value because there’s no future expected returns from that. Owning a financial instrument, like a stock or a bond, often only has financial value in that there’s no additional utility that comes from owning that asset. Something like owning a home has both — both usage value (if you live in that home) and some amount of financial value from a potential future sale of that home or from renting the home out to others.

The financial value of an asset can be tricky to reason about because there are a few different interlocking concepts:

  • Financial value
    • True future income stream — this is the “true” schedule of future returns from owning the asset. This is unknowable except in retrospect.
    • An individual’s fundamental valuation — this is the net present value of the assets future expected returns.
      • An individual’s belief’s about the future income stream of an asset (i.e., what’s going to happen in the future)
      • An individual’s net present value function which will vary depending upon the individual’s belief’s about the future and their risk tolerance (the value of money today vs money tomorrow)

I want to say that there exists a true future income stream of an asset that exists but that it is unknowable, and lots of different individual’s fundamental valuations. Individual’s valuations may differ because they have different beliefs about the future or because they have different discount rates.

One thing that complicates this picture is that to any given individual, there are two different ways that an owner can realize the value of an asset:

  • Receiving a stream of income into the future according to the assets true future income stream
  • Selling the asset to someone else at the prevailing price

The second point is what can lead to confusion about what the value of a financial asset is — since owners of an asset can always sell the asset at the prevailing price, it can be tempting to say that the price reflects the asset’s value.

When I’m talking about financial assets, I want to define price and value distinctly.

  • Price: in financial markets, price reflects the weighted average beliefs of all the individuals in the market about the financial value of the asset

So, price does not impact the financial value of an asset in terms of future income streams, but it can affect the value to an owner of the asset, because if the price is higher than the owner’s individual fundamental valuation, they can (and should!) sell the asset on the market and come out on top.

The point that I want to keep clear is that the price of an asset can move around a lot (e.g., in the stock market) without the underlying value of the asset shifting at all — sometimes they both move together, sometimes the value can change while the price stays the same. But they’re too distinct concepts and hopefully this clarifies how they’re distinct.

Too late

The rate at the which my world has changed over the past 3 weeks is really astounding. At the beginning of March, a mere 24 days ago, I was still planning on traveling from Mexico to go to a friend’s wedding (remember those?!) that would have been this week. Back then, this plague was still called the “corona virus” and I truly thought it wouldn’t hit America — it’d fizzle out in the east just like SARS and MERS had before.

Never before in my life have my beliefs and expectations about the future shifted so rapidly.

It’s very difficult for humans to reason about exponential growth — our brains just simply don’t work in a way that allows us to extrapolate exponential growth into the future using the normal mental tools we have. The way this has registered for me has not been in terms of the numbers — I can read the numbers off of the charts and project those into the future and say “10 thousand deaths” aloud just as well as anyone else — but rather in terms of the time scale. Going from 100 deaths to 10 thousand deaths only takes 3 weeks if your death rate is doubling every 3 days. That’s the difference between “what virus?” and bodies piled up in the halls of hospitals in 21 days.

We are just simply not built to process that amount of change that rapidly — when I talk to people here in Mexico who don’t seem that worried about the plague, I like to point out that Mexico right now has the same number of cases and deaths that New York did three weeks ago. Too late is coming. Too late is already here. Too late comes so fast that it’s difficult to comprehend even as I’m very closely watching it happen to my friends in New York.

The rate at which the world is changing makes it impossible to plan — all of my plans and goals for the year are out the window. I’ve experienced changes to my plans in the face of unexpected events, but in the past I’ve always been able to assess the situation, adjust my plans, and keep pushing forward. But right now that feels impossible — I can’t adjust because the ground is shifting under me in a way that doesn’t feel remotely forecastable. I really don’t know how to think about what the world will be like two weeks from now.

So for now I’m in a holding pattern. It’s too late to fix what’s broken and still too early to figure out what’s next.

An ode to memorization

In the last few years I have invested _a lot_ of time in pure, rote memorization. Over this period, I’ve changed my opinion 180 degrees from how I thought about memorization when I was in high school. Back then, I thought that any topic that was “pure memorization” wasn’t worth learning, and now I believe that memorization is a necessary condition for many types of learning, and we should re-orient the way we structure education to shift the emphasis to memorization.

First, some justification: memorization is required for fluency. It may be fluency in a language, for which you will need to memorize many thousands of vocabulary words. It may be fluency in mathematics, for which you will need to memorize many identities and relationships. In order to be fluent in a software program like Excel you’ll need to memorize many different formulas and how they work. If you want to be fluent in Python or R, you’ll need to do the same.

But what is fluency and why does it matter? In a language, it’s obvious — most people have a clear understanding of what it means to be fluent in a language and why that’s important for mastery. In mathematics, maybe less so.

When I was taking math in middle- and high-school, I remember absolutely refusing to memorize the identities and formulas we were supposed to. I’d, surely obnoxiously, ask the teachers why it was important to memorize these formulas when it was always possible to just go look them up? I really firmly believed that if I understood the concept behind the formula or the identity, then I didn’t need to memorize it. This is flat out wrong.

We can make a point clear with a simple example: the times tables — you probably have memorized that 6 x 7 = 42 (or at the very least you can get there very quickly via 36 + 6 or some other rapid shorthand that combines other smaller memorized quantities). Knowing this value cold, via pure rote memorization, is much more useful than simply understanding the concepts that underly the identity — simply knowing how multiplication relates to addition , and that 6 x 7 = 6 + 6 + 6 + 6 + 6 + 6 +6 = 7 + 7 +7 +7 + 7 +7  just doesn’t actually help you that much when you need to solve more abstracted problems.

If you were working on a complex math problem and every time you saw a multiplication symbol you had to go back to your addition rules to work out the value, this would slow you down considerably — you’d probably lose your train of thought on whatever larger procedure you were trying to work out, and you’d end up way too mired in the details — this betrays a lack of fluencyBy memorizing these identities, your brain does not get “distracted” by working out the computations and you are able to focus more on the higher-level, more abstracted quantities.

And this is why memorization is so important! It allows you to gain fluency, which allows your brain to focus on other higher-level more complex problems. In mathematics, if you don’t have important identities memorized (like 6×7=42 or that sin^2 + cos^2 = 1 or whatever) then when you encounter these relationships in the wild, you will end up losing the thread of whatever problem you’re actually trying to solve while you work out the details of these values from first-principles.

Mastery of a subject can only be acquired through great fluency — you have to allow your brain to work with higher-order abstract concepts and not get bogged down in the details, and that means being able to use higher-level abstractions (which have been memorized) in order to solve much more complex problems. If you never do the memorization (i.e., you don’t have the higher-level abstractions to work with) then it will be much more challenging to solve these complex problems and at some point you won’t be able to solve them at all.

When I’m encountering a new topic that I want to gain mastery of, I actually invest in memorization even before I invest in understanding — knowing the identities and being able to use them is more important than understanding where they come from or why they work the way they do. I know that I can always come back to learn about the why, but I’ve found that going to memorization first tends to help me gain a better working knowledge of whatever topic I’m interested in which in turn helps me understand the system as a whole.

Dating App Strategy

I, unfortunately enough, have spent a lot of time on dating apps. I was an okcupid user back in the early 2010s and have since then (excluding a few happy years coupled) been a longtime Bumble/Tinder/Hinge user. Over that time I’ve seen lots change about the dating apps — from Tinder’s incredible swipe interface innovation that turned finding a lifelong mate to cherish into a sort of erotic candy crush you can play on the subway, to Bumble’s important girls-go-first innovation that has shifted how (some) people think about early-relationship dynamics in interesting ways (for what it’s worth, women in Mexico seem to find it a lot more unusual / challenging than women in NYC did).

Having seen now, conservatively, many thousands of dating profiles I’ve developed some (apparently) non-standard beliefs about the best way to craft a dating profile to optimize for good matches. I’ll share those here in the hopes that they may be of use to all of those other poor souls swiping through the hellscape of dating apps here in 2020. Obviously, given my personal and conspicuous lack of success, take all of this advice with at least a few large granules of salt.

First, we should align on what the goal of a dating app is — for me, it’s to find people with whom I enjoy spending a lot of time with. Ideally at least one of those people will turn into a long-term partner with whom I can build a deep relationship with, but I’m actually happy to meet people on the apps whom I could plausibly become friends with or at least wouldn’t regret spending a few hours with on a first date. That is, I want to effectively separate the people I’m most likely to get along well with from the mass of folks with whom I mostly likely won’t.

If your goal is something different, maybe maximizing the number of dates you go on, or identifying high-probability one-night-stands, then you can disregard all of the rest of the advice in this blog post (or maybe consider actually inverting the strategies described here).

So, with that goal in mind, I want to craft a profile that:

  • Attracts potential partners whom I’m most likely to get along well with
  • Repels potential partners whom I’m least likely to get along well with

I think of this as  a signaling problem where I want to use my profile to signal most effectively to my type of people that I’m their type, and signal to the rest that I’m not. I want to use the space in my profile to send very strong signals to both of those parties — ideally each piece of information in the profile sends the correct signal to both of those two groups.

Let’s consider the single most common thing I see in dating profiles:

“I love to travel”

Because it’s so common, it therefore immediately does a terrible job of separating high-probability partners from low-probability partners. Everyone loves to travel! This sentence tells you nothing about the person writing it and sends no signal to anyone. Because one can easily imagine two people who both love to travel not only not getting along, but also not enjoying the same travel experience, we can improve it by adding more specificity. Here are three examples:

  • I love local culture, so when traveling you can always find me sampling the street food, the spicier the better (no matter the gastrointestinal consequences).
  • I’m a shoestring traveler — I love backpacking and meeting new people in hostels. For me, travel is all about late nights with new friends.
  • I’m all about treating myself — my ideal vacation involves a whole lot of spa time and incredible meals with people I love

All three of these people have a valid love of travel, but one can easily imagine them not enjoying the same trip at all. By adding a lot more specificity, we can attract the sort of people who have similar tastes and highlight our differences to the rest.

Here’s another example of a thing that I enjoy that would do a good job of sending clear signals to the relevant groups:

“I love to go to art museums alone and crank Radiohead through my headphones”

This is both true and highlights some unique things about me that will interest people who I think are good potential partners and turn off bad potential partners. If you’re not into art museums and you’re annoyed by people who like Radiohead, this profile is not going to appeal to you! However, if you’re the sort of person who finds this intriguing / endearing you should email me.

If your goal is to identify the most high-quality matches from the rest of the world, you should try to pack your profile with points like these — you can use your photos to make these points as well, so be strategic! If we all get better at doing this sort of signaling, we’ll all be able to find each other more quickly.

Happy hunting.

How I Use Anki

My number one best tool for learning Spanish has been the spaced repetition flashcard app Anki. I’ve written about Anki as a key component of my general approach to learning Spanish, and have talked about how important memorizing massive amounts of vocabulary is to any endeavor, but today I’m going to write in a bit more detail about the tactics I’ve developed for efficient Anki usage.

Anki is a very powerful software program, but the learning curve is very steep, so I’m not at all surprised that more people haven’t been able to get into using it effectively. Hopefully some of these tips will help you get started and make your study sessions as productive as possible.

  • Study your flashcards every day — it’s critical that you not skip days or weeks or you’ll end up so far behind that it’ll feel impossible to catch back up
  • Target an amount of study time that you can do every day! Adjust your review schedule so that you’re consistently studying the same amount every day (I target between 20 and 30 minutes, but if you can only do 10 that’s fine — just make sure you aren’t falling behind on your review cards)
  • Use decks to group together similar cards — I keep my Spanish “dichos” (sayings) separate from my Spanish vocabulary, which is obviously separate from my deck of mathematics identities.
  • You can even nest decks — so I have one top-level deck for “vocabulary” and then sub-decks like “vocabulary from Pedro Páramo” and “vocabulary from Las Muertas” and “vocabulary from news articles”
  • Get to know the most important options for an Anki deck. I generally only change:
    • New cards per day — how many new cards are you shown each day?
    • Interval modifier — how much faster (or slower) should cards re-appear relative to the standard Anki algorithm? I put relatively lower numbers for more challenging decks and relatively higher numbers for easier decks
  • Use the “custom review” to get extra review time for cards that you’ve forgotten
    • Every three days, I do a “custom review” and review all the cards that I’ve forgotten in the last three days — this helps me get enough exposure to the cards that I’m struggling with to actually get them implanted in my memory
    • Anki really isn’t designed for learning something from scratch, but when it comes to learning vocabulary, I’m often seeing words for only the second or third time when they show up in my Anki review session (the first time when I read them originally, the second time when I looked them up to create the Anki card, and the third time in a review session), so this is a bit of a “hack” to help with learning in addition to just maintaining long-term memory
  • Once a month, review cards that have been suspended
    • Anki keeps track of how many times you fail a card that is in “review” mode — by default, once that counter hits 8, Anki will mark that card as a “leech” and suspend it (i.e., will stop showing the card)
    • In general, if a card is suspended or marked as a leech, it means you’re going to need to put in additional effort to get it to stick in your brain. Once a month I review all of the cards that have been suspended and dedicate myself to extra study for those cards (often creating a host of additional cards that include the word or phrase in order to give myself enough context to get it to stick in my brain)

Hopefully these tricks will help you get started with Anki — to be honest, at this point, it’s pretty difficult for me to imagine learning a language without Anki. It’s so crucial to the way that I study and the progress that I’ve made that I hate to imagine trying to learn a language without it.

If you have the patience to learn the tool, and the dedication to commit to studying every day, Anki is an incredibly powerful study aide. Let me know if you have other tips or suggestions for using Anki for studying!

Saying Thank You

One habit that I’ve built up over the years that I count as one of my super powers is training myself to say “thank you” reflexively anytime anyone criticizes or corrects me. If I’m cooking something in the kitchen and someone says “you’re doing that wrong”, or if I’m showing someone some code I wrote at work and they say “that’s the wrong way to do that”, or if I’m speaking in Spanish and someone corrects my grammar / pronunciation / word choice, the immediate first response is always the same: “Thank you.”.

And when I say “thank you”, I actually mean it! It’s a sincere “thank you”, not a sarcastic one. (Those who know me that I loathe sarcasm generally, and always advocate for more earnestness, directness, and honesty in communication with others.)

Now, saying “thank you” to criticism or correction is of course easiest when the criticism is delivered in a kind and supportive way — the better the person is at giving criticism or feedback, the easier it is to say “thank you.” in response. However, for me at least, it’s important to say “thank you” even when the criticism isn’t delivered in the nicest way. In fact, it’s even more important to use this response when the criticism isn’t delivered optimally — when it pisses me off and when the first response that I want to give is “hey if I wanted your help I’d ask for it.”

The reflexive thank-you response does a few things:

Most importantly, it encourages people to give me more feedback — this is super important to me in pretty much all aspects of my life. I want people to tell me when I could be doing something better. I might not always agree with them, and therefore I might not actually change the way I do something, but I always want to know if someone has an idea for me. Reflexively responding with “thank you” communicates to the other party that their criticism is welcome and that I’m listening. This is especially key for me with Spanish — reflexively saying “gracias” when someone corrects my Spanish helps encourage more corrections (or, as I like to think about them, free Spanish lessons).

In cases where the tone or delivery of the criticism isn’t optimal, reflexively saying “thank you” immediately pushes the tone of the conversation towards being collaborative instead of adversarial — this is crucial in situations where it might not be obvious (or even be the case) that the other party has your best interests at heart. By declaring up front that I’m happy to receive the criticism and that I’m interested in learning from it, most of the tension or negative emotion in the conversation is immediately alleviated.

Finally, and again important in situations where the criticism isn’t especially well delivered, saying “thank you” buys me time to think more deeply about how I want to respond. It might be that I say “thank you”, pause for a beat, and respond with “but I still think I’m right for reasons x, y, and z”. The use of “thank you” both resets the tone (as described above) and gives me time to think more clearly and reasonably about my response instead of responding from a place of pure emotion / anger / frustration.