Austin, TX, 8:15. Morning walk around the neighborhood. There are many good Halloween decorations, but this one is particularly entertaining. #adayinthelife

Given what happened to the other saguaro, it makes me fear that this one will fall too.

I took this photo of a saguaro a month ago. It had been partially burnt during the fires that happened north of Phoenix earlier this summer. This weekend I realized it had fallen. According to Wikipedia, saguaros grow their first arm around 75–100 years of age. 👋🌵😔

Custom URL Handler for Files With Unique IDs

Yesterday I read a series of posts on custom URL scheme handlers on the Zettelkasten forums. The handler registers itself to open links like zettel://202006061337, where the number is the unique ID of a zettel (here it’s a timestamp). I’m not sure everyone realized the magnitude of what that means.

Combining a custom URL with a unique ID means notes and links can become entirely independent from your apps. Only the handler needs to know about the apps you’re using.

That alone is very nice, but then I thought: URLs can have query parameters… That means I can have URLs like zettel://202006061337&edit that open in my text editor of choice: TextMate, BBEdit, WriteRoom, FoldingText, etc. Or zettel://202006061337&preview to open in Marked. Or I could even pick the app interactively with zettel://202006061337&pick.

Handling the ID alone is pretty easy since the common Zettelkasten-like apps respond to a URL scheme to search and open files. The Archive uses thearchive://match/ID, nvUltra uses x-nvultra://find/ID, and nvAlt uses nvalt://find/ID.

But how to open in a text editor or in Marked given only the ID? With Spotlight. I used mdfind -name ID to find the file. This could be further refined with the -onlyin FOLDER option but I didn’t need it. Then it’s a matter of calling open -a Marked FILEPATH.

There’s also a zettel://create special case that will create a new zettel with the current time stamp (YYYYMMDDHHMM). It’s always done with the default Zettelkasten app because the script doesn’t know where to write the file but the app does.

I wrote the handler in Applecript because it’s the easiest way I know to create something that macOS considers an “app” and that can therefore handle URLs.

The full script is below. To use it:

  1. Open Script Editor and paste the code below in a new file.
  2. [Optional] Modify values in the Configuration section to pick a different URL prefix, default Zettelkasten app URL, editor, and previewer. You can add as many apps as you’d like in the appChoices array.
  3. Save as “Application”. You can save it anywhere. Make sure none of the boxes are checked.
  4. Register the app as a URL handler. You can do it with the SwiftDefaultApps Preference pane, or using the instructions provided by Christian Tietze in the forums:
    • Locate the application file you just created
    • Right-click the app, select “Show Package Contents”
    • Inside, open Contents/Info.plist with a text editor
    • Paste the following bit of XML on a blank line right below the <dict> line. Replace zettel with the URL prefix you’ve chosen.
    • Launch the app to register the URL handler.
<key>CFBundleURLTypes</key>
<array>
    <dict>
        <key>CFBundleURLName</key>
        <string>Zettel Link Opener</string>
        <key>CFBundleURLSchemes</key>
        <array>
            <string>zettel</string>
        </array>
    </dict>
</array>

Here’s the full script:

-- Zettel Link Opener
-- Created by Alexandre Chabot-Leclerc
-- Source: https://alexchabot.net/2020/06/06/custom-url-handler-for-zettels/
-- URL Handler for zettelkasten unique IDs, e.g., zettel://202006061017
-- Handles options after the ID to open different apps:
--    zettel://202006061017&edit to open is a text editor like TextMate
--    zettel://202006061017&preview to open in a preview app like Marked
--    zettel://202006061017&pick to open a menu of apps to pick from

-----------------------------------------------------------
-- CONFIGURATION
-- URL prefix for your custom URL, e.g. zettel://ZETTEL_ID
property urlPrefix : "zettel"

-- Default URL to call to open a note with a given ID. The ID will be appended
property defaultZkAppUrl : "thearchive://match/"
--property defaultZkAppUrl : "nvalt://find/"
--property defaultZkAppUrl : "x-nvultra://find/"

-- URL to use to create a new zettle with the current timestamp YYYYMMDDHHMM
property urlForCreation : "thearchive://matchOrCreate/"
--property urlForCreation : "nvalt://make?txt="
--property urlForCreation : "x-nvultra://make?txt="

-- Apps to use for the different query parameters
property editApp : "FoldingText" -- App to used with "&edit" query parameter
property previewApp : "Marked" -- App to used with "&preview" query parameter 

-- List of app to display in the menu with with &pick query option
-- The apps will appear in the order defined here
property appChoices : {defaultZkAppUrl, editApp, previewApp, "TextMate"}
property defaultApp : {defaultZkAppUrl}
-----------------------------------------------------------

on splitText(theText, theDelimiter)
	set AppleScript's text item delimiters to theDelimiter
	set theTextItems to every text item of theText
	set AppleScript's text item delimiters to ""
	return theTextItems
end splitText

on removeUrlPrefix(original)
	-- Remove URL prefix so we're left with only the ID and the optional query parameter
	return do shell script "echo " & quoted form of original & " | sed 's;" & urlPrefix & "://;;'"
end removeUrlPrefix

on getIdAndOption(resouceAndQuery)
	-- Split the zettel ID and the optional parameter
	-- For example 202006061012&edit or 202006061012&preview
	set theItems to splitText(resouceAndQuery, "&")
	if length of theItems is 1 then
		-- Append an empty string if there's no option so this
		-- function always returns an array of 2 elements
		copy "" to the end of theItems
	end if
	return theItems
end getIdAndOption

on findFilepath(zk_id)
	-- Finds the filepath using Spotlight.
	-- It's easier than finding the proper filename given only the zettel ID
	return do shell script "mdfind -name " & zk_id
end findFilepath

on createZettel()
	set newZkId to do shell script "date +'%Y%m%d%H%M'"
	do shell script "open " & urlForCreation & newZkId
end createZettel

on openInChoosenApp(zkId, zkFilepath)
	--	From Simple List Handler by Patrick Welker <http://rocketink.net>
	-- Promp the use for the app to use
	set selectedApp to item 1 of (choose from list the appChoices with title "Available App" with prompt "Which app do you want to use?" default items defaultApp)
	if selectedApp is false then
		-- Exit prematurly if the user clicked Cancel
		error number -128
	end if
	
	-- Open the URL directly, or open by app name
	if selectedApp contains "://" then
		do shell script "open " & selectedApp & zkId
	else
		do shell script "open -a " & selectedApp & " " & quoted form of zkFilepath
	end if
end openInChoosenApp

on open location thisURL
	set resouceAndQuery to removeUrlPrefix(thisURL)
	set idAndOption to getIdAndOption(resouceAndQuery)
	set zkId to item 1 of idAndOption
	
	if zkId is "create" then
		createZettel()
		return
	end if
	
	set zkFilepath to findFilepath(zkId)
	
	if item 2 of idAndOption is "edit" then
		do shell script "open -a " & editApp & " " & quoted form of zkFilepath
		-- Exit the script immediately so we don't also open in the default app
		return
	else if item 2 of idAndOption is "preview" then
		do shell script "open -a " & previewApp & " " & quoted form of zkFilepath
		-- Exit the script immediately so we don't also open in the default app(
		return
	else if item 2 of idAndOption is "pick" then
		openInChoosenApp(zkId, zkFilepath)
		return
	end if
	
	-- Fall back to the default handler if there was no option
	-- or the option was invalid
	do shell script "open " & defaultZkAppUrl & zkId
end open location

What does %matplotlib do in IPython?

TLDR; Use %matplotlib if you want interactive plotting with matplotlib. If you’re only interested in the GUI’s event loop, %gui <backend> is sufficient.

I never really understood the difference between %gui and %matplotlib in IPython. One of my colleagues at Enthought once told me that at some point in his career, he more or less stopped reading documentation and instead went straight to the code. That’s what I did here. But let’s do a bit of history first.

In the “beginning”, there was pylab. It (still) is a module of matplotlib and was a flag to IPython designed to facilitate the adoption of Python as a numerical computing language by providing a MATLAB-like syntax.1 The reference was so explicit that before being renamed to pylab on Dec 9, 2004, the module was called matplotlib.matlab. IPython adopted the rename on the same day.2 With the ‑‑pylab flag or the %pylab magic function, IPython would set up matplotlib for interactive plotting and executed a number of imports from IPython, NumPy and matplotlib. Even thought it helped a few people transition to Python (including myself), it turned out to be a pretty bad idea from a usability point of view. Matthias Bussonnier wrote up a good list of the many things that are wrong with it in “No Pylab Thanks.”

For the 1.0.0 release of IPython in August 2013, all mentions of %pylab were removed from the examples (in a July 18, 2013 commit) and were replaced by calls to the %matplotlib magic function, which only enables interactive plotting but does not perform any imports. The %matplotlib function had already been introduced in a 2013 refactoring to separate the interatice plotting from the imports. The %gui magic command had already been introduced in 2009 by Brian Granger to “manage the events loops” (hint hint).

Now we know that the (my) confusion with %gui and %matplotlib started in 2013.

This analysis refers to IPython 7.8.0 and ipykernel 5.1.2.

Our entry point will be the %matplotlib magic command. Its source code is in the IPython.core.pylab.py file. The essential call is to shell.enable_matplotlib(gui), which is itself implemented in IPython.core.interactiveshell.InteractiveShell, and does five things:

  1. Select the “backend” given the choice of GUI event loop. This is done by calling IPython.core.pylabtools.find_gui_and_backend(gui). It encapsulates the logic to go from a GUI name, like "qt5" or "tk", to a backend name, like "Qt5Agg" and "TkAgg".
  2. Activate matplotlib for interactive use by calling IPython.core.pylabtools.activate_matplotlib(backend), which:
    1. Activates the interactive mode with matplotlib.interactive(True);
    2. Switches to the new backend with matplotlib.pyplot.switch_backend(backend);
    3. Replaces the matplotlib.pyplot.draw_if_interactive method with the same method, but wrapped by a flag_calls decorator, which adds a called flag to the method. That flag will be used by the new %run runner that’s introduced below at point #5;
  3. Configure inline figure support by calling IPython.core.pylabtools.configure_inline_support(shell, backend). This is where some very interesting stuff happens. It first checks that InlineBackend is actually importable from ipykernel.pylab.backend_inline, otherwise it returns immediately. But if it’s importable and the backend is "inline", it:
    1. Imports the ipykernel.pylab.backend_inline.flush_figures function, and register it as a callback for the "post_execute" event of the shell. As we’ll see later, callbacks for "post_execute" are called after executing every cell;
    2. If the backend was not "inline", it’ll unregister the flush_figures callback;
  4. Enable the GUI by calling shell.enable_gui(gui). This method is not implemented in the IPython.core.interactiveshell.InteractiveShell base class, but rather in IPython.terminal.interactiveshell.TerminalInteractiveShell. If a gui as specified, it gets the name of the active_eventloop and its corresponding inputhook function using IPython.terminal.pt_intputhooks.get_inputhook_name_and_func(gui). The active_eventloop is just a string, such as 'qt', but the inputhook is more interesting. It’s the function to call to start that GUI toolkit’s event loop. Let’s dig further into get_inputhook_name_and_func(gui). That function checks a few things, but it essentially:
    1. Imports the correct inputhook function for the chosen GUI by importing it from IPython.terminal.pt_intputhooks.<gui_mod>. For example, the Qt inputhook is imported from IPython.terminal.pt_intputhooks.qt. Later on, when inputhook is executed for Qt, it will:
      1. Create a QCoreApplication;
      2. Create a QEventLoop for that application;
      3. Execute the event loop and register the right events to make sure the loop is shut down properly. The exact operations to start and stop the loop are slightly different for other GUI toolkits, like tk, wx, or osx, but they all essentially do the same thing. At this point we’re ready to go back up the stack to enable_matplotlib in %matplotlib;
  5. Replace IPython’s default_runner with the one defined in IPython.core.pylabtools.mpl_runner. The default_runner is the function that executes code when using the %run magic. The mpl_runner:
    1. Saves the matplotlib.interactive state, and disables it;
    2. Executes the file;
    3. Restores the interactive state;
    4. Makes the rendering call, if the user asked for it, by checking the plt.draw_if_interactive.called flag that was introduced at point #1.3 above.

As for the other magic, %gui, it only executes a subset of what %matplotlib does. It only calls shell.enable_gui(gui), which is point #4 above. This means that if your application requires interaction with a GUI’s event loop, but doesn’t require matplotlib, then it’s sufficient to use %gui. For example, if you’re writing applications using TraitsUI or PyQt.

The Effect of Calling %gui and %matplotlib

Let’s start with the “simplest” one, %gui. If you execute it in a fresh IPython session, it’ll only start the event loop. On macOS, the obvious effect of this is to start the Rocket icon.

Animation of the Python rocket icon starting because of a call to `%gui`.

At that point, if you import matplotlib and call plt.plot(), no figure will appear unless you either call plt.show() afterwards, or manually enable interactive mode with plt.interactive(True).

On the other hand, if you start your session by calling %matplotlib, it’ll start the Rocket and activate matplotlib’s interactive mode. This way, if you call plt.plot(), your figure will show up immediately and your session will not be blocked.

Using %run

If you call %run my_script.py after calling %matplotlib, my_script.py will be executed with the mpl_runner introduced above at point #5.

Executing a Jupyter Notebok Cell When Using the "inline" Backend

In the terminal the IPython.terminal.interactiveshell.TerminalInteractiveShell.interact() method is where all the fun stuff happens. It prompts you for code, checks if you want to exit, and then executes the cell with InteractiveShell.run_cell(code) and then trigger the "post_execute" event for which we’ve registered the ipykernel.pylab.backend_inline.flush_figures callback. As you might have noticed, the flush_figures function comes from ipykernel, and not from IPython. It tries to return all the figures produced by the cell as PNG of SVG, displays them on screen using IPython’s display function, and then closes all the figures, so matplotlib doesn’t end up littered will all the figures we’ve ever plotted.

Conclusion

To sum it up, use %matplotlib if you want interactive plotting with matplotlib. If you’re only interested in the GUI’s event loop, %gui <backend> is sufficient._ Although as far as I understand, there’s nothing very wrong with using %matplotlib all the time.

  1. Basically, no namespaces, and direct access to functions like plot, figure, subplot, etc. [return]
  2. The earliest commit I found for the IPyhon project was on July 6, 2005 by Fernando Perez, 7 months after the name change. Its Git hash is 6f629fcc23ba63342548f61cc7307eeef4f55799. But the earliest mention is an August 2004 entry in the ChangeLog: “ipythonrc-pylab: Add matplotlib support,” which is before the offical rename in matplotlib. [return]

2019 Holiday Newsletter

Every year since 2017 I send my colleague a long email about ideas and media I’ve come across during that year. This is the 2019 edition. You can read the 2017 and [2018]() editions.

Hi all,

Time for installment #3 of my Holiday Newsletter! For once, I started a bit earlier collecting links and summarizing why things are interesting. Let me know if you’d be interested in receiving something like this more than once a year. It’s something I’m considering doing in 2020.

I hope you find some time to read/listen/watch something good over the break, even if it’s not from this list. :) Happy Holidays!

-Alex

If You Read Just One Thing

The End of Bureaucracy by Gary Hamel and Michele Zanini at HBR is the most mind-blowing thing I’ve read this year. It features a Chinese company named Haier, which is the worlds largest appliance maker with $35 billon/year in revenue. The business is structured around 4,000 independent microentreprises (MEs) made of 10 to 15 employees. Each ME provides different of services to other MEs: marketing, R&D, design, manufacturing, HR, etc. Each ME is free to contract with any other ME, or go to an external provider if they don’t think they’d be well served internally. It’s a setup that flies in the face of pretty much any other large company. Definitely worth a read.

Good Reads

Dan Barber (chef and co-owner of Blue Hill at Stone Barns) writes Save Our Food. Free the Seed. at the NYT. It’s an inspiring, and also infuriating, article about seeds and how they’re at the root of our entire food supply chain. I didn’t realize that seed “designers” make their seeds work hand in hand with insecticides, which are often made by the same companies (e.g., Monsanto). That wouldn’t matter too much if we weren’t down to just two mega seed producers, and if the seeds weren’t patented.

I still haven’t read any of Michael Lewis’ books (somehow), but every time I read one of his articles I’m completely sucked in. Here are two good ones I read this year. In Jonathan Lebed’s Extracurricular Activities, Lewis tells the story of a 15-year old “stock manipulator.” In Portrait of an Inessential Government Worker talks about Art Allen, the only oceanographer inside the U.S. Coast Guard’s Search and Rescue division, and the person who’s basically single-handedly responsible for mankind’s ability to find things lost a sea. Lewis also started a podcast this year called Against the Grain. Recommended.

In Stab a Book, the Book Won’t Die Craig Mod (real name) speaks of different forms of content and their respective “contracts” with their consumers. Books have edges, a clear “I buy you, I read you or I don’t” contract. Online publications, like Netflix, Twitter, and Instagram don’t have such clear contracts. They’re infinity pools. I really liked his observation that with print newspapers, only the front page (often just above the fold) needs sensational headlines to get people to buy it. Internal headlines can afford to be factual. Online, all headlines are on the front page.

Cormac McCarthy, in his “free time”, does editing for faculty and post docs at the Santa Fe Institute. Van Savage and Pamela Yeh publish McCarthy’s advice on writing scientific papers at Nature. It’s equally good advice for writing documentations and reports.

The Japanese addressing system is strange and wonderful. When written in Japanese, the addresses start with the largest geographical entity and proceed to the most specific one.

Someone who’s name I can’t figure out writes Becoming a magician, with an interesting approach to “personal growth”: surround yourself with people who feel to you like they’re magicians.

Technology & Software

Julia Evans’ What does debugging a program look like? is a great read about the multiple ways to approach debugging. Julia started writing zines full-time this year. If you’re interested in having a look, I have one on my desk. And on the topic of debugging, Greg Wilson recommends Why Programs Fail: A Guide to Systematic Debugging by Andreas Zeller.

In Open Source is not about you, Rich Hickey, creator of Clojure, writes a wonderful article about expectations one should have about open source software (I still find strange people who “blog” using Github repos and Gists…)

As a user of something open source you are not thereby entitled to anything at all. You are not entitled to contribute. You are not entitled to features. You are not entitled to the attention of others. You are not entitled to having value attached to your complaints. You are not entitled to this explanation.

If you have expectations (of others) that aren’t being met, those expectations are your own responsibility. You are responsible for your own needs. If you want things, make them.

And on a related topic, License Zero has a thoughtful look at funding models for open source.

I award “Data Structure of the Year” to Conflict-free Replicated Data Types (CRDTs). I learned about them through Ink and Switch’s article Local-first software: You own your data, in spite of the cloud, where they use them to write a real-time syncing engine that doesn’t require a central server. But then, in pure Baader–Meinhof, it seemed like everyone was talking about them, e.g., the Xi editor, and Figma.

Basecamp published Shape Up, the “manual” for their software development process. It’s a 1-hour read that I highly recommend. It’s a variation on Agile, with thoughtful additions, structures, and rationales for why the do things in certain ways.

Lowtech Magazine hosts their website on a solar-powered Raspberry Pi. Warning, their site might be down when you visit if it’s been cloudy for a while in Barcelona. Roel Roscam Abbing wrote about the technical details.

Jakob Nielsen explains Why You Only Need to Test with 5 Users.

Here are some people I discovered this year. They are so inspiring, and also so frustrating, because they make you wonder “And what have I done today?”

Podcasts

My “Podcast of the Year” is the The Amelia Project, a fiction podcast about Amelia, an organization that helps clients fake their death and then reappear as whatever they want. It’s original, beautifully produced, endearing, and funny. Do start with the 1st episode, and go get yourself a hot cocoa. And if you’re new to fiction podcasts, check out Limetown, and Passenger List.

Michael Lopp (author of Managing Humans, which was in last year’s email) has a podcast called The Important Thing. If you’re interested in management and leadership, I recommend the The One About Management (Pt. 1) and The One About Management (Pt. 2) for a good discussion of what it means and what it’s like to be a manager.

The Nevermore, Amazon episode of Rework (by Basecamp), talks about about the economics of bookstores, and how selling new hardcovers for $15 (like Amazon does) would put them out of business. Independent bookstores are the best. If you prefer the convenience of shopping online but want to support independent bookstores, use Biblio. Cool fact: they offset the carbon emissions of their shipping.

Food, Drinks & Restaurants

For the Austinites, and people visiting Austin, go eat at:

  • Nixta Taqueria: yes, they’re hipster tacos, but they’re also delicious.
  • Launderette is still our favorite restaurant in Austin. Laura Sawicki is the best pastry chef in town.
  • Get ice cream at Manoli’s. Just do it.
  • Everything I drank from Vista Brewing this year was delicious.
  • Get some pastries at Le Politique’s Patisserie. I recommend scheduling your afternoon coffee break for the 3–4PM window, when pastries are half price.
  • I finally went to a Home Slice and it’s really that good.
  • Bombay Express makes delicious india food. Get the pani puri, and the chole puri, and the carrot halwa, and a dosa, and… get all the things. It’s worth going on the weekend just for the Thali.

Make bread at home. It’s easy and it’s better than anything you can get in Austin (except maybe from Sour Duck, and from the late Miche Bread). Just buy Flour Water Salt Yeast by Ken Forkish, and make the Saturday Overnight White bread. Bring the extra loaf to the office, please.

Music

It was a good year. It’s always a good year. Here’s a playlist of every song I “liked” in 2019, and one of every album. The music on those playlists wasn’t necessarily released in 2019 though. Just to name a few albums that were release in 2019:

  • Charlie Hunter and Lucy Woodward - Music!Music!Music! (Spotify)
  • MADMADMAD - Proper Music (Spotify)
  • Dan Tepfer - Natural Machines (Spotify)
  • Rhye - Spirit (Spotify)
  • voljum - Cyberglove (Spotify)
  • Bibio - Ribbons (Spotify)
  • Archive - 25 (Spotify)
  • Lite - Multiple (Spotify)
  • Canine - Dune (Deluxe) (Spotify)
  • Lambert - True (Spotify)
  • Himura Yoshiteru - View from Bottom (Spotify)
  • Janus Rasmussen - Vin (Spotify)
  • The Comet Is Coming - Trust in the Lifeforce of the Deep Mystery (Spotify)
  • Battles - Juice B Crypts (Spotify)
  • Anna Meredith - FIBS (Spotify)
  • Trentemøller - Obverse (Spotify)
  • And So I Watch You from Afar - ASIWYFA Live 10 Year Anniversary (Spotify)
  • Eprom - AIKON (Spotify)
  • Floating Points - Crush (Spotify)
  • Bon Iver - i,i (Spotify)
  • Jacob Collier - Djesse Vol.2 (Spotify)
  • And many more…

Movies, TVs, and Videos

  • Woman at War tells the story of a choir conductor and eco-activist who plans to disrupt the operations of an aluminum plant in Iceland, but whose plans are temporarily interrupted by the opportunity to finally adopt a child from Ukraine. It’s funny, beautiful, and strange.

  • The Marvelous Mrs. Maisel and Killing Eve were the two best series we watched this year. One great quality of Killing Eve is that it ends, in only two seasons.

  • Simple Made Easy by Rich Hickey. Simple is the not the same as easy.

  • In his 2019 State of Mozilla talk, Mike Hoye makes this wonderful point about caring about making it possible to care. Paraphrasing from his talk: There’s no Keurig machine in the Toronto office because there’s no amount of care that will make it produce good coffee. I like that a lot (coffee, but also this statement).

2018 Holiday Newsletter

This is the 2nd installment of an email I sent to my Enthought colleagues just before the Christmas holidays. The other installment is 2017.

Hi everyone,

This is installment #2 of Alex’s annual Holidays newsletter. I decided to send it company-wide this year. It’s a collection of interesting reads/listens/watchs I “consumed” this year.

Programming & Technology

Getting better at it:

I found out what people meant by level up about three and a half years into my programming career when I played my first game of Dungeons and Dragons. […] It sounds like most folks picture leveling up as a little avatar of themselves advancing up a ladder of 20 levels, moving up to the next rung incrementally as they gain experience. This creates this visual of our skills improving linearly with our time spent in tech. I’ll henceforth call this the ladder interpretation.

I pictured, instead, what I will call the derivative interpretation. It comprises a series of maybe three levels.

  • Level One: Getting better, adding skills
  • Level Two: Improving at getting better/adding skills
  • Level Three: Getting better techniques for improving at getting better/adding skills

I think if there’s one technology we should be “concerned” or excited about, as Python users, it’s Javascript (there are many reasons to be afraid, I know). This year, Michael Droettboom compiled NumPy to WebAssembly. The project is called pyodide

Management

As a newly minted manager, I read quite a few management books this year. Here are my favorites. I highly recommend reading Managing Humans, even if you’re not a manager.

  • First Time Manager: A high-level overview of what the new responsibilities are.
  • Managing Humans by Michael Lopp aka Rands. Now at Slack, previously at Palantir and Apple, he is an amazing storyteller. The story form also makes the lessons stick better.
  • High Output Management by Andy Grove, Intel CEO. Ian Tien wrote a good summary. It’s often quoted as the source of much of modern management in tech companies.
  • The Five Dysfunctions of a Team by Patrick Lencioni. It’s a fable about management and leading a team. It’s a bit corny, and yet it was a page-turner.
  • It’s Your Ship: Management Techniques from the Best Damn Ship in the Navy by D. Michael Abrashoff. Don’t let the title (and the cover) fool you, it’s great. Not all lessons apply to the Enthought, but it’s amazing what he managed to do in the Navy.

Reads

Podcasts

You know, there has to be a section about podcasts!

  • This American Life - NUMMI: It’s from a few years back but it’s relevant to us, to what Enthought is doing with JSR/TEL/AK/Exxon. It’s the story of the NUMMI plant, a joint venture between GM and Toyota, of the transformative experience of the immersion process, and about what do to with the people who did not go through the immersion process.
  • New York Times - The Caliphate a fascinating story by Brokmini, who’s been following ISIS for years. It should be more widely known. It’s leaps and bounds better than Serial.
  • 99% Invisible - 330 - Raccoon Resistance on the design of Toronto’s raccoons-resistant compost bins. 99pi at its best. Funny and informative.
  • The Dollop - 342 - The John Paul Getty’s: a hysterical account of the Getty family (yes, the museum Getty’s).
  • The knowledge project - The Kids are Worth It with Barbara Coloroso. I’m not a parent, but if I were I’d probably re-listen to this once a week until my kids are 18.
  • Limetown, season 2, and 1! The best “radio-drama” I’ve listened to. Season 2 started on Halloween this year and it’s now over. You can binge listen to it while walking on dark Albuquerque streets at night and freak out!
  • Reply All - 131 - Surefire Investigations, a Yes-Yes-No that showcases Gritty, the new Philly Flyers’ mascot.
  • Reply All - 114 - Apocalypse Soon: 2018 started strong.

Books

Things I learned

Don’t use sed to replace spaces with new lines, use tr instead. (tr ‘ ‘ ‘ ’ < input_file), and join newlines with paste (paste -s -d ‘ ‘ < input_file)

On macOS, increase your keyboard repeat speed with defaults write -g KeyRepeat -int 1 # normal minimum is 2 (30 ms)

Software

  • Kakoune, code editor
  • Kitty, a terminal emulator
  • Feedbin, an RSS read that gives you a customer email to subscribe to newsletters. Game changer.

Watch

Highlight from Trainer reports

[…] led us to discover a bug in IE. The crux of the issue is that cowsay uses angle brackets in its speech bubble, leading to text that looks like:

 _____ 
< moo >
 ----- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/
                ||----w |
                ||     ||

Internet Explorer interprets the “< moo >” as an unclosed html element, which breaks its rendering of the page. Both Firefox and Chrome ignore moo tags.

Both Firefox and Chrome ignore <moo> tags.

Happy Holidays!

-Alex

Manually Merging Day One Journals

My first Day One entry is from January 24, 2012. I used it often to take note about what I was doing during my PhD with the #wwid tag (what was I doing, an idea from Brett Terpstra, I think), and sometimes to clarify some thoughts.

When Day One went The Way of the Subscription, I didn’t bother too much because Dropbox sync still worked. Until it didn’t. I somehow didn’t realized it and kept adding entries to both the iOS and the macOS versions. Not good. It’s been on my to do list for a while to find a way to merge the two journals. I could probably subscribe to the Day One sync service and have it figure out the merging but I didn’t want to subscribe just for that.

I learned somewhere that Day One 2 could export journals as a folder of photos and a JSON file. I figure I could probably write a script to do the merging. So I downloaded Day One 2 on my iPhone and Mac, imported my Day One Classic journals, exported them as JSON to a folder on my Mac, and unzipped them. I also created a merged/ folder where to put the merged journal. The hierarchy looks like this:

$ tree -L 2
.
├── Journal-JSON-ios/
│   ├── Journal.json
│   └── photos/
├── Journal-JSON-ios.zip
├── Journal-JSON-mac/
│   ├── Journal.json
│   └── photos/
├── Journal-JSON-mac.zip
├── merge_journals.py
└── merged/

I first copied the photo folder from Journal-JSON-ios/ to merged/ and the photos from Journal-JSON-mac/photos/. I was pretty confident that I would end up with the union of all the photos because Day One uses UUIDs to identify each photo. The -n option to cp prevents overwriting files.

$ cp -r Journal-JSON-ios/photos merged/
$ cp -n Journal-JSON-mac/photos merged/photos/

I then ran the merge_journals.py script (below) to do a similar merge of the entries, based on the UUIDs. The merging happens by building a dictionary with UUID of each entry as the key and the entry itself as the value. It’s two loops over the iOS and the macOS entries. Entries with the same UUID should have the same contents, unless I’ve edited some metadata on one platform but not the other. I’m not too worried about that.

The output dictionary will be written to the Journal.json file. The entries are sorted chronologically because that’s how it was in the exported journal files, but I doubt it matters.

The output dictionary is written to disk without enforcing the conversion to ASCII since the exported journals are encoded using UTF-8. The indent is there to make the output more readable and diff-able with the exported journals.

import json

with open('./Journal-JSON-ios/Journal.json') as f:
    ios = json.load(f)
with open('./Journal-JSON-mac/Journal.json') as f:
    mac = json.load(f)

# Extract and merge UUIDs
uniques = {entry['uuid']: entry for entry in ios['entries']}
for entry in mac['entries']:
    uniques[entry['uuid']] = entry

# Create the output JSON data structure
output = {}
output['metadata'] = mac['metadata']
output['entries'] = list(uniques.values())
# I'm not sure it matters, but Day One usually exports the entries
# in chronological order
output['entries'].sort(key=lambda e: e['creationDate'])

# ensure_ascii print unicode characters as-is.
with open('merged/Journal.json', 'w', encoding='utf-8') as f:
    json.dump(output, f, indent=True, ensure_ascii=False)

The last step is to zip the journal and photos together, which tripped me up a few times. The Journal.json and the photos/ folder must be at the top level of the archive, so I zip the file from within the merged/ folder and then move it back up one level.

$ cd merged
$ zip -r merged.zip *
$ mv merged.zip ..

I could then import merged.zip in Day One, which created a new Journal, and delete the old one.

I guess I could somewhat automate this to roll my own, DIY, sync between versions of Day One, but I’d rather pay them money once I decide to use Day One frequently again. Still, I really appreciate that the Day One developers picked formats that could be manipulated so easily.

2017 Holiday Newsletter

This is the first installment of what became a yearly email I sent to my colleagues at Enthought.

Hi everyone,
I’ve been collecting these things for a while with the intent of sharing with some of you, but I decided why not share them with more people. I thought now’s a good time because you will maybe have some time during the holidays. It’s a grab bag of interesting things. Some of them are tangentially related to what we do here, some others not at all. I hope you find one or two good ones.

Happy Holidays,
Alex

Podcasts

Some great podcasts episodes:
- This episode of The Knowledge Project with Naval Ravikant, CEO and co-founder of AngelList.
- This two-part series from Reply All. A a telephone scammer makes a terrible mistake. He calls Alex Goldman. Seriously, listen to part 1. And then there will be part 2.
- The Dollop talks about Uber. You probably know how terrible they are, but they’re terribler than that.
- And an “old” one, one of my favorite 99% Invisible episodes. It’s about high heels.

Blogs and particular posts

Loren Shure is probably the most well known public face of The MathWorks. She’s been blogging forever about “cool things one can do with MATLAB”, or nice features of MATLAB. It runs every two weeks. I think it’s a great example of content marketing.

A nice application of the Heath brothers’ “SUCCESS” approach from “Made to Stick” (Simple, Unexpected, Concrete, Credible, Emotion, Story) and how it applies to teaching: How to break the first rule of systems thinking | thinkpurpose.

Some articles about leadership, from the great Michael Lopp, aka Rands, who was a manager at Apple, Palantir, Pinterest, and now Slack. Also the author of “Managing Humans” and the “The Nerd Handbook”. Here are some recent good posts. His Don’t Skip This is also a good place to start.

Laurel Norris wrote a great post on “Working-Learning Research” called Robust Responses to Open-Ended Questions. It’s a bit of a sales pitch for SmileSheets.com, but it’s convincing. It’s about picking good questions for post-presentation and post-class surveys.

This article by Marc Cranney from Andreesen-Horowitz provides an interesting framework for doing enterprise sales. I particularly liked how the language changes when one targets executives vs VP/directors vs group leads/regular employees.

Joey H is the developer of git-annex, was a core contributor to Debian, writes a really interesting blog, and lives off-grid in the North East (he used to live in a yurt). He wrote this really nice post about finding old bitcoins, and receiving positive user feedback. Here’s what he added at the end of his bug report template: “Have you had any luck using git-annex before? (Sometimes we get tired of reading bug reports all day and a lil’ positive end note does wonders)”. He shares some of the replies.

Books

New Directions in Open Education is the transcript of a Keynote Mike Caulfield gave at Metropolitan State’s TLTS conference in Denver, CO. Here are my notes from the talk:

  1. The student’s sense of belonging is extremely important. Belonging increases engagement and enhances learning. Belonging is affected by how the class is taught, but also by the class content. The material should be targeted to the audience. A drawing class to kids who like manga should be about manga, not renaissance painters. The students should be able to relate to the materials.
  2. Students should be able to share the result of their work. It gives them an audience, and then their output can itself be used as teaching materials. He mentions a class where students had to do an assignment (do something that makes you uncomfortable and then write about your experience and why you were uncomfortable) and post the results in a bank of assignment outcomes. The next batch of students could see those assignments and either “replicate them” (do the same thing that makes them uncomfortable) or simply get inspired. If they choose to replicate an assignment, then we get two “solutions” to the same assignment. If they choose to do something else, the assignment bank just got bigger. That’s genius!

Again, this is what I think about, when I think of this human core of open:

  1. We are encouraged to modify materials to create a sense of local belonging
  2. We use the power of the open internet to create work that is relevant and impactful, with a real audience
  3. We see the diversity of our students not as challenge to be solved, but as potential to be tapped

By far the best thing I’ve seen at Disney World so far.

Bread #2.

The Modern, Fort Worth.

The weight of the world.

By Giacometti.

There’s sometimes action out of my window.

Got a new room, with view on the East River. Much upgrade.

Wind tunnel model. Most beautiful item at the Musée des Confluence.

Anna going a bit overboard: crunchy speculoos, crème fraîche, and… maple syrup.

Party prep (Marshmallows + Jello powder)

Place des moulins, Marseille.

“The Cloud” of Lyon’s Musée des Confluences, seen from the roof.

A holiday sight.