Lobsters

I’m now the sysop of Lobsters, a social news site focused on technology.

Its creator jcs announced three weeks ago that he was ready pass the torch. I’ve been very active on the site, submitting tons of stories to help get the flywheel turning on a solid community. I want to see this good community continue and grow. With the support of the irc regulars, I stepped up to become the new administrator.

We planned the migration and executed it last weekend. There’s a handful of tidying-up tasks on the to-do list, but everything’s running smoothly on the new server.

The Lobsters codebase has been open source for years and I’m now maintaining it. There have been some great contributions from community members and I’ve tagged some issues to encourage new contributors. The new ansible playbook for configuring the site is also public, so hopefully it’ll now be very easy for more people to set up their own sites using the codebase.

I’ve been getting a lot of questions about managing communities in the last week, but I think I’ve already expressed most of my thoughts about it on Lobsters over the years with comments on what I think Lobsters is for, the environment I want to foster, difficulties with brief text on Twitter, and my approach to effective moderation.

If you’re reading this blog because we’ve met or spoken, please email me for an invitation. Otherwise, check the user list for someone you know or ask in chat (mention what you want to contribute or link a personal site/github so they know you’re not a random spammer). Hope to see you there!

Posted in Life at 2017-10-23 10:48 | No comments | Tags: , ,

Attending Recurse Center

I have been accepted to the Recurse Center to spend three months on collaborative, self-directed study of programming. I’m planning to continue studying Haskell and dependent types, proof assistants, and category theory. Maybe Coq, Idris, or TLA+ if I can find someone else interested.

Recurse encourages experienced devs to start or work on open source projects. For a couple years I’ve been taking notes on git’s command-line interface and joking about becoming the first CLI UX specialist, so I think I’m going to start that project as part of my Haskell practice. I don’t plan to re-implement git, I plan to wrap it with more cohesive mental models and useful features for day-to-day development work. If you have thoughts or resources in this area, I’d love to hear about them.

I’ll be attending the Fall 2 session in New York City from September 25 to December 15. If you’re in the NYC area those dates and would like to meet up, reach out, I’d love to meet up. And if you’re in Chicago, well, you won’t see me at Code and Coffee for a few months.

Posted in Life at 2017-08-29 15:22 | 4 comments |

MiscPodcast

I have some random episodes of podcasts laying around waiting to get listened to from podcasts I don’t (yet) care to subscribe to. Maybe they had an interesting guest or topic, or came recommended. These downloads will lay around on my computer for months because they’re not in my podcasting app, so they’re not really in my listening queue.

So I hacked up a PHP script to grab those mp3s and serve a valid (if spartan) podcast feed.

You can find the code on GitHub. It’s designed to be dropped into place rather than robustly deployed. Then I download mp3s to that folder and they’re delivered as my own personally-curated podcast feed.

Thanks to Shubham Jain for his PHP-ID3 library so the script can read some metadata into the feed.

Posted in Code at 2017-08-27 16:02 | 1 comment | Tags: , ,

Arithmetic Wrap in GMS2

I’m learning GameMaker Studio 2 because my 10-year old nephew wants to make video games (and the 10 year old inside of me wants to make video games, too). It’s a nice toolkit and IDE for games, very beginner-friendly, with a friendly community. It’s even been used in some highly polished and popular games. If you’re curious, there’s a ~90 minute tutorial playlist that’s easy to skim as a demo.

Pricing is reasonable, it’s $100 to export desktop games, couple hundred more to export to iOS + Android, HTML 5, even PS4 and Xbox. And this week version 1.4 (with a free trial and discounted upgrade to version 2) is on sale cheap.

The games are coded in “GameMaker Language”, which is approximately PHP 3. Little more OO, little less coercion, but it feels similarly very focused on its niche without much experience behind it. I’ve been mostly coding Haskell the last few weeks, so there’s a bit of whiplash moving between the two.

There’s some great video tutorials on YouTube showing how to make games like NES Zelda, Farming RPG, and Platformer. All three of those channels are worth clicking around on as they have other good playlists or one-off videos.

On that last channel I saw a video on Useful Scripts for GMS2 presenting five code snippets. In GML, a “script” is roughly a singleton static method object used to centralize game state or encapsulate snippets of functionality.

The third script presented at 5:18 in the video was wrap(value, min, max) which wraps values that exceed the min or max back around to the other side. So wrap(5, 0, 9) is 5 because it’s in the bounds, but wrap(11, 5, 9) is 6 because it wraps 2 past 9. There’s a visual in the video at 5:30 that makes it real clear. There’s also a screenshot of the code:

/// @description Wrap(value, min, max)
/// @function Wrap
/// @param value
/// @param min
/// @param max
// Returns the value wrapped, values over or under will be wrapped around

if (argument0 mod 1 == 0)
{
	while (argument0 > argument2 || argument0 < argument1)
	{
		if (argument0 > argument2)
			argument0 += argument1 - argument2 - 1;
		else if (argument0 < argument1)
			argument0 += argument2 - argument1 + 1;
	}
	return(argument0);
}
else
{
	var vOld = argument0 + 1;
	while (argument0 != vOld)
	{
		vOld = argument0;
		if (argument0 < argument1)
			argument0 = argument2 - (argument1 - argument0);
		else if (argument0 > argument2)
			argument0 = argument1 + (argument0 - argument2);
	}
	return(argument0);
}

There’s a lot going on there. This is probably hot-path code that runs every frame, but it has branches inside a loop for something that could almost certainly be an arithmetic one-liner. So for practice with GML and the IDE in general, I rewrote it.

I saved the above script as original_wrap and created my own wrap implementation:

/// @description wrap(value, min, max)
/// @function wrap
/// @param value The value to wrap into the bounds
/// @param min Minimum bound, inclusive
/// @param max Maximum bound, inclusive
// Returns the value wrapped to the range [min, max] (min and max can be swapped).
// Calls floor() on reals, but GML's modulo is doing something weird and original_wrap just hangs indefinitely on some values anyways so oh well.

var value = floor(argument0);
var _min = floor(min(argument1, argument2));
var _max = floor(max(argument1, argument2));
var range = _max - _min + 1; // + 1 is because max bound is inclusive

return (((value - _min) % range) + range) % range + _min;

Some oddities, like JavaDoc instead of a function signature, so arguments have automatic names (reminds me of perl 5). In dev I got a compiler error for referencing argument1 before argument0 – I’m not sure what that could be but look forward to reading the manual. I can’t unimport/shadow the global max and min, the convention seems to be to use a leading underscore for colliding names.

To test it, I created another script called test and invoked it from an object’s create event. Which felt a little roundabout for specifying that I wanted it to run at startup, but I’ve barely touched the manual so I’m probably missing something obvious. I know there’s 3rd-party test library but I wanted to hand-roll to see more moving pieces. I generated a golden master test suite to exercise a bunch of test data, though I didn’t go all the way into property-based testing:

// make this easy to spot in the build output
show_debug_message("HELLO &&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&");

for(i = -5; i < 25; i++) {
  var orig = original_wrap(i, -2, 5);
  var new = wrap(i, -2, 5)
  if (orig != new) {
    show_debug_message("int failed " + string(i) + " orig " + string(orig) + " new " + string(new));
  }
}

//for(i = -5.0; i < 19.2; i = i + 1.1) {
//  show_debug_message(i);
//  var orig = original_wrap(i, -2.1, 5.3);
//  var new = wrap(i, -2.1, 5.3)
//  if (orig != new) {
//    show_debug_message("real failed " + string(i) + " orig " + string(orig) + " new " + string(new));
//  }
//}

The commented-out testcase is, I think, about the broken behavior with regards to reals. I’m pretty sure the (argument0 mod 1 == 0) exists to type-check if argument0 is an integer or real. I’m not sure because GML has typeof for typechecks (is_real in 1.4, which feels very PHP 3), but this is probably well-copy-pasted newbie code. When I tried to test the behavior, I ran into a bug where original_wrap hung indefinitely on some inputs (the existing test triggers this, if you want to uncomment and run). I didn’t want to keep tinkering, so I dropped in some floor calls and moved on.

Anyways, this was some fun tinkering. I’m looking forward to working through the official tutorial with my nephew and maybe making some small games. (Oh, and I release this into the public domain, feel free to use it with or without credit.)

Posted in Code at 2017-08-06 21:50 | No comments | Tags: ,

Redshift With Cloudiness Adjustment

A Lobsters story on the bright blue light of displays reminded me I should post this. I use redshift to adjust the color temperature of my monitor at night so I sleep better, and I wrote a custom wrapper script to include an adjustment for how overcast it is.

 
#!/bin/bash
 
set -e # exit on error
#set -x # debugging
 
if [ `pidof /usr/bin/redshift` ] ; then exit ; fi
 
# Chicago
LAT=41.94
LONG=-87.65
 
MAX=5700
DOWNTO=4200
# Tungsten: 2700K
# Halogen: 3400K
# Fluorescent: 4200K
# Daylight: 5500K
MODE=vidmode
GAMMA=0.8
 
# figure out how overcast it is, and adjust temperature to match
TODAY=`date +%Y-%m-%d`
wget -q -O/tmp/clouds.xml "http://graphical.weather.gov/xml/sample_products/browser_interface/ndfdXMLclient.php?product=time-series&sky=sky&lat=$LAT&lon=$LONG&begin=${TODAY}T00:00:00&end=${TODAY}T23:59:59"
OVERCAST=`echo -e 's/[^0-9]*([0-9]+)<\/value>/1/m\nt xxx\nd\n:xxx' |sed -rf - /tmp/clouds.xml |tail -n 1`
TEMP=`echo "$MAX+${OVERCAST}0" |bc`
 
redshift-gtk -l $LAT:$LONG -t $TEMP:$DOWNTO -g $GAMMA -m $MODE
Posted in Code at 2017-01-25 12:32 | No comments | Tags: ,

Hard Lessons

Having worked on email-related code before, I have been morbidly fascinated by one of the founders of handmade.network writing an email client. Handmade Network is trying to reinvigorate programming by emphasizing small teams and from-scratch performant code. It’s a great way to write small, self-contained projects (games, libraries, utilities) that can be done, but fell out of favor two decades ago for complex user-facing software.

This update included a few sentences I’ve been waiting for:

The biggest lesson is that not everyone is RFC-compliant. It was a shock seeing some companies accept ill-formed e-mail addresses, developers showing their best-but-still-inaccurate regular expressions for compliance, and security agents from company’s mail servers trumping simple IMAP requests that should have yielded a proper response, but didn’t. Look, I always knew commercial software packages don’t fully adhere to a spec—not even language compilers achieve 100% accuracy—but seeing violations led to unfortunate wrinkles and hard-coding in specific recovery points when I try to talk to some servers.

From what he lists, he’s only seen the tip of the iceberg. For example, he hasn’t mentioned some of the fun problems of IMAP or talked about the woes of email encoding and attachment. Specifically, this strategy of “seeing violations led to unfortunate wrinkles and hard-coding in specific recovery points when I try to talk to some servers” is really, really not going to scale. C is a tough language for the tower of abstractions he’s going to build and rebuild in the face of unexpected inputs and dusty corners of the spec.

And email is a particularly hard domain because it’s old and *looks* simple, so there’s an incredible amount of errors you have to cope with from version 0.1. Users will never accept “Yeah, you just can’t read email from people using Outlook, it’s Microsoft’s bug.” And then on top of all that, many emails are shifting to HTML-only with increasing expectations of CSS support and you’re implementing or talking to a huge browser engine. Email was a big factor in ending my support for Postel’s maxim.

It might be another 5 months before I reach a working prototype for that [GUI], and probably another two months of polish before I consider the possibility of releasing some build publicly.

I wish him a lot of luck and there’s a tiny, windmill-tilting bit of me that hopes he’ll succeed, but I’m watching this race for the crash.

Posted in Code at 2016-09-28 10:39 | No comments | Tags: ,

Queue Zero

Almost exactly a year ago, I posted about Sizing Up My Queue to count up how much video and audio I had downloaded to watch. The final tally?

queue: 877 files, 674007 seconds = 7d:19h:13m:27s total duration

I’ve been running that script almost every day, and for the first time it said:

queue: 0 files, 0 seconds = 0d:00h:00m:00s total duration

I did track values over time, and after a lot of frustration LibreOffice permitted this hideous graph – the Y axis is how many days of media remain:

media-queue
  • Most of the early drop was me shrugging and saying “yeah, OK, I’m really not interested enough in that podcast to actually listen to it”
  • The half-day jump in December is when I fixed the script to include .mov files
  • Big gap and accumulation in March/April is when I was working on my talk and book
  • About half of the drop at the end was archiving a video site I finished scraping
  • I almost exclusively listen to podcasts when doing chores or playing video games, so I’d have hit zero a month earlier if I didn’t play ~65 hours of Crypt of the Necrodancer
  • The last file was the recording of my 2015 RailsConf talk – watching my own presentations really makes me squirm, though it’s invaluable for improving as a speaker

This was a fun little project. There’s still a few thousand files in ~/queue. It’s a bit of a junk drawer (games waiting for me to have Windows again, photos to file away, archived web pages), but the majority of it is books and papers. I suppose next I could write a script to take the word count of epub/mobi/pdf/html files… it’d be a bit of fiddling running different commands to dump word counts from the various formats, but it could work.

Well, it could work in that, yes, I could technically write that script. I’ve known for decades that my to-read list has been growing faster than I read.

Posted in Life at 2016-06-15 09:32 | No comments | Tags: , , ,

Vim: highlight word wrap column in insert mode

I like vim’s colorcolumn for highlighting where word wrap will occur, but I consider it a distraction when I’m not in insert mode. After some tinkering, I wrote this in my .vimrc:

" highlight textwidth column in insert mode
highlight ColorColumn ctermbg=0*
function! HighlightOn()
  if &textwidth > 0
    " the +1 feature in the 'colorcolumn' docs doesn't work for me
    let &colorcolumn=&textwidth + 1
  else
    let &colorcolumn=""
  endif
endfunction
autocmd InsertEnter * :call HighlightOn()
autocmd InsertLeave * let &colorcolumn=""

That note about +1 is me working around a bug. I should be able to just write:

" highlight textwidth column in insert mode highlight ColorColumn ctermbg=0* autocmd InsertEnter * let &colorcolumn=+1 autocmd InsertLeave * let &colorcolumn=""

Unfortunately, some tweak or plugin breaks this feature for me. I wrote this workaround rather than diagnose and fix it properly because the process just seemed too tedious.

Posted in Code at 2016-06-09 13:07 | No comments | Tags:

Recursive Sum

In #ruby on Freenode, platzhirsch asked about how to total an array of Transactions when the Transactions may have parents. The two obvious approaches have pitfalls when there are a lot of Transactions, and he said he expects to have 23 million that may be deeply nested. Here’s his sample code:

 
class Transaction
  attr_accessor :amount, :parent
 
  def initialize(amount, parent)
    self.amount = amount
    self.parent = parent
  end
end
 
t1 = Transaction.new(100, nil)
t2 = Transaction.new(250, t1)
t3 = Transaction.new(300, t2)
 
current = t3
all = []
while current != nil
  all << current
  current = current.parent
end
total = all.inject(0) { |total, transaction| total + transaction.amount }

Spoiler warning: last chance to solve it for yourself before I dig into the solutions.

This code sample expressed the first obvious solution: build a list of all the transactions. The problem is that you’ll spend RAM and time building a data structure you expect to use once.

One person offered an addition to Transaction to solve it recursively, the second obvious approach:

 
class Transaction
  def total_amount
    (parent ? parent.total_amount : 0) + amount
  end
end

This pitfall is that this risks blowing the stack when Transactions are deeply nested: recurse too many times and you’ll run out of RAM. It’s also super-specialized, if you want to do anything else with every Transaction you’ll have to write another custom method. And despite the specialization, you might end up writing this logic again if you have a collection of child transactions:

 
t4 = Transaction.new(400, nil)
 
total = [t3, t4].inject(0) { |sum, t| sum + t.amount }

Here’s my approach:

 
# because Transaction doesn't have any logic, I made a shorter version:
Transaction = Struct.new(:amount, :parent)
 
# And I like using powers of two when testing recursion, because the sum
# will come out obviously different for different combinations of items:
t1 = Transaction.new 1, nil
t2 = Transaction.new 2, t1
t3 = Transaction.new 4, t2
t4 = Transaction.new 8, nil
 
TransactionEnumerator = Struct.new(:collection) do
  include Enumerable
 
  def each
    collection.each do |t|
      yield t
      yield t while t = t.parent
    end
  end
end
 
ts = TransactionEnumerator.new [t3, t4]
total = ts.inject(0) { |total, transaction| total + transaction.amount }

This little wrapper doesn’t recurse, doesn’t duplicate Transactions, and doesn’t build a data structure. It can work on any collection of Transactions that exposes each, the sum logic is expressed only once and separately from the control flow, and it provides the powerful Ruby Enumerable interface.

Hope you enjoyed this little puzzle! If you had an alternate solution, please wrap it in <code></code> tags below.

And let me tag on my own fun exercise: add an ID field to Transaction and implement TransactionEnumerator#uniq yield the transactions exactly once, so this returns true (Array has #uniq, but you shouldn’t assume the collection is an Array):

 
TransactionEnumerator.new([t1, t1]).uniq == [t1]
Posted in Code at 2016-02-13 09:36 | No comments | Tags: ,

2016 Media Reviews

I’ve appreciated when people take the time to write reviews and highlight connections to other good works. This post will be regularly updated through 2016. Previously: 2014 2015

Continue this post…
Posted in Life at 2016-02-04 13:26 | No comments | Tags: , , ,
  • Twitter: pushcx

  • More tweets below and @pushcx