Macbook Pro Setup for fast.ai MOOC

Background

I decided I wanted to run the exercises from the fast.ai MOOC on my local computer. The recommended approach is to run everything on a t2 AWS instance to make sure everything is setup correctly and then run the final product on a p2 AWS instance. While it still makes sense to run your final job on something with a lot of GPU horsepower (like an AWS p2 instance or Floydhub), I didn't particularly feel like using an AWS instance with no GPU to setup the job. Thus the quest for setting up my home machine - a Macbook Pro running MacOS Sierra with a NVIDIA GeForce GT 750M 2048 MB card - for this purpose. 

The process

1. Install xcode 8.2 command line tools from apple’s site from here: http://adcdownload.apple.com/Developer_Tools/Command_Line_Tools_macOS_10.12_for_Xcode_8.2/Command_Line_Tools_macOS_10.12_for_Xcode_8.2.dmg. (Note: as of this writing, CUDA 8.0 requires Xcode 8.2 on Mac OS Sierra Other versions of Xcode lead to a nvcc error - nvcc is the nvidia compiler)
2. Make the Xcode command line tools the default: sudo xcode-select --switch /Library/Developer/CommandLineTools/
3. Install CUDA 8.0 from here: https://developer.nvidia.com/cuda-downloads. Make sure you check if your particular graphics card is supported. There's a support matrix on the CUDA site.
4. Install cuDNN 5.1 for CUDA 8.0 (https://developer.nvidia.com/compute/machine-learning/cudnn/secure/v5.1/prod_20161129/8.0/cudnn-8.0-osx-x64-v5.1-tgz). (Note: although higher versions of cuDNN are available, keras 1.2.2 seems to only work with cuDNN 5.x)
5. Install anaconda using the anaconda installer from https://www.continuum.io/downloads,
6. Install theano (pip install theano)
7. Install keras v1.2.2 (pip install keras==1.2.2)
8. Install kaggle-cli

Caveats

The list above is from memory and judicious use of the bash history command, so I cannot guarantee I haven't missed anything. If you happen to get stuck at some point, try looking at the script here http://files.fast.ai/files/install-gpu.sh to figure out if you can spot the missing piece. The steps below are based off that script.

Makers and Dreamers

The world is filled with two broad types of people: the ones that care and the ones that don’t. Makers and Dreamers make up the ones that care. Makers, as the name suggests, are the people that build things with their hands. Often, though not always, makers make things that dreamers dream up. The dreamers are always optimistic. They are sure that somehow, by some transcendental intervention, their beautiful dream will see the light of the day. Not only that, it will be well received and adored by all that come across it. The Makers, on the other hand, are pessimistic, almost cynical. However, they try their best to at least create a horse where the dreamers saw a unicorn. 

It is fascinating to watch the interactions between these two types. One cannot succeed without the other, but each sees the other as something to be endured instead of a necessary part of the ecosystem. Wonderful things can happen however, when the dreamers understand the makers and vice versa. If the dreamers acknowledge the world for what it is, warts and all, they can, by their sheer positivity motivate the makers to outdo themselves. When the makers, on their part, buy into these seemingly wild flights of fancy, they can build things of a subliminal beauty that wows the onlooker. 

Multiple github accounts on one computer

Generate a key

ssh-keygen -t rsa -C "your-email-address"
Save this key in a different file than the default. e.g. ~/.ssh/id_rsa_TWO.pub

Attach keys

Login to your second GitHub account and attach the newly created key  (~/.ssh/id_rsa_TWO.pub) . On Github this should be under the Account Settings -> SSH Keys.

Next, because we saved our key with a unique name, we need to tell SSH about it. Within the Terminal, type: ssh-add ~/.ssh/id_rsa_TWO. If successful, you’ll see a response of “Identity Added.”

Create SSH configs

Create and ssh config file under ~/.ssh/config

  1 #Default Github

  2 Host github.com

  3     HostName github.com

  4     User git

  5     IdentityFile ~/.ssh/id_rsa

  6

  7 Host github-TWO

  8     HostName github.com

  9     User git

 10     IdentityFile ~/.ssh/id_rsa_TWO

Create a repository

Create a repo on github using the add repo button

Now for using the 2nd user do the following:

In the directory you wish to push to github:

  $ git init

  $ git remote add origin git@github-TWO:USER/remoterepo.git

  $ git add .

  $ git commit -m "initial commit."

  $ git push origin master

Inspired by: http://net.tutsplus.com/tutorials/tools-and-tips/how-to-work-with-github-and-multiple-accounts/

Android AutoCompleteTextView suggestion list from an external Web Service

Android offers an AutoCompleteTextView widget. If you peer into the source code of the AutoCompleteTextView widget you'll notice that it contains a ListView that renders the suggestion list. This ListView is generally fed by an ArrayAdapter. It's fairly trivial to stick an array like data structure in this ArrayAdapter to display suggestions. This works well for static data. However, if you plan to poll an external Web Service to build your list of suggestions, things get tricky. My first thoughts were to build a custom ListAdapter that would go out to the Web Service on every getView() call. I quickly discarded that idea since it wouldn't mesh well with the ListAdapter/ListView interaction model. I finally settled on using a custom Filter to build the suggestions list from a Web Service. An added advantage of using a custom Filter is that the filter() method is run on its own worker thread. This obviates the need to make the web service call on an AsyncTask. Here's the code:

    private class AutoCompleteAdapter extends ArrayAdapter {

        @Override

        public Filter getFilter() {

            //This is important.

            return new MyFilter();

        }

        

        final class MyFilter extends Filter {

            @Override

            protected FilterResults performFiltering(

                    CharSequence constraint) {
                //This method gets called on a worker thread
                //so no AsyncTasks required here

                FilterResults result = new FilterResults();

                try {

                    URL feedUrl = new URL(...);

                    URLConnection conn = feedUrl.openConnection();

                    InputStream is = conn.getInputStream();

                    //Read from InputStream

                    //Build a List of suggestions

                    result.values = suggestions;

                    result.count = suggestions.size();

                } catch (MalformedURLException e) {

                    e.printStackTrace();

                } catch (IOException e) {

                    e.printStackTrace();

                } 

                return result;

            }

            @Override

            protected void publishResults(CharSequence constraint, FilterResults results) {

                clear();

                if (results.count > 0) {

                    for (String s : (List) results.values) {

                        add(s);

                    }

                }

                notifyDataSetChanged();

            }

        }

    }

When my little boy grows up

In no particular order, when my little boy grows up:

I'll miss...

.. how he runs and hugs me when I come home

.. how he looks at me with big curious eyes when I explain something

.. how he assumes that his daddy can do anything and everything

.. how he loves it when I hold him close and read to him

.. how he endlessly plays silly little games

.. how he dances without getting self conscious

.. how he learns everything as quickly as he does

.. how much he loves his mother

.. how there's so much trepidation and excitement in his eyes about something new

.. how he runs around the house without a break

.. how he loves to hide behind things and play peek-a-boo

.. how the most silly thing makes him laugh out loud

.. holding him close to me when he's scared

.. how much he loves to ride on my back

.. how he takes the most incredible things at face value

.. how he never lies, even if that gets him into trouble

.. how he's the most maddening person ever by far

.. how he holds my finger when we go out for a walk

.. how he holds on to my finger from his crib when I lie down next to him on the floor

.. how his mother and I cannot stop talking about him

.. how he walks around the house with his stuffed dog in his arms

.. how much he depends upon me

.. how he makes me smile

.. his cute mispronunciations

I'll miss that little boy when he grows up.

Android In-app Billing notes

In the notes below, I have distilled the knowledge I gained implementing Android In-app billing. All of this is current and correct as of this writing (Sept 2012). However, given the pace at which the In-app billing framework seems to evolve, a lot of this might not be applicable by the time you read this. Take it for what it's worth.

Development

- Read this http://developer.android.com/guide/google/play/billing/index.html carefully. All of it. Multiple times.

- Don't start from scratch. Grab the in-app billing sample that ships with the SDK and re-purpose it. Create a library from the sample app after stripping out things you don't need. Include that in your main application.

- Changes made in the Play developer console don't take effect instantaneously. Sometimes for hours. Don't despair if you see unexpected responses from the Play service.

- If your app is about subscriptions rather than plain old one-time purchases, do yourself a favor and implement a service on your backend to keep track of subscriptions. This will eliminate a whole host of edge cases that arise if you don't have a backend to keep track of active subscriptions and cancellations.

- Make sure that the ITEM_TYPE property in the request bundle passed in sendBillingRequest is correct. If there's a mismatch between the product type configured in Play and your request you'll just get an ITEM_UNAVAILABLE response with no explanations.

- Stackoverflow is your friend. See the in-app-billing tag. 

- This is another great resource to save you some heartache.

Developer testing

Pay careful attention to the table in this section: http://developer.android.com/guide/google/play/billing/billing_testing.html#billing-testing-static.  The table is fairly confusing, but the bottom line is this: use a developer account for testing responses from Play.

Here's a good way to test your app during development:

1. Make sure you have the com.android.vending.BILLING permission in your Manifest.

2. Upload the app to the Play Console and leave it unpublished. Make sure the versionCode of the latest uploaded APK in Play is the same as in the app you test with. This is important. You won't get signed responses unless these match.

3. Make sure the primary account on the device that you plan to test with is configured as a developer in your Google Play account. This is especially important if your app has never been published before. You won't get a signed response otherwise.

4. Don't test till at least a few hours have elapsed from the time you uploaded your app. For best results test it the next day.

5. Test your app's flow using static test product IDs e.g. android.test.purchased.

6. If you wish to test it with a real product ID make sure you wait for some time after creating the product in Play before testing with that ID.

QA

Final testing just before launch

1. Bump up your app's versionCode and upload the APK again. Leave the app unpublished.

2. If this app has never been published before, test using a developer account. If your app has been published before you can use any account. 

Gotchas

- Can't stress on this enough. Changes made in the Play console don't necessarily take effect right away. If you don't see an updated response from the Play service, give it some time to catch up.

- If you upload an app and leave it unpublished, unless you are added as a devloper in the Play dev console, you'll always see an empty signature string. 

- Google Play doesn't let you query published products for pricing information. If you plan to display the price of a product in your app, make sure there's a way to keep the price published in Play in synch with what you display in the app. This should be via a CMS or something similar.

- Google Play will not let you edit products once they are created. Publish with care.

Shaving the Yak

All I wanted, was to choose the right wordpress theme for some blog I was helping set up. To do this I had to install MySQL. To do that I ended up upgrading my Ubuntu install from 12.04 to 12.04.1. All of this to decide which wordpress theme to mooch off of. Talk about Yak Shaving.

Remapping the Show Desktop key combination on Gnome

I use both Ubuntu and Windows XP desktops at the same time. I am used to hitting Win+M on windows to minimize all my windows and show the desktop. It bugs me no end that I cannot do the same on Ubuntu. But some quick searching led me to 2 forum posts that helped me nail this. The default key combination for showing your desktop on Ubuntu is Ctrl+Alt+D. So the first thing you'll have to do is to assign a new sequence to that action. You do this by using either the gconftool command line tool or the gconf-editor GUI. The command line method works thus:


gconftool-2 -t str --set /apps/metacity/global_keybindings/show_desktop "<Super>m"


If you decide to use the gconf-editor tool instead, navigate to apps -> Metacity -> global_keybindings and look for the "show_desktop" key in the right side pane. Here replace the existing value with "<Super>m" (without the quotes).

If you have compiz installed the "Negative" plugin in compiz inverts your screen colors on hitting Win+M and restores your colors on hitting Win+M again. You'll have to disable this. If you use gconftool from the command line, here's the incantation that'll do the trick:


gconftool-2 -t str -s /apps/compiz/plugins/neg/allscreens/options/screen_toggle_key ""
gconftool-2 -t str -s /apps/compiz/plugins/neg/allscreens/options/window_toggle_key ""


If you choose to use the GUI gconf-editor instead, you'll have to navigate to the right key and update it's value, similar to the show_desktop example above.

And there you have it.