How to Think Like a Software Developer

Any sufficiently advanced technology is indistinguishable from magic.

Arthur C. Clarke

When most ordinary people think of software and computing machines they usually perceive it as some sort of complicated and mysterious system of engineering, a black box of wonder capable of making the impossible seem possible.

Introduce a child to a touchscreen device and a virtual reality experience and the most likely form of reply he will give you is that it is a form of magic, something beyond the stories he has heard of from old books and weary relatives.

As the people that work on these machines we must seem like a sort of bizarre crowd.

We spend our days in solitude doing most of the following:

  • Staring at a glowing screen as if looking for an answer.
  • Reading technical books full of unusual terminology.
  • Talking about cryptic languages most people do not understand or use.
  • Writing hundreds of lines of code that make no sense to the average eye.
  • Communicating with a group of foreigners via a worldwide network of devices.

Does this seem strangely familiar to you?

Upon further thought, have we come far enough as to blur the line between reality and fantasy in terms of what we can achieve using our knowledge?

Software as a Form of Magic

I believe that a good software developer is like a good magician in the classical, fantastical, sense of the word.

He is someone who endeavors to bring change into the universe through words and ritual alone. Using a specific set of words with very special properties and occult meaning: a programming language.

These words serve as his sorcery; being no less mesmerizing than the grey cloak and the Wizard’s hat that have captivated our writers for centuries.

Fortunately for us, our modern tools are deterministic. Which means that our words will always have a known and intended effect every time they are interpreted by our machines in the same ritual of execution.

The Realm of Problems

In order to bring change into the world, a developer must be confronted with a problem or (if the problem is too broad) a set of problems.

There are, broadly, three categories of problems defined around the existence or non-existence of some thing:

  • Something that does not exist.
  • Something that exists but is not working properly or desirably.
  • Something that exists but is unknown or incomprehensible.

A problem serves as a trial, or an opportunity for the developer both to solve a problem and grow as a practitioner.

A developer can bring something into existence by arranging his words into a cohesive abstract system of moving parts.

He can delve into someone’s work, understand his particular system of parts, re-arrange it, and expand the system to meet new expectations.

He can create his own implementation of something that already exists, for the sake of knowledge and experience as if to gain some sort of acquired power.

The applications are endless for the software developer.

He examines a problem that can be solved by the application of technology, devises a solution, implements it by a series of elegant machine commands, and changes a small (or large) part of the world forever.

How to Conquer a Problem

Now that the role and the craft of a developer are understood, what is left is a manner of dealing or thinking about problems effectively.

As powerful as a software developer can be, his knowledge and tools can only be as effective as his understanding of the problem.

If he does not understand a problem his solution will not understand his requirements as well, and we should not assume otherwise.

In this case, I defer to Mat Helme’s work on problem solving, which should be considered our guide to thinking like a developer in a practical sense.

Briefly, a general way of dealing with problems can be summarized in the following four points, known as the Four P’s of Problem Solving:

The Problem Solving Process

The first step is prep, this is where we understand and diagnose the problem.

Then our next step is plan. This is where we organize everything before acting.

The third step is perform. We simply put the plan into action.

Then our final step is perfect. This is when we check to see if our end goal has fulfilled the problem. If not we can go back, review and rework the solution.

Laced throughout this process is the concept of experimentation.

There can be no problem solving without both curiosity and experimentation. This is where the act of software development approaches science in spirit.

Problem Solving Is Science

In fact, the scientific method itself can be seen as a cyclic or recursive application of the Four P’s of Problem Solving:

  1. The preparation stage is used for making observations about a part of the world (a system).
  2. The planning stage is used for formulating hypotheses and predictions.
  3. The performing stage is used to gather data, results, and study the outcome.
  4. The perfecting stage is used to revise, refine, or discard hypotheses in accordance with real-world results.

The parallels seem to suggest that the developer as a problem solver is not very different from the scientist.

They both make observations about a system, formulate hypotheses or guesses as to why the system works a particular way, postulate predictions about future behavior under certain circumstances, and test them under a controlled environment to examine their validity.

The difference is that the developer does not need thousands of dollars in government funding to carry out his experiments.

He can do them at will, in quick succession, and at little to no cost in most cases. This itself is another magical property of the work of the developer, in that it usually does not have any physical constraints and thus, no limits to the amount of work that can be carried out.

More Than a Thought

To put into practice the act of thinking and acting like a software developer, we must continually approach problems armed with the following:

  • A problem-solving methodology (the four P’s).
  • A set of tools (programming language, libraries, framework, etc.).
  • A set of rules to reason about the system we wish to change or create (logic, induction, deduction, and pattern-recognition).
  • Time, hard work, perseverance, curiosity, and experimentation.

Additionally, good software developers tend to be good computer scientists, which mean that beyond their thought process, they also have knowledge of algorithms, data structures, and design patterns in particular, as well as a good grounding on best practices.

These are the secrets behind the magic that allow the individual to write code that can both change the world and be used by others in their own path to build a different vision of reality.

How to Make It as a Mediocre Software Developer

If you’re in the software world, chances are you’ve felt overwhelmed with the amount of work and effort required to be competent and keep your skills up to date.

As a developer, this thought shouldn’t scare you. Making good software is hard. Humans make mistakes, and machines remind us of that. It’s all so unpleasant, you see.

Therefore, it completely makes sense that you wouldn’t want to spend the majority of your time on a never-ending struggle with a machine and would rather enjoy yourself more often than not.

And that is totally fine. Thousands of other developers are already doing just that.

What follows is my best advice on how to become a professional enterprise software developer and enjoy a comfortable career with minimal effort.

1. Don’t Ask Any Questions

The best people are the ones that ask the right questions. But forget that, being the best means working hard. And working hard means, well, working.

The best technique is to say nothing and nod confidently in understanding.

If someone attempts to pry, you can condescendingly reply with a nonchalant ‘I know what I’m doing’ or scoff at them with slight disbelief.

Doing so would get you into trouble if you were, say, a brain surgeon or an emergency anesthesiologist. But you’re a developer and you have three indispensable tools at your disposal which almost guarantee that you don’t really need to know what you’re doing:

  • Google
  • Stack Overflow
  • Ctrl + C / Ctrl + V.

Which is exactly why you should move on to the next point on this list.

2. Say ‘Yes’ to Everything Your Boss Says

Nobody likes saying the word ‘No’. But you know what people hate even more than that? Hearing the word ‘No’.

If there’s anything you should learn from politics is that naysayers stay at the bottom of the ladder. You want to be the Yes-man.

It doesn’t matter that the boss or client has no idea of what they want, or that there hasn’t been any product design planning made beyond a few minutes of conversation, or that nobody has any clue of the complexity of the problems that your team will be facing or even the tech stack that you’ll be handling.

Just say ‘Yes’ and smile. If it works for management it’ll work for you.

3. Let Your Teammates Handle It

A great asset for disguising your mediocrity is other people. You should always seek to work in the largest teams because as more people tend to work on a project it becomes harder for management to keep track of who is responsible for what.

And more people means you can use your flattery to extract help and hide behind the accomplishments of your teammates.

Whenever something goes wrong or you have no idea what to do, you can seek out one of the more competent people you’ve buttered up over the past couple of weeks and ask ‘what they would do in your position’ and steal their ideas for your own use.

Their poor social skills in most cases will work for you and you’ll be able to bill yourself on your résumé as a ‘strong team player’.

4. When in Doubt, Blame I.T.

As a developer, you know you’ve made it when you see one of the lowly I.T. “people” have to painstakingly deal with broken Windows upgrades or Hard Disk Drive failures while you sit in your comfy padded chair with your colorful IDE open and a few Reddit tabs on the background.

‘Heh’ you might say to yourself while you shrug and get back to “work”. But stop right there, for this situation presents a unique opportunity for the rare times when you are asked to explain yourself.

You can now blame the I.T. department.

From everything to data loss, password resets, hardware “failures”, administrator access, user permissions, and more. ‘Waiting on I.T.’ or ‘I.T. screwed up again’ is the perfect excuse to enjoy your lack of productivity at someone else’s expense.

5. Hide Your Development Tools

Development tools are designed to be efficient and save time. They are not your friends. If you can cut down on building or deployment time, that is time that you will be expected to be performing additional work.

This is of course, highly undesirable. However, with a little cunning you can make your laziness work for you. Perhaps make one of your co-workers help you write a script to automate some mind-numbing task or get on GitHub and find a gist that does what you need to skip some long repetitive work.

The key is to keep these tools and hacks hidden away from management. You can now enjoy the rest of the time that these tasks would regularly take and maybe even take some credit for finishing earlier than expected.

Oh, and definitely do not share your development tools with your co-workers. If they can’t keep quiet about them then they don’t deserve them.

6. Don’t Document Your Work

Documentation usually means that there is some sort of forethought or emphasis on the future at your company. You should absolutely be discouraging this sort of behavior from your employer. It is just completely unacceptable.

You should never write more than two lines of comments or a few words in your commit messages, or even follow standard design patterns or coding conventions.

Those are things for the lead engineers to worry about. You’re a professional and an adult, you don’t need to explain yourself to anyone and much less waste time worrying about how the system is going to scale, the maintainability and efficiency of your code, or other trivialities.

7. Spend All of Your Free Time on Entertainment

Finally, here is where you get to enjoy the fruits of your labor: a lifetime of choice entertainment, constant outings with ‘the boys’, a long string of drunk nights out, and enough stories to make the bigwigs over at lunch fall off of their seats with riotous laughter.

Or maybe you’re the more introverted type. In that case, you get to look forward to a big Steam library, terabytes of obscure Chinese cartoons, and Lord knows what else you might end up having lying under some AES-256 encryption in your SSD.

Don’t worry about reading technical books, contributing to open source software, listening to podcasts, taking online tech courses, or even having some personal passion project at home. Those things are for sweaty geniuses, autists, and future mass shooters.

You know better than that.

The Four Commandments for Keeping Your Team Out of Integration Hell

“Michael: There’s nothing here to fear.
Lucifer: Well, there’s always the truth.”

-Mike Carey

Source control can be hard. Source control can be terrifying. And worst of all, source control can be a real Biblical hell.

For those beautiful souls fortunate enough to have never touched upon the issue, allow me to briefly explain:

Integration Hell refers to the point in production when members on a delivery team integrate their individual code.

In traditional software development environments, this integration process is rarely smooth and seamless, instead resulting in hours or perhaps days of fixing the code so that it can finally integrate.

Source

By now, Git and Git-Flow (if you’re not using it, you undeniably deserve to burn) are probably the most popular source control tools on the planet. You’re likely at least familiar with them.

But tools and workflows can only be as good as the people using them.

And if it’s true that C can make it easy to shoot yourself in the foot, Git can sometimes make suicide seem like a more appealing proposition than your last three failed relationships.

So, in order to fulfill my divine duty and save you from the lake of fire, I bring you four commandments that will keep you and your team on the pathway to Heaven and good Fortune:

I. Commit Early, and Commit Often

Each developer should be able to break down his work into simple, logical chunks suited to a small commit; and should commit those changes as soon as possible.

II. If It’s Not Pushed, It’s Not Done

Each commit should be pushed into the remote repository.

Hard drives can fail, programs can crash, IT can be presumed to be incompetent, and mistakes will be made.

If your work hasn’t been pushed, it hasn’t been done. Period.

III. Use It or Lose It

Each unit of work should be defined as an individual feature branch diverging from a main development branch.

Your work should be integrated within the main development branch or branches in no longer than a certain time period. It is up to you and your team to determine that time period based on the scale of work that your repository usually handles.

But normally I’d recommend as frequently as each day and as infrequently as each week.

This means that each development task should be intelligently designed and broken down into many small features (branches) in order to keep the work continually integrated with the main codebase.

If one of the features needs to exceed this given amount of time, especially see the next commandment.

IV. Do the Pull, or Be a Fool

As a developer, it’s your responsibility to make sure your work not just works, but also plays nice with everyone else’s work.

That means that if your feature branch has been isolated from the main branches for longer than reasonable, you have to pull changes from the development branches via a merge or a rebase in order to make your branch usable.

Failing to do so can only guarantee that things will break down the line, and make using the codebase a foolish thing to do.

Don’t be that person.

Making a 4chan Twitter Bot with Groovy in 8 Easy Steps

First off, if you’re reading this you’re probably already wondering: why would anyone want to make such a thing?

“Just what is wrong with you?” might be one of the thoughts racing through your head right now. And I understand, but I’ll ask you to suspend your disbelief if only for a minute and indulge me for one moment.

I Promise I’m Not Evil

As it stands, 4chan is one of the most unique places on the Internet, with a long and rich history of angst-fueled mischief, wildly irreverent humor, unabashedly offensive locker-room banter, the occasional problem with law enforcement, and much, much more.

However, more importantly, I believe it has also acted as an incubator for mainstream Internet culture, providing us with countless memes, phrases, and attitudes that either directly or indirectly shape the sort of behavior we can expect to find both online and offline.

Therefore, and especially for those of us with a fascination for culture, wouldn’t it be interesting to reach into the “brain” of this dynamic, anonymous hive mind and grab just one momentary thought?

Wouldn’t it be worth it to sift through all the erratic noise to find one genuinely thought-provoking gem of commentary? Or… at least, a sincerely silly one!

That’s where the idea for this project came from.

And if you just want to see the real-life application of it, hop on over to Twitter to take a look at @r9k_b, a 4chan Twitter Bot that runs on this very same source code to tweet original user-content from 4chan’s /r9k/ board.

Getting Started

Now, without further ado, let’s dive into the code. With this, you’ll be able to make your own Twitter Bot to pull tweetable commentary from any 4chan board you’re interested in.

You’ll be using the Groovy programming language to write a simple .groovy one-off script. If you already know Java, you basically already know Groovy. If not, now might be a great time to get familiar with it.

Groovy is extremely fun, flexible, and productive to use. Plus, it will grant you access to the entire Java ecosystem of tools and libraries.

If you’re not interested in the process of building the bot, you can go on ahead and view the complete source code directly at the r9k_b GitHub repo.

What You’ll Need

  1. An Installation of the JDK and the Groovy SDK
  2. A Twitter Account
  3. The 4chan API
  4. The Jsoup Library
  5. The Twitter4j Library

First Things First

Before you attempt to proceed with making a Twitter Bot, you first need to register for Twitter’s API. Make sure your account has a valid phone number (for account activation) associated with it, otherwise you will not be able to use it as a bot.

To achieve this you must:

  1. Visit the Twitter Apps page.
  2. Navigate and click on the “Create New App” button.
  3. Complete the application form.
  4. Move on with the “Create Your Twitter Application” button.
  5. Go to the tab that reads “Keys and Access Tokens”
  6. Click on “Create My Access Tokens” at the bottom.

Now, take note of the following:

  • The Consumer Key (API Key)
  • The Consumer Secret (API Secret)
  • The Access Token
  • The Access Token Secret

With these values on hand, create a twitter4j.properties file in the same directory as your Groovy script. Paste in the following code in the file and fill out the values with your own:

oauth.consumerKey=Your_Consumer_Key
oauth.consumerSecret=Your_Consumer_Secret
oauth.accessToken=Your_Access_Token
oauth.accessTokenSecret=Your_Access_Token_Secret

What You’ll Be Doing

  1. Using Grape annotations to quickly and easily add your library dependencies to your script.
  2. Making a 4chan API request for the catalog of active threads in a board of your choosing.
  3. Parsing the JSON response.
  4. Iterating over the catalog in order to store the thread IDs for future retrieval of each thread’s user posts.
  5. Making another 4chan API request for each of the stored threads.
  6. Parsing the JSON response.
  7. Iterating over each of the parsed threads in order to store the “tweetable” (length <= 140) user posts/comments.
  8. Picking and tweeting a stored user comment to your account.

Step 1: Adding Imports and Dependencies

This code snippet contains all you need to get started with writing the actual code in your script. It uses Grape to fetch the .jar files for you, so you don't have to worry about dependency management and you can focus on writing your code:

import groovy.json.JsonSlurper
import java.net.URL
import java.text.SimpleDateFormat

@Grab(group='org.twitter4j', module='twitter4j-core', version='4.0.5')
import twitter4j.Status
import twitter4j.Twitter
import twitter4j.TwitterException
import twitter4j.TwitterFactory

@Grab(group='org.jsoup', module='jsoup', version='1.9.2')
import org.jsoup.Jsoup

Steps 2-4: Retrieving Catalog and Thread Board Data

These steps require some underlying knowledge of the 4chan API and the intuitive Groovy JsonSlurper class. It is worth mentioning that the left shift operator is overloaded to support addition of elements into a List object.

As implied in the beginning, this example will use the /r9k/ board as a data source.

final String BOARD_NAME = "/r9k/"
final String THREAD_CATALOG_REQUEST_URL = "https://a.4cdn.org" + BOARD_NAME + "threads.json"

println "Retrieving " + BOARD_NAME + " data..."

threadCatalogData = new URL(THREAD_CATALOG_REQUEST_URL).getText() // Makes the HTTP request.
List threadCatalog = new JsonSlurper().parseText(threadCatalogData) // Parses JSONObjects into Maps and JSONArrays into Lists.

// Store all active threads from the board.
listOfThreads = []
numOfPages = threadCatalog.size()
numOfPages.times { i ->
    Map catalogPage = threadCatalog.get(i)
    List threadsInPage = catalogPage.threads
    numOfThreadsInPage = threadsInPage.size()
    numOfThreadsInPage.times { j -> listOfThreads << threadsInPage.get(j).no }
}

Steps 5-7: Retrieving Tweetable User Comments

These steps are very similar to the previous three. Additionally, you will be using a different method to loop for a specified number of iterations. You can find out more about these and other methods used to loop over stuff in the Groovy language here.

listOfComments = []

// Retrieve all posts from 20% of the most recent threads in the board catalog.
1.upto(listOfThreads.size() / 5) { i ->
    chosenThreadNo = listOfThreads.get(i)
    threadPageRequestUrl = "https://a.4cdn.org" + BOARD_NAME + "thread/" + 
        chosenThreadNo + ".json"

    threadPageData = new URL(threadPageRequestUrl).getText()
    Map threadPage = new JsonSlurper().parseText(threadPageData)

    List posts = threadPage.posts

    // Grab all the thread comments of Twitter-able length (with no URLs or web links).
    final int CHARACTER_LENGTH = 140
    numOfPosts = posts.size()
    numOfPosts.times { j ->
        Map post = posts.get(j)
        if (post.com != null) {
            comment = Jsoup.parse(post.com).text() // Removes HTML from comment.
            if (comment.length() <= CHARACTER_LENGTH && !comment.contains("www") 
                && !comment.contains("http"))
                listOfComments << comment
        }
    }

    Thread.sleep(2000) // Respect the 4chan API rules.
}

Step 8: Posting the Tweet

// Choose a comment for Twitter posting.
    chosenCommentIndex = new Random().nextInt(listOfComments.size())
    chosenComment = listOfComments.get(chosenCommentIndex)

    twitter = TwitterFactory.getSingleton()
    twitter.updateStatus(chosenComment)

    Date date = new Date()
    SimpleDateFormat today = new SimpleDateFormat("EEE, d MMM yyyy HH:mm:ss Z")
    println today.format(date) + " - Updated Twitter status with: " + chosenComment

    // Wait for a while (15 mins) until the next tweet.
    Thread.sleep(900000)

Final Step: Putting It All Together

To finish off the script, we wrap all the code logic inside a while loop that continually executes until we decide to close down the console:

import groovy.json.JsonSlurper
import java.net.URL
import java.text.SimpleDateFormat

@Grab(group='org.twitter4j', module='twitter4j-core', version='4.0.5')
import twitter4j.Status
import twitter4j.Twitter
import twitter4j.TwitterException
import twitter4j.TwitterFactory

@Grab(group='org.jsoup', module='jsoup', version='1.9.2')
import org.jsoup.Jsoup

while(true) {
    final String BOARD_NAME = "/r9k/"
    final String THREAD_CATALOG_REQUEST_URL = "https://a.4cdn.org" + BOARD_NAME + "threads.json"

    println "Retrieving " + BOARD_NAME + " data..."

    threadCatalogData = new URL(THREAD_CATALOG_REQUEST_URL).getText() // Makes the HTTP request.
    List threadCatalog = new JsonSlurper().parseText(threadCatalogData) // Parses JSONObjects into Maps and JSONArrays into Lists.

    // Store all active threads from the board.
    listOfThreads = []
    numOfPages = threadCatalog.size()
    numOfPages.times { i ->
        Map catalogPage = threadCatalog.get(i)
        List threadsInPage = catalogPage.threads
        numOfThreadsInPage = threadsInPage.size()
        numOfThreadsInPage.times { j -> listOfThreads << threadsInPage.get(j).no }
    }

    listOfComments = []

    // Retrieve all posts from 20% of the most recent threads in the board catalog.
    1.upto(listOfThreads.size() / 5) { i ->
        chosenThreadNo = listOfThreads.get(i)
        threadPageRequestUrl = "https://a.4cdn.org" + BOARD_NAME + "thread/" + 
            chosenThreadNo + ".json"

        threadPageData = new URL(threadPageRequestUrl).getText()
        Map threadPage = new JsonSlurper().parseText(threadPageData)

        List posts = threadPage.posts

        // Grab all the thread comments of Twitter-able length (with no URLs or web links).
        final int CHARACTER_LENGTH = 140
        numOfPosts = posts.size()
        numOfPosts.times { j ->
            Map post = posts.get(j)
            if (post.com != null) {
                comment = Jsoup.parse(post.com).text() // Removes HTML from comment.
                if (comment.length() <= CHARACTER_LENGTH && !comment.contains("www") 
                    && !comment.contains("http"))
                    listOfComments << comment
            }
        }

        Thread.sleep(2000) // Respect the 4chan API rules.
    }

    // Choose a comment for Twitter posting.
    chosenCommentIndex = new Random().nextInt(listOfComments.size())
    chosenComment = listOfComments.get(chosenCommentIndex)

    twitter = TwitterFactory.getSingleton()
    twitter.updateStatus(chosenComment)

    Date date = new Date()
    SimpleDateFormat today = new SimpleDateFormat("EEE, d MMM yyyy HH:mm:ss Z")
    println today.format(date) + " - Updated Twitter status with: " + chosenComment

    // Wait for a while (15 mins) until the next tweet.
    Thread.sleep(900000)
}

Now that you've got the whole script, save it (if you haven't already) as filename.groovy. You can execute it with a groovy filename command or simply give it a double click to run if you installed the Groovy SDK properly.

Congratulations! You now have your very own operational 4chan Twitter Bot.

Now sit back, run the script, and be prepared to be made very uncomfortable, repeatedly.

Update: The original r9k_b script has been Groovy'fied by none other than GR8Conf founder Søren Berg Glasius. You can view the updated script over in the main branch of the r9k_b repo.

A Good Time For A Welcome

Mian Labs is now officially an established brand. It is no longer an idea, but a living, breathing piece of technology with a hold on my brain that is uniquely its own. And it cares only for one thing: good software.

It exists to secure the future of better code. And I exist to aid it in that mission. Follow us along while we reach our initial goal of making the Android platform our home for inventive and memorable experiences; we all might just learn quite a lot.

I plan to welcome the coming of Mian Labs with its first software release, an Android application called Pkdx (pronounced “Pikadex”).

pkdx_p1

Pkdx will offer its users the chance to board a vessel in time back to the simpler, older days of the Pokémon franchise, where catching Pokémon was accompanied by a sense of childish wonder and amazement, delivered by a technology powered by only two AA batteries.

A place where the menus were cluttered and the text was cramped, and yet for all of our modern design today we still can’t quite feel the same endearment whenever we tap to accept another friend request or click to ‘like’ whatever sort of media bit popped up on our news feed.

I believe that these sorts of meaningful experiences, even more so than the code that powers them, is what Mian Labs is about. It is not only my brand, my label, and perhaps later on even something more, but it is also my humanity – put forth to you: the reader, the user, and the friend.

You will see more about it and the tech behind it as it is released.

Update: Pkdx has been released. You can get it here.