Programming Analytics

Why did x86 never die?

Tue, 05 Mar 2013 15:29:00 -0500

This article on ITWorld about potential x86 replacements proposes that the reason x86 remained superior is that it had established software and that the potential replacements lacked compilers.

Balderdash.

In addition to the potential replacements Intel offered, the market offered hundreds of other possible replacements to the x86, many with phenomenal compilers. MIPS, PowerPC, DEC Alpha, and tons of others all offered excellent performance (I won’t list SPARC because SPARC was in fact a terrible design and remains one to this day).

What kept Intel’s x86 line on top of the heap was Moore’s law. Throughout the 1990s, everyone always knew they could switch to a different chip and see a 50% performance increase immediately. However, they also knew, by Moore’s law, that they could wait six months and get a 50% performance increase by buying the latest Intel chip six months in the future.

Even better, Intel kept playing a phenomenal game of catch-up. Whatever feature competitive CPU architectures built, Intel built into the next version of x86. The P6 CPU was, in fact, a very advanced RISC chip with a nifty-quick microcode decoder attached to the front of its pipeline. So Intel’s chips rapidly caught up the 50% performance gap.

So everyone had two options: Buy a replacement platform now and rebuild your code, or wait six months and get equivalent performance with no rebuilding. I know which choice I made back then.

Keep in mind that buying a replacement platform was often 10x more expensive, since none of the alternate CPUs available at the time had sufficient volume to keep prices down. So if your choice was between buying a $10,000 DEC Alpha and buying a $3000 Pentium Pro with half the performance, … you’d buy a Pentium Pro, wait two years, and buy another one.

Getting back to today - the reason Intel’s x86 is finally threatened is because, due to Android smartphones, ARM chips now have sufficient production volume and performance demands. Chipmakers are competing to create the highest performing ARM chips, and they’re producing millions of them at a time, so they can afford to invest in performance boosts.

Intel can keep reducing the power usage of x86 to compete, but this is a different type of challenge, and I honestly think x86 chips will not always dominate against ARM.

Programming is like building a house?

Fri, 01 Feb 2013 12:20:00 -0500

I saw this article today which compares programming to building a house. It’s a noble sentiment.

However, one of the advantages of having (I can hardly believe it) 22 years of professional software development expertise is that I have seen trends like this come and go. Every few years, this theme of “Programming should be engineering!” resurfaces. And afterwards, even though some progress was made, all goes back to the way it was before, and the companies that do “release-early, release-often” win out over the companies that do rigorous design.

Why is this?

Programming is, unfortunately to say, not like engineering in the way we wish it was.

When you are designing a house, or designing a bridge, or any other gigantic physical work, you have complete knowledge of your materials available. If you say that you wish this roof to be held up by two steel beams or by a wooden frame, you can look up the quality of the materials and know exactly how it works.

When you are doing mechanical engineering, physically changing something has a known time and cost and often is impossible once a system is built. In software engineering, changing a component is virtually costless.

In programming, every detail is just as unique as the overall design of the application itself. Every detail can suffer from the same problems that can cause the program as a whole to fail.

I think it is much more fair to compare software to law. A programmer is like a lawyer. They have rules to follow and a flexible set of limits to push against. But at every step of the way, the rules and the limits are subject to interpretation. At every step of the way, the results of something you thought you understood can surprise you.

Managing Receivables

Sun, 16 Dec 2012 15:34:00 -0500

Saw an interesting discussion on Slashdot today and thought I’d share some of my personal experience about receivables. I’ve been involved with lots of startups and this is a common occurrence.

Delayed payment is normal. Large companies have very complex rules about how to make payments and how to process invoices. You must be extremely persistent and gracious in order to get things resolved. Each company will respond differently, but I encourage you to make use of some (if not all) of these following tactics:

Get a “Master Vendor Agreement” in place with the customer that states invoicing terms. This contract may take months to negotiate and require guidance from a lawyer. Once this is done, all of your projects should be addendums to this original master vendor agreement. This reduces the amount of paperwork the large multinational company has to do to validate each of your invoices and speeds them up.

Provide both a discount for early payment and a penalty for late payment. Annotate these discounts & payments on each invoice. If you carefully track your effort, you can know how much it costs you to track long term overdue payments. You can use this to determine how much of a discount you can offer for prompt payment.

Designate someone within your company as the “Accounts Receivable” person. It is their job to contact each customer with an overdue payment once per month (or week). They should very carefully take notes on all of their conversations and correspondence, but they _must_ be friendly and relaxed. The goal is to establish a positive rapport with the “accounts payable” person on the other side. It may take dozens of polite phonecalls to get routed to the correct person though, so you absolutely must be willing to put in the effort while not creating bad will.

Be gracious when payment is offered. Many times, companies may refuse to pay late payment fees; you can simply say, “I’ll remove the late payment fee if you wire the money by tomorrow”.

If desired, you can contact your bank to find out if they will finance your receivables. Some banks will provide you with cash up front for a fee since they know how this process works.

Don’t harass your point of contact until the invoice is more than a reasonable amount late. Generally, in a big company, the person who signs the contract doesn’t even know the person who actually pays the bill. You want to avoid harassing your point of contact (who is usually your biggest fan) until you really need their help getting the bill paid.

Know your customers’ “approval limits”. Generally, executives at a large company will have specific approval levels - for maybe $500 they can simply file an expense report; for $2500 they have to file one form with one signature, and for more than that they have to get approval from a VP level person. If you can keep your projects small enough, you can bypass some of the challenges.

Once you’ve read lots of advice on slashdot and picked a strategy, contact a lawyer before doing anything. Most lawyers will be able to confirm whether your plans follow the law quickly. It’ll only cost you a small amount.

And finally, remember, “managing receivables” is part of the cost of doing business with large companies. Factor it into your project costs.

Tinting Buttons in WPF

Thu, 06 Dec 2012 19:33:00 -0500

While working on the UI of a WPF application, someone told me, “If you’re going to change the background color of the form, why not re-tint all the buttons to match it?”

So I thought I should write a tiny piece of code that takes as input a brush, lightens it by 50%, and assigns that to all controls of a certain type. Here’s how to do it:

private void RecolorChildren(DependencyObject dep, System.Windows.Media.Brush b)
{
    for (int i = 0; i < VisualTreeHelper.GetChildrenCount(dep); i++) {
        DependencyObject child = VisualTreeHelper.GetChild(dep, i);
        if (child is System.Windows.Controls.Button) {
            System.Windows.Controls.Button button = child as System.Windows.Controls.Button;
            button.Background = b;
        } else if (child is System.Windows.Controls.ComboBox) {
            System.Windows.Controls.ComboBox cbx = child as System.Windows.Controls.ComboBox;
            cbx.Background = b;
        } else {
            RecolorChildren(child, b);
        }
    }
}

To tint a brush:

private static System.Windows.Media.Brush LightenBrush(System.Windows.Media.Brush b, int percent)
{
    SolidColorBrush scb = b as SolidColorBrush;
    System.Windows.Media.Color c2 = scb.Color;
    c2.B = (byte)(c2.B + ((double)(255 - c2.B) / 100.0 * percent));
    c2.G = (byte)(c2.G + ((double)(255 - c2.G) / 100.0 * percent));
    c2.R = (byte)(c2.R + ((double)(255 - c2.R) / 100.0 * percent));
    b = new SolidColorBrush(c2);
    return b;
}

AltDev Student Summit 2012

Sun, 18 Nov 2012 14:50:26 -0500

AltDev Student Summit 2012:

My presentation at the AltDev Student Summit was about what new graduates should expect from the games industry. It’s a modified version of the presentation I gave to the San Diego State University ACM - http://acm.sdsu.edu/ - last month.

This is a very light presentation with some interesting stories about the games industry.

New Article: Fix Your Pebbles

Sun, 21 Oct 2012 20:31:01 -0400

Just recently published my latest article, this time about my programming philosophy. This article talks about how I teach my team members to deliver results and apply their intelligence to problems.

http://www.altdevblogaday.com/2012/10/21/fix-your-pebbles/

New Article: The Elements of Comment Style

Wed, 19 Sep 2012 20:24:11 -0400

I wrote a new article for AltDevBlogADay regarding the use of comments in programming. I’m rather happy with the way this article turned out; I find that the research and categorization I did when reading through my past work was quite useful in understanding how I use comments.

Hopefully all of you will find the article fun reading too!

SSIS

Thu, 30 Aug 2012 15:13:09 -0400

Never before in the history of mankind have so many connection strings been erroneously reset by such a simple project.

With apologies to Winston Churchill.

Open source database schema comparisons!

Tue, 21 Aug 2012 10:53:21 -0400

I’ve really missed my old Red Gate SQL Compare product - I’ve been using it for years and it was a fantastic tool. But for working on independent projects, I just didn’t want to pay for a personal use license. So that’s why I’m excited to find this project:

http://code.google.com/p/sql-dbdiff/

Despite the awkward user interface, this program does appear to get a thorough database schema from a Microsoft SQL server, and it appears to be fully capable of doing a difference analysis and creating a changescript. With a bit of cleanup, this program would be a fantastic replacement!

As a note, it looks like it’s a fork of this program:

http://opendbiff.codeplex.com/

I’ll need to do some more analysis to figure out which one is a more appealing start. However, they both appear to have ended development sometime in 2010 - I wonder why?

Type Conversions!

Fri, 17 Aug 2012 15:09:56 -0400

A far better way of writing “string-to-type” conversion is here:

http://www.hanselman.com/blog/TypeConvertersTheresNotEnoughTypeDescripterGetConverterInTheWorld.aspx

Here’s the terrible thing about C# - it changes and grows so fast there’s tons of fantastic stuff in it that often slips below the radar. This is by far the best little type conversion kit I’ve seen!

Automated SSIS Decompiler

Tue, 14 Aug 2012 00:40:27 -0400

I’ve started an open source project to create an Automated SSIS Decompiler. This program is so far capable of reading in a DTSX package and writing, as output, a C# source code file and an accompanying configuration file. I’ll continue to develop this program as long as it helps me research my current administration / maintenance process; anyone who is interested in providing assistance will be welcome to join in!

SSIS is not a useful product.

Fri, 10 Aug 2012 20:01:37 -0400

Read this article today while researching ways to migrate an SSIS package into a C# program. It would really be neat if there was a way to programmatically convert an SSIS abomination into a readable, intelligible C# / .NET program. Maybe that will be a useful project ;)

http://ayende.com/blog/2637/ssis-15-faults

So here’s the thing: why is SSIS bad? Honestly, when I used to use Data Transformation Services back in the late ’90s, there was nothing wrong with it. DTS was a simple way to automate a complex, bulk data transformation process. It was limited, but that was okay - because it had a specific function. It imported and exported data to and from SQL server.

Nowadays SSIS supports everything and the kitchen sink. What used to be straightforward (Import -> Select source -> Select columns -> Go!) is now a massive undertaking. I thought it was really neat how DTS had the ability to save a package so you could repeat it again in the future. If I’d have known that it would become this complex and impenetrable, I would have walked away.

Introduction to Regression Analysis

Fri, 10 Aug 2012 01:50:54 -0400

I’ve been teaching people how to use basic regression for a while now, and I finally wrote it all up as an article here:

http://www.altdevblogaday.com/2012/08/10/business-analytics-with-regression/

There are lots of tools available that are far more complex than you’d ever need, but fortunately there are open source tools that can really help you get started smoothly. Good luck!

C# Command Line Wrapper Library

Wed, 08 Aug 2012 14:11:57 -0400

So I got tired of writing and rewriting the same command line user interface over and over again.

I have lots of programs that I write to be “automatable” - i.e. so that an operations team member can put it into a script and execute it, or schedule it, or batch it up.

Rather than continue rewriting this stuff over and over, I wrote a library that accomplishes universal execution automation via reflection:

http://code.google.com/p/csharp-command-line-wrapper/

You can view the code here, and drop it into your project at any time by copying and pasting this code:

http://code.google.com/p/csharp-command-line-wrapper/source/browse/trunk/CommandWrapper/CommandWrapLib/CommandWrapLib.cs

It produces nice clean output and parses your variables on the command line.

JPL was able to snap a picture of Curiosity as it landed

Mon, 06 Aug 2012 16:39:42 -0400

This is truly the coolest photo I’ve seen in a while. Nice camera work. ;)

http://www.nasa.gov/mission_pages/msl/news/msl20120806b.html

C# Library for Automating VMWare

Mon, 06 Aug 2012 15:17:00 -0400

This looks surprisingly useful! Why not have an Amazon-style scripted toolset for deploying images to your internal cloud?

http://www.codeproject.com/Articles/31961/Automating-VMWare-Tasks-in-C-with-the-VIX-API

This would have been fantastic back when we were deploying virtual GamePulse images every week.

QuickDraw Source Code

Fri, 27 Jul 2012 16:21:26 -0400

Ever wonder how the Mac was possible? Now you can actually download the 68000 assembly source code for Bill Atkinson’s QuickDraw, one of the most impressive toolkits you never really thought about:

http://www.computerhistory.org/highlights/macpaint/

Your Task "Inventory"

Fri, 27 Jul 2012 13:20:17 -0400

Joel Spolsky wrote a cool article on the problems with letting your programming backlog get too big.

http://www.joelonsoftware.com/items/2012/07/09.html

Over the past five years I suffered heavily from this. There were always hundreds of bugs being reported; there we always years worth of backlog programming ideas in the queue. At the time, I felt it was useful to have a massive backlog of ideas.

But of course, at the same time, we kept our programming velocity up: We had a daily scrum process and we identified the most promising changes that could get into our codebase right now. Because of that, we were able to deliver incredibly rapid change and keep our product at the top of the market even when we had fewer programmers than any of our competitors.

I like to think of it as “banking” your changes. Any code you write is ephemeral until it is launched. As you work, you build up lots of potential improvements. But if you don’t deploy your code - real artists ship - your changes never benefit anyone. We kept up our velocity by shipping all changesets that were ready, deploying code to live about once every three weeks.

About Year-Week Dates

Fri, 27 Jul 2012 12:59:00 -0400

I did a lot of work in sales data processing. In America, sales data is reported monthly; and months are handled in one of two ways: either as a calendar month or as a NRF 4-5-4 calendar month, which is used to make year-over-year month comparisons consistent. Basically, the NRF calendar allows you to compare January 2011 to January 2012 in such a way that each month contains the same number of weekdays and weekend days.

When I started working with European data, I was overjoyed to discover that European data was reported weekly. Even better, all the weeks appeared to start on Sunday. I could simply load data from all the European countries, who provided for each data point the year and week numbers. I wrote a tiny function in my code that took these two numbers and produced a date, and a separate function that did the opposite. Here’s what they looked like:

-- Determine the week number of this date
SELECT DATEPART(week, GETDATE())

-- Starting from January 1st, backtrack to Sunday; then add 29 weeks 
-- to get the start date for week 30
SELECT DATEADD(week, 29, 
    DATEADD(day, 
    1-(SELECT DATEPART(weekday, '2012-01-01')), '2012-01-01'))

This worked great until I started receiving data from provider A which said “Week 53 of 2008” whereas the same data from provider B said “Week 1 of 2009”. All my carefully constructed logic blew up. Because these providers would not explain what algorithm they used to calculate their year-week numbers, I had to infer their logic from how their calendars changed. I wrote this post to hopefully explain to you the challenge involved in using year-week numbers; I’m sorry it’s not more helpful.

Here are some of the definitions I have seen used:

Weeks start Sunday OR weeks start Monday.
The first week that contains January 1st of year X is considered week 1 of year X OR the first week whose first day starts after January 1st of year X is considered week 1 of year X.
In some cases, I’ve worked with some data companies that worked on a “balance of days” principle, where the first week that has three or more days within year X is considered week 1 of year X.

With this in mind, you can determine year-week numbers in any way you want. Just remember that the numbers are not standardized unless you publish your definition of how you calculate year week numbers.

C# CSV Reader

Fri, 27 Jul 2012 01:05:59 -0400

Code that reads data from comma-separated files is quite tricky. Because most beginners simply split on the presence of a comma, and because this mostly works, it’s easy to get started with CSV files - but actually quite hard to perfect them.

I wrote a library back in 2006 to parse these files, and have been modifying this for a long time. I just realized it’s really silly of me not to open source it. So here goes:

https://code.google.com/p/csharp-csv-reader/

Enjoy!