Programming Analytics

A feed of interesting tidbits from IT, software engineering, business intelligence, and videogaming.

So this was surprisingly tricky. We have hundreds (thousands) of PDF forms, and we wanted to scan them quickly to see what fields were editable in each one. The code winds up looking like this:

using (var pdfReader = new PdfReader(infile)) {
    foreach (var f in pdfReader.AcroFields.Fields) {
        var n = f.Value.GetMerged(0).GetAsNumber(PdfName.FF);
        if (n != null && 
            ((n.IntValue & (int)PdfFormField.FF_READ_ONLY) > 0)) {
            // Field is read only
        } else {
            // field is editable
        }
    }
}

Once you’ve done the hard work of determining which fields are editable, you can write out a spreadsheet listing names of fields and so on by using “f.key” (this is the name of the field).

Keep in mind that PDF logic is a little bit unusual. All instances of the same named field have the same properties, whether they appear on multiple pages or a single page. So if the same field repeats multiple times, you’ll have to decide how you want to handle this.

This article on ITWorld about potential x86 replacements proposes that the reason x86 remained superior is that it had established software and that the potential replacements lacked compilers.

Balderdash.

In addition to the potential replacements Intel offered, the market offered hundreds of other possible replacements to the x86, many with phenomenal compilers. MIPS, PowerPC, DEC Alpha, and tons of others all offered excellent performance (I won’t list SPARC because SPARC was in fact a terrible design and remains one to this day).

What kept Intel’s x86 line on top of the heap was Moore’s law. Throughout the 1990s, everyone always knew they could switch to a different chip and see a 50% performance increase immediately. However, they also knew, by Moore’s law, that they could wait six months and get a 50% performance increase by buying the latest Intel chip six months in the future.

Even better, Intel kept playing a phenomenal game of catch-up. Whatever feature competitive CPU architectures built, Intel built into the next version of x86. The P6 CPU was, in fact, a very advanced RISC chip with a nifty-quick microcode decoder attached to the front of its pipeline. So Intel’s chips rapidly caught up the 50% performance gap.

So everyone had two options: Buy a replacement platform now and rebuild your code, or wait six months and get equivalent performance with no rebuilding. I know which choice I made back then.

Keep in mind that buying a replacement platform was often 10x more expensive, since none of the alternate CPUs available at the time had sufficient volume to keep prices down. So if your choice was between buying a $10,000 DEC Alpha and buying a $3000 Pentium Pro with half the performance, … you’d buy a Pentium Pro, wait two years, and buy another one.

Getting back to today - the reason Intel’s x86 is finally threatened is because, due to Android smartphones, ARM chips now have sufficient production volume and performance demands. Chipmakers are competing to create the highest performing ARM chips, and they’re producing millions of them at a time, so they can afford to invest in performance boosts.

Intel can keep reducing the power usage of x86 to compete, but this is a different type of challenge, and I honestly think x86 chips will not always dominate against ARM.

I saw this article today which compares programming to building a house. It’s a noble sentiment.

However, one of the advantages of having (I can hardly believe it) 22 years of professional software development expertise is that I have seen trends like this come and go. Every few years, this theme of “Programming should be engineering!” resurfaces. And afterwards, even though some progress was made, all goes back to the way it was before, and the companies that do “release-early, release-often” win out over the companies that do rigorous design.

Why is this?

Programming is, unfortunately to say, not like engineering in the way we wish it was.

When you are designing a house, or designing a bridge, or any other gigantic physical work, you have complete knowledge of your materials available. If you say that you wish this roof to be held up by two steel beams or by a wooden frame, you can look up the quality of the materials and know exactly how it works.

When you are doing mechanical engineering, physically changing something has a known time and cost and often is impossible once a system is built. In software engineering, changing a component is virtually costless.

In programming, every detail is just as unique as the overall design of the application itself. Every detail can suffer from the same problems that can cause the program as a whole to fail.

I think it is much more fair to compare software to law. A programmer is like a lawyer. They have rules to follow and a flexible set of limits to push against. But at every step of the way, the rules and the limits are subject to interpretation. At every step of the way, the results of something you thought you understood can surprise you.

Saw an interesting discussion on Slashdot today and thought I’d share some of my personal experience about receivables. I’ve been involved with lots of startups and this is a common occurrence.

Delayed payment is normal. Large companies have very complex rules about how to make payments and how to process invoices. You must be extremely persistent and gracious in order to get things resolved. Each company will respond differently, but I encourage you to make use of some (if not all) of these following tactics:

  • Get a “Master Vendor Agreement” in place with the customer that states invoicing terms. This contract may take months to negotiate and require guidance from a lawyer. Once this is done, all of your projects should be addendums to this original master vendor agreement. This reduces the amount of paperwork the large multinational company has to do to validate each of your invoices and speeds them up.

  • Provide both a discount for early payment and a penalty for late payment. Annotate these discounts & payments on each invoice. If you carefully track your effort, you can know how much it costs you to track long term overdue payments. You can use this to determine how much of a discount you can offer for prompt payment.

  • Designate someone within your company as the “Accounts Receivable” person. It is their job to contact each customer with an overdue payment once per month (or week). They should very carefully take notes on all of their conversations and correspondence, but they _must_ be friendly and relaxed. The goal is to establish a positive rapport with the “accounts payable” person on the other side. It may take dozens of polite phonecalls to get routed to the correct person though, so you absolutely must be willing to put in the effort while not creating bad will.

  • Be gracious when payment is offered. Many times, companies may refuse to pay late payment fees; you can simply say, “I’ll remove the late payment fee if you wire the money by tomorrow”.

  • If desired, you can contact your bank to find out if they will finance your receivables. Some banks will provide you with cash up front for a fee since they know how this process works.

  • Don’t harass your point of contact until the invoice is more than a reasonable amount late. Generally, in a big company, the person who signs the contract doesn’t even know the person who actually pays the bill. You want to avoid harassing your point of contact (who is usually your biggest fan) until you really need their help getting the bill paid.

  • Know your customers’ “approval limits”. Generally, executives at a large company will have specific approval levels - for maybe $500 they can simply file an expense report; for $2500 they have to file one form with one signature, and for more than that they have to get approval from a VP level person. If you can keep your projects small enough, you can bypass some of the challenges.

  • Once you’ve read lots of advice on slashdot and picked a strategy, contact a lawyer before doing anything. Most lawyers will be able to confirm whether your plans follow the law quickly. It’ll only cost you a small amount.

  • And finally, remember, “managing receivables” is part of the cost of doing business with large companies. Factor it into your project costs.

    While working on the UI of a WPF application, someone told me, “If you’re going to change the background color of the form, why not re-tint all the buttons to match it?”

    So I thought I should write a tiny piece of code that takes as input a brush, lightens it by 50%, and assigns that to all controls of a certain type. Here’s how to do it:

    private void RecolorChildren(DependencyObject dep, System.Windows.Media.Brush b)
    {
        for (int i = 0; i < VisualTreeHelper.GetChildrenCount(dep); i++) {
            DependencyObject child = VisualTreeHelper.GetChild(dep, i);
            if (child is System.Windows.Controls.Button) {
                System.Windows.Controls.Button button = child as System.Windows.Controls.Button;
                button.Background = b;
            } else if (child is System.Windows.Controls.ComboBox) {
                System.Windows.Controls.ComboBox cbx = child as System.Windows.Controls.ComboBox;
                cbx.Background = b;
            } else {
                RecolorChildren(child, b);
            }
        }
    }
    

    To tint a brush:

    private static System.Windows.Media.Brush LightenBrush(System.Windows.Media.Brush b, int percent)
    {
        SolidColorBrush scb = b as SolidColorBrush;
        System.Windows.Media.Color c2 = scb.Color;
        c2.B = (byte)(c2.B + ((double)(255 - c2.B) / 100.0 * percent));
        c2.G = (byte)(c2.G + ((double)(255 - c2.G) / 100.0 * percent));
        c2.R = (byte)(c2.R + ((double)(255 - c2.R) / 100.0 * percent));
        b = new SolidColorBrush(c2);
        return b;
    }
    

    My presentation at the AltDev Student Summit was about what new graduates should expect from the games industry. It’s a modified version of the presentation I gave to the San Diego State University ACM - http://acm.sdsu.edu/ - last month.

    This is a very light presentation with some interesting stories about the games industry.

    Just recently published my latest article, this time about my programming philosophy. This article talks about how I teach my team members to deliver results and apply their intelligence to problems.

    http://www.altdevblogaday.com/2012/10/21/fix-your-pebbles/

    I wrote a new article for AltDevBlogADay regarding the use of comments in programming. I’m rather happy with the way this article turned out; I find that the research and categorization I did when reading through my past work was quite useful in understanding how I use comments.

    Hopefully all of you will find the article fun reading too!

    Never before in the history of mankind have so many connection strings been erroneously reset by such a simple project.

    With apologies to Winston Churchill.

    I’ve really missed my old Red Gate SQL Compare product - I’ve been using it for years and it was a fantastic tool. But for working on independent projects, I just didn’t want to pay for a personal use license. So that’s why I’m excited to find this project:

    http://code.google.com/p/sql-dbdiff/

    Despite the awkward user interface, this program does appear to get a thorough database schema from a Microsoft SQL server, and it appears to be fully capable of doing a difference analysis and creating a changescript. With a bit of cleanup, this program would be a fantastic replacement!

    As a note, it looks like it’s a fork of this program:

    http://opendbiff.codeplex.com/

    I’ll need to do some more analysis to figure out which one is a more appealing start. However, they both appear to have ended development sometime in 2010 - I wonder why?