Tech Trained Monkey

Everyday Problem Solvers

Monthly Archives: March 2012

How about an hourly build?

The benefits of a daily build are well understood by now.  McConnell even cites the book Showstopper! as an extreme example circa 1993:

Who can benefit from this process? Some developers protest that it is impractical to build every day because their projects are too large. But what was perhaps the most complex software project in recent history used daily builds successfully. By the time it was released, Microsoft Windows NT 3.0 consisted of 5.6 million lines of code spread across 40,000 source files. A complete build took as many as 19 hours on several machines, but the NT development team still managed to build every day. Far from being a nuisance, the NT team attributed much of its success on that huge project to their daily builds. Those of us who work on projects of less staggering proportions will have a hard time explaining why we aren’t also reaping the benefits of this practice.

I think the main argument against daily builds was, frankly, a technological one– it simply took too long. In this age of blazingly fast 64-bit processors– and even cool distributed build stuff like IncrediBuild—  time should no longer be a factor.

Most of us will never work on a project as large as Windows NT, so our builds should be near-instantaneous. And we should do better than a daily build. I believe, for all but the largest projects, you should be building multiple times per day. This is also known as continuous integration:

Developers should be integrating and releasing code into the code repository every few hours, whenever possible. In any case never hold onto changes for more than a day. Continuous integration often avoids diverging or fragmented development efforts, where developers are not communicating with each other about what can be re-used, or what could be shared. Everyone needs to work with the latest version. Changes should not be made to obsolete code causing integration head aches.

I’m at the point now that I get aggravated when other developers leave files checked out overnight. Perhaps it’s a question of programming style, but I believe your code should almost always be in a compilable state. It may not do much, but it should successfully compile. I strongly believe that an aggressive checkin policy leads to better code. So does Martin Fowler:

One of the hardest things to express about continuous integration is that makes a fundamental shift to the whole development pattern, one that isn’t easy to see if you’ve never worked in an environment that practices it. In fact most people do see this atmosphere if they are working solo – because then they only integrate with themself. For many people team development just comes with certain problems that are part of the territory. Continuous integration reduces these problems, in exchange for a certain amount of discipline.

The fundamental benefit of continuous integration is that it removes sessions where people spend time hunting bugs where one person’s work has stepped on someone else’s work without either person realizing what happened. These bugs are hard to find because the problem isn’t in one person’s area, it is in the interaction between two pieces of work. This problem is exacerbated by time. Often integration bugs can be inserted weeks or months before they first manifest themselves. As a result they take a lot of finding.

With continuous integration the vast majority of such bugs manifest themselves the same day they were introduced. Furthermore it’s immediately obvious where at least half of the interaction lies. This greatly reduces the scope of the search for the bug. And if you can’t find the bug, you can avoid putting the offending code into the product, so the worst that happens is that you don’t add the feature that also adds the bug. (Of course you may want the feature more than you hate the bug, but at least this way it’s an informed choice.)

Now there’s no guarantee that you get all the integration bugs. The technique relies on testing, and as we all know testing does not prove the absence of errors. The key point is that continuous integration catches enough bugs to be worth the cost.

The net result of all this is increased productivity by reducing the time spent chasing down integration bugs. While we don’t know of anyone who’s given this anything approaching a scientific study the anecdotal evidence is pretty strong. Continuous Integration can slash the amount of time spent in integration hell, in fact it can turn hell into a non-event.

I’m not making any conscious effort to use so-called Extreme Programming; my concern is a more practical one. I simply can’t think of any justifiable reason for a developer to hold on to code that long. If you’re writing code that is so broken you can’t check any of it in for more than a day– you might have some bad programming habits.  I believe it’s far healthier to grow or accrete your software from a small, functional base and use aggressive daily checkins to checkpoint those growth stages.

How much does a 0-day vulnerability cost?

The market for exploits for zero-day vulnerabilities has exploded in the last year, says Adriel Desautels, the founder of Netragard, a penetration testing and vulnerability assessment outfit that, among other things, acquires and develops exploits.

The number of buyers and the money they are willing to pay for working exploits has dramatically increased, and so has the number of exploits offered for sale each month, he says. Also, the purchase deals are made much more quickly than in the past.
Obviously, the whole economy around this “product” has matured.

As a legitimate company, Netragard must be very careful when selling its exploits. According to Desautels, the firm rejects the majority of those who want to buy them.

“Realistically, we’re selling cyberweaponry,” he points out, but does not share how the vetting process is performed or the price that specific exploits can reach.

It is very well known what some software vendors offer for them through their own bug bounty programs, as well as the prizes offered for working exploits to participants in hacking contests such as Pwn2Own and Pwnium.

These sums are considerably smaller that the ones that can be earned by enterprising vulnerability researchers and hackers if they choose to sell exploits to other organizations, and that’s counting in the fee for the intermediary.

The Bangkok-based security researcher that goes by the handle “the Grugq” is one of these mediators. His contacts in various governments and knowledge of the matter at hand make him eminently suitable for brokering such deals.

He is also careful when choosing to whom to sell the offered exploits, and that’s mostly US and European governments and agencies. Ethical considerations aside, they simply pay much more than a Middle Eastern or Asian government can offer.

The Chinese government doesn’t need his services, he says, because its huge number of hackers usually sell their exploits exclusively and directly to them. He also says that he has no contacts in the Russian government, and that “selling a bug to the Russian mafia guarantees it will be dead in no time, and they pay very little money.”

So how much does a working exploit go for? Well, the price depends on a number of things.

An exploit of a vulnerability in a widely used piece of software is more costly than that of one in a less popular one, and the same goes for those that take advantage of vulnerabilities in the latest software versions. Exploits for software that is more difficult to crack is also more pricey.
Taking all this in consideration, it’s easy to see that an exploit for Windows will be more expensive than one for breaking into a Mac OS X machine, and that the tougher security features of iOS will raise the price for its exploits above that for Android.

According to Andy Greenberg, the current rough price list looks like this:

Exploit Prices List“Each price assumes an exclusive sale, the most modern version of the software, and, of course, not alerting the software’s vendor,” he says.
“Some fees might even be paid in installments, with each subsequent payment depending on the vendor not patching the security vulnerabilities used by the exploit.”

Event though considered unethical by some, these sales and acquisitions are sure to continue for the time being.

Demand creates supply and, according to the Grugq, banning the sale of exploits would have the same effect that the war on drugs has had on eliminating drugs – none.

Hiring interns

When it comes to hiring interns most people suddenly trust HR and their recommendations. Not me. In fact I trust HR as little as possible. Why? Latest example is: My own HR tried to hire me! and the bad thing is that the HR sits in FUCKING FRONT OF ME!!! Anyhow… lets move on…
The basic reason I don’t trust HR in the hiring process is because they don’t know is valuable in a candidate or not… Communication skills crap aside, HR can’t find people who posses what I most value when hiring interns: availability, minimum knowledge and WILL to learn.

Why? Internship is the place to learn how something is done in the real world. In order to learn something new you need time, because studying and observing takes time. Minimum knowledge because if you can’t read, im sorry you’re as good as…. better not say it, but resuming, your useless. Will because not always things will go as you wanted, you will get frustrated, maybe even angry and you might even question yourself. If you really want this, you find a way to cope with all that and overcome everything. All for the sublime feeling that is seeing a computer doing exactly what you expected, and trust me this feeling is GREAT!

The thing about hiring interns is that they are usually worried about getting the college degree and stuff like that… in Brazil there are very few things more undervalued that a bachelor degree in computer science. Dont get me wrong, the knowledge is highly appreciated but the degree itself not. The trick is to find the enlighten one that comprehend that the college will not prepare them for the real world and fish them out before someone else does. When someone one can see that what happens in real life will force, and focus you, into studying something that college will only give it a quick brush something magical happens: They get on board of the success train. Some might fall out along of the way. The family traditionally can’t appreciate the value of real world experience when it comes to education, and that is a big pressure to handle. Combined with the WILL to learn and overcome you have a golden boy.

My real trick is not only find this guy, but also teach him. Today im working in a new prospect: I found a guy highly interested in learning how to program, and he has a major in Graphic Design! He’s on the first year of computer science. He’s a blank stylish book. Can you imagine the possibilities?

About starting this blog

If you actually look, this blog was created on may/2011 but, until march/2012 nothing relevant was posted. Back then i wasnt commited and had no idea how hard it is to create and evolve actual textual ideas… im very good at this in a code level, but a text level i actually suck. i find it very hard to carry somthing to the very end.

I have one of those minds that are completaly obssessed about somthing or is ignoring it completally. Something like:on-offMy big problem here is that i switch very fast between obssessions… A week ago i was obssessed with the possibility of developing applications for Azure. A week before that i was thinking about a physics engine, and before that i was looking into developing a CUDA application… Point is: i have a real hard time focusing. This is why I need a job. A job kinda forces me into following something to the end. I really love discovering new stuff but i really think i need help in that area.

So when i came to start this blog i did because a good friend had started his, and i kinda like to write stuff… i enjoy teaching, and making presentations and most of all i love discussions. I love discussions so much that I often play the “devils advocate” part just to inflame the other part. Not as a troll would. I do it in a healthy way… i think so at least. So, for a week, or less, i really wrote stuff and edited and did the whoel 12 steps, and then schrodinger cat died (lol) and i forgot the blog… i forgot that i owned it…

About a week ago i read about how having a blog might help people that are hiring you discover more about you. So i decided to start writing again. But im being very real here… not soft corners… no more hiding (pls dont comment with “get out of the closet” jokes… its just idiot kind of jokes…)! And i actually fell good about this…

Im not doing this for comments, or popularity or to sell adds… or profit directaly from the blog (if someone wants to hire me because they see something in the blog, im not going to say no…). Im doing this because I want, because somehow i fell that the internet really gets me…HEHEHEHEHEE

About managing risks

A little time ago I was proposing to my manager a tool and a methodology change for our PMO. The tool and the methodology were based on risk mitigation and tracking. I am a firm believer that “Issues” don’t just come up. Issues that just come up were once risks that went unnoticed and eventually, like any living (yes I belive that risks are living beings) thing, evolved to Issues.

When I exposed the principle that the project manager had to invest time discovering risk, instead that divert time to write down risks that presented themselves (btw, the risks that become god-like-issues don’t present themselves) and that map all risks was a vital part of that, he mocked me saying something like: “So when I see that an airplane is about to crash in my building im supposed to run and create a new risk entitled plane crashing at the office?” I said no, you’re not. But the risk of loosing your primary work station should be increased to 99% and the risk of being without resources should also be increased. I also said that if you were a good project manager you would also contact HR and ask for new candidates.

Of course the example above is an extreme event. The fact is: It’s uncommon that a risk escalate so fast (like living things, once again). Risks tend to grow very slowly, being fuelled by negligence, incompetence and lack of proper organization/tracking. Most risks that become problems were actually mapped and “handled”. A perfect example is: an ex-manager of mine called the client warning that he’s servers were outdated. By doing so he actually believed that he handled the risk of not being able to deploy the product due outdated software. Fact is that the deploy date came and the deploy did not happen on date because the servers were outdated. He’s mistake was to belive that because someone was warned he would take any action. What he really should have done is called day after day and only close the risk when the client confirmed that the servers were updated, and if he did not updated by the day of the deploy escalate the risk to issue.

The tool that i proposed would handle situations like that perfectly. OK its was not a third part software but a customization on Sharepoint 2010. The tool principle is that you would not only register the risk/issue. You would have to register also what actions you as project manager took in order to solve this issue, and how effective these actions were. Yes there’s an overhead but besides a complete follow-up on the evolution of the risk/issue, a PMO manager could actually see how good each project manager is based on the effectiveness of his actions. It was a thing of art. If we evolved to a “better practise” and could assign a cost value to each task we would be able to measure how much each risk/issue costed and by implication how much each actions costed. And knowing how effective an action was, we could designate a cost-benefit value. It is utopia for most of the managers.

I must disclose that I am not a project manager. I am a developer. I develop solutions for leaving… and of course… a boy can dream…

Motivation Part I

Yesterday i read a quote by Bill Gates that goes something like this: “Success is a lousy teacher. It seduces smart people into thinking they can’t lose“. Some time ago i saw a movie named A Good Year (Russel Crowe) and the movie said something like “You’ll come to see that a man learns nothing from winning. The act of losing, however, can elicit great wisdom. Not least of which is, uh… how much more enjoyable it is to win. It’s inevitable to lose now and again. The trick is not to make a habit of it.” and quite a few more on the same idea… and it got me thinking how it sucks to lose, how important power and success (or the image of) is to us and manly why can’t we all just be.
As a man of science, specifically computer science i find it quite hard to comprehend some behaviors, like faith for example, but nevertheless I try. Thinking real hard about why loosing sucks i came by the thought that it only sucks because when you lose you don’t get the bug, cof! cof! cof!, i mean, the big prize, and i came to understand that it’s a moment thing! I wonder how one stops “accepting the defeat” and become a “looser”… by that logic, winners are sour losers, but losers nevertheless… which leave us to the question: Are there winners?
Some argue that winners indeed are losers that didn’t accepted the defeat, worked hard on to something and become a winner… to that i always say “the dispute was not over when someone was declared a looser, and should be declared as under-advantage (os something like that) instead”, because once someone is declared winner of the race, he’s the winner and everyone else is a looser, therefore there is no general concept of winner, therefore everyone can be a winner, but not at the same time. Being cruel, the second place is the first user, cof! cof! cof! i mean, the first looser.
By now you must be thinking that there’s nothing motivational about this post. I don’t care about what you think. Motivation is not “Go for it! You can make it!”, that’s cheering! Motivation is something that drives you to do something. Motivator is the agent of motivation, like love, passion, hunger, fear, anger, etc… Some say that Fear is the most powerful motivator, idiots say that love is (if you really think about it, fear motivates a lot more than love.. like fear of losing someone you love… you don’t move a mountain because you love someone, you move it because you are afraid that if you don’t, that person will go away…). In my opinion the most effective motivator is hunger (hunger is like a bitchy woman who will bitch until you satisfy her). Fear, Love, Anger are powerful, but very unstable, a sunny/rainy day might ruin the whole motivation. But REVENGE… oh sweet revenge… is the most powerful! Very dangerous!
Think about it… success might push you forward… loosing might stop you… but revenge sweet revenge will certainly get you to completely plan and eventually destroy someone. Quite a dark ending…

Ideas

I dont always have good ideas for my blog, but when i do, i have a billion at the same time

The “build me a pair of pants” Problem

For a long time ive been hearing about scrum and agile methods and what i like to call “deliver piece by piece” methodologies… and ive been also studying it (my boss seems to like it… joking… i was courious about how this “magic” works…) and i must say its quite… … … un-rational in my opinion. I cant get pass the fact that 1. The client will decide what i will build in which order and 2. If i was the client i would actually buy a shotgun, and unload it with mighty wrath, when someone come to and try to convince me that “paperwork” is a deliverable. Clients buy Solutions/Products, or promises of such. Nobody buys a project (I will talk about outsourcing another day). To express what i mean i came up with the “Build me a pair of pants”, as it goes like this:

A costumer come to you and say that he/she needs a single solution for 2 problems: “the too many items in the hands” problem and the “im feeling cold im my legs”. As you look at the situation you notice that hes/shes out of pants (ill leave to your imagination other details that are not relevant)… the answer is clear: He/She need pants. The pockets will solve the hands availability issue and the jeans will solve the cold issue… right? You decided that, as always, scrum will be applied. So you build the log and it consists of 2 items, build pockets and build the legs. As a good “master” you ask the client which one he wants first, and as a client with no pants on he answers: “i dont care, i have my hands full and im feeling cold in my legs!” so, the question lies here: which one you build first? The pockets or the pants?

If you build the pants first, ok you finish the cold issue… but how does he gonna put on the pants? “hands full” issue blocks it… and lets say, that ok, he put everything down and got into the pants somehow, when the pockets are ready, you will force him to stop again, drop everything, “attach” the pockets and ok lets go on? OR youll build the pockets, dont solve any problems (he now carries pockets on the hands) and make him stop only once and attach the pockets on the legs, put them on and move on?
The main problem is yet to come: You may not realize, but when you deliver “piece-by-piece” software you abdicate the “OMG moment”. When you deliver part number 1, he might be gratefull (notice the might) but he will focus on what he doesn’t have… yes, clients are glass half-empty kind of people. And when he get the last part, he will be used to the other parts, and therefore “no wow for old-news” comes in and slap you in the face.
What is the “right” way to do it then? you might be asking… Get a “closed-scope” contract, assume risks and charge a bit more to compensate? What most people fail to understand is that when you sell a “software project” your actually building a product specifically to one client, at least, and that the client you pay you for the product! not the project! the client could not care less about your methodology, internal documentation (they do recognize the value of user manuals, and training material and a few others) or other any other crap (DISCLAIMER: clients attorneys might think differently about that). Stop thinking project and start thinking product. Don’t listen to the sales people. Great products do a lot more revenue than great projects, and great products are not necessarily the result of a great project. Actually as far as I can remember, the greatest products in the world are the result of a mega-failed project (think space shuttle program: 150 billion cost overhead, Sydney Opera House 14x: cost overhead and over 10 years delay, and trust me there are a lot more).
i really don’t want to end this post with a statement or something… i would like to hear back…