Now that I am no longer working for Nutanix, I feel like I can talk about what I learned about Hyper converged infrastructure and cloud computing in the last 2 years. Let’s look at how hyper converged architecture is formed and then we will discuss the advantages of structuring your IT infrastructure this way. Most of what is mentioned here is a simplification of Nutanix Bible.
What is Hyper converged infrastructure ?
For a system to be hyper converged, it needs to have at least 4 components :
Hardware
Hypervisor
Distributed storage software
Management plane that can manage all of the above
So, let’s look at it from Nutanix’s point of view.
Hardware
All software needs hardware to run on and for hyper converged platforms, this is not different. You can buy hardware from Nutanix, Dell-EMC, Lenovo etc. Nutanix hardware is usually standard x86 servers from Supermicro ( although that may have changed ) with Nutanix branding. This hardware is not special, which is why you can swap it out with Dell-EMC, Lenovo gear.
Hypervisor
Hypervisors are usually purpose built operating systems that can create virtual machines ( VMs ). So, on the hardware, first you need to install a hypervisor. Here again, you have choice. You can install Nutanix’s own AHV , or you can install VMWare ESX or Microsoft Hyper-V.
Distributed Storage
Using the hypervisor above you can create a VM that has direct access to all the disks on the computer. This VM ( called a Controller Virtual Machine or a CVM ) talks to other CVMs in the data center, forming a cluster. On this VM, a set of software is installed which aggregates all the disks on the computer ( and the disks on all the other computers in the cluster ). This is then presented back to the hypervisor as a single unit. This component is the absolute minimum that must come from Nutanix if you want to use Nutanix platform.
Management plane
Some management software is installed on the CVM which can be accessed by going to https://CVM_IP_ADDRESS:9440 using a web browser. For Nutanix, this management plane is called PRISM. Besides serving the user interface, it also serves a RESTful API which can be used to automate tasks on the cluster. PRISM is a must for managing storage in Nutanix cluster, but you can still manage your VMs from VMWare vCenter for your ESX cluster.
Advantages of Hyper converged infrastructure
So why go through this at all ? This seems much more complicated than just running your applications directly on hardware. It is even more complicated than running applications on a virtual machine on the hardware. ‘Distributing storage ? Presenting the storage back to the hypervisor ? You are out of your mind, sir !’ I can hear you saying. Here are some reasons :
Self healing : Hyper converged systems offer automatic data replication. When VMs write data to disk, the platform writes a copy elsewhere. So if your hard disk fails, just rip and replace it when you have time. The system notices that there is now only one copy of all the data that was in that hard disk and automatically builds a second copy. Same applies when a single computer in the cluster fails.
Expandability : You can start with just a few computers and then grow from there as your computing needs grow. You don’t have worry about to buying a large number of computers upfront and then not needing it.
Easier to manage : Hyper converged systems offer a single interface to manage all your IT infrastructure. Manage storage, compute etc from the same web based console.
Efficiency : The platform can notice when duplicate data is being written to disk by virtual machines. For example, your VMs serving live traffic may be cloned by developers to debug issues or to develop new features. Most of the data on these VMs may be similar and can be deduplicated saving storage space.
Security : Data can be encrypted automatically by the platform before they are written to disk.
There are many more advantages of course, but that should justify the extra complexity.
UPDATE: This post should not be considered an endorsement of Fideism
First of all, let me say that I am not an official spokesperson for the Roman Catholic Church. I tell this story because I feel it needs to be said, but it may contain inaccuracies.
Part I : French Revolution
In 1689, St. Margaret Mary Alacoque receives a vision of Jesus. Jesus reveals to St. Margaret that despite the love he has shown to men we only repay Him with ungratefulness and wickedness. He orders St. Margaret to ask the King of France to consecrate France to His sacred heart. The King Louis XIV ignores this order.
Painting of Jesus appearing to Saint Margaret Mary, Church of San Michele by Giovanni Destefanis
The Napoleonic wars severely depopulates France with a large number of people perishing in Napoleon’s thirst for power
Catholic Church suffers major setbacks : it’s nuns and priests turned out, leaders killed or put in exile, it’s property seized and controlled by enemies
There is an attempt to de-Christianize France
However, the effects were partially reversed in time. Some effects still remain to this day. Most important of these are “liberalism” and the idea of replacing God with reason. See : The Cult of Reason.
France has corrupted the universe, one day it will be punished. The faith will die out in France: three quarters of France will not practice religion anymore, or almost no more, the other part will practice it without really practicing it. Then, after [that], nations will convert, the faith will be rekindled everywhere.
A great country in the north of Europe, now Protestant, will be converted; by the support of this country all the other nations of the world will be converted
I see that there are many versions of this text and some readings of these messages will lead you to consider the notion that there is nothing much to it. Be warned that interpreting these messages are difficult for the best of us, but at least we can see that half of this has already come true. If you look at religion in France today, only a quarter believe in God.
So now, the question is which is that great country to the north of Europe ?
Part III: Our Lady and Fátima
In 13 May 1917, three children ( among them Lúcia Santos )in Fátima, Portugal reported a vision of Mother Mary.
Lúcia Santos (left) with her cousins Jacinta and Francisco Marto, 1917
The Lady revealed Three Secrets to St. Lúcia. Here is some text from the second secret :
The war is going to end: but if people do not cease offending God, a worse one will break out during the Pontificate of Pope Pius XI. When you see a night illumined by an unknown light, know that this is the great sign given you by God that he is about to punish the world for its crimes, by means of war, famine, and persecutions of the Church and of the Holy Father. To prevent this, I shall come to ask for the Consecration of Russia to my Immaculate Heart, and the Communion of reparation on the First Saturdays. If my requests are heeded, Russia will be converted, and there will be peace; if not, she will spread her errors throughout the world, causing wars and persecutions of the Church. The good will be martyred; the Holy Father will have much to suffer; various nations will be annihilated. In the end, my Immaculate Heart will triumph. The Holy Father will consecrate Russia to me, and she shall be converted, and a period of peace will be granted to the world.
To me it looks like the ‘war that is going to end’ is World War I and the other one that ‘breaks out during the Pontificate of Pope Pius XI’ is World War II.
Our Lady fulfills her promise, asking “for the consecration of Russia to Her Immaculate Heart, promising its conversion through this means and the hindering of the propagation of its errors
At Rianjo, Spain in August 1931, Our Lord communicated to Sister Lucia His dissatisfaction. Here is what He said :
Make it known to My ministers, given that they follow the example of the King of France in delaying the execution of My requests, they will follow him into misfortune.
They did not wish to heed My request! Like the King of France they will repent of it, and they will do it, but it will be late. Russia will have already spread its errors in the world, provoking wars and persecutions against the Church. The Holy Father will have much to suffer.
Many consecrations of Russia are attempted ( in 1942, 1952, 1964, 1982 ). St. Lúcia informs that none has fulfilled the conditions ( the conditions are that all bishops in union with the Pope and Russia must be specially mentioned )
In March 25, 1984, Pope John Paul II, “united with all the pastors of the Church in a particular bond whereby we constitute a body and a college,” consecrates “the whole world, especially the peoples for which by reason of their situation you have particular love and solicitude.” Both the Pope and Sr. Lucia initially seemed uncertain that the consecration has been fulfilled, but shortly thereafter Sr. Lucia tells the papal nuncio to Portugal that the Consecration is fulfilled.
1991, Dissolution of Soviet Union
Technically Russia is to the north of Europe and technically the country is Protestant, in the sense that Orthodox Christians do not accept the authority of the Pope. Since the fall of Communism religion is returning to Russia.
Part IV: Why did I tell this story ?
Regardless of what you think about the interpretation of these events, what you have to accept that there are currently two major threats to our faith :
Replacing God with the State
This is mostly a Communist idea. We can see echoes of this idea in modern left wing and liberal movements. Since I went to a government college and grew up in a Communist influenced state, I know this is a bad idea already. I do not think governments are very good at their jobs. There may be good people in the government, but mostly it is an instrument used by people with power to take advantage of people without power. Incentive structures within the government are setup so that people get paid well even if they mistreat the people who have to deal with these bureaucrats.
Replacing God with Reason
This is the modern vision put forward by New Atheists. We are required by our professions to maintain and upgrade our capacity to reason, and for a long time, we have done so. We have built our minds into a Fortress of Reason, and this benefits us. Unfortunately, to maintain our relationship with God, it is upon this Fortress of Reason that we must commence our siege.
External targeted contributions. For example, most people submitting patches to open source software are facing some issue as a user. If they are experiencing an issue, others may also be experiencing similar problems. This is a much better way of driving development, compared to doing it post expensive market research.
Marketing
Recruiting vehicle
Increasing the size of markets. For example Google is trying to bring more people online with Android to increase the size of search market.
Agile is a combination of two processes that are designed to develop software. The two methodologies that form Agile are : Scrum and Kanban.
Key to understanding agile is understanding the objective tree. Here is an example objective tree for a company like amazon.com which aims to build an e-commerce website.
Organization Objective Tree
There are many people who argue about the number of levels in the tree and how they should be called. But the key insight here is that :
a) Objectives form a tree
b) When child objectives are done, the parent objective is automatically done
There are two methodologies within Agile : scrum and kanban.
Scrum
Process
Objectives are broken into user stories. A prioritized list of user stories is called a backlog. Product management owns the backlog.
Objectives are achieved by working on them in 2 weeks sprints. Team pulls the maximum number of stories that can be completed into the sprint backlog and starts working on them when the sprint starts. Sprint backlog is owned by the team.
Every day of the sprint there is a 15 minute standup meeting where team members describe 3 things :
What they worked on yesterday
What they are planning to work on today
Blockers ( if any )
This is an opportunity for micro-course-correction. If a story was done, team members can move to help with another story. If there is something that blocks a team member, management can move to unblock them. If a story might spillover, it can be called out.
At the end of the sprint there is a sprint retrospective to re-evaluate and course correct at a higher level.
Every N sprints ( for a finite value of N ) there is 1 refactor sprint. No new features are delivered in this sprint. Architectural runway is prepared for incoming stories and code is made more maintainable.
Why is the process like this ?
Reason for product management owning the backlog is to communicate clearly what the current priority is. This can be challenging in a changing environment. For example, management may ask the team to support an application on Windows 2012 platform and mark it high priority since largest customer base is on that platform. Afterwards it could turn out that highest paying customers are on RHEL 7.1. Management can communicate this information to engineering by adding a new story to support RHEL 7.1 and deprioritizing the story for Windows 2012 support.
Team owns the backlog because the team is responsible to ensure that the features ( stories ) committed at the beginning of sprint is done by end of sprint. This is why it’s the team that need to pull the stories into the sprint ( and not the manager ) so they feel a sense of ownership.
Standups give an idea about the pulse of the sprint : whether the team is on track, if it needs adjustment etc. Since software development is a collaborative process, it can get blocked on other people. For example : UI team can get blocked on UX mockups, developers can get blocked on APIs that were promised by other team members, one team member may be blocked for code review from another etc. Daily meetings help with dealing with these problems as they arise. It also enables the team to help others, for example, if someone airs a database problem, someone else on the team may have encountered and solved the problem before. Standup meetings should be short and sweet. If they are too long, since they occur everyday, they become a productivity drag.
The sprint retros are an opportunity to improve things at a higher level. For example, developers may notice that the CI system could run a larger portion of automated tests, reducing QA burden. Developers may point out that the code is getting too complex in one area and request refactor stories.
Expectations
1. If a story is in the backlog, it may get done. If a story is not in the backlog, it will never get done.
Why is this expectation there ? Sometimes product management may request a feature. Engineering does not know if that feature was important, whether it was a priority etc. Moreover, there could be multiple managers requesting different priorities for stories fuelling confusion. This expectation sets up management to close out the relative priority of objectives before people set out to achieve them.
There is no such thing as an implicit requirement. Management owns the product backlog.
2. Target spillover for any sprint is zero
Why is this expectation there ? For hard software engineering problems it is required that the team is a highly committed one. An uncommitted team member will usually bring down the whole team since the job is collaborative in nature. Target spillover of zero ensures that the team remains focussed on the problem at hand.
3. No activities are planned mid-sprint
Why is this expectation there ? This is to help the team remain focussed so that the cost of context switching is lowered. It is called a “Sprint”, not a “Trot” or “Gentle Walk”. Management is expected to defend the team from external influences. Course corrections are possible at the end of sprint. Therefore, it is important to keep the sprints as short as possible to get a meaningful amount of work done. The length of your sprint should be the minimum amount of time it takes you to respond to external influences. Usually, it is 2 weeks.
4. QA should raise a ticket if there is a problem.
Why is this expectation there ? It is possible that QA notices a problem and communicates it verbally or through email. Now people may go on vacation etc cutting the team’s ability to deliver.
There is no such thing as an implicit ticket. Raise it in issue management software.
5. No story is done until :
a) All code is reviewed
b) There are no open tickets
c) Documentation is complete
Why is this expectation there ? Code reviews are important because it lets someone else get familiar with that code base allowing them to take over in the event of the author’s absence. It is important to close all open tickets before calling a story done since otherwise your development process simply turns feature requests to bugs. Documentation should be developed lock step with code since that is when it is easiest to write it.
6. Do not test from feature branches
Why is this expectation there ? Someone could merge something else into the development branch making the feature tested in feature branches to be unviable in development branch. Development branch is what ultimately gets shipped to users so that is where things should be tested.
7. Stories are of the form “As a [user] I want to do [task] so that I can [accomplish goal]”. If they are not in that form, it can be safely deleted.
Why is this expectation there ? Often stories may be created with no clarity on what the next step is. Often developers may misunderstand the requirement and deliver a different feature requiring multiple unnecessary round trips and feedback loops to complete the feature. If the story is not in the prescribed form, it can be deleted and considered non-existent since it motivates people to state the objectives of the story explicitly.
A few points
In Scrum the keywords are focus and course correction.
Manager – employee relationship is co-operative. The team does everything to make the manager look good ( by delivering high quality features that delights users ) and managers does everything to setup the team and employees for success : removing impediments, giving them tools and access they need to meet the objectives.
Dev – QA relationship is quasi-adversarial. QA tries to defend the customer demanding issues be fixed within reason. Dev team appreciates QA for finding the bugs they have missed that could have caused them to deliver an inferior product.
Agile requires self motivated team. An under-motivated team may over inflate estimates or under commit into a sprint.
There is no such thing as half-agile. Agile gives management a certain amount of power and engineers certain amount of power. This is to improve efficiency by reducing friction and miscommunication. If management says : ‘I will accept these parts of the process that gives me power and ignore these parts that require me to relinquish power’ this process will fail. It requires humility on the part of management and engineering.
Kanban
This is usually for support. After product is developed, support is trained on resolving bugs that get into production. Support can’t wait 2 weeks to resolve issues. When tickets come into the queue people take them on and try to get through them as fast as possible.
Scrum is more proactive and Kanban is more reactive. Kanban is for situations where a 2 week turnaround time is unacceptable.
Waterfall
Agile came about as a response to the Waterfall model of software development. It helps solve the most important problems with waterfall such as quicker course correction, improving focus and reducing miscommunication.
In waterfall, the software developed in following phases ( one after other ) :
Gathering requirements
Designing solution
Implementing solution
Testing solution
Problems:
A mistake in early phase cannot be detected until last phase.
There is little focus as work drags on for years.
People working on each step only communicates with people in previous step increasing communication errors.
Should Indian Railways be privatized ? Looking up the answer to this question on the Internet would lead one to conclude that no, it should not be privatized. I am not convinced, I think it should be privatized.
Lets look at the arguments against privatization, one by one.
It is not clear how it will improve situation
The argument is that privatization and increased competition will improve the quality of service, drive down prices and spur innovation. It should reduce corruption and force employees to do their best. For a government owned company the employees have no reason to work hard, treat its customers well and figure out how to make everything better. They get paid regardless. This causes them to mistreat their customers and generally be lazy about their work.
The Indian Railways lost ₹30,000 crore (US$5 billion) in 2015. This is despite the fact that it is a high demand business with a government sanctioned monopoly. Booking a train ticket is so painfully difficult that it a subject of so many degrading memes on the Internet. The compartments are noisy and sleeper classes are filled with bed bugs. The toilets are dirty. The staff is unpleasant to deal with. The train stations are riotous and unclean. It smells of shit and urine.
I wonder how many levels of incompetence it takes to suck so badly at a job.
Privatization has helped the Indian telecommunication sector and aviation sector. It is possible for it to work in the rail industry as well. Even if it doesn’t help, it is at least worth a try.
Look at the state of American and British Railways
They are significantly better, aren’t they ? But even if it isn’t, there are a thousand reasons it could work better in India. Yes, I know there are a thousand reasons why it couldn’t work also. But why not give it a try ?
Concerning America, their landmass is 3 times that of India and they are a very affluent and urbanized country. This means that most of the travel is very long and hence very inconvenient to be conducted by train. So rail is used mostly for goods and human beings pay a little more to travel by planes.
It is a natural monopoly due to high cost of entry and exit
This is partially true. The high cost of entry and exit is largely due to high cost of building and operating railways, so we may need to have some government assistance with regard to this.
One solution to this problem is to let the rails themselves be owned by one company and combine this with multiple companies owning the individual trains. The train companies lease the rails from the rail company to provide their service.
Sure, some government assistance will be needed to acquire more land, ensure that there is stiff competition etc but these are all problems that have well known solutions. It can definitely not be a fire and forget endeavor.
Also I don’t understand how it becomes harder for individual investors to exit the rail business as most companies are traded on the public stock market. If you want to enter the rail business buy a few shares of the company at the Mumbai Stock Exchange. Want to exit ? Sell them at the stock exchange.
Some routes may be under serviced because they may be unprofitable
Form a government company to service only these routes using tax payer money. There is no need to keep the entire business in the public sector and cause incalculable misery to everyone in India to benefit a few.
The railway employs a lot of people these jobs may be lost in privatization
Privatization does not automatically cause loss of jobs. If a private railway business can operate with fewer number of people and automate away a lot of these jobs it would benefit the customer with lower fares. This money saved by customers can be used to buy new things which creates new jobs. See the parable of broken windows. Another possibility is to tax the rail companies and pay unemployment benefits to these people directly. At least the public will be spared from their laziness and callousness.
Defense concern. Countries like China could buy up all Indian Rail businesses and in the event of a war, discontinue them all
We can repossess these businesses in the event of a war. The other problem is if they sabotage it silently while they are holding it. This problem can be solved by stronger policing.
National pride and unity
A lot of arguments center around how Indian Railways is a matter of pride for Indians. Sorry, I would rather travel comfortably than with pride. Also, I don’t think it is a matter of pride given how absurdly mismanaged it is.
So anyway, I think most of these arguments are bogus or ill-informed. I have similar thoughts about the power industry ( power cuts are uncommon in Western countries ) since there is no innovation in solar, wind, nuclear power in India despite us having no serious oil reserves. I think India is a poor country on account of power. But more on that later.
What do you think ? Let me know in the comments section.
So last night we watched “Batman v Superman”. This got me thinking about the prequel : “Man of Steel”. I just rewatched it and found that it is in fact a very good movie. Let me tell you why.
Superman has always been one of my favorite characters. This is not to say that I have watched and enjoyed every version of Superman in existence. In fact, I may not even like some incarnations of this character. But the fact remains that the essence of the Superman story is very interesting. So I am always surprised to find that many people dislike Superman deeply.
One reason I have heard for this strange phenomenon is that they think Superman is too powerful. He seems invincible. It is true that a story would be uninteresting if the hero had no chance of losing or if he was pitted against a weak villain ( “Merlin” TV series anyone ? ). This is not the case for “Man of Steel”. Zod is every bit as powerful as Superman. It is in fact a vital part of the story that Superman is outgunned and outnumbered in this fight. He has to fight many Kryptonians who are just as strong as him but born and bred to be soldiers. These Kryptonians trained their entire life to be warriors and you can see Superman struggling to match them. The truly awesome power of Superman is the hook for the story. But this does not make it dull if he spends the majority of his time standing up to guys a lot stronger than himself.
A lot of people are angry that at the end of the movie Superman kills Zod. I think it is an important plot point : this implies that Superman was never invincible to Zod, Kryptonians can kill other Kryptonians on Earth with their bare hands. The fight could very much have ended with the death of Superman, it is unclear how he survived at all. Maybe Zod was still distraught over losing his purpose and his people. Maybe his heart wasn’t fully in the fight. Maybe it was just plain luck.
I think it is very brave that a simple farm boy is willing to take up such dreadful odds to do the right thing. He didn’t sign up for any of this.
There is also another aspect to this story. When Zod asks Superman to join him, it would have been the easiest thing in the world to say ‘Yes’. The movie depicts throughout, the loneliness that Clark feels. He feels like an outsider, a freak. It would have been a welcome relief to live on the ‘New Krypton’ and feel belonged. So why does he say ‘No’ ?
The simplest answer would be to say that he didn’t want to kill all those humans. But he did finish off the Kryptonian race ( his own ) in the process. Is it any better ? This is where the back story kicks in again. Every Kryptonian’s life is predestined, every man and woman born to a single predefined purpose. There is no place for chance or choice. With such a system, the ‘New Krypton’ would inevitably meet the original Krypton’s fate : infinite self-destruction. This is why Jor-El does not leave Krypton with Kal-El. When a race is doomed, it can only be wiped out so that a new one can have a hope of survival. This point, however cruel is a fact in the story universe.
Human beings may not make the same mistakes Kryptonians made, which means they have a shot at survival and expansion, not the infinite destruction that Kryptonians face. This is also why Jor-El was OK with sending Kal-El away from Krypton : he was a natural birth. He is not part of the system. Jor-El may have tried to bargain with Zod to let the Kryptonians live alongside humans in a last ditch attempt, but he knew that only humans have a serious chance.
Finally, I also want to make the point that it may not be Superman a lot of people hate. This point has been argued before, but I think we find it difficult to accept even a fictional character who is both good and successful. The reason for this is that he reminds us of our own shortcomings. We feel that we are justified in doing evil, so that we can be successful or happy. We feel it is impossible to live in this unjust and unfair world by being good. It is this lie that we tell ourselves that is challenged when we watch a Superman movie. We are not angry at Superman, we are angry at ourselves. Sometimes we project this anger towards all that is good. If that is true, we have to do better than that, don’t you think ?
Here is my theory on an algorithm to build great software :
Figure out how the software ought to work.
This is the most important part in the software design process. It is important that the user experience be nailed down before we begin all other processes. If we don’t do this first, people will find the easiest way to make the software work. Such an approach may not deliver the best quality experience to the user. It is important to note that it is essential, to involve the users in this step. We need to clearly understand the problems they are facing. How should the optimal solution to their problem work ? Is it even possible with the available resources and at the current state of the art ?
Make it work
This is the part where you build the software according to the expectations set by the user. At this point you do not have to worry about code quality, performance etc.
Make it better
In this step you have to make sure that the code is extendable, has high quality tests and is secure
Make it pretty
In this step, make the software beautiful. Make the text conversational and friendly. Introduce fancy graphics and other visual elements if possible.
Make it fast
Finally it is time to seek out the worst performing parts of the software and make it fast.
If you follow these steps, and in that order, you should have a high quality piece of software in your hands.
Polymorphism is touted as one of the awesomest features of Object Oriented Programming. But what exactly is the point of polymorphism ?
Lets take the problem of trying to automate Google search. Lets say your test case is :
Open the browser
Navigate to “www.google.com”
Type “facebook” to the search text box
Click on “Google Search” button
Wait for the page to load
Check if the results page contain a link to “www.facebook.com”
Clearly we need to find some way to drive the browser to do these actions. But how will the automation software know where the ‘search text box’ is ? How will it know where the ‘Google Search button’ is ? So, there is a technology called Xpath that lets you identify a particular element in a web page.
So here is the procedural code that solves the problem ( Language : C ):
Now lets look at the object oriented solution to the problem. Please keep in mind the Liskov Subsitution Principle : base class pointers can point to derived class objects. The code is in Java, so if you are unfamiliar with Java, just keep in mind that classes that implement an interface can be thought of as a derived class of said interface :
public interface BrowserDriver {
public void navigate( String URL );
public void type ( String textBoxXpath, String textToType );
public void click ( String buttonXpath );
public void waitForPageLoad();
public boolean searchForLink();
}
public class FirefoxBrowserDriver implements BrowserDriver {
public void navigate( String URL ) {
// Code to navigate Firefox browser to the URL
}
public void type ( String textBoxXpath, String textToType ) {
// Code to type "textToType" into text box identified by textBoxXpath in Firefox
}
// etc
}
public class ChromeBrowserDriver implements BrowserDriver {
public void navigate( String URL ) {
// Code to navigate Chrome browser to the URL
}
public void type ( String textBoxXpath, String textToType ) {
// Code to type "textToType" into text box identified by textBoxXpath in Chrome
}
// etc
}
public class GoogleSearchFaceBookTest {
public static boolean testGoogleSearchFacebook( BrowserDriver driver ) {
driver.navigate( "www.google.com" );
driver.type( "//searchbox", "facebook" );
driver.click( "//searchbutton" );
driver.waitForPageLoad();
return driver.searchForLink("www.facebook.com");
}
}
public class TestMain {
public static void main( String [] args ) {
BrowserDriver browserDrivers[] = { new ChromeBrowserDriver(), new FirefoxBrowserDriver() };
for( browserDriver : browserDrivers ) {
if(!GoogleSearchFaceBookTest.testGoogleSearchFacebook( browserDriver )) {
System.out.println( "GoogleSearchFaceBookTest has failed!" );
}
}
}
}
If you have a very large test suites, there are a couple of advantages to this approach :
This code is a lot less copy paste and hence less error prone.
Code is easily extensible : Want to add a new test ? Just add it to the main for loop. New test runs with all supported browsers. Want to add a new browser to the test suite ? Just add the driver object to the browserDrivers array. Your entire test suite now runs against the new browser.
How does this magic happen ? If you look at these lines of code :
you can see that the code behaves differently ( polymorphism ) based on the type of the driver object. If the driver object is of type ChromeBrowserDriver, calls get dispatched to ChromeBrowserDriver methods. If driver is of type FirefoxBrowserDriver, calls get dispatched to FirefoxBrowserDriver methods. This is the power of polymorphism and it allows us to write modular code.
One important question that could come up at this point is : ‘My favorite language is C. Do I need to switch to Java to write object oriented code ?’. And my answer is : No, it is not the language that is important, but the technique. You can find object oriented code in C. Look at the write() system call in Linux :
This is an object method operating on the file object pointed to by the “fd” parameter.
Since C is a simpler language compared to Java, if modularity problems are not common in the problems that you solve, use C. I believe we must use the simplest tool that gets the job done. This reduces chances for bugs and creates more performant software.
On the other hand, if modularity problems are common in the problem you are attempting to solve, it maybe helpful to use a language with support for object orientation. For example, if you say that ‘class A implements interface B’ and while implementing class A, if you forgot to implement a method specified in interface B, you get a nice compile time error in Java. In fact, if you use a good IDE, it will report the problem immediately, before you even decide to compile the program.
How would you want manage a software company ? Software is unlike many other industries in the sense that it is a creative process. In other industries ( say a Coca-Cola bottling plant ) there is a linear relationship between labor and result. If it takes 1 hour for a person to fill 100 Coca-Cola bottles, it will most probably take him 2 hours to fill the a total of 200 bottles and so on. If you want 200 Coca-Cola bottles filled in an hour, you should therefore employ 2 people. It is fairly predictable how much time it would take to fill 300 Coca-cola bottles with one person : 3. None of this translates well into software. In software, once a software is developed, it is trivial to create a copy of it. Software companies make money by making new things : adding features, fixing bugs etc ( I am excluding SaaS business models for the moment, though they also require new features to stay on top of competition ). One important feature of this type of creative work is that, by its very nature, it is unpredictable. If you know what code will solve your problem, and you know that it will work, typing in the code is the simplest thing in the world. Majority of software development involves figuring out howto solve the problem. And this process is venturing into the unknown, taking educated guesses, trying what works and on failures start fresh. What kind of management does this lend itself into ?
First off, it is pointless to ask people to put more time at work. When you try something and fails, it is important to take a step back and relax. The problem needs to be emptied from your mind for a while so that on the next approach you will not go down the same rabbit hole you went before. Software companies that require employees to put their bums on their seats as much as possible needs to go out of business, and fast.
Secondly, it is very hard ( and most times, impossible ) to estimate how much time it would take to do something. To give you an example, lets say I want you to make me some “Japanese Kuhoo Kuhoo”. I won’t tell you what it is, or how to make it, but I want you to tell me how much time it would take to make it. You don’t know what you would need to make it. You don’t even know if it is a real thing or something I made up entirely to mess with you. How will you know how much time it would take ? You can’t. You can’t plan for the unknown. If you absolutely want to know how much time it would take, please consult an Oracle. Because their estimates can have as much bearing on reality as anyone else’s.
Thirdly, you must never reward drama. Drama is always a sign of dysfunction, not capacity. Imagine two teams. Team 1 is a well run team. There are unit tests for almost all important functionality. Commits that break unit tests are disallowed automatically from nightly builds. Team 2 gives zero fucks to any sort of automated testing. Result is that days before release, testing reveals lots and lots of bugs. Developers are asked to stay back late to fix issues. Management sees that Team 2is toiling away, burning the midnight oil while people from Team 1 goes home on time. Management rewards Team 2 with bonuses. Until of course the house of cards built by Team 2 just crumbles under its own weight.
Fourthly, and this is not specific to software companies. Accountability should never get lost in layers and layers of management. For example, imagine people found critical security issues during testing just before release. Management is not supposed to say ‘We will focus on these trivial stylistic changes which are more visible to my manager and release the software with security problems’. In big companies it is very much possible for this to happen. Managers know that even if word gets out that the software was released with known security defects, the resulting inquiry would probably find nothing.
Fifthly, there should be penalty for risk-avoidance. For example, lets say that an employee comes to internal IT department with a request to install the Operating System on their Solid State Device. The IT department refuses to install it saying that it is a risk for them since they do not know how to configure backups properly. The Operating System being installed on the SSD can save developer time which is valuable to the company. But the problem here is that there is no downside for IT department to refuse the request. They have only upsides. Ideally, there should be a penalty for the internal IT in the company so that if the company is spending the money for an SSD, if IT is refusing to make use of it they should get penalized.
Sixthly, people should not be allowed to screw company for short term gain. For example, a new CEO can cut R&D spending for software companies if those projects would only lead to products in a few years. Cutting those projects would mean that the expenses would be reduced ( with no reduction in immediate profits ). Slump in profits comes a few years later when the company has no more new products coming out of the pipeline by which time the executive would have left the company. This problem can probably be solved by paying the executive even after him leaving the company; based on the performance of the team he used to manage.
Seventhly, responsibilities must be handed off properly. Lets say ‘Project A’ uses library built by ‘Project B’. When ‘Project B’ obsoletes a version of the library, ‘Project A’ must be given permission to take over maintenance of the library. Or the manager for ‘Project A’ should force the engineers to move away from ‘Project B’ library before it goes out of support.
Eighthly, you must not use metrics such as ‘number of cases solved’ as a metric for productivity. This is the Coca-Cola bottle mentality. If you do this, you will get the behavior that you incentivize. People will pick up easier cases since they can be solved faster. This is bad since when faced with a problem, it is the skill of your programmers that will decide how well and in how much time it gets solved. This skill is like any other muscle, it gets better the more you use it. It is probably a better idea to encourage your programmers to pick hard problems. ‘Number of cases solved’ in no way represents the amount of work done : the quality of the solution matters, the complexity of the problem matters.
Ninethly, you mustn’t assume that if you put more people into a project it will run into completion faster. Putting more people into a late project makes it later. See ‘Mythical Man Month’.
Tenthly, you must not waste your employee’s time. People who want to live meaningful lives will rue it when their work is wasted. They want to do a good job and when they see their work is duplicated, or wasted because of say, management indecision, they are less amused. This problem is compounded by the recent ‘Agile Movement’, which has taken to mean, the managers can change their mind whenever they want. Project direction, once set, should only be changed if market conditions change.
Glen and Sally were having coffee at the coffee shop.
“If you could have one wish what would it be ?” Sally asked.
“I would wish to have infinite wishes”, Glen said without hesitation. He had thought of this before.
“I mean apart from that” Sally said.
“I would wish to have another wish without the restriction to have infinite wishes and then wish to have infinite wishes, but I get your point” Glen said. “I would wish for perpetual perfect regeneration with the ability to end my life whenever I please” he continued.
“That is very particular. Why ?” Sally asked.
“Well, first I would have wanted to live forever, but then what would be the point if you age to say 90 and then live in that state forever ? Definitely you would want to stay young. But then, what if someone cuts off your arms and legs ? You wouldn’t want to live like that forever. See, immortality brings its own set of curses. You wouldn’t want to be forced to end your life because your are handicapped. You would want everything to return to normal. So the ideal way to phrase a good immortality wish is ‘perpetual perfect regeneration’. But what if someone caught you and catapulted you into space ? You would float through space forever having nothing to do. So you would probably want to end your life then. You would probably want to do that anyway at some point.”
“But what would you do with immortality ?” Sally had asked.
“Well, first off I would quit my job.” Glenn said with a laugh.
“Of course” Sally laughed “And then what would you do?”
“Well I would go out and learn everything about everything” Glen said.
“Everything ?” asked Sally.
“Yes, everything.” Glen replied “Physics, Chemistry, Philosophy, Psychology, Sociology, Voodoo magic, you name it.”
“And then what ?” Sally asked.
“Well, I have no idea how long that will take” Glen replied “Step two would be to obtain power and money. I bet being immortal would lend itself nicely to doing really dangerous jobs that has a potential for high pay off. But I am really hoping that there would be some connection in all the knowledge that is available in the world that nobody has made. A connection nobody can possibly make in one lifetime. A source of large amount of power.
After that I would have to try and take over. See, I would have lots of time. All I would need to do is wait for right opportunities. I bet the best places to take over first are the war torn areas in Africa where a power vacuum develops every few decades. The point is try to install people who are obliged to you to serve as your puppets.
Form a secret society, with a hierarchical tree like structure. At each level, you have a number of people overseeing the work of one or more people. Anyone power hungry can join, and take a pseudonym. Nobody knows each others name, how many people there are at each level or how many levels there are. You can kind of ask these people to help each other and act as a power broker. It is important to stay away from the light though”
“Why ?” asked Sally.
“Well, that kind of power would simply attract too much attention. I can definitely see people sacrificing their entire lives to make my life miserable if they can detect my involvement” Glen replied.
“OK” Sally was now starting to regret ever having asked the question.
“Well the next step is to optimize” Glen continued on, unaware of the uneasiness developing in Sally, “Obviously, I feel the worst thing in the world is war. When people are constantly fighting each other they have nothing to look forward to. First thing I would do is stop all the wars. When people feel safe, and they know what they build won’t be taken away from them using force, they will start building for the future. Next step is to educate. The benefits of Enlightenment should percolate to the whole world. Just imagine, billions of people in the developing and under-developed nations joining the quest for Scientific Inquiry. How much more the average human condition would improve! How the international productivity would go through the roof!”
“What then ?” asked Sally
“Well, the next step is to expand outwards, of course. Colonize the moon. Colonize Mars. It might be possible to sustain human life in harsh conditions by then. Expand outward. We have no idea how vast the Universe is. I could be doing this for a while”, Glen finished with a smile.
There was a silence. “Do you know what you are planning to do ? You are planning to become God!” said Sally.
“Not really, for all we know God’s powers could be infinite” said Glen. He was an atheist himself, but he did not want to offend Sally’s feelings.
“But do you think that what you asked for is right ?” asked Sally.
“Ah, I thought you wanted to hear about the most interesting wish I could come up with” said Glen taking a sip of his coffee. “I was not thinking about the morality aspect of it. I wouldn’t do this if the opportunity was real. If a genie appeared before me and granted me one wish, I would first ask him ‘Can you go back in time and fix stuff there ?’, if his answer is ‘Yes’, I would wish for ‘Go back to Big Bang and fix everything from there onward, if there was time before that fix that also, if there are other Universes fix everything there as well. FIX EVERYTHING'” he said waving his hands to express his enthusiasm. “If he can’t go back in time and fix stuff – I mean what a crappy genie, right ? – then I would ask him to fix everything instantly”.
“OK” Sally was smiling approvingly now.
“That is the only moral thing you can do, Sally. Everything else is immoral. Since we are just chatting, I just inferred the question as ‘What is the most interesting thing you would wish for?'” said Glen and then he smiled as well.
**************
Sally walked into the room where G.E.N.I.E ( Generic Electronic Nature Influencer Engine ) was housed. She wanted to take Glen’s advice on what to wish for before entering her wish. Sally had felt a huge burden of responsibility when her research was complete and she realized that they could build exactly one machine that could in fact grant someone one actual wish. The machine could not do wish recursion and therefore wishing for more wishes was futile, that was the first thing she tried. But now she had figured out the ideal wish. The only wish any moral person could make.
She pulled out the console for G.E.N.I.E and typed in her wish :
