Wednesday, July 31, 2013

The Stars At Night...

Normally, I don’t filter my thoughts or chose my words with particular care. I tend to say what I am thinking without sugarcoating it. However, I spent a lot of time thinking about how I wanted to phrase a sentence. I wanted to be fair, honest, and hopefully not too offensive. So after a week or two of thinking very hard on the subject, here is what I came up with... Australians are fucking retarded when it comes to economics. Seriously, that is the very best that I can do and I think it is completely unfair. Unfair to the mentally handicapped. Mental retardation is the result of a serious genetic issue or severe brain trauma. Meanwhile, the belief that property doubles every seven to ten years is willful ignorance. It requires ignoring every lesson learned from the destruction of the US and European housing bubble and to cover one’s ears while screaming, “La, la, la!” when confronted with irrefutable evidence (http://www.prosper.org.au/wp-content/uploads/2011/07/PhilipSoosBubblingOver1.pdf, http://www.demographia.com/dhi.pdf, or plenty of more). It is dumber than believing in perpetual motion or cold fusion.

Even worse than the belief that property can continue to grow at an unsustainable rate forever is the belief that exorbitant home prices is a good thing. The average Aussie looks at home prices that cost roughly 2-3x what their American counterparts are paying and see this as some kind of vindication in the superiority of living in Australia. If trying to come up with a king’s ransom to every month just to cover the mortgage is vindication, then I will pass. It is not polite to say this, but when we lived in Australia, Julie and I were well within “the 1%”. And yet, there was no way we could afford to live there. I simply could not understand it. If we could not afford to live there, how could the other 99% afford it? The answer... They can’t. Instead, it is a constant cycle of indebtedness and borrowing more to dig themselves out of debt by borrowing against home equity, ironically putting themselves further into debt.

For those that bought roughly twenty years ago, they cannot stop patting themselves on the back for being so “smart”. They see their wealth as a virtue and proof positive that their sacrifice was worth it. To those who bought ages ago, not owning property is a sign of laziness or selfishness. If only we were as hard working and smart as them, we would be wealthy too. Except it ignores the fact that the price of homes of more than doubled in REAL (not nominal) terms while wages have remained stagnant. Buying at these levels is not only stupid, it is impossible. Yet, sadly, for those who do not own property, they do not see a problem. For, they believe, that property will always continue to appreciate at 7-10% annually and that when they finally buy, they will benefit.

And I sat there, on a different continent thousands of miles from home, for weeks and for months feeling like I was crazy. I kept on looking at this insanity and saying to myself, “This does not add up!” Yet everyone I talked to gave me blank stares as if I were the one who were crazy. Finally, as I was days away from going into a full melt down and rocking autistically in a corner trying to come up with a financial model that makes sense, I stumbled across http://www.macrobusiness.com.au. Although I stated previously that Australians are fucking retarded when it comes to economics, obviously that was a sweeping generalization. The reality is that 99.999% of Australians are fucking retarded when it comes to economics. That 1 in 100,000 Australian that is not a fucking retard goes to macrobusiness and contributes mightily.
Finding like minded individuals did wonders for me. It was like I had unplugged from the matrix that is the mainstream media in Australia and found a society of people capable of skepticism and rational thought. Through this community, I felt a sense of validation that something was not right, but I also began to learn about what was causing this distortion. It comes down to government control of the land. The crippling, unsustainable house prices in Australia comes down to a colossal failure by the government of epic proportions.

In Australia, to keep cities “livable”, the government has set up extremely restrictive land usage zones around their cities. Land is released extremely slowly and is grabbed by developers. The developers then do nothing with the land. They wait. And wait. And wait. Finally, when infrastructure is built the developers, ever so slowly, build homes. The homes are released for purchase at a snail’s pace which does not ease the pressure of supply and demand or help to lower the overall price of housing. It is in the developer’s best interest to keep home prices as high as possible to maximize profits.

Most participants at macrobusiness reference studies from the think tank Demographia. If the policy of land release I described above seems absurd and the affects of said policy seem unjust, then the next logical question is what is the right policy? If one wanted to see what happens if you do the exact opposite, then one could take a look at how the state of Texas handles land use. According to some of the smartest people I have ever debated with, the good people of the state of Texas have the best policies for land use IN THE WORLD.

As if my head was not hurting enough at this point; I had discovered I had moved halfway around the world to a country with a worse housing bubble than the US, that the mainstream media and the government was regularly lying to the citizens, and the growth story of Australia was about as real as Ireland’s pre-burst. Now I had to deal with the smartest people in the country almost worshipping the policy of... Texas.

It reminded me of the scene in “The Empire Strikes Back” where Darth Vader reveals to Luke that he is indeed his father. “No! That’s not true! That’s a lie! That’s impossible!”

I am far from an intellectual and do not consider myself to be a cultured person. I did not go to boarding schools on the East Coast nor am I the descendent of several generations of WASPs who can trace their ancestry back to the Mayflower. No, I am a second generation American raised in what was once a small backwater town of Phoenix, AZ. And even I have stereotypes of Texans being hicks stuck in my head. Yet, the more I looked into it, and the more I thought about it, the more I came to realize the truth - just like Luke Skywalker.

It has taken me a very long time to get to a point in my life where I can take a more nuanced view of the world. Things are seldom black and white. Often there is no right or wrong. I identify myself as being politically right of center. Yet although I often disagree with the American Civil Liberties Union (ACLU), I feel like we have a better democracy because of them. Even though I don’t like just about everything they do; I find their members to be passionate, intelligent, and their voice makes a meaningful contribution to society. However, on other issues my black/white, good/evil paradigm still holds. On the issue of gay rights, there is no compelling argument against equality. I have tried very hard to see the opposition’s viewpoint. I can’t. And when it comes to land usage, I find my old paradigm useful. Australia is wrong, parts of the United States are right. The evidence for this bold assertion? The only number that matters, the median multiple (the median sales price of a house divided by the median income).

From http://www.demographia.com/dhi.pdf (I put the link in earlier, seriously stop reading this crap and read this study in its entirety):



Rating
Median Multiple
Severely Unaffordable
5.1 & Over
Seriously Unaffordable
4.1 to 5.0
Moderately Unaffordable
3.1 to 4.0
Affordable
3.0 & Under

On the macro-level, looking at median multiples in major markets with populations over 1,000,000 people, Australia sports a “Severely Unaffordable” rating with a median multiple of 6.5 while the United States is ever so close to “Affordable” with a rating of 3.2. Of course, all real estate is local. So I will present more micro-level numbers:



City
Median Multiple
Austin, TX
3.6
Dallas-Ft. Worth, TX
2.9
Houston, TX
3.0
Seattle, WA
4.8
Los Angeles, CA
6.2
San Diego, CA
6.4
San Francisco, CA
7.8
Melbourne, Australia
7.5
Sydney, Australia
8.3

The central hypothesis of all of Demographia’s research is that the way local governments and municipalities release land affects the price of the homes. As much as I have complained about Australia, there are several California cities that are near or even worse than Melbourne and Sydney. Even my adopted home of Seattle is sporting a “Seriously Unaffordable” rating. Even worse, the people of Seattle have learned nothing from the collapse of home prices in 2006 and I believe that we have re-entered a bubble.

Based on my experience in Australia, high house prices crush the economy. Mortgage payments take away disposable income that could be spent on other things. As a homeowner, it might be fun to sit around congratulating myself on my new found wealth due to the irrational bubble, except I have no real access to said wealth unless I borrow from my house, restart my mortgage, and get further into debt. These bubbles lead to economic booms and busts which make my life as a consultant unstable. I always seem to muddle on through it, but it is difficult and stressful. So what to do?

Well, what if someone came in and said, “I will pay for all of your transaction costs - realtor fees, excise taxes, and moving expenses?” If my house is truly overvalued, I would really have to consider it. What if this person offered my wife a job in an area that is now world famous for NOT  exposing their citizens to housing bubbles by pursuing a sensible land release policy while being radically pro-business? We would have to consider it, wouldn’t we?

WE’RE MOVING TO TEXAS!

A few years ago, I would have never considered it, but now... I don’t know, my eyes have been opened and I think low house prices and a pro-business environment is the best thing for sustainable growth. My heart may long for California, but let’s face it - California is a mess. I am also quite comfortable with my adopted home of Seattle, but there are problems on the horizon here too. Although the natural beauty of the area from the trees, lakes, and mountains will be difficult to replace; this area is stuck in a boom and bust cycle. It’s not helping to live in an area where Microsoft is a large employer. I am not predicting that Microsoft will go out of business, but where I once was a tin-hat wearing conspiracist, I can now count amongst my friends senior level people who have left and publicly called for a 30% layoff which will have deep repercussions on the local economy. Then there is the magical offer that eliminates a lot of the transaction costs of selling and moving, thrown in with the opportunity to significantly lower our mortgage while simultaneously getting a better house... Well, it made us stop and think. Why not?

The Austin Factor

So, we told people slowly, here in our bluer than blue state. Most often we were met with disgusted faces and shocked outrage, “Texas? You’re going to leave this for Texas?”

Then we say, “Austin.”

And the disgust and outrage drains right out of them. “Oh, well Austin is pretty cool.”

Sustainable Growth

Back when I was a Boy Scout, some of the older kids taught us how to syphon gas out of our scout masters’ cars. And syphon we did. We would find a spot away from adult supervision, throw a couple of logs into a circle, douse it in gasoline, and watch it burn. The flames were high and spectacular and then just as quickly, they were gone. It required us to get more gasoline or to eventually give up and find something else to do. These fires were fun for us burgeoning pyromaniacs, but they were also useless. They did not provide heat nor were they useful for cooking.

At some point, we always wandered back to adult supervision and proceeded to build fires the “right” way. We would carefully place kindling in the bottom with smaller sticks around it. Then giant logs would eventually catch fire. We tendered and nurtured these fires for hours and they provided light, warmth, and a way to cook our food.

When it comes to economics and housing policy, I have seen first hand the effects of handing out first home owners grants, choking land supply, or whatever myopic and stupid policies are put in place to prevent the collapse of a bubble. All of this is tantamount to us boys throwing gasoline on fires. It’s fun, but useless, dangerous, and wasteful. I am happy to be going someplace that recognizes the necessity of sustainable growth.

My Last Warning on the Subject

I once stated at a dinner party amongst friends that I wanted to see the Northern Territory and Uluru. Someone I genuinely like responded by saying, “But the Northern Territory is full of Aboriginals.”

To which I said, “So?”

“And not the good kind,” she expanded.

Here I was at a quandary as I found her near blatant racism offensive. On the spot I came back with, “Do you know the difference between an Australian racist and an American racist? (Pause) An American racist knows they are a racist.”

This remark was very poorly received. No one wants to be called a racist. The United States elected a half black man to be our leader and that does not absolve us of our extremely racist history nor does it mean we no longer have a race problem, however, we as a country just might be the greatest mix of races and cultures the world has ever seen. Over a hundred years ago, we opened up our borders taking in just about anyone. My grandparents were kicked out of every nation in Europe before showing up in New York. We had the land and the desire to support growth, and grow we did becoming the world’s largest economy and the third largest nation by population. It has been immigration that is largely responsible for this growth with 40% of the Fortune 500 being founded by an immigrant or one generation removed.

In more recent times, the United States screwed up our economy and did so quite badly. In an effort to stabilize things; there were bank bailouts, zero percent interest rates, “quantitative easing”, and a return to more immigration. As a country, we know that an expanding population base is going to create growth.

Anecdotally, we left for a few years only to find that our area had gone under a rather obvious demographic shift. My son goes to school with children named Umong, Sushant, and Rajesh. The children may look different from my son, but they play Skylanders, root for the Seahawks, and watch Ben 10. They are little Americans just like my son.

Their parents may speak with accents and miss Cricket, but they know they have a better life here in the United States than they did in India. They know their children will have better lives as well. And yes, the parents do represent competition to me, but I do not believe I am entitled to a job. If we are up for the same position, let the best person win. Their presence here makes me work that much harder, makes me learn more on my own time, and it keeps me on top of my game. I have an advantage - I was raised here. I am used to American culture. I speak the language. If I am beat out by an immigrant, so be it. That immigrant is going to earn money here, pay taxes here, and contribute to the economy here. Their presence is going to bring more opportunities to me as the competition is not a negative sum game. I respect these immigrants who have left their extended families behind and put their faith in the United States. It’s not easy to be an immigrant and their presence is helping us out of the economic nightmare of our own making.

Australians need to learn this lesson. Australia should not be a country and continent for white people. It needs to be a country for all people who are willing to contribute. Australia needs to wake up to the fact that competition is good and more people leads to growth. There are people who are literally risking their lives to enter Australia - the least densely populated country in the world and the government and planners ring their hands and ask, “Where will all these people go?”

These people will become contributors - if you let them. They will build cities. They will enrich schools. They will earn incomes and pay taxes. Australia is on the verge of an economic collapse due to the unsustainable model and anti-growth policies that have been pursued for the last century. To ease the pressure of the housing crisis and continue economic development, the urban growth containment policy simply has to be abandoned. At the same time, this small country of 23 million or so people could easily double the population in twenty years through immigration. New cities can be created. People can start doing things instead of flipping real estate to each other in hopes to keep the Ponzi scheme alive.

Already in less than a year since we have left, the currency has fallen by 20%. Even the mainstream media is finally clueing in that the mining boom (which I claim never existed) is over. It is time for the country to grow up, shed its racist past, and acknowledge that decades of bad policy has enslaved its people in debt servitude - not driven true sustainable growth. Oh, and we are moving to Texas.

Tuesday, July 9, 2013

SharePoint is a Colossal Piece of Shit and Should Not Be Used by Anyone

I have a hard time, occasionally separating my facts from my opinions. That being said, I think I can make a compelling case that SharePoint is a colossal piece of shit and should not be used by anyone(tm). First, a bit of my background... I started programming at about eight years old writing simple BASIC programs on a TRS-80. I graduated to writing assembly language and then stopped coding for several years. I took one course in Computer Science at the United States Air Force Academy, got an A, and then never bothered setting up my PC again for several years. I graduated from college, got a job, and then I found out how hourly rates for people who could program ABAP (SAP’s proprietary programming language which looks an awful lot like COBOL). I spent a good year trying to convince people that in spite of the fact that I do not have a Comp Sci degree that I am a good programmer.Somehow, I succeeded in my quest and spent several years as a professional developer in SAP. Y2K came and went and so did my programming career.

Several years later, I was asked to develop a simple expense submission application in SharePoint. SharePoint was selected as a platform because the company I was working for liked to follow every worst practice by Microsoft. Later, I worked on several large scale implementations of SharePoint in Australia and I can say, unequivocally, that SharePoint was always the wrong platform and while it does a lot of things, it does all of them poorly. So, in order of importance...

SOURCE CONTROL AND DEPLOYMENTS
My background in SAP is going to show here. SAP is far from perfect, but one of the things it does exceptionally well is version control and deployments through their basis system. As a developer, every change I made in the dev environment was attached to a transport. When I was satisfied that my configuration changes or code changes were working correctly, I could release my transports. These changes would then be migrated to the test environment. If all went well, the changes would then be migrated to production.

On my largest of several SAP projects, there were regularly scheduled migrations on Tuesdays and Thursdays at 4PM. At about 3:50, someone would walk around saying, “Save your changes, we’re going to release transports in ten minutes!” I would try to finish up whatever I was doing and then I would walk away. I’d go outside or get a cup of coffee or whatever. I would leave my computer for about fifteen to twenty minutes, and every time I came back, the system was ready for me to start working again. Each of these scheduled deployments were a complete non issue. The fact that I am praising this may seem strange, but I have worked on plenty of projects where deployments could take hours or even days. Each deployment seemed like a herculean effort and they never seemed to go right. It seems obvious, but being able to set up distinct environments for development, integration testing, user acceptance testing, production,and optionally sandboxes is not always done and yet it is critical to good development.

In the .NET world, the preferred method of source control is Visual Studio Team Foundation Server (TFS). There are mechanisms for doing deployments to a server and managing it through TFS, however, there is no way to do this for SharePoint. At best, a developer can set up a Virtual Machine that is based off of a production server and then do development work in the VM. Everything can then be bundled into a .WSP file and that .WSP file can be uploaded to a production server, but this mechanism is inexcusably sloppy.

Ideally, any change that is made in development should be available for deployment from code to a configuration file. Since it is only possible to bundle code, it is entirely possible and extremely probable that a configuration file would be modified in development and not on production. Therefore, the .WSP bundled file may work perfectly in dev but not where it counts. Note that I am only talking about dev and production here as the mechanism for deployment is so bad that I can’t possibly imagine a dedicated testing environment.

I have seen an extremely gifted programmer come up with a solution that was comprised of several hundred lines of powershell script to handle deployments. As I said, the developer was very good, but there was always drama with the deployments. The fact that all the custom script had to be written in the first place shows how immature SharePoint is as a platform. If the platform does not have a mechanism for creating environments and handling code AND data migrations, it should not be considered. Period!

Sadly, it is really easy to do some things in SharePoint. Want a new team site? Click a few buttons and voila! No problem. This gives the impression that you can create entire elegant solutions quickly and easily, but going back to my point here, if you cannot manage your environments, then you wind up doing your development and testing in production or find things that work fine in one environment don’t work when you try them in production. It almost forces developers into worst practices while setting expectations from management sky high.

Lists are not relational
Creating lists that can persist data and do rudimentary validation is an absolute breeze in SharePoint. That’s too bad because, again, it gives the illusion of being useful, but lists are awful to work with and lead to horrible applications.

A leading retailer in Australia created a list to handle stock transfers. In a sane system, there would be two tables - a stock transfer header and stock transfer items. The header would contain some basic information about store transfer from, store transfer to, requested by, created on, etc. The items table would then contain the quantity and product SKUs. In the sane, relational world, it would not matter how many items there were. A stock transfer could have one item or it could have fifty. The model will handle either scenario equally well.

Alas, the sad retailer chose to do this on SharePoint and it was done in a list. The list had 150 columns in it and allowed for 30 items in the transfer. The columns asked for the information I identified as belonging in the header and then had columns like SKU1, quantity1, SKU2, quantity2, SKU3, quantity3, and so on. The form the user entered had a hard coded 30 fields regardless if there was one item in the transfer or 30. If there were more than 30, then a separate transfer had to be completed.

So what, you might ask? Yes, it might look ugly, but what does it matter. Well, it matters in the sense that if you ever wanted to ask a business question like what is the most requested item for transfer, it couldn’t be done. In the relational world, the SQL statement to answer the question is trivial. In the list/non-relational world, a product could be in any one of up to 30 columns and the query would have to look in every column to provide the answer.

Speaking of querying lists... Lists reside in SQL Server, but you cannot directly query them. If one writes a C# application that needs to access SharePoint list data, it can be done. In SharePoint 2007, it was done through a goofy implementation called CAML query which, while not particularly difficult to write, is painful to test or modify (but when you’re developing in production, who cares about testing). In SharePoint 2010, the CAML query was more or less dropped in favor of Linq (not to be confused with Microsoft’s corporate instant messenger application Lynq) which was neither better nor worse but different. However, neither system lends itself to simply exposing the list data in a way that can be accessed by SQL. It makes accessing said data difficult to write and test and seems a bit pointless. And yet, it gets worse as lists don’t scale particularly well and Microsoft recommends not using lists larger than 5,000 items - the default list view threshold established in SharePoint 2010.

SharePoint looks like shit
Like most web applications, SharePoint has a master template that can be modified to give the site a particular look and feel. Except SharePoint throws certain things into the default master and it’s not very obvious or intuitive as to what can or cannot be changed. Out of the box, SharePoint looks awful, so just about everyone wants to change it, and yet it takes a ton of effort. Meanwhile, if one were developing on another platform, Twitter’s open source Bootstrap is gaining a lot of traction. There are many free Bootstrap themes that are available, but for less than $20 per theme, one can choose templates that create a very professional look and feel with tons of choices (https://wrapbootstrap.com/). If your site was developed on Python, Rails, Java, or even .NET; I would put aside a few hours to integrate the theme. On SharePoint? Good luck...

SharePoint is not cheap
It’s usually a bad sign when there is no accurate way to get an estimate for what a SharePoint implementation is going to cost. At a minimum, it requires a SQL Server, a SharePoint Server, and a $100/ year/ user license. Then, since it looks shitty and doesn’t really do anything, there is the added cost of implementing it. Then since most implementors are hacks working on a crappy platform and all customizations are not supported, there is the ongoing costs of maintenance.

I have an inside source who used to work at Microsoft Consulting Services who claims that 35% of all SharePoint licenses are “naked CALs”, or put in other terms, companies buy the license and then never actually install it because it costs way too much and it is a pain in the ass. In fact, my friend went on to write applications that work on SharePoint pitching the product to CIOs as a way to claim a successful implementation.

Intangibles
Even now, in its fifth major release, SharePoint does not work on Windows. That’s right, SharePoint only works on Windows Server. If you are developing SharePoint applications, you have to create a VM on your workstation and do your development inside the VM. While not necessarily a show stopper, it seems clumsy and creates some administrative overhead and requires additional setup times and issues with connectivity.

Further, there are plenty of scenarios in SharePoint that have not changed over the years. Google did an amazing job back in 2004 with showing the world how to use AJAX (Asynchronous Javascript And XML). Essentially, when a user loads the gmail client, the page does not refresh, only the elements do. This was absolutely amazing nine years ago, but SharePoint has not caught on. Go to a standard document library and click new. The browser refreshes and a new page is loaded. The user can select a document and upload it only to get redirected to the original page. Maybe I’m nitpicking here, but the product really should take use of AJAX, but I doubt the product managers at Microsoft really care. They are too busy integrating the Office toolbar or Silverlight or something else no one cares about.

Conclusion
Since SharePoint does not have a good mechanism for maintaining multiple environments and since lists are not relational and do not scale; SharePoint is best used as a document repository. Yet, it looks bad and requires a lot of work to make it semi-decent. In my humble opinion, the best thing to do would be to use Dropbox and integrate it with Active Directory at a fraction of the cost and none of the headache.