Friday, July 29, 2016

Sometimes Things Work Out Perfectly or Bypassing the MBAs

So one of my good friends and former/future band mates, Brian is a high level UNIX security guy for Lowe’s Home Improvement. He’s not only a great drummer, but plays guitar better than I do and apparently is so good at his job that they can’t/won’t move him up. In typical corporate fashion they also won’t listen to the recommendations of a guy who has been-there-and-done-that, instead taking direction from MBA type managers.
So a few weeks ago he gets a call from a former supervisor who had jumped ship and gone to The Home Depot in Atlanta. They offer to nearly double his salary and even better promise to give him some authority to implement his own suggestions. Almost too good to be true right?
So on September 9, 2014, Krebs on Security broke the news that Home depot may have been hacked an compromised millions of credit card numbers going back months to April or May of 2014. Not wanting to sign on to a sinking ship, or even worse, get “housecleaned” when upper management comes through and has to make some kind of changes to show that they are in control and on top of the situation. Yes, everyone knows they are neither in control nor on top of any situation and they are just smart enough to have a reasonable expectation of jumping out of bed in the morning and being able to find the floor in the morning.
Not only does Brian turn down the job, but he gets to work the next day to find that Lowe’s knows he has been in Atlanta looking.
Uh-Oh.  So when the MBA types confront my buddy about interviewing with the competition, Brian lets them have it. First he gives the “you are lucky this security breach wasn’t you” speech followed up with “you all couldn’t secure a piece of candy from a baby” (I’m paraphrasing of course) and finally ended up with “there is one guy in this entire organization who can help you and that is me.”
Lowe’s and behold (yea, I went there) they listen. For the first time in recorded human history the guy that actually does the work got listened to over the MBA types who walk around with dirty bums because they can’t figure out how not to have dirty bums. Brian now reports directly to Chief Information Security Officer (CISO), has his own handpicked team to lead, a budget, and best of all bypasses all the demonstrably useless levels of management between those who point direction and those who cut the gears that make the world spin.

Sometimes, things work out perfectly.

Wednesday, July 27, 2016

How To NOT Get Scammed

Ten days ago, I get an email from lokoofficefile@gmail.com asking if my company can handle website design and "do you accept credit cards or check ??" The email closes with "kindly get back to me ASAP so i can send you the job details."

The first problem with this is that web application development is what my company does. The second is the use of the word "kindly." I despise that word and it is only ever used by English as a Second Language (ESL) speakers. Third, the email address is pretty hokey. Fourth, bad grammar and punctuation. Fifth, "do you accept credit cards or check ??" No, I prefer to run my company on a pro bono basis and as such we don't like to get paid. But, business is business so I wrote back and asked him for the details. He responds:


Hi, Here is the job details
I have small scale business which i want to turn into large scale business now and my company is based on importing and exporting of Agriculture products such as Kola Nut, Gacillia Nut and Cocoa so i need a best of the best layout design for it. Can you
handle that for me ?. so i need you to check out this site but i need something more perfect than this if its possible.http://www.agroamerica.com.... the site would only be informational, so i need you to give me an estimate based on the site i

Note:
  1. I want the same number of pages with the example site i gave you to check excluding videos and blogs.
  2. I want only English language
  3. I don't have a domain yet but i want the domain name as
    thecertifiedfarmproduct.com
  4. you will be updating the site for me.
  5. i will be proving the images, logos and content for the site.
  6. i want the site up and running before ending of next month.
  7. My budget is $4000 to $8000

Kindly get back to me with:
  1. an estimate
  2. your cell phone number
  3. And will like to know if you are the owner ??

Regards

Jeff Brown



Hm. What difference does it make that I'm the owner, I answered the email and speak for the company. The company phone number is published on our site at Sentia Systems.  I responded that we can do his project and that I would send over a contract for him to sign and return and we could get started. I copied the legal department and they sent me back a contract for him to sign on the following Monday. He signed it and sent it back. I was shocked. He also sent back this response:


Monte,

Attached is the copy of the sign contract, your estimate cost meet up with my initial budget and also ready to make down payment so that you can start work on the website design for my new company and do you have a manual terminal machine for the processing of my credit card. please advice so we can proceed with deposit.
Await your response.

Thank You

Jeff Brown



Now my spidey sense is really really tingling.

Is the credit card stolen? What does he have to gain by a manual entry process? I respond that he can use his credit card to PayPal me the deposit or he can snail mail me a check. He responds:


check work for me, i will need full name and address for check mailing.

Kindly get back to me so we can get the ball rolling.


Again, with the 'kindly' that sets my teeth on edge. I reply back with "the name of the company is Sentia Systems, Inc, and the address is on the contract." I informed him that we would wait for the check to clear and then begin on his site.

I haven't heard from him since. I get up this morning to a text message from the legal department with this link detailing the exact same scam, just with a different front man.

What is the moral of the story? Like they said at the end of the pre-shift meeting on Hill Street Blues "Let's be careful out there."

Yes, I am old and I remember watching Hill Street Blues in prime time.


This post originally appeared on http://sentiasystems.com/WebLog/Details/3009, today, 7/27/2016.

Monday, July 25, 2016

Software Design: Why You Can't Do Things By Committee

Have you ever noticed that the Government in the United States doesn’t really DO anything? Well, there are lots of meetings and paper shuffled and money changing hands and decisions made, but the work is done by someone else. Yes, all you military people, I know, you are a special case, the exception that proves the rule. So it seems to me that we need a little more military, which has proven extremely effective and a little less politics. Bear with me, the following three vignettes about the original pony car, the Mustang, illustrate my point.  Don't worry if you aren't a car person, you don't have to be.  It is about the process of producing a product.

By now, you should know that we don’t talk about politics and government here, the above is simply an illustration. Here is another: In 1962 a few engineers at Ford decided that they had a vision for a new direction at the company. With several British roadsters and the Chevrolet Corvette for inspiration they designed and built a two seat, V4, mid-engine, space frame super car. They called it the Mustang after the P51 fighter from World War II. Two were produced and the second was a fully functional race car that debuted at the United States Grand Prix in 1962 at Watkins Glen. The car was too complex for regular production so the then CEO at Ford, Lee Iacocca, Instantly created a new genre of automobile by telling those same engineers to produce a new car, but use existing technology (and add a back seat) but keep the same spirit of a sports/racing tradition. In 1964 The Mustang we all know and love hit the showrooms across the nation. The new Mustang was the most successful car produced to that date in terms of velocity of sales, moving 22,000 units the first day, proving that the confluence of brilliant design and execution with strong leadership produces great results.

In 1971 ford launched a new initiative, “Project 80” to produce two new prototypes for the Mustang, the “Ohio” based on a 1970 Maverick platform and the “Arizona” based on a 1971 Pinto platform. Tens of thousands of market research calls were made and 200 handpicked potential customers said that if Ford produced the Arizona they might be interested. The same thing happened with 700 more people in Long Beach. The Arizona was the basis for the new Mustang. In early 1973 the engineers simply couldn’t figure out how to make a Pinto handle like a mustang and came up with a rubber isolation disc that averaged out the loads and dubbed the “toilet seat” for its looks. This allowed production to go forward for the 1974 model year. In September 1973 the Mustang II hit the showrooms and buyers felt betrayed. There was no fire breathing V8. A fully optioned notchback topped $4500, more than most buyers could afford to pay. Barely 18,000 cars sold in the first month, compared to over 22,000 on the first day for the 1964 Mustang.

In 1973 Ford undertook a project to consolidate the European and American versions of various cars they built and make one platform that was sporty and hold 4 or 5 passengers. The code named it The Fox. Taking the new Fox platform and running with it, engineers designed a new body with a lower nose and higher cowl that produced a 6% reduction in drag, giving better fuel economy and better performance. A McPherson strut type suspension replaced the “toilet seat” and was cheaper, more durable and rode and handled better. Because of the lessons learned in the Gas Crisis of the 70s, the Fox Mustangs did come with a 4 cylinder engine, but you could get the 302 V8 and a turbo charged 2.3L that was better balanced and weighed less that the V8, giving similar performance with better fuel mileage. From 1979 to 2005 (26 years!) Ford built and sold 4,243,284 Fox bodied Mustangs making it one of the most successful designs of all time.

What lessons can we learn from all this? First and obviously, you can’t accomplish anything great by committee, of course. You get bogged down in font size and background color and never get any real work done. Second, let the smart people do smart things. If you don’t understand what the smart people do, how they do it, or why they make the decisions they make, you shouldn’t be pointing out the directions of their efforts. Here at Sentia, we call that Manageritis, where the Manager Gland gets swollen and inflamed taking over all normal brain functions. Third, we have to come up with a way to prototype new work in some reasonable amount of time that will let us prove the worth of our new design without spending a lot of time and effort to produce a product that isn’t what the consumer either wants or needs. Ford did this in three months with the Mustang concept in 1962 and that is why we do the things we do here at Sentia. That is why we designed and built the tools to easily generate the software that our clients want and need and to easily get the data in and out of these new applications. We don’t want, and you can’t afford for us to build you a Mustang II. That is why we can do it better, faster and less expensively than anyone else.

Thursday, July 21, 2016

Software: That's Not How This Works. That's Not How Any Of This Works.

I was out Friday night and discussing the current state of the universe with a friend, Aaron Burton, who was a very talented software developer. Notice the was. Even though developing software is physically easy and pays particularly well, Aaron found that like Beatrice in the Esurance commercial who taped pictures to her living room wall, business doesn’t have a clue about the production of quality software, “that’s not how it works, that’s not how any of this works.”

Nope, Aaron decided he would give up his high paying cushy job and start a brewery. Sure, brewing beer with your buddies can be fun, but it is a huge risk, a lot of sweaty work and doesn’t pay particularly well. There is only one Jim Koch in the world, after all. Aaron explained to me that every manager he had ever had was more interested in feathering his nest than getting the job done. I’ve experienced this myself where well intentioned people would walk around shouting “XML! Web Services! Entity Framework! Netezza! nHibernate! WebMethods! MQSeries!” and other things they didn’t understand but had been taught were the way to build good software, but ignoring the big picture. It’s kind of like a “manager” at Ferrari walking into a design engineer’s office with a screwdriver and beating the engineer about the head and shoulders with it shouting “You have to use this, it’s the best!” 

 Well, no, it's not kind of like that, it’s exactly like that.

A few weeks ago in the same kind of venue as last Friday, I made the rather unlikely statement that I could run Bank of America with 20% the people they use. My friend Heather, who works for Wells Fargo and has worked for BofA was understandably curious and asked how I would do that. I told her that first, I would stop development on ALL new projects and let the ship sail under the steam it had built up for about 18 months. Out of all the thousands of developers that would make idle, I’d pick about 20 to help write ONE application that does everything that BofA does. We would literally go to every department head and figure out exactly what that department does and automate it. You don’t need people to run a bank. In fact, people make bad decisions, so they gum up the works at the bank. If you disagree, you may proceed directly back to 2008 and stay there. Do not pass go. Do not collect $200. 

 I know that is a little oversimplified, but I have seen the way these places, and that particular place, is run and I think at least half of what they do is monkey motion. If you aren’t an executive pointing the way into the future, an actuary figuring out how to get to the future that gets pointed out, or literally communicating with a client, you don’t need a job at a bank. Or an insurance company. Or anywhere else. I will add that in a manufacturing environment you have to have someone to bolt the bumper on to the Chevy, but even a lot of that can (and is (and should be)) automated.

If my name were Moynihan this post would make me livid. If your name is Moynihan, you probably are livid, but at the wrong person. I would be mad at the people who were so inefficiently running my company that some half-baked software developer could do it for 20% of what you are spending. You are probably mad at me, and there’s the rub. ”If your company has a bunch of “managers” running around poking screwdrivers at engineers, call me. If you are a “manager” and you walk around poking screwdrivers at engineers, you need to do a few things:
  • Stop
  • Put down the screwdriver, Beatrice
  • Call me, because “That’s not how it works, that’s not how any of this works.”

In defense of business, yes this stuff is hard, and yes there is a lot of it, and no there isn’t anyone that you know you can trust not to lead you astray. That’s why you need to call me.

I have no interest in brewing beer.  I'm less sure that you have no interest in making me want to brew beer.




Originally posted on http://sentiasystems.com

Wednesday, July 20, 2016

Homogeneous Environments: Pick One Technology, Build One Application.

You know, if I needed a truck, I’d probably just go buy one. If I needed a truck that nobody built, I’d probably pick a manufacturer and order all the parts I needed and bolt them together. Given the multitude of trucks of various configurations built today, that is a pretty unlikely scenario, but in the software world I have seen this problem in every business I have ever been in. If you need to build a truck, you don't form a committee to pick and engine and another to pick a transmission and all the things you need.  Small companies use whatever is cheapest or whatever an employee is most familiar with, and large companies use whatever they are sold on the golf course. We end up with Mom and Pop using manual processes and QuickBooks (Never, EVER use QuickBooks for anything, your data is in there for good and you can’t get it out and can’t integrate it with anything else) and the big companies use literally hundreds of small developed and computer-off-the-shelf (COTS) applications with no idea how they work and a cast of thousands to support them.  Even worse, most of them are built for one user and have no security outside of "lock your machine when you leave." Further, some of the COTS applications are amazingly expensive. Moses Cone in Greensboro has spent over $130,000,000 (yes kids, $130 MILLION) to implement Epic Electronic Medical Records (Click here for details) and has little to show for it.  I would let them use ours for free.  Novant Medical Group is also implementing Epic and has over 600 smaller applications to maintain and secure, built in goodness-only-knows what development languages. This is an impossible task for less than an army of support and administration people. Those people are expensive. That doesn’t count the consultants that they have to hire to integrate these COTS applications and make them communicate with each other, or the productivity lost if they don't integrate and just duplicate effort everywhere.  I mean how many times do you have to write your name, address and phone number when you go to the doc/hospital?

There is really only one ‘manufacturer’ of software development tools that vends a complete suite of tools that allows a programmer to build a complete application and that is Microsoft. With SQL Server and Visual Studio you can design and build applications for the web or Windows desktop without having to buy or find any other products. We are even building hybrid applications for mobile devices with minimal Android or iOS code by writing a mobile web application and showing it in a browser control on your phone. Microsoft’s products are a little more expensive, but there is a fairly decent chance that even a junior developer has a familiarity with everything needed to produce an application. If you don’t want to maintain your own servers there are any number of hosting solutions that will take care of all of the server maintenance and patching for you for a fee comparable to a Unix/Linux environment, so it isn’t that much more expensive. Arvixe charges an additional $5/month for Microsoft over LINUX.

Outside the Microsoft world there is a bewildering array of offerings. Of course there is Oracle database engine, but it is even more expensive than SQL Server, Microsoft’s database engine (and Oracle is not offered by Arvixe, consequently). Further, Oracle is difficult to set up, with text files laying all over the network and on client machines, and most Oracle developers use a third part tool called TOAD (I kid you not) to access the database visually. So developers wanting to go down this road usually pick MySQL or PostgreSQL, which are NOT free, contrary to popular belief. These are also difficult to set up and if not set up correctly suffer horrible performance issues. Outside the database issue there is a bewildering array of products to design and build the User Interface: PHP, Perl, Python, SSI, CGI, Ruby on Rails, Flash, and the list is nearly endless. 

What does all this mean for someone who commissions new software or buys it off the shelf? That’s a tough question. If you learn everything you need to know to make a truly informed decision, you may as well just write the software yourself. My recommendation is to use a single technology stack probably Oracle/Java/PHP or Microsoft, and simplify the development of everything you do. You only need one knowledge base, one staff and literally one application, unless you can find the off-the-shelf application that does everything your business needs (and you can't). At Sentia we picked the Microsoft stack and designed and implemented a set of standards that allow us to secure our applications and ease development. We were literally copying and pasting old code into a new window and changing the names to produce new code. Then we came up with a way to automate the copying and pasting. Then we came up with a way to generate the database code for create, read, update, delete and search that we had been writing by hand. Then we came up with a way to look for related information in other tables, like students and classes. We can get a list of all the students in a class or all the classes a student is taking without typing any code at all in that example. Microsoft adopted an existing technology a few years ago called Model/View/Controller (MVC) that will generate web forms based on a model , that thing we were copying and pasting above . Yes, that is the Model in MVC. So from a relational database design Sentia (and Microsoft, let’s be fair) can generate about 80% of a new application. This is why we use one technology stack. This is why you should use one technology stack. This is why you should throw rocks at any other consulting firm that tells you anything that you haven’t read right here first, or who tries to sell you expensive COTS software, because after they sell you expensive COTS software they are going to sell you "integration services" which is going to be a bunch of H-1Bs sitting around copying and pasting data from one application to another for $200 per hour.

I've said for years that I could run Bank Of America (yes, I've worked for them) with about 20% of the people they use.  They provide a service.  The only people the truly need are leaders saying "THIS is the way we do things" and customer service people: tellers.  Every other big building you see with BofA on the side of it is worthless,spending money you give them to do things that don't need to be done, unless you are in the business of having meetings, drinking coffee and eating donuts.  I hear that's not very profitable.  Sure, you need a data center and people to administrate the one application we would build to do what they do, and you have to have traders to make money on the back end.  You CAN automate trading, you SHOULD not.  Some things need a little bit of intuition that you can't teach a computer.  Those people are the 20%.  So what does that mean?  If they did things the way we propose, they could cut their rates by 80% and make the same amount of money.  Your 5% APR Mortgage could be had for 1%.  Pretty soon this new way of doing things would put everyone else out of business.  Winter is coming, BofA, remember that you heard it here first. 

Before you start to feel badly for BofA remember that you are doing things exactly the same way they are, emailing spreadsheets and having little accounting, inventory, Customer Relationship Managers, Point-of-Sale applications et cetera and  ad nauseum that don't communicate with each other and aren't secure.  Ever hear of anyone ever having a data breach?  This is why.  Here is a visual of the biggest data breaches since 2004.  All these companies are doing things the same way you and BofA are.  We know a better way.  Keep it simple, sweetie.

Don’t be Novant Health or Bank Of America.  Lower your costs by 80% and don't appear as a bubble on someone's list of monumental screw-ups.

Call somebody who literally wrote the book on how to do this stuff.

http://sentiasystems.com

Tuesday, July 19, 2016

Replacing Wasted Effort, Real 'Outside the Box Thinking'

Monday, we revealed that we have a way to generate about 80% of a new application.  Today we are going to consider the impact of having that ability.  First though a little clarification: We are NOT using Entity Framework (EF).  EF is a train wreck and an abomination with horrible set based code and leaving access to the table structure open for a SQL Injection attack or worse.  All our software uses stored procedures and has as little middle tier code as possible to get the job done. Back to our regularly scheduled program.

In the beginning, there were ledgers and spreadsheets and these were large pieces of paper filled out by people like Bob Cratchit. Time progressed and seemingly the same people were doing the same things just with the help of a computer. Robert P. Cratchit XIII is still filling out spreadsheets, just without getting ink on his hands. The information from these spreadsheets comes from any number of sources and is delivered to Bob so he can type it in. There must be a better way.

Now, for the first time, we have the ability to generate nearly complete applications that will automate almost every aspect of a business, mostly eliminating the need for all the analysts and the support and data entry people and streamlining the entry, use and analysis of data. Instead of having an Enterprise Resource Planning (ERP) system, like SAP, and an human resources management system, like PeopleSoft, a warehouse management system like Maximo, ad nauseum, all costing six, seven or even eight figures (a CERNER installation will cost a good sized hospital $150,000,000), why don’t we use some of this new technology to generate an application that does precisely what we need to do?

In the Robert Cratchit world of the 21st century, someone has a great idea, gets some funding, buys a ton of equipment and then calls Accenture, Deloitte or Infosys to come build them systems to account for time and materials. The consulting firm comes in, installs and configures all the various system and then leaves a team of developers there to “integrate” the data from these disparate sources and email the data to Mr. Cratchit. Each of these developers bill out between $100 and $200+ per hour and are there for the duration. Integration seemingly is an ongoing project.  It seems to be ongoing because it is ongoing.  You may literally have a bunch of H-1Bs (look it up) sitting around copying and pasting data from one system to another.  Yes, I have seen that happen.  Deloitte pays a resource $15/hour charges your business $200/hour and pockets the difference.  This work is never done.

Now, however, since we can generate the applications that business needs, we don’t need these amazingly expensive, huge applications or the legions of $200/hour 'developers' to make them work and communicate, we simply generate an application. Then we can build customized reports to support whatever it is that the business needs to look at to make Enterprise decisions. In fact there is a business that will generate software for free and only charge the client a small fee per month to use it and house the data.

Why isn’t everyone using this model? In the land of the blind, the one eyed man is going to try to explain ‘purple’ to someone and because it is outside the realm of their experience, they will discount what he says. Different is not always better, but it’s time for business to stop mouthing the tired clichés and actually start “thinking outside the box.”

So with a web based, generated, enterprise application we can now have all our data in one spot, from around the world, at our fingertips. We've automated the efforts of all the people in a company, and now that company spends a couple of hours a month sending out (generated of course) bills. Now THAT'S progressive. 

Monday, July 18, 2016

Conclusion: How To Automate the Production of Software

Over the past few weeks we have been discussing best practices in producing software.  Some of the highlights include:
  • Pick a technology and stick with it.  We choose Microsoft since they are the only vendor that produces everything we need from server to desktop OSs to tools that support database and procedural code.
  • Applications are database driven.
  • The table structure is hidden from everyone except the administrative users, and all data access is done through stored procedures
  • Set based code is the best and the fastest, write as little procedural code as possible and then only to 'objectize' the database entities for consumption by the User Interface.
On to the good stuff. Let's talk about what we are gong to do with all this new code.


Automation

Since all of this is database-centric, we can literally write code to write the procedures and middle tier objects for us and use MVC (or another piece of code we make to consume the middle tier for the desktop, the Data Form Base) to generate the web forms/views for the UI.  We are not going to provide examples of that. Suffice it to say that we can literally generate about 80% of a new application given a properly designed database.  Not only that, but this design is less than half the size and complexity and about twice as fast and efficient as code you might get from one of the big consulting firms like Accenture, Capgemini, Deloitte or others.  Factor in that our rates are about a quarter of theirs and you find that we can build better, faster software for about a tenth of what they might charge, if they could do things of this complexity at all.  Since the code is generated, we don't have to worry about coding standards or checking developers to make sure they are doing what they are supposed to do, or even in many cases, need developers at all.  The last project we completed is in testing now and we spent about 4 hours designing it, 15 seconds to generate the code and a couple of weeks to make it pretty.

Complexity

Above we just mentioned complexity.  The classic salesman problem states that as the number of stops a salesman has to make increases the amount of time to calculate the shortest route also increases.  As the number of stops approaches infinity, the mount of time it takes to solve the problem also approaches infinity.  The only way to break this law is to be able to generate a solution that solves the problem for you.  Being able to generate most of a piece of software means that tackling a really complex job becomes more simple and easy to build and maintain.  With this automated process of generating software we could produce one application that documents everything a business does.  Not only that, but then we could generate reports that would detail the state of the business at any juncture.  The benefit is that you can relate all your information together, like Human Resources, Capital Assets, Structures, Consumables and even processes and what it takes for each step.  This gives you, for the first time, the ability to know what each widget you produce or each service you provide costs before you decide on a price or to accept the price your client offers.
 
What does all this mean?
Everyone has heard of US Air.  What many don’t know is that US Air was the subject of one of the largest non-financial bankruptcy in history.  US Air failed for one reason: they didn’t and still don’t know how much it costs to fly you from Point A to Point B.  They have history.  They know how much it cost last week, last month or last year, but not today or in the future.  If they designed and built one application that documented everything they do, pilots, flight attendants, price of the particular load of kerosene, amortizing the airframe, maintenance costs, consumables consumed on that particular flight, they could have printed a report that showed the cost of every seat on that flight.  If you add, say, 20% on top of that, you have a tidy little profit and you will never go bankrupt.  Even better, with this reporting ability you could find out where the money was going.  There is a model for how much each phase of a process should cost and it would be easy to find which phases were overrunning their budget and fix it. Best of all, every other airline who competes on price will eventually go out of business.  Let's say that again: If your company doesn't do things this way, someone in your industry will and that company will put you out of business.

Let's close with this example: One of our subsidiary companies, Sentia Health has used this process to produce a single application that allows doctors to document the patient's visits, in English, and then take that documentation and pay for the encounter with no other effort needed.  They have made the insurance industry and all its associated costs, obsolete.  Patients can log in and update demographic information, take questionnaires (avoiding the sheaf of papers you have to fill out when you go to the doc) schedule appointments, email the practitioner and have our new health insurance company pay for the encounter in real time, obviating the need for any of the wasted motion the  insurance industry inflicts on patients and practitioners. Gone is the paper and the medical coding and arriven is the Dr. House on your laptop, sans bad attitude and cane.

This is the future and we are building it.

Friday, July 15, 2016

Digging Into the Nitty Gritty: The Graphical User Interface

Over the past two weeks, we have discussed design philosophy database design and development and procedural or middle tier design and development.  Today we are going to discuss the Graphical User Interface (GUI) or Front End.  This is the only part the user sees, so a word to the wise: hire a graphic designer if you can afford one.  we are talking about development here, but you have to make it pleasing to the eye, trust us.  That said, here we go:

Graphical User Interface (UI or GUI) 
For the purposes of this discussion we are going to stick with Microsoft’s Model/View/Controller (MVC) framework since a discussion of both desktop and web is beyond the scope of this book. If you aren't familiar with it, click here.


Singular Objects  Singular objects should have partial views for CRUDS operations that feed the various complete views for the object. Collections related to the singular objects in the same view should be represented in a table/list/dropdown or grid. That means that a credit union object would have a table for the associated branches, with a link to go to the singular branch view in each.

Collection Objects  Collection objects should be the result of the search for the singular object. That means if we want a list of Cars with from a certain manufacturer we instantiate a new Car object, set its ManufacturerID property appropriately, pass it into the constructor that takes a SessionGUID and a Car object and it returns the list of cars from that manufacturer from the database. This is represented in the UI as a table/(dropdown) list box.

Reporting  

It now becomes possible to generate reports automatically as well. Well, semi-automatically.  We use SQL Server Reporting Services (SSRS).  There is no surprise there given our bent toward picking ONE technology.  No more spreadsheets, no more manual entry, no more copying and pasting. Once the report is defined, it just works. As executives, our efforts are dictated by reports. Sales reports, cost analysis, fixed/variable cost ratios, profit and loss are all things that do not require employees to produce and run. With this model these are all automated and can be run at any time. One note: SSRS does NOT play well with MVC.  You have to add a traditional ASP.NET web page inside your MVC application and call it from your controller, passing it the various parameters for your report.  That is probably another post.


Conclusion This all is probably a little confusing and it is complicated.  We've recommended some design decisions I'm sure some of you do not agree with.  there is a method to the madness.  In Monday's post (convenient that we cliffhang you over the weekend then drop the bomb first episode the new season, huh?) We are going to describe the reason we do things the way we do them: Automation. Yup, in Monday's post we are going to show you how we have automated the process of software development to produce about 80% of any given application in seconds with nothing more than a database design.  The last application that was commissioned took about four hours to design, 15 seconds to develop and a couple of weeks to make pretty.  ONE guy for a couple of weeks.

Stay tuned, Same Bat Time, same Bat Channel.

 

Thursday, July 14, 2016

Digging Into The Nitty Gritty: MiddleTier Design and Development

Over the past couple of weeks we have been discussing the most effective way to write software and defining a framework to make that a reality.  Yesterday, we discussed best practices at the database level.  Today we are going to discuss the lowest procedural level, the middle tier. For a detailed explanation of the terms used go here, though we here at Sentia do NOT agree with the discussion of what does and should happen in the middle tier, but we are discussing that below.  Grab a fresh cuppa and enjoy.
 
Middle Tier 
The middle tier should be nothing more than an Object Oriented (OO) wrapper for the database objects and a way to create, read, update, delete and search (CRUDS) them. Advanced OO principles like Dependency Injection or SOLID are basically a way to save state without the use of a database. The database is where state should reside to make is durable, or alternatively as with Inversion of Control (IoC) these truths are self-evident and don’t need to be documented. Below is a discussion of what the middle tier should consist of.
 
 
Stored Procedures
While technically part of the middle tier, as they are not a data store and do contain logic, they should be discussed here. Basically, the rule on procedures is "use them." Do not do any data processing, outside of data validation in the middle tier.

BusinessObjectBase
The purpose of the BusinessObjectBase class is to provide a common interface for all Singular Business objects. It is actually an abstract class as it provides some default housekeeping logic for objects that derive from it, like a string called UniqueID that is a string representation of the EntityID so that you can search a collection of the objects by ordinal or by ID simply by casting the ID int to a String.
 
BusinessObjectCollection
The purpose of the BusinessObjectCollection is to house a one or more objects derived from the BusinessObjectBase class and to Automate CRUDS operations to the database. It has methods like Add, Update and Delete that must be overridden in the derived class.
 
DataAccessor Class
Many of you will have noticed the reference to the DataAccessor class at the top of the two business objects classes. This is instantiated ‘just in time’ in the various overridden add/update and delete methods in the base classes. It depends on a Static class GlobalConnectionString that doesn’t require instantiation and keeps the connection string to the database in a ‘sticky’ fashion, that is, if you don’t want to look in a config file and set the property every time, this class remembers the string for you and uses the last setting automatically. Notice further that there is no coding for ad hoc SQL. This is by design and it wouldn’t work anyway since the user who should be logging in has no access to the tables in the database.

 
BusinessObjectEnumerator Class
This is to allow iteration of objects derived from BusinessObjectBase from objects derived from BusinessObjectCollection.

 
Singular Objects
Note: that is not Singleton objects.  We find those usually create more problems than they solve.  The purpose of the singular object is simple: it represents a row in a table of a database. It has the ability to prepopulate itself from the database given an ID, but has no methods only properties corresponding to the columns in the database. It derives from the BusinessObjectBase class and an example of the CreditUnion object discussed above can be seen in Appendix A - CreditUnion Singular Object Example. There are several things to note here. First, notice the Constructor designed to return a CreditUnion object based on a CreditUnionID and SessionGUID. Second notice the SessionGUID itself and recall that we can limit the credit unions a user sees based on the user type or on which credit union they belong to.

 
Collection Objects
The purpose of the collection object is to take care of the housekeeping for rows in the database (represented by the singular objects), and to automate CRUDS (Create, Read, Update, Delete and Search) operations. Further, at the top of every collection object there is an enumerator which describes a way to retrieve a collection of singular objects by the related information in the database. That means that I automatically can get a list of credit unions by providing a BranchID, or a little more appropriately get a collection of Car objects (Cars) who have a particular type if tire installed simply by providing a TireID. This is what we mean by leveraging the power of the database. In the bad old days we might have returned a list of cars which is inefficient across the network, and a list of tires and iterated through both looking for matches. This is the epitome of procedural programming and why it is not optimal. The set based approach used here returns only the objects needed saving both network traffic and processing time. For small data sets, this could be a degree of magnitude faster, and as the two datasets increase in size the performance gap gets wider and wider.
 
Web Service
The object model described above is perfect for a web service. For a web application, there is no need for a web service however and having an installed desktop application doesn’t make much sense with a remote database, except in certain instances where what is necessary simply can’t be done in a browser.
 
 So yes, the Enumerator class is an artifact of the .NET framework.  Yes, you can use a plain old generic list object (List<T>) in place of a collection class, but you are going to have to eventually write some kind of custom code that you don't have access to in the List<T>.  Even better, when you dimension a new collection object you make it

Tires tires = new Tires(CarID);

instead of

Tires tires = new List<Tire>.....uh how do we get a list of tires for this particular car?

Tomorrow we will take a look at the (Graphical) User Interface (GUI) and how best to tackle developing the things that should be there.

Wednesday, July 13, 2016

Digging In To The Nitty Gritty: Database Development and Best Practices

For the past week or two we've been delving into Overall Design and Design Philosophy.  Today we are going to give specifics about what to do and what not to do in the Database tier and why.

Database

There are thousands of books written about the philosophy of database design.  The fact is that each design has its own strengths and weaknesses.  We will start with the assumption that the reader is familiar with the Boyce-Coddnormal forms. 

Again, Use English


Table names are Singular
We realize that every table is a collection of rows, we don’t have to have the additional ‘s’ or pluralize it to know that. 

Don’t Hate Vowels

When we eliminate vowels, we promote the use of codes.  Our job is to make applications that are easy to understand, easy to use and require no specialized knowledge.  Make Table and Column Names descriptive of what they contain and easy to understand. 

Don’t Use Prefixes or Suffixes

The table containing student information should be named Student.  If you have a need to group the table with other tables do it in a Schema.  PersonalInformation.Student instead of PIStudent

Primary Keys

Primary keys are either integers (int or larger bigint) or Globally Unique Identifiers (GUID). If you use int or bigint. Use IDENTITY to generate the keys.  If you use a GUID, set a default value of NEWID(). The keys have no intrinsic or extrinsic value, that is, they uniquely identify a row in a table and contain no other information.  This is vitally important.  For example, in the Retailer table the StoreID should never be shown to the user.  If it is the user will begin referring to the Montgrove Rd. location as Store number 27.  Which sounds great until that location closes, and the new location is 11379 and there isn’t a spot on the Distribution Center line for a box labelled 11379 so it sits in the floor while there is a hole in spot number 27.  Pickers in the center literally walk from box 26 to the end of the line drop products into the 11379 box and walk back to the 28 box in the DC.  Yes, this is going on right now in every Ross Distribution Center in the world. 

Discrete v. Continuous Values

In case you aren’t familiar, a discrete value is something you might pick from a list of values, that is, Zip Code, Gender or Transaction Type.  A Continuous value is everything else, like length, Total Amount or First Name. 

Discrete values should always have their own lookup table in the database. Yes, this sounds like a lot of extra work, but we will show you how to avoid every bit of this work and to have the flexibility to modify that table later.  Yes, I know some things will never change, like gender.  What?  Gender is more than Male and Female now?  Never say never.

Relationships

Database relationships are one of only two reasons we use the relational database (the other is indexes).  Generally any relationships will be on indexed values in a well-designed database so they work together.  SQL Server enforces relationships.  This is so some user can’t put an alternator in your car’s tire table.  Relationships also allow you to use the abovementioned lookup tables.  Sometimes it is less than convenient to translate the TireID of 35216 in the CarTire linking table to the TireName of ‘Michelin Pilot Sport Cup2 345/30ZR20’ in the Tire table.  If you find a need to use this table frequently, build a view to return the information the user expects to see instead of the ID column.

Be Additive

An update is a delete and a re-add.  A delete is incredibly expensive and should be avoided except in instances where it is necessary.  Instead of updates and deletes, add state instead of change state.  When the student in our example registers for classes in the spring, we don’t delete or update the fall registration, we add the new classes to the StudentClass linking table with a new SemesterID

Linking Tables

We’ve made several references to linking tables above without really defining them.  With the normal forms come two kinds of relationships One-to-Many where the Primary Key of the linked table is inside the primary table as a Foreign Key, and Many-to-Many where there is a table between the primary and the linked table.

One To Many Relationship Example
 
 
Many To Many Relationship Example
 

There are a couple of things to note here:
·         The StudentClass table is named according to which tables it links
·         The StudentClass table does not have a compound primary key, but has its own key generated at the time of insert.  This avoids having a key that encompasses all of the columns and probably doesn’t really describe the data anyway
·         Notice that StudentClass and Class both have Foreign Keys that point to other tables instead of implicit, hardcoded values (SemesterID and MajorID not shown)
·         The ‘Student’ in front of FirstName is a preference and not part of the framework.  Don’t use it if you don’t like it and cite the “don’t use prefixes and suffixes” rule.  The ID column however probably should use the prefix since it will be used to relate tables together.  We don’t want to relate ClassID with ID in the Class table, nor do we want to have ID several times as foreign keys to several different tables.

Security

Security is a very very sticky wicket in the modern world and we hear about security breaches almost every day.  The steps outlined here will secure the data in software designed according to the principles in this book.  That means that your company will NEVER appear on the front page of the Wall StreetJournal for a security breach and you won’t lose your job or worse your company for having one.  ...a data breach, not a job.

Create a Database User/Login with only guest permissions

Standard has been that this login is named <databasename>User.  Allow that user to see and login to the database as a guest but with no other permissions.  Deny 'DENY ALTER, CONTROL, DELETE, INSERT, REFERENCES, SELECT, TAKE OWNERSHIP, UPDATE, VIEW CHANGE TRACKING, VIEW DEFINITION’ on all user defined tables. Grant execute on Additive Stored procedures.  There is a procedure in Appendix A - Set Permissions Stored Procedure that will do this automatically.
 

Create a [User] (or equivalent) Table

Include whatever information you like about the user, but make a column that holds a GUID identifying the User’s session.  Traditionally I have called it SessionGUID.
 
[User] Table Example
 
 
In this example we are modeling users at credit unions.  Since we have sensitive information about clients, we want to limit what each Credit Union Employee can see to the Clients of his or her particular credit union.

Create a LogIn Stored Procedure

The purpose of this procedure is to check the username and password of the requesting user and to deny them access in case of failure or to set the SessionGUID and allow them to see what they are supposed to in the application. 

Create a LogOut Stored Procedure

The purpose of this procedure is to set the logout time and delete the SessionGUID from the [User] table effectively locking the user out of the application until such time as they log in again.  An example LogOut procedure can be seen in Appendix A – LogOut Stored Procedure.

Create a ValidateUser Stored Procedure

The purpose of this procedure is to validate every incoming call to the database. It takes as a parameter the SessionGuid and returns the UserTypeID which can in turn be used to limit what the user can and cannot see in the database.


Create a TimeOut Stored Procedure

The purpose of this procedure is to check on a schedule whether any users have not had activity in the timeout period (see ValidateUser to see and set the timeout period).  If not, the procedure sets the LastActivity and SessionGuid in the [User] table to null and Logs the Logout in the UserAudit table.  Discussion of the UserAudit table is beyond the scope of this book.  Schedule the TimeOut procedure to run automatically in SQL Server Agent every <time out minutes value> -1 minutes.   
What we have accomplished here is a simple way to completely secure your data with a custom security process at the very lowest level.  This combats every kind of data theft including SQL Injection and should be virtually impervious to any kind of attack.  This does not require the use of Enterprise Service Bus (ESB), Message Queueing (MQ) or any other kind of external permissions tracking devices.
Tomorrow, We are going to look at the various and sundry best practices for your middle tier objects.  Stay tuned: Same Bat Time, Same Bat Channel.