|By Jesse Randall Warden||
|April 15, 2009 01:00 PM EDT||
Five weeks ago, I started a project with Enablus, a firm that works with mainly startups to build products. I loathe service work, and love product work, so am really glad to be working with them. Even better, they are only 30 minutes from my house. I don’t really need to go more than 2 to 3 times a week, but I like how close they are when I do. One thing that struck me in the initial project discussions was that Enablus was really big on process and lingo. Darrell Ross, the project manager and partner I’m working with, although hard to read initially, still showed signs of uneasiness when I didn’t give him the answers he was wanting to hear in the initial discussions. For example:
“Do you utilize Continuous Integration?”
“Uh… what the hell is that?”
“You know, everyone on the team checking in code into a source control system every day, utilizing Test Driven Development practices, automating your builds, etc.”
We’d constantly go through this. He’d ask a serious question like, “Do you utilize OOP and an software framework(s)?” and I’d respond with some cynical response questioning the merits of when OOP and frameworks are applied, or what I did and did not like about a particular framework, and how I don’t always officially follow their implementations… never really answering his question and thus not giving him confidence. I was constantly amazed at how structured Darrell made things sound, and he probably wondering how the hell a positive impression of “Jesse Warden” was formed in his head in the first place based on the answers I was giving him.
In all instances, we’d discuss the details of my responses, and he seemed to get the details he needed, feel better, and move on. Too ease his stress, I opened up and said that I’ve never really worked in places that were above a Level 3, and even those I felt never followed the processes he was mentioning, specifically Waterfall or Agile development, to the letter. Later in my consulting and contracting career, I had moved to using some concepts of Agile such as iterative development, and having builds of the software regularly on a schedule (usually weekly) even if the client didn’t need one for weeks or months. It had nothing but positive ramifications in the long run, so I stuck with what worked and saw other successful people doing similar things. It’s really a simple process (my version): Have a build working by Friday, every Friday.
I’m not sure Darrell believed me that everywhere I’ve worked, whether full-time or consulting/contracting, did not use a standardize process such as Agile. Maybe it did and I just wasn’t privy to the details? Again, I never cared till this year. Either way, nothing sounded as strict and defined as Darrell made it sound as he described the Agile process to me. Hence the point of this series of posts.
I’ve never been interested software development processes. To me, it’s always been more important to be really good at the 3 main things that count: knowing your platform, knowing your tools, and knowing the language you code in. The better you know those 3 things, the more capable you’ll be at developing software, helping those on your team develop software, and thus ensure your team will be successful. As Dave Wolfe has said, it takes a village; a great team can ensure success. Whether the project actually succeeds isn’t really up to the software team at that point; hence the second rule to the “8 of 10 software projects fail” is “9 out of 10 are perceived as failures”. Even if you create an awesome piece of software, management, stakeholders, and/or clients can still perceive it as failing.
As I’ve found throughout my career, there have been more pressing issues than “process”, such as:
- getting the dang design comps
- making a workflow where fonts will compile on both machines (Mac and PC)
- coming to a consnsus with team members on how the framework you are using should be implemented
- finding out what the heck we are building really is
- finding out how the heck what we are building really works
- resolving source control issues
There are tons of other things, but each has the capacity to reach fire drill status, and thus become a major time sink and productivity killer. As you know, losing time is the LAST thing you want to have happen in software development.
It also depends on the type of work. Working with Design Agencies, there was no process. Not because Agencies are dumb, but rather because the project timelines Agencies have are typically significantly shorter than traditional software or product development. On average, 2 weeks vs. 4 months. While a deadline can stretch to 4 months, you only work 10 man days on it, and of those 10, 5 are done initially, and the other 5 are done over the course of 2 or 3 months on client changes. In short, if you have 5 days to complete a complicated coding project, you’re more focused on getting things done than you are worrying about encapsulation, continuous integration, and frameworks. Writing code on day 1, and wondering who in the heck has time for UML and test cases. Agencies are deadline driven, whereas in software development, nothing is ever on time.
In consulting, you’re typically working within the constraints of your client’s company. It can go down two different ways. Either you get treated like a service shop, and requirements are thrown over the wall, and you complete how you see fit. Otherwise, you integrate with an existing team, and work within their processes, trying to improve them where you can. Contracting is usually the same way, although, you usually have little to no say in the process.
Even including regular work (W2 at a software shop), all instances can significantly have their chances of success improved through skill. If you’re skillful in the aforementioned 3 areas (platform, tools, language), it doesn’t matter how screwed up things are; you can always depend on skill to make or break a project.
…or so I thought. Going back to the 2nd rule of software development, “9 out of 10 software projects are perceived as failures”, I started to have more of the second rule happening vs. the first. In those projects, more of my time would be spent doing non-software things, like pitching a tent in the project manager’s office, discussing details with stakeholders, drilling sales on what they were really trying to accomplish, etc. In the end, trying to define success to ensure my team and I weren’t setup to fail from the beginning. Being setup to fail, yet having a team with mad skill, results in you months later outside the office smiling on your umpteenth beer regaling all in the pimp architecture you created that never did, nor will, see the light of day. Creating awesomeness with no result is extremely more frustrating than flat out failing. The positive to failing is you can learn from it. Failing, even though technically you succeeded, is just straight ridiculous, and it was at those points I started to become more interested in the software development process. I realized I needed to find a team who knew all these processes I’d read about on the internet blogs.
Enablus is one of those teams. They clearly know Agile, as far as I know it 5 weeks in.
I got Darrell to explain how Agile, specifically the Scrum part, works. This is my paraphrased, and re-interpreted version of our discussions, and my working with the process for a few weeks. If you have a good idea, do market research to ensure it’ll sell, then you just need a good team to execute it. The execution on the software side for product development utilizes an Agile approach.
The Agile approach, for the engineer, starts by defining “User Stories”. User stories are a set of sentences that describe what the user experiences while using the software. These can be derived from the wireframes; design comps aren’t a requirement. A user story can be expressed like so:
Login: User inputs Username and Password and clicks “Start Application”. Login is validated. For all other login instances, the Dashboard is displayed.
Notice there is no implied technology or design construction details. The user story merely describes what the user see’s, what interactions the do, and what the results of those interactions are. That same user story could apply to Flash, Silverlight, AJAX, PHP, etc. It’s technology agnostic; the point here is to document a user experience to ensure we’ve captured it well enough to be understood, and thus executed upon. The same goes for design. It doesn’t say “2 text input fields for username and password, with bold headings to the left, aligned center”. It’s left up to the interpretation of the designer to execute that user story, and hopefully provide direction for the GUI programmer through design comps assuming the user stories were derived from the wireframes, and not the design comps.
Notice also, it’s not a “feature”. It doesn’t say, “The user can login”. That leaves things massively open to interpretation, and solves no business value even if the feature is “completed”. As we all know, not all logins are the same. Documenting features, and executing them is one of the number one ways to be setup to fail, and is also one the main reasons Waterfall doesn’t work. If a software spec says, “The user must be able to log into the system”, but doesn’t describe registration, single sign-on, error scenarios, and how it’s supposed to look, you’ll end up with software that the client doesn’t like.
Some user stories are extremely high level. These are sometimes called meta-stories because they actually encompass a wide array of smaller user stories. Describing a login scenario can have a multitude of smaller user stories within it such as “user is presented with an error dialogue stating that they must first activate their account via the confirmation email before gaining access to the entire application’s feature set”. The level of detail only matters in so much as to make sense to the designer and engineer executing it. If a user story encompasses weeks of coding, it’s probably a meta-story and should be broken down into more manageable user stories that are easily defined, and more easily tackled in a reasonable time frame. What’s reasonable and doable is different for each team.
You create user stories for your entire application. What does, rather what SHOULD, the user be able to see, do, and experience?
These user stories go into a “Backlog”. This backlog is a list of all the user stories, say 50… or a hundred. How many user stories you have are irrelevant. What’s important is that they are all documented in one place called the Backlog. You need to ensure all of the user stories are valid with the stake holders. It also helps to go over them with your entire team to ensure they make sense to everyone involved.
As you move through your project, you may find you’ll need more user stories to account for things you didn’t think about, or for additions/modifications to existing user stories. You then just add them to the back log. It’s normal for a backlog to grow during a project life cycle. However, you do not have to complete the entire backlog to complete the project. A lot of times, the stakeholders will make a business decision that 40 of the 60 user stories is enough to have a great product, and say “ship it”. That’s one of the key benefits to Agile is that you have working software throughout the project, and at anytime can make the determination of when it’s “done”… or is just a version 1.
Once your done with your backlog, you then have your team go through and assign points to each user story. To do that, you need a Point System. A point system is defining a metric for “how challenging something is”. In simple terms, a 1 is easy and a 5 is hard. It doesn’t have to be 1 through 5; it could be 1 through 10 or whatever. It’s best if they are numeric values because you use the points to measure a variety of statistics about your project. Additionally, each team may have their own definitions of what is challenging. While it’s best to use the same 1-whatever metric for the client and server team, the server team will most likely think creating a new server-side method a “1-easy” whereas a client developer a “5-hard”.
Once you’ve defined your point system and gained consensus amongst the team, you then go and assign a point value to each user story. This in essense gives each user story a “value”; how many points a user story worth. If you have multiple team members working on different parts of the system, say server-side and client side, you then have a point value added to each, such as “2 - mostly easy” for the server-side portion and “3 - doable” for the client side portion. The combined value gives the value of the user story, in this case 5.
From these point values, the project manager can then go back to the client, and ask, “With 20 points, what features do you wish to have completed in the first Sprint?”
The client can “purchase” user stories using those 20 points. A Sprint, which I elaborate on below, is an uninteruppted time period in which your team works on completing their assigned user stories. You make a guess in Sprint #1 on how many points your team can complete per Sprint. As time goes on, you start to see a pattern emerge of how many points per Sprint is the team completing. The client and managers can use this metric for planning and resource purposes. You can forecast, somewhat, how many Sprints you have to do to complete 60 points worth of features, and when you’ll have them completed. The chosen user stories that the client purchased are what your team is working on in Sprint #1.
Assigning points is a very important step for the engineering team. You typically don’t change point values mid-project, only assinging points to modified user stories or new user stories.
As I mentioned, Sprints are a period of time that your team works on their assigned user stories. A sprint is however long your team decides it should be to get a working build with the user stories assigned. This could be one week, two, or a month. My current team is using 2 week cycles. It’s important to note the “uniteruppted” part. The client and project manager won’t interject user stories mid-sprint, nor modify your work load. Basically, they don’t bother you. This allows your team to focus on the task at hand, prevent fire drills, and stay productive.
In the examples provided, your team has 2 weeks to complete 20 points worth of user stories. Now, a sprint isn’t just, say 10 days of straight working. You usually have 3 special days. The first day is the kick off. You determine the user stories you and your team members will tackle. A couple days before the last is your “merge day”. It’s when everyone merges the code together from their different branches (or you just make sure trunk works if not using branches, more on that later) to get an idea of where the build is at, and where your team is at with their user stories. This is a final checkpoint before…. the UAT. UAT stands for user acceptance testing and in this case, it’s going over the final working build. Your team collectively goes over each user story, and identify if the build satisfies the user story or not.
As I said, our sprints are 2 weeks in length. The first Monday is spent confirming what user stories are in the current sprint and who’s working on them. We try to work in our own personal branches so the 2nd Wednesday is “Merge Day”, where I take any branches I and my teammate have, merge into trunk, and work out the kinks. We make a mad dash the rest of Wednesday and all of Thursday to finish our remaining user stories, or fix issues with existing ones, or just the build in general. Again, over the course of a sprint, you can start to get a group average of how many points your team is capable of doing in a sprint.
We’ve been using a shared Google Spreadsheet with each Sprint’s user stories in its own tab. We collaboratively use this all the time for reference.
We have a call everyday at 1 pm EST / 10 am PST. Our server-side time + client is out west in Cali, and our Flex team + UX + designer + project manager + rest of the team for other areas is east in Atlanta. We each go around talking about what we completed yesterday, this morning, and what we plan on going the rest of the day. We also cite any roadblocks or issues, and if that involves another team member, we just schedule a separate call, or time to talk over IM, with that specific person. This results in our calls usually lasting 5 minutes each day. I’ve heard of other projects at Enablus having their daily standup lasting longer than 5 minutes, but our team is pretty focused; we know what we need to do.
The majority of my road blocks involve a web service not working like expected suddenly, or questions for the designer on how a part of the design comp is supposed to work. Most of these I just solve on my own by calling the team member outside of the meeting time.
Pros & Cons
There you have it, what Agile is based on what I’ve been told and experienced so far. So how’s it working out so far? Here are the pro’s and con’s for this first entry in the series. I’ll elaborate in future articles, but here’s the quick rundown.
- I’ve never been this stressed in the first two weeks of a project before.
- coding with a “git-r-done” mentality; fast and furious. Not everything has to be uber-architected.
- each developer has their own branches; this makes checkins & merges a lot easier and less stressful
- It’s nice to have something real after just 2 weeks. Gives a great feeling of validation very quickly
- a lot of re-factoring
- creating new user stories later can break older ones
- merge day is stressful
- “need to see it working”; the desire to see working functionality before important functionality decisions can be made isn’t solved
In conclusion, I really like Agile methodology that we’re using so far as it allows me to code some things extremely fast and not have to worry about uber-architecting it. I just have to get it working which I’m good at doing quickly. For the things that matter, I can spend an allocated time which I feel good about doing because I can more easily see over time which areas deserve architecture and which ones do not. The merge day is extremely stressful, but overall I like having an allocated day for it and using separate branches which prevent daily SVN drama.
I’ve never been this stressed in the first two weeks of a project before. The typical scenario is you get excited, and can’t wait to dive in. It’s a happy feeling filled with anticipation. Then, as the deadline approaches, you steadily get stressed out and things become less fun, albeit extremely frustrating. This isn’t always the case, but I’ve continually expected fun in the beginning, stress at the end.
It’s been really nice to be stressed at the beginning; this has implied to me the project won’t be so traditionally bipolar. The same thing happened to my mood once I capped myself at 2 cups of coffee a day. Think of it, you have 2 weeks to create working software; ready, GO! That’s insane, and really cool at the same time. What could you do in 2 weeks? It’s really nice to know that even after just 2 weeks of work on a 4 month (ish) long project, even the first 2 weeks will produce something valid and working.
Stay tuned for #2 in the Agile Chronicles series where I elaborate on the re-factoring challenges.
Everyone knows that truly innovative companies learn as they go along, pushing boundaries in response to market changes and demands. What's more of a mystery is how to balance innovation on a fresh platform built from scratch with the legacy tech stack, product suite and customers that continue to serve as the business' foundation. In his General Session at 19th Cloud Expo, Michael Chambliss, Head of Engineering at ReadyTalk, discussed why and how ReadyTalk diverted from healthy revenue and mor...
Dec. 7, 2016 04:30 PM EST Reads: 1,670
The many IoT deployments around the world are busy integrating smart devices and sensors into their enterprise IT infrastructures. Yet all of this technology – and there are an amazing number of choices – is of no use without the software to gather, communicate, and analyze the new data flows. Without software, there is no IT. In this power panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, Dave McCarthy, Director of Products at Bsquare Corporation; Alan Williamson, Principal...
Dec. 7, 2016 04:15 PM EST Reads: 316
The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal ...
Dec. 7, 2016 03:45 PM EST Reads: 2,245
You have great SaaS business app ideas. You want to turn your idea quickly into a functional and engaging proof of concept. You need to be able to modify it to meet customers' needs, and you need to deliver a complete and secure SaaS application. How could you achieve all the above and yet avoid unforeseen IT requirements that add unnecessary cost and complexity? You also want your app to be responsive in any device at any time. In his session at 19th Cloud Expo, Mark Allen, General Manager of...
Dec. 7, 2016 03:30 PM EST Reads: 1,767
Successful digital transformation requires new organizational competencies and capabilities. Research tells us that the biggest impediment to successful transformation is human; consequently, the biggest enabler is a properly skilled and empowered workforce. In the digital age, new individual and collective competencies are required. In his session at 19th Cloud Expo, Bob Newhouse, CEO and founder of Agilitiv, drew together recent research and lessons learned from emerging and established compa...
Dec. 7, 2016 02:30 PM EST Reads: 939
"IoT is going to be a huge industry with a lot of value for end users, for industries, for consumers, for manufacturers. How can we use cloud to effectively manage IoT applications," stated Ian Khan, Innovation & Marketing Manager at Solgeniakhela, in this SYS-CON.tv interview at @ThingsExpo, held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
Dec. 7, 2016 02:30 PM EST Reads: 4,306
Bert Loomis was a visionary. This general session will highlight how Bert Loomis and people like him inspire us to build great things with small inventions. In their general session at 19th Cloud Expo, Harold Hannon, Architect at IBM Bluemix, and Michael O'Neill, Strategic Business Development at Nvidia, discussed the accelerating pace of AI development and how IBM Cloud and NVIDIA are partnering to bring AI capabilities to "every day," on-demand. They also reviewed two "free infrastructure" pr...
Dec. 7, 2016 02:15 PM EST Reads: 1,110
Financial Technology has become a topic of intense interest throughout the cloud developer and enterprise IT communities. Accordingly, attendees at the upcoming 20th Cloud Expo at the Javits Center in New York, June 6-8, 2017, will find fresh new content in a new track called FinTech.
Dec. 7, 2016 02:15 PM EST Reads: 2,193
Information technology is an industry that has always experienced change, and the dramatic change sweeping across the industry today could not be truthfully described as the first time we've seen such widespread change impacting customer investments. However, the rate of the change, and the potential outcomes from today's digital transformation has the distinct potential to separate the industry into two camps: Organizations that see the change coming, embrace it, and successful leverage it; and...
Dec. 7, 2016 02:15 PM EST Reads: 3,382
"Dice has been around for the last 20 years. We have been helping tech professionals find new jobs and career opportunities," explained Manish Dixit, VP of Product and Engineering at Dice, in this SYS-CON.tv interview at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Dec. 7, 2016 01:45 PM EST Reads: 1,108
SYS-CON Events has announced today that Roger Strukhoff has been named conference chair of Cloud Expo and @ThingsExpo 2017 New York. The 20th Cloud Expo and 7th @ThingsExpo will take place on June 6-8, 2017, at the Javits Center in New York City, NY. "The Internet of Things brings trillions of dollars of opportunity to developers and enterprise IT, no matter how you measure it," stated Roger Strukhoff. "More importantly, it leverages the power of devices and the Internet to enable us all to im...
Dec. 7, 2016 12:30 PM EST Reads: 726
Extracting business value from Internet of Things (IoT) data doesn’t happen overnight. There are several requirements that must be satisfied, including IoT device enablement, data analysis, real-time detection of complex events and automated orchestration of actions. Unfortunately, too many companies fall short in achieving their business goals by implementing incomplete solutions or not focusing on tangible use cases. In his general session at @ThingsExpo, Dave McCarthy, Director of Products...
Dec. 7, 2016 12:30 PM EST Reads: 844
"At ROHA we develop an app called Catcha. It was developed after we spent a year meeting with, talking to, interacting with senior citizens watching them use their smartphones and talking to them about how they use their smartphones so we could get to know their smartphone behavior," explained Dave Woods, Chief Innovation Officer at ROHA, in this SYS-CON.tv interview at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Dec. 7, 2016 12:30 PM EST Reads: 630
We are always online. We access our data, our finances, work, and various services on the Internet. But we live in a congested world of information in which the roads were built two decades ago. The quest for better, faster Internet routing has been around for a decade, but nobody solved this problem. We’ve seen band-aid approaches like CDNs that attack a niche's slice of static content part of the Internet, but that’s it. It does not address the dynamic services-based Internet of today. It does...
Dec. 7, 2016 12:00 PM EST Reads: 1,120
"ReadyTalk is an audio and web video conferencing provider. We've really come to embrace WebRTC as the platform for our future of technology," explained Dan Cunningham, CTO of ReadyTalk, in this SYS-CON.tv interview at WebRTC Summit at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Dec. 7, 2016 11:30 AM EST Reads: 678
20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
Dec. 7, 2016 11:15 AM EST Reads: 2,295
IoT solutions exploit operational data generated by Internet-connected smart “things” for the purpose of gaining operational insight and producing “better outcomes” (for example, create new business models, eliminate unscheduled maintenance, etc.). The explosive proliferation of IoT solutions will result in an exponential growth in the volume of IoT data, precipitating significant Information Governance issues: who owns the IoT data, what are the rights/duties of IoT solutions adopters towards t...
Dec. 7, 2016 11:01 AM EST Reads: 217
WebRTC is the future of browser-to-browser communications, and continues to make inroads into the traditional, difficult, plug-in web communications world. The 6th WebRTC Summit continues our tradition of delivering the latest and greatest presentations within the world of WebRTC. Topics include voice calling, video chat, P2P file sharing, and use cases that have already leveraged the power and convenience of WebRTC.
Dec. 7, 2016 10:30 AM EST Reads: 1,712
Unsecured IoT devices were used to launch crippling DDOS attacks in October 2016, targeting services such as Twitter, Spotify, and GitHub. Subsequent testimony to Congress about potential attacks on office buildings, schools, and hospitals raised the possibility for the IoT to harm and even kill people. What should be done? Does the government need to intervene? This panel at @ThingExpo New York brings together leading IoT and security experts to discuss this very serious topic.
Dec. 7, 2016 10:00 AM EST Reads: 433
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life sett...
Dec. 7, 2016 09:15 AM EST Reads: 7,258