|By Jesse Randall Warden||
|April 15, 2009 01:00 PM EDT||
Five weeks ago, I started a project with Enablus, a firm that works with mainly startups to build products. I loathe service work, and love product work, so am really glad to be working with them. Even better, they are only 30 minutes from my house. I don’t really need to go more than 2 to 3 times a week, but I like how close they are when I do. One thing that struck me in the initial project discussions was that Enablus was really big on process and lingo. Darrell Ross, the project manager and partner I’m working with, although hard to read initially, still showed signs of uneasiness when I didn’t give him the answers he was wanting to hear in the initial discussions. For example:
“Do you utilize Continuous Integration?”
“Uh… what the hell is that?”
“You know, everyone on the team checking in code into a source control system every day, utilizing Test Driven Development practices, automating your builds, etc.”
We’d constantly go through this. He’d ask a serious question like, “Do you utilize OOP and an software framework(s)?” and I’d respond with some cynical response questioning the merits of when OOP and frameworks are applied, or what I did and did not like about a particular framework, and how I don’t always officially follow their implementations… never really answering his question and thus not giving him confidence. I was constantly amazed at how structured Darrell made things sound, and he probably wondering how the hell a positive impression of “Jesse Warden” was formed in his head in the first place based on the answers I was giving him.
In all instances, we’d discuss the details of my responses, and he seemed to get the details he needed, feel better, and move on. Too ease his stress, I opened up and said that I’ve never really worked in places that were above a Level 3, and even those I felt never followed the processes he was mentioning, specifically Waterfall or Agile development, to the letter. Later in my consulting and contracting career, I had moved to using some concepts of Agile such as iterative development, and having builds of the software regularly on a schedule (usually weekly) even if the client didn’t need one for weeks or months. It had nothing but positive ramifications in the long run, so I stuck with what worked and saw other successful people doing similar things. It’s really a simple process (my version): Have a build working by Friday, every Friday.
I’m not sure Darrell believed me that everywhere I’ve worked, whether full-time or consulting/contracting, did not use a standardize process such as Agile. Maybe it did and I just wasn’t privy to the details? Again, I never cared till this year. Either way, nothing sounded as strict and defined as Darrell made it sound as he described the Agile process to me. Hence the point of this series of posts.
I’ve never been interested software development processes. To me, it’s always been more important to be really good at the 3 main things that count: knowing your platform, knowing your tools, and knowing the language you code in. The better you know those 3 things, the more capable you’ll be at developing software, helping those on your team develop software, and thus ensure your team will be successful. As Dave Wolfe has said, it takes a village; a great team can ensure success. Whether the project actually succeeds isn’t really up to the software team at that point; hence the second rule to the “8 of 10 software projects fail” is “9 out of 10 are perceived as failures”. Even if you create an awesome piece of software, management, stakeholders, and/or clients can still perceive it as failing.
As I’ve found throughout my career, there have been more pressing issues than “process”, such as:
- getting the dang design comps
- making a workflow where fonts will compile on both machines (Mac and PC)
- coming to a consnsus with team members on how the framework you are using should be implemented
- finding out what the heck we are building really is
- finding out how the heck what we are building really works
- resolving source control issues
There are tons of other things, but each has the capacity to reach fire drill status, and thus become a major time sink and productivity killer. As you know, losing time is the LAST thing you want to have happen in software development.
It also depends on the type of work. Working with Design Agencies, there was no process. Not because Agencies are dumb, but rather because the project timelines Agencies have are typically significantly shorter than traditional software or product development. On average, 2 weeks vs. 4 months. While a deadline can stretch to 4 months, you only work 10 man days on it, and of those 10, 5 are done initially, and the other 5 are done over the course of 2 or 3 months on client changes. In short, if you have 5 days to complete a complicated coding project, you’re more focused on getting things done than you are worrying about encapsulation, continuous integration, and frameworks. Writing code on day 1, and wondering who in the heck has time for UML and test cases. Agencies are deadline driven, whereas in software development, nothing is ever on time.
In consulting, you’re typically working within the constraints of your client’s company. It can go down two different ways. Either you get treated like a service shop, and requirements are thrown over the wall, and you complete how you see fit. Otherwise, you integrate with an existing team, and work within their processes, trying to improve them where you can. Contracting is usually the same way, although, you usually have little to no say in the process.
Even including regular work (W2 at a software shop), all instances can significantly have their chances of success improved through skill. If you’re skillful in the aforementioned 3 areas (platform, tools, language), it doesn’t matter how screwed up things are; you can always depend on skill to make or break a project.
…or so I thought. Going back to the 2nd rule of software development, “9 out of 10 software projects are perceived as failures”, I started to have more of the second rule happening vs. the first. In those projects, more of my time would be spent doing non-software things, like pitching a tent in the project manager’s office, discussing details with stakeholders, drilling sales on what they were really trying to accomplish, etc. In the end, trying to define success to ensure my team and I weren’t setup to fail from the beginning. Being setup to fail, yet having a team with mad skill, results in you months later outside the office smiling on your umpteenth beer regaling all in the pimp architecture you created that never did, nor will, see the light of day. Creating awesomeness with no result is extremely more frustrating than flat out failing. The positive to failing is you can learn from it. Failing, even though technically you succeeded, is just straight ridiculous, and it was at those points I started to become more interested in the software development process. I realized I needed to find a team who knew all these processes I’d read about on the internet blogs.
Enablus is one of those teams. They clearly know Agile, as far as I know it 5 weeks in.
I got Darrell to explain how Agile, specifically the Scrum part, works. This is my paraphrased, and re-interpreted version of our discussions, and my working with the process for a few weeks. If you have a good idea, do market research to ensure it’ll sell, then you just need a good team to execute it. The execution on the software side for product development utilizes an Agile approach.
The Agile approach, for the engineer, starts by defining “User Stories”. User stories are a set of sentences that describe what the user experiences while using the software. These can be derived from the wireframes; design comps aren’t a requirement. A user story can be expressed like so:
Login: User inputs Username and Password and clicks “Start Application”. Login is validated. For all other login instances, the Dashboard is displayed.
Notice there is no implied technology or design construction details. The user story merely describes what the user see’s, what interactions the do, and what the results of those interactions are. That same user story could apply to Flash, Silverlight, AJAX, PHP, etc. It’s technology agnostic; the point here is to document a user experience to ensure we’ve captured it well enough to be understood, and thus executed upon. The same goes for design. It doesn’t say “2 text input fields for username and password, with bold headings to the left, aligned center”. It’s left up to the interpretation of the designer to execute that user story, and hopefully provide direction for the GUI programmer through design comps assuming the user stories were derived from the wireframes, and not the design comps.
Notice also, it’s not a “feature”. It doesn’t say, “The user can login”. That leaves things massively open to interpretation, and solves no business value even if the feature is “completed”. As we all know, not all logins are the same. Documenting features, and executing them is one of the number one ways to be setup to fail, and is also one the main reasons Waterfall doesn’t work. If a software spec says, “The user must be able to log into the system”, but doesn’t describe registration, single sign-on, error scenarios, and how it’s supposed to look, you’ll end up with software that the client doesn’t like.
Some user stories are extremely high level. These are sometimes called meta-stories because they actually encompass a wide array of smaller user stories. Describing a login scenario can have a multitude of smaller user stories within it such as “user is presented with an error dialogue stating that they must first activate their account via the confirmation email before gaining access to the entire application’s feature set”. The level of detail only matters in so much as to make sense to the designer and engineer executing it. If a user story encompasses weeks of coding, it’s probably a meta-story and should be broken down into more manageable user stories that are easily defined, and more easily tackled in a reasonable time frame. What’s reasonable and doable is different for each team.
You create user stories for your entire application. What does, rather what SHOULD, the user be able to see, do, and experience?
These user stories go into a “Backlog”. This backlog is a list of all the user stories, say 50… or a hundred. How many user stories you have are irrelevant. What’s important is that they are all documented in one place called the Backlog. You need to ensure all of the user stories are valid with the stake holders. It also helps to go over them with your entire team to ensure they make sense to everyone involved.
As you move through your project, you may find you’ll need more user stories to account for things you didn’t think about, or for additions/modifications to existing user stories. You then just add them to the back log. It’s normal for a backlog to grow during a project life cycle. However, you do not have to complete the entire backlog to complete the project. A lot of times, the stakeholders will make a business decision that 40 of the 60 user stories is enough to have a great product, and say “ship it”. That’s one of the key benefits to Agile is that you have working software throughout the project, and at anytime can make the determination of when it’s “done”… or is just a version 1.
Once your done with your backlog, you then have your team go through and assign points to each user story. To do that, you need a Point System. A point system is defining a metric for “how challenging something is”. In simple terms, a 1 is easy and a 5 is hard. It doesn’t have to be 1 through 5; it could be 1 through 10 or whatever. It’s best if they are numeric values because you use the points to measure a variety of statistics about your project. Additionally, each team may have their own definitions of what is challenging. While it’s best to use the same 1-whatever metric for the client and server team, the server team will most likely think creating a new server-side method a “1-easy” whereas a client developer a “5-hard”.
Once you’ve defined your point system and gained consensus amongst the team, you then go and assign a point value to each user story. This in essense gives each user story a “value”; how many points a user story worth. If you have multiple team members working on different parts of the system, say server-side and client side, you then have a point value added to each, such as “2 - mostly easy” for the server-side portion and “3 - doable” for the client side portion. The combined value gives the value of the user story, in this case 5.
From these point values, the project manager can then go back to the client, and ask, “With 20 points, what features do you wish to have completed in the first Sprint?”
The client can “purchase” user stories using those 20 points. A Sprint, which I elaborate on below, is an uninteruppted time period in which your team works on completing their assigned user stories. You make a guess in Sprint #1 on how many points your team can complete per Sprint. As time goes on, you start to see a pattern emerge of how many points per Sprint is the team completing. The client and managers can use this metric for planning and resource purposes. You can forecast, somewhat, how many Sprints you have to do to complete 60 points worth of features, and when you’ll have them completed. The chosen user stories that the client purchased are what your team is working on in Sprint #1.
Assigning points is a very important step for the engineering team. You typically don’t change point values mid-project, only assinging points to modified user stories or new user stories.
As I mentioned, Sprints are a period of time that your team works on their assigned user stories. A sprint is however long your team decides it should be to get a working build with the user stories assigned. This could be one week, two, or a month. My current team is using 2 week cycles. It’s important to note the “uniteruppted” part. The client and project manager won’t interject user stories mid-sprint, nor modify your work load. Basically, they don’t bother you. This allows your team to focus on the task at hand, prevent fire drills, and stay productive.
In the examples provided, your team has 2 weeks to complete 20 points worth of user stories. Now, a sprint isn’t just, say 10 days of straight working. You usually have 3 special days. The first day is the kick off. You determine the user stories you and your team members will tackle. A couple days before the last is your “merge day”. It’s when everyone merges the code together from their different branches (or you just make sure trunk works if not using branches, more on that later) to get an idea of where the build is at, and where your team is at with their user stories. This is a final checkpoint before…. the UAT. UAT stands for user acceptance testing and in this case, it’s going over the final working build. Your team collectively goes over each user story, and identify if the build satisfies the user story or not.
As I said, our sprints are 2 weeks in length. The first Monday is spent confirming what user stories are in the current sprint and who’s working on them. We try to work in our own personal branches so the 2nd Wednesday is “Merge Day”, where I take any branches I and my teammate have, merge into trunk, and work out the kinks. We make a mad dash the rest of Wednesday and all of Thursday to finish our remaining user stories, or fix issues with existing ones, or just the build in general. Again, over the course of a sprint, you can start to get a group average of how many points your team is capable of doing in a sprint.
We’ve been using a shared Google Spreadsheet with each Sprint’s user stories in its own tab. We collaboratively use this all the time for reference.
We have a call everyday at 1 pm EST / 10 am PST. Our server-side time + client is out west in Cali, and our Flex team + UX + designer + project manager + rest of the team for other areas is east in Atlanta. We each go around talking about what we completed yesterday, this morning, and what we plan on going the rest of the day. We also cite any roadblocks or issues, and if that involves another team member, we just schedule a separate call, or time to talk over IM, with that specific person. This results in our calls usually lasting 5 minutes each day. I’ve heard of other projects at Enablus having their daily standup lasting longer than 5 minutes, but our team is pretty focused; we know what we need to do.
The majority of my road blocks involve a web service not working like expected suddenly, or questions for the designer on how a part of the design comp is supposed to work. Most of these I just solve on my own by calling the team member outside of the meeting time.
Pros & Cons
There you have it, what Agile is based on what I’ve been told and experienced so far. So how’s it working out so far? Here are the pro’s and con’s for this first entry in the series. I’ll elaborate in future articles, but here’s the quick rundown.
- I’ve never been this stressed in the first two weeks of a project before.
- coding with a “git-r-done” mentality; fast and furious. Not everything has to be uber-architected.
- each developer has their own branches; this makes checkins & merges a lot easier and less stressful
- It’s nice to have something real after just 2 weeks. Gives a great feeling of validation very quickly
- a lot of re-factoring
- creating new user stories later can break older ones
- merge day is stressful
- “need to see it working”; the desire to see working functionality before important functionality decisions can be made isn’t solved
In conclusion, I really like Agile methodology that we’re using so far as it allows me to code some things extremely fast and not have to worry about uber-architecting it. I just have to get it working which I’m good at doing quickly. For the things that matter, I can spend an allocated time which I feel good about doing because I can more easily see over time which areas deserve architecture and which ones do not. The merge day is extremely stressful, but overall I like having an allocated day for it and using separate branches which prevent daily SVN drama.
I’ve never been this stressed in the first two weeks of a project before. The typical scenario is you get excited, and can’t wait to dive in. It’s a happy feeling filled with anticipation. Then, as the deadline approaches, you steadily get stressed out and things become less fun, albeit extremely frustrating. This isn’t always the case, but I’ve continually expected fun in the beginning, stress at the end.
It’s been really nice to be stressed at the beginning; this has implied to me the project won’t be so traditionally bipolar. The same thing happened to my mood once I capped myself at 2 cups of coffee a day. Think of it, you have 2 weeks to create working software; ready, GO! That’s insane, and really cool at the same time. What could you do in 2 weeks? It’s really nice to know that even after just 2 weeks of work on a 4 month (ish) long project, even the first 2 weeks will produce something valid and working.
Stay tuned for #2 in the Agile Chronicles series where I elaborate on the re-factoring challenges.
SYS-CON Events announced today that Tintri Inc., a leading producer of VM-aware storage (VAS) for virtualization and cloud environments, will exhibit at the 18th International CloudExpo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, New York, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
May. 27, 2016 04:15 AM EDT Reads: 2,263
SYS-CON Events announced today that BMC Software has been named "Siver Sponsor" of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2015 at the Javits Center in New York, New York. BMC is a global leader in innovative software solutions that help businesses transform into digital enterprises for the ultimate competitive advantage. BMC Digital Enterprise Management is a set of innovative IT solutions designed to make digital business fast, seamless, and optimized from mainframe to mo...
May. 27, 2016 01:45 AM EDT Reads: 2,084
SYS-CON Events announced today that EastBanc Technologies will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. EastBanc Technologies has been working at the frontier of technology since 1999. Today, the firm provides full-lifecycle software development delivering flexible technology solutions that seamlessly integrate with existing systems – whether on premise or cloud. EastBanc Technologies partners with p...
May. 27, 2016 01:30 AM EDT Reads: 2,141
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, wh...
May. 27, 2016 01:30 AM EDT Reads: 1,951
Companies can harness IoT and predictive analytics to sustain business continuity; predict and manage site performance during emergencies; minimize expensive reactive maintenance; and forecast equipment and maintenance budgets and expenditures. Providing cost-effective, uninterrupted service is challenging, particularly for organizations with geographically dispersed operations.
May. 27, 2016 01:00 AM EDT Reads: 1,883
SYS-CON Events announced today that Alert Logic, Inc., the leading provider of Security-as-a-Service solutions for the cloud, will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. Alert Logic, Inc., provides Security-as-a-Service for on-premises, cloud, and hybrid infrastructures, delivering deep security insight and continuous protection for customers at a lower cost than traditional security solutions. Ful...
May. 27, 2016 12:45 AM EDT Reads: 2,708
The IoT is changing the way enterprises conduct business. In his session at @ThingsExpo, Eric Hoffman, Vice President at EastBanc Technologies, discuss how businesses can gain an edge over competitors by empowering consumers to take control through IoT. We'll cite examples such as a Washington, D.C.-based sports club that leveraged IoT and the cloud to develop a comprehensive booking system. He'll also highlight how IoT can revitalize and restore outdated business models, making them profitable...
May. 27, 2016 12:45 AM EDT Reads: 2,773
The essence of data analysis involves setting up data pipelines that consist of several operations that are chained together – starting from data collection, data quality checks, data integration, data analysis and data visualization (including the setting up of interaction paths in that visualization). In our opinion, the challenges stem from the technology diversity at each stage of the data pipeline as well as the lack of process around the analysis.
May. 27, 2016 12:15 AM EDT Reads: 1,359
Designing IoT applications is complex, but deploying them in a scalable fashion is even more complex. A scalable, API first IaaS cloud is a good start, but in order to understand the various components specific to deploying IoT applications, one needs to understand the architecture of these applications and figure out how to scale these components independently. In his session at @ThingsExpo, Nara Rajagopalan is CEO of Accelerite, will discuss the fundamental architecture of IoT applications, ...
May. 27, 2016 12:00 AM EDT Reads: 1,149
SYS-CON Events announced today that Commvault, a global leader in enterprise data protection and information management, has been named “Bronze Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY, and the 19th International Cloud Expo, which will take place on November 1–3, 2016, at the Santa Clara Convention Center in Santa Clara, CA. Commvault is a leading provider of data protection and information management...
May. 27, 2016 12:00 AM EDT Reads: 3,046
With major technology companies and startups seriously embracing IoT strategies, now is the perfect time to attend @ThingsExpo 2016 in New York and Silicon Valley. Learn what is going on, contribute to the discussions, and ensure that your enterprise is as "IoT-Ready" as it can be! Internet of @ThingsExpo, taking place Nov 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA, is co-located with 17th Cloud Expo and will feature technical sessions from a rock star conference faculty ...
May. 26, 2016 11:30 PM EDT Reads: 2,564
As cloud and storage projections continue to rise, the number of organizations moving to the cloud is escalating and it is clear cloud storage is here to stay. However, is it secure? Data is the lifeblood for government entities, countries, cloud service providers and enterprises alike and losing or exposing that data can have disastrous results. There are new concepts for data storage on the horizon that will deliver secure solutions for storing and moving sensitive data around the world. ...
May. 26, 2016 11:00 PM EDT Reads: 1,249
SYS-CON Events announced today that MangoApps will exhibit at SYS-CON's 18th International Cloud Expo®, which will take place on June 7-9, 2016, at the Javits Center in New York City, NY. MangoApps provides modern company intranets and team collaboration software, allowing workers to stay connected and productive from anywhere in the world and from any device. For more information, please visit https://www.mangoapps.com/.
May. 26, 2016 10:45 PM EDT Reads: 685
18th Cloud Expo, taking place June 7-9, 2016, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy. Meanwhile, 94% of enterprises are using some...
May. 26, 2016 10:45 PM EDT Reads: 3,104
SoftLayer operates a global cloud infrastructure platform built for Internet scale. With a global footprint of data centers and network points of presence, SoftLayer provides infrastructure as a service to leading-edge customers ranging from Web startups to global enterprises. SoftLayer's modular architecture, full-featured API, and sophisticated automation provide unparalleled performance and control. Its flexible unified platform seamlessly spans physical and virtual devices linked via a world...
May. 26, 2016 10:15 PM EDT Reads: 2,081
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, will provide an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life ...
May. 26, 2016 10:00 PM EDT Reads: 1,894
In his session at 18th Cloud Expo, Bruce Swann, Senior Product Marketing Manager at Adobe, will discuss how the Adobe Marketing Cloud can help marketers embrace opportunities for personalized, relevant and real-time customer engagement across offline (direct mail, point of sale, call center) and digital (email, website, SMS, mobile apps, social networks, connected objects). Bruce Swann has more than 15 years of experience working with digital marketing disciplines like web analytics, social med...
May. 26, 2016 09:00 PM EDT Reads: 1,266
Join us at Cloud Expo | @ThingsExpo 2016 – June 7-9 at the Javits Center in New York City and November 1-3 at the Santa Clara Convention Center in Santa Clara, CA – and deliver your unique message in a way that is striking and unforgettable by taking advantage of SYS-CON's unmatched high-impact, result-driven event / media packages.
May. 26, 2016 08:00 PM EDT Reads: 2,333
SYS-CON Events announced today that ContentMX, the marketing technology and services company with a singular mission to increase engagement and drive more conversations for enterprise, channel and SMB technology marketers, has been named “Sponsor & Exhibitor Lounge Sponsor” of SYS-CON's 18th Cloud Expo, which will take place on June 7-9, 2016, at the Javits Center in New York City, New York. “CloudExpo is a great opportunity to start a conversation with new prospects, but what happens after the...
May. 26, 2016 07:15 PM EDT Reads: 1,012
The 19th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit y...
May. 26, 2016 07:00 PM EDT Reads: 1,952