Spoiled Tool Shop Staff Walkthrough

Posted on 07.08.2019by admin

Spoiled Tool Shop Staff Walkthrough

Request Missions - Naruto Shippuden. Xbox 360 Cheats. Top Wiki Contributors. Spoiled Tool Shop Staff Client: Tool Shop Staff.

The stories I hear about User Interface (UI) test automation are often fantastic or tragic. Some of those fantastic projects have thousands of tests running on parallel virtual machines that start with every new code check in. These stories usually involve sophisticated tools to discover whether a test is flaky, or if there is really a bug in the product. I have also heard stories of UI test automation projects gone wrong, such as the horrors of building a framework that has more bugs than it finds. People also talk about constant battles with management. Conflict arises over the return on investment of automating a user interface. Developers are sometimes asked to even hide problems to make reports to management look good, and to “just do what they are told”.

Some of these success stories are so fantastic that they are hard to believe, while the tragic stories are full of stress and disappointment. There seems to be a tendency for the tales of UI test automation to be either an unbelievably brilliant unicorn or utterly terrible. However, I’d like to tell you a different story based on my experience, about reasonable UI automation that is neither a technical wonderland or in complete shambles. This story demonstrates a context where UI test automation can be successful without the intense hype or tragedy, and the value that it brings.

Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.

We will never share your email. 1-click unsubscribes.

The Context

Our story starts with a software product that has been around for years, that is currently in maintenance mode. Over time, the development team produced a main development branch, a few code branches that were for specific customers, and a few customer specific configuration branches. We tested three or four versions of the software for each release, and discovered problems whilst integrating features into the different customer branches. Little did we know that we were in for a few surprises in ways we could never have imagined.

It is fairly common practice today to perform some type of testing close to the code, normally unit testing, in addition to more customer-focused testing in the user interface. This product is built on a technology stack that is not very popular anymore — stored SQL procedures, a pinch of CSS, and a dash of JavaScript. Unit testing stored SQL procedures can still be done, but it is difficult, time consuming, and requires changes to the code to the point that that the tests become ineffective. The programming language and design was not built with test-ability in mind, but the software has paying customers so a rewrite is not in the cards any time soon. As a result, we had some unit tests, but not as many as we needed, and not at the level of effectiveness required.

Modern software shops usually aim to deliver new products to customers every couple of weeks. This strategy makes it easier to manage risk, easier to plan a release, and generally improves quality of life for the development team. Our release schedule tends to be quarterly, and sometimes longer. We release a new version of the product to our customers approximately once every three months. Rather than frequent releases with small changes, we have a longer release cycle with a lot of changes, along with the risk and uncertainty that goes with it.

The user interface of this product is relatively mature. It rarely changes, and when it does, it is not by much. Of course, there are occasional updates to add new features, and to refactor old features that are suffering from performance problems. The direction of development involves chasing what the customer needs, not the hottest JavaScript library or layout style. My automation project is not at risk of massive failure and refactor because someone decided to change from AngularJS to React, or a customer decided that a new feature is not what they imagined.

The Work

My daily work as an automation person can be broken into three broad categories:

Building tests
Refactoring
Bug hunts and investigation

As of today, I have a suite of about 150 tests that run nightly on three different environments. One Microsoft SQL Server release branch, one Oracle release branch, and one Oracle environment that we are preparing to release. Each test run takes about 2.5 hours to complete.

First thing in the morning I do two things; I check the email containing results from each run, and I read the application logs looking for anything interesting to investigate. The nightly test results give a list of everything that failed overnight. This includes some combination of software bugs, and tests that failed, either due to a problem with the test itself or because the product changed.

For me, bug hunts start with the application logs. The logs help me to associate a software problem to a failed test using the time stamp. When there is something interesting in the application logs, I’ll copy the time stamp into a text file and find the tests that were running around that time. I will start the test and observe the flow while watching the logs to discover exactly where the problem is. This usually gets me close to isolating the problem. After watching the test and looking for obvious failures, I’ll perform the test myself. This helps me observe the product, explore, and find exactly what is triggering the exception.

Running through the rest of the failed tests, leads me to refactoring. The tests normally fail because of a software change or a problem in the test itself. Last week, the development team made a change to our datepicker control. It was time for a new one, the old datepicker was slow and unattractive. The new datepicker was opened by clicking on an icon rather than setting focus on a date field, and it also had different IDs in the DOM. I spent 3/4 of a day updating test methods that used the datepicker to get them working again. Other tests that fail intermittently are usually from timing issues; this means that the test code is trying to manipulate the web browser somehow before the DOM is fully rendered. Refactoring is a regular activity, namely updating tests to make them stable again, and removing old tests that are no longer meaningful. This protects the test suite from decay and obsolescence. My test suite is small by some standards, with approximately 150 tests in total. This small number is what makes the suite manageable. I have enough time in a day to assess failures, and perform some refactoring for stability and performance. This isn’t possible when test suites are allowed to grow uncontrollably.

Adding new tests is done occasionally with careful consideration. Typically, the development team and I meet to discuss missing coverage. We try to design a test that will discover important problems in the product, while at the same time being as simple as possible. After building the test, I review it again with the development team to make sure it isn’t missing anything important. I get it checked into our source code repository and add it to the nightly test suites. There is a reasonable argument here that we are building simple, shallow tests because that’s what automation in the user interface affords. I combat the shallowness by exploring, layering coverage, and being alert to the different signals bugs can send (application log errors, and intermittent test failures for example).

The Value

Last month the development team made a change to a data lookup tool. This tool is used in many different places throughout the product; particularly in modal dialogs. The email from my nightly test run showed a high number of failures the morning after this change was introduced. Changing that lookup spoiled the layout of the modal dialogues where it was used. My automated tests failed every time they tried to find the field and enter a text string.

Legacy products often have a problem where it is common for new software changes to create new, and surprising problems. That creates a new kind of work, or I would say re-work, that only exists because the change failed in the first place. In our Lean Software Testing course, we call this “Failure Demand”; demand that only happens because of a failure. This UI automation suite helps us discover those problems much faster; a problem found the day after it was introduced is easier to track down and correct. The suite performs a nightly reconnaissance mission over a significant portion of the product. In the morning, I review the findings, and let people know about any important problems.

Ideally, consistency and quick coverage are key. This is of course, different from what a person investigating software can provide, but it is certainly no less valuable.

Reasonable Automation

Automating the User Interface doesn’t make sense in a lot of contexts. This strategy often introduces considerable costs to development organizations with questionable value. Reasonable automation isn’t always flashy, and doesn’t always use the latest tool set or technology. What it does provide is consistent value to the development team in the form of confidence and early, accurate bug reports.

This is a guest posting by Justin Rohrman. Justin has been a professional software tester in various capacities since 2005. In his current role, Justin is a consulting software tester and writer working with Excelon Development. Outside of work, he is currently serving on the Association For Software Testing Board of Directors as President helping to facilitate and develop various projects.

Join 34,000+ subscribers and receive a relevant software quality article each week!

One-click unsubscribes. We'll never share your email.

Please try again.

Comments

If you work with a group of people, you've most likely needed to share or collaborate on files from time to time. It's rarely as easy and straightforward as it ought to be, but using file-syncing utility Dropbox, you can configure a collaborative shared file space that removes most of the hassle from the equation. It's simple to set up and could save you a ton of time. Here's how to do it.

Spoiled Tool Shop Staff Walkthrough

Why Dropbox Is Great for Collaboration (and When It Isn't)

When you work with others, you often need to share files. Whether it's a straight-up word processing document, presentation, web site mock up, or maybe even a music project, you've likely needed to pass your project along to a coworker for their input, contribution, or use. Even over a local network this can get complicated because you either need to set up network storage that everyone can access or allow every else some kind of access to your personal machine so they can share files with you.

The problem with using a local network storage device is that it only exists in one place, and if you want a nearly bullet-proof backup system locally you need to back everything up both locally and remotely. There's nothing wrong with this option, but it can be a bit much if you're not sharing huge amounts of data.

The problem with making everyone provide access to their machine for everyone else is that it's hard to maintain and doesn't work well when sharing a particular file with multiple people. Like the previous method, it requires some effort.

Dropbox doesn't have these issues. Here's why Dropbox wins as a collaborative tool:

All files are automatically sync'd to everybody's computers as soon as the files are added to the shared Dropbox.
People can either access the files by installing Dropbox and logging in using the username and password for the shared Dropbox account or that account can be shared with them as a shared folder.
Everyone has access to the shared Dropbox via the web (on dropbox.com), so they can access files easily from any computer should they need to work on the go. (If you want to take it a step further you can also sign up for DropDAV).
Backup becomes irrelevant because the shared Dropbox files will be on at least two computers and they'll be synced up to the cloud. That gives you at least two local copies and one off-site copy of every file in the Dropbox. The larger the number of people syncing up with the shared Dropbox, the greater the chance that the world will have to end in order for your data to be completely eradicated.
Regardless of where you are, Dropbox will stay in sync so long as you have a connection to the internet. If you work remotely like we do at Lifehacker, this is enormously beneficial. If you simply want easy access to work files when you're away from the office, or need to deliver a file to your coworkers when you're not around, this is also very helpful.
Dropbox saves versions of your files, so if someone accidentally deletes a file you can generally restore it easily through the web site. They also have a pack rat add-on (for purchase) that will save an unlimited number of versions of the file, rather than just the last one.

Here's where Dropbox can be problematic as a collaboration tool:

Dropbox is free for up to 2GB of space (plus referrals), but that isn't a lot. On the plus side, it's only $10 or $20 per month for 50GB or 100GB of storage (respectively) if you do pay. This works fine if you're not storing a ton of data. If you are, you can still get over 1TB of data with Dropbox Teams, but you can also expect to pay thousands for it. For massive amounts of storage, it's not always the best solution. If you're dealing with smaller files, however, this isn't much of a concern.
You need a reasonably fast connection to the internet if you're sharing large files, or all this file sharing will happen far too slowly. Most basic broadband connections are good enough, but if your upload speed is less than 768kbps and your download speed is less than 5mbps, you'll need to be a little patient.
If a file is open by more than one person at the same time, conflicts happen. (Dropbox doesn't 'lock' files like some local servers you may have used.) While Dropbox is fairly good at resolving these conflicts by saving multiple copies of the file and designating which copy belongs to which computer, merging the changes made to both files can often be tedious and annoying.

How to Set Up a Shared Dropbox

Now that you know the pros and cons of using Dropbox as a collaborative work tool, we're going to take a look at putting together a shared Dropbox that overcomes the downsides (as much as possible) and exploits the upsides for more efficient and productive work.

Signup and Setup

To get started with a collaborative Dropbox account, you may want an independent account for the shared Dropbox. Why not just create a folder in your existing Dropbox and share it with other people? That works when everyone else already has a Dropbox account, but if they don't (and don't want their own account) they can simply use the one you've created. Some people also have free Dropbox accounts that can only hold so much data, and adding a shared folder could put them over their storage allotment. Additionally, if the shared account has primary ownership over the shared folder, you never have to worry about transferring ownership later on. (Dropbox lets you do this, but if you ever delete your shared folder it can be very easy to accidentally delete it in everyone else's Dropbox as well. If you leave ownership with the shared account, this isn't even a concern).

Once you've signed up for the Dropbox account you're going to share, create a root folder in the account (through the web site) and share that with yourself and anyone else you're collaborating with who already has a Dropbox account. Provide everyone on your team with the username and password to the independent Dropbox account so they can install Dropbox and use that account if they do not want an account of their own (or simply want to log in online to view files, restore them, or share access with another member of the team down the road).

Organization

Organization of the file system is important to make sure the Dropbox doesn't become a huge mess. When you have a lot of people using the same space, it's very easy for things to quickly get out of hand. We'll take a look at the basic structure of the Lifehacker shared Dropbox in a moment, but first it's important to note that communicating how things are organized is important. You can create a highly organized Dropbox, but that doesn't mean everyone else is going to understand what you did. After you organize everything, make sure you explain it to everyone else to avoid any confusion.

The Lifehacker Dropbox is split up into three sections: Assets, People, and Projects. Assets contain files we use regularly, such as photos, logos, background music for videos, and so on. People folders include an individual folder for every member of the team. These folders include things we're working on individually but want to provide others with access. It's also a place to share a file with a specific person. Lastly are Project folders, which contain files for projects we work on as a group. While these are not the only folders in the Dropbox, they make up the majority. Every group will have individual needs, but every group is likely to need Asset, People, and Projects folders.

How are these folders labeled? While you can simply create three folders for each category and include sub folders inside, we do it like this:

Assets - Graphics
Assets - Music
Assets - Video
Etc.

Why? In part it's a personal preference to avoid drilling down through several directories just to find a specific file. I also find that when you have multiple folders with the same prefix it's easier to spot what you're looking for. Whatever you prefer is just fine.

Sharing Files and Notifications

Once everybody knows how things are organized, it's really easy to notify them of a shared file. All you have to do is send a quick email, IM, text, or whatever makes sense for the circumstance, to let them know you've added the file to the Dropbox. Additionally, Dropbox is good about posting notifications (via the system tray in Windows and Linux or Growl on the Mac) when new files are added from a computer that is not your own. If you're around to notice, you'll often receive a passive alert when new files show up.

We've even had some fun with this in the form of screenshots. For example, we have a group chat room that we're logged into during the day. One day most of us were having a conversation and wondered if someone else was watching the conversation. Rather than responding, a screenshot of the conversation suddenly appeared in the Dropbox. It was called 'I SEE EVERYTHING.jpg.' This is to say that a collaborative Dropbox can be fun, too.

Solving Problems

The two main problems you'll run into when using a shared Dropbox are the amount of time it may take to sync larger files and conflicts that arise when multiple people are editing the same file. There are no perfect solutions to these problems, but there are a few ways you can alleviate the trouble they cause.

First, when it comes to slow connections you can't really make them faster. What you can do, however, is work on the same network whenever possible. Because Dropbox has a nifty feature called LAN sync, which syncs the files locally before uploading them to the cloud, your fellow workmates will receive the files quickly despite the speed of your connection. This won't be helpful to telecommuters, but if all or most of you are in a central workspace it'll make a huge difference.

Second, when you're dealing with syncing conflicts due to people using the same file you just need to make sure that doesn't happen. If you're running Mac OS X, you can use the labels feature built in to the OS to set files to a certain color to denote that they're in use. If you're not using Mac OS X or syncing with multiple operating systems, you can just add some text like [IN USE] or [AD EDITING] (AD being my initials) to the end of the file so people know not to open it.

Neither of these are perfect solutions, but they definitely help so implement them as needed.

That's all there is to it. Using Dropbox as a collaborative tool is simple to do and it makes life much easier. If you decide to give this a shot or are already using Dropbox as a collaborative tool and have some additional tips, be sure to share them in the comments.

You can follow Adam Dachis, the author of this post, on Twitter and Facebook. If you'd like to contact him, Twitter is the most effective means of doing so.