Shopping for a vacuum shouldn't be guesswork, but you can't know which model is best for your needs just by looking at the specifications. For this reason, we have developed and refined a sophisticated methodology for testing cordless sticks, canisters, and upright vacuums. Our standardized approach for assessing the performance of each vacuum focuses on six metrics: carpet cleaning, hardwood cleaning, pet hair, filtration & suction, ease of use, and battery performance (for cordless stick vacuums). This allows us to directly compare each model in conditions that closely simulate what you would experience cleaning surfaces, upholstery, and nooks and crannies in your home.
Carpet Peformance
The ability of a vacuum to clean a carpet is arguably one of the most important, if not the most important metric, in assessing a vacuum. This is because carpet comes in different fiber lengths (also known as low pile to high pile) and introduces a significant vertical component into floor cleaning that isn't evident when cleaning a hardwood floor. For this reason, our testing procedures for carpets were the most elaborate. The most crucial aspect, however, is that our carpet performance testing is designed to mimic the real-world use of a vacuum while ensuring objective ways of recording performance quantitatively and comparing different vacuum models.
Our test procedure involved preparing a debris mixture comprised of 1/2 cup each of sand, rice, and Cheerios. By weight, this was 200g of sand, 112 grams of rice, and 15 grams of Cheerios. We weighed each debris type separately and logged the combined weight for each test to determine precisely how much of the mixture each vacuum picked up. Our protocol involved operating each vacuum at its highest setting in a 24-second timed window, which emphasizes the effectiveness of each vacuum rather than effort (i.e., high numbers of passes). Each vacuum had a standard test lane of 3ft by 9ft (27 square feet) on a section of carpet that was side by side with identical lanes for each vacuum. This also represents a recently updated test feature (increased surface area), allowing us to get more information about the vacuum cleaning characteristics. After each test, we weighed the contents of the dust bin and calculated the percentage of debris captured. Each vacuum was tested twice in separate trials, followed up with an additional trial if there was a notable variation in the results.
We further verified the performance of each vacuum by recording the results from multiple passes and noting if there was a difference in directional performance (forward vs. backward). We also examined the carpet closely after cleaning, which is critical considering that a carpet may look clean even though small particles have worked down toward the base of the fibers. We also looked at how well debris was retained by each vacuum since, in a few cases, a pile of debris that was sucked up by the vacuum was released after we shut off the vacuum. Yikes! Carpet performance is also based not only on how much material the vacuum picks up but also on the engineering of each vacuum. In other words, some vacuums do a great job in one direction while struggling in the other direction. That being said, we carefully used the same technique with each vacuum to maintain an apples-to-apples comparison.
Along with our capture percentages and performance verification, we recorded qualitative observations. We noted any additional observations about vacuum behavior or our experience (i.e., the more powerful vacuums produce so much suction that it often takes more effort to operate).
Hardwood Performance
Due to the growing popularity of hardwood flooring in homes and the unique challenges of cleaning hard surfaces, we next assessed the performance of each vacuum on hardwood. Without carpet fibers to trap debris, materials are easily scattered (including during the vacuuming process!), and there is also the risk of scratching or damaging floors. In particular, there is a need for effective edge cleaning, crevice cleaning, and preventing debris scattering, which not only lengthens the cleaning job but can also cast material into hard-to-reach areas.
Our hardwood cleaning performance sub-metric (70% weighting) involved a cleaning test with the same debris mixture as our carpet performance test. Again, the backbone of our procedure was knowing precisely how much debris was scattered onto the floor and precisely how much was picked up in 24 seconds of vacuuming. We conducted two trials with each vacuum and added a third trial if there was significant variance in the results.
We then dove into an updated test protocol (15% weighting) for determining how well a vacuum cleans crevices by vacuuming a bamboo mat with slats (with a measured amount of sand beneath the slats). We made eight total passes, four forward and four backward, and weighed the amount of sand that was picked up.
We assessed edge-cleaning capability (15% weighting) to determine how well a vacuum can pick up debris in tight spaces. Stairs, corners, and baseboards pose a hurdle for vacuum heads to grab at the end of their roller. It's not as straightforward as one would hope. To assess performance, we vacuumed perpendicular to a wall (we called this head-on cleaning). We also ran the vacuums parallel to the wall as if you would be running the vacuum along the baseboards of a hallway. In this case, we distributed 1/8 cup of coffee grounds in two 16-inch strips at the intersection of the flooring with the wall and then visually assessed performance.
Lastly, we evaluated whether a vacuum performed better in one direction or the other, how it scattered debris during cleaning, and if the vacuum head mimics a snowplow during the cleaning process. We also looked at how well a vacuum holds onto debris after cleaning. Furthermore, we looked closely at the hardwood before and after cleaning to determine if the brush roll or vacuum head was scratching or marking the surface. Vacuums with special features, such as hard floor modes and suction control, were included in our overall assessment. The performance of different brush roller types and designs rounded out our testing on hardwood surfaces.
Pet Hair
All of us at GearLab love our pets, so we were particularly excited to test each vacuum's effectiveness in cleaning pet hair. This metric is also important enough that we tested it separately from general carpet performance. Our methodology is based on our combined years of experience dealing with the specific challenge of pet hair removal and evaluates both surface pickup and brush roll hair management.
Our testing procedure involved weighing out 4 grams of pet hair, wiping and rubbing it across thirteen square feet of carpet (both low and high pile carpet were tested), and then distributing the remaining fibers across the area. We added one gram of long hair extensions and scattered that over the top of the pet hair, then pressed the hair into the carpet fibers with a heavy roller to mimic foot traffic to ingrain all hair into the test carpet fully. We then ran each vacuum for 12 seconds back and forth, followed by weighing the hair captured in the dustbin to determine the percentage captured. The low pile carpet was 60%, and the high pile was 40% of the total score.
Considering how insidious pet hair can be, scoring included our observations of brush roll wrapping, the extent to which embedded hair was removed from the carpet, and the need for multiple passes. Additionally, we assessed anti-wrap features, hair retention in the vacuum (where hair was picked up but became trapped and wasn't delivered to the dustbin), and the requirements for cleaning maintenance. We stand behind all of our testing protocols, but we are especially confident in our pet hair testing; our results offer an exceptional perspective on which vacuums in the lineup are best suited for dealing with pet hair.
Filtration and Suction
Our testing methodology for measuring filtration and suction underwent significant scrutiny and refinement before our latest round of vacuum testing. Our rationale behind these developments was based upon the importance of quantitative assessment of factors that are difficult to observe, such as indoor air quality and allergens, which profoundly impact our well-being.
For testing suction, we measured unsealed suction pressure at the vacuum head with a specially designed platform and manometer, which measures internal pressure in Pascals (i.e., one newton of force applied over one square meter). We conducted this test multiple times with each vacuum. We focus on the power at the vacuum head rather than the handheld or tube. The location is key as the initial readings close to the motor do not factor in the design head and how it interacts with the floor. Part of the magic lies in the product design efficiency by creating an effective seal to carpet and hard surfaces.
We then measured the filtration of each vacuum in a closed room with two Dylos particle monitors that provided constant read-outs of the concentration of small particles (2.5-0.5 micrometers) and large particles (>2.5 micrometers). The smaller size is equivalent to mold spores and bacteria, while the larger size is equivalent to pollen and dust. Over a 30-second interval, each vacuum was used to clean up ⅓ cup of flour that was applied to a specific spot on the floor. We went to great lengths to filter the air in each room between tests and establish a particle count baseline prior to testing.
Our scoring considerations assessed not only the vacuum motor but also the effectiveness of the filtration systems, the quality of the seals in the vacuum, and the nature of the internal airflow as debris is transported to the dustbin or collection bag.
Ease of Use
We drilled down on the real-world practicality of a vacuum in our ease of use testing. This characteristic influences the cleaning effectiveness of each vacuum since you'll more frequently use a vacuum that is a pleasure to operate.
Imagine testing over thirty vacuums on a slalom course of traffic cones with a decibel meter and a stopwatch, and you get an idea of just the first hour of ease of use testing. We then moved on to an obstacle course of furniture and tested how each vacuum handles tight corners and low clearances. Some vacuums have an ingenious folding feature that allows them to reach under furniture, giving our backs a break from an arduous chore.
Additional factors in our testing included evaluating dustbin capacity, the experience of emptying the dustbin, the ability to clean the brush roll, the versatility of the included tools, and the maintenance of various components.
We also assessed the effectiveness of upholstery cleaning performance. Here, we again employed coffee grounds, with 1/16 cup on a pillow and 1/16 cup on an ironing board, and then did one pass with the upholstery tool supplied with each vacuum.
Battery
Battery longevity is a critical factor for the cordless stick vacuums we tested, but it doesn't apply to canister or upright vacuums. A good battery system affects a vacuum's usability and functionality and determines its cleaning area coverage.
Our battery testing procedure involved timing a vacuum on the low power setting after fully charging it. We operated each vacuum until it lost power. After fully recharging the battery, we timed how long before it lost power, but in this case, we ran it at the maximum setting. Our testing conditions for each vacuum were consistent and didn't involve any attachments or modifications.
Additional considerations when assessing the battery included the arrangement for recharging the vacuum, whether it comes with a stand, how easily it leans against the wall of a storage closet, or whether it requires installing a wall mount for storage.
