The idea of analytics in security applications is not a new one. In fact, some in the industry have been (over)promising its capabilities for years.
“AI and deep learning were launched around five years ago with much hype rather than an educational approach, which left both systems integrators and end users a bit confused on the differences between AI, machine learning and deep learning,” says Jason Burrows, Western sales director at IDIS America, Coppell, Texas. “Some early offerings disappointed as they were launched prematurely, before engines were fully trained and able to recognize objects reliably and accurately. The cost and complexity of early-to-market AI applications made users question the value of deployment, configuration and operator use. This was compounded by privacy concerns and a greater storage burden. As a result, many end users were reticent, and systems integrators were unclear on which solutions would be best suited to meet their customer’s needs.”
Many of these concerns are quickly fading, though. IDIS’ 2020 global survey of more than 700 security integrators, consultants and end users showed that more end users had already begun adopting deep learning or had short- to medium-term plans to do so. The survey also showed that both integrators and end users had a clearer understanding of the main security challenges deep learning-powered analytics can overcome.
SDM’s 2021 Industry Forecast Study showed similar changing attitudes. Of those surveyed, 45 percent said they currently offer video analytics, while 26 percent are planning on offering them in the next one to two years, 9 percent are planning on offering them in three to five years, and only 20 percent have no plans to offer them. In addition, 55 percent of security professionals surveyed expect revenue from analytics to increase over the next year.
“What we saw last year was of course an exception, but it also sparked new use cases for analytics — and there were a bunch of them,” says Florian Matusek, product group director, video analytics, Genetec, Montreal.
Many of these use cases went beyond what we typically think of as security, expanding the usefulness of traditional security solutions.
“Cameras themselves and video analytics have really crossed into being a powerful technology that goes beyond just security,” says Will Kelso, vice president of enterprise business intelligence at Interface Security Systems, Earth City, Mo. “Now you’re able to get real positive change throughout an entire organization, so not only are you able to share the spend, but make a big impact across the whole institution.”
Rob Merchant, president of MTS Intelligent Security Solutions, East Brunswick, N.J., featured on this month’s cover, has been an integrator of security analytics for more than 15 years, deploying license plate recognition (LPR) and face detection systems as early as 2006. “Algorithm maturity, application of AI and deep learning and more competition in this space [have changed in the past year],” Merchant says. “I think advances in computing software have allowed these complex algorithms to be able to be scaled to run on more commodity-based platforms. You are seeing these analytics work in more relevant conditions. Aside from LPR and other analytics that have been around for a while, most of these had very niche roles. They are becoming more affordable and more applicable across a greater user base.”
Over the past year, awareness and understanding of AI has increased, both in regards to machine learning and deep learning, says Ryan Gregory, director of solutions and services at Axis Communications, Chelmsford, Mass. “Accordingly, many end users have heightened expectations around the performance of analytics, mostly based on trends and experiences in the consumer electronics space. In many instances, people have come to rely upon these analytics in their daily lives — speech recognition for audio commands or text conversion; optical character recognition for extracting data from print; and facial ID to unlock computer devices. These more common and practical uses have created greater interest, expectations and potential use cases for analytics.”
Aaron Saks, product and technical manager, Hanwha Techwin America, Teaneck, N.J., says there are two pieces to the puzzle that explain why analytics have grown so popular: COVID-19 and AI. “With COVID-19, all of a sudden there was a shift to different people doing different jobs — maybe you had someone watching a camera at a desk, but working from home they’re doing it differently. Also, COVID-19 called for specific types of analytics people wanted to see — looking for face masks, social distancing, whether there’s a car in the parking lot.
“The bigger picture regardless of COVID-19 is the whole AI part. It’s only been these past couple of years that the cost, performance and processing power for AI has been effective and efficient. AI has been around for years, but now we’re in a place where AI can be cost-effectively used.”
A large part of this increased performance and decreased cost is due to the chip size, says J. Matthew Ladd, president of The Protection Bureau, Exton, Pa.
“Technology is able to advance because of the ability to put more analytics onto smaller and smaller chips and concurrently the power and the capabilities of the chips increase,” Ladd explains. “When analytics first came out, you had to connect the camera to a computer — now the camera is making the decisions itself.”
Aaron Simpson, president and chief technology officer at Stone Security, Salt Lake City, Utah, SDM’s 2020 Systems Integrator of the Year, says that analytics are finally getting to a place where he feels comfortable vouching for them. “We have been very cautious with analytics over the years. We have watched many companies over-promise and under-deliver, and often go out of business. That said, we are very encouraged by the advancements in analytics and have adopted many solutions in the past year or two that are delivering impressive results.”
Navigating the Ethical Battleground of Analytics
Years ago, privacy wasn’t much of a concern in the security industry. Most of the time, you could hardly even recognize your own cousin’s face in the grainy video surveillance footage of yore. But now, with facial recognition and other analytics making headlines throughout the world, ethics should be top-of-mind for everyone in the industry.
“Ensuring analytics are ethically used in security applications has been a major focus for the industry,” says J. Matthew Ladd of the Protection Bureau. “During the pandemic, one concern centered around the use of the data taken from thermal imaging cameras to recognize an elevated body temperature in a person. How should this data be handled? If someone has an elevated skin temperature, does that information get reported to the human resources department? Is that information retained in any way? Or is that individual simply turned away at the door?”
Why are end users so concerned about the privacy of their personal information? AMAG’s Jonathan Moore explains: “One of the biggest fears is whether governments or other entities in positions of power will use analytics — specifically facial recognition — to unethically or illegally drive an agenda. This subject will continue to be sensitive and will be litigated at all levels of government and in industry regulatory bodies for the foreseeable future. If those in power can prove the analytics will be used ethically, then the arguments around this topic will start to fade and the use of analytics will grow more quickly.”
Racial bias is also a concern when it comes to some analytics, including facial recognition.
“End users need to be cautious of the technology and vendors they work with and, most importantly, understand what data has been used to train algorithms,” says Jason Burrows of IDIS America. “For example, many early iterations of facial recognition solutions had trained algorithms to distinguish faces of Uyghur people, a predominantly Muslim minority ethnic group in China. Other solutions were developed using enormous data sets of people’s faces for military and counter-terrorism surveillance. Unsurprisingly, they revealed a high level of racial bias when deployed in commercial and public space settings, and other algorithms that were developed using data sets gleaned from the internet without consent.”
Some manufacturers, like Genetec, have been working to come up with solutions for privacy concerns.
“We have a product called the Privacy Detector that pixelates people in video, but you can still see what happens,” says Florian Matusek of Genetec. “Many times you don’t need to see people, but if you need to, you can unlock that. This year, in 2021, the Privacy Protector has been the fastest growing analytic in our portfolio, specifically in the U.S. where customers are becoming increasingly aware — especially in the public safety field, like city councils.”
If other manufacturers don’t follow suit voluntarily, outside forces may force them to.
“There will have to be pressure put on the manufacturers and users of the technology when designed for, and used in, an unethical manner,” says Michael Trask, director of sales, North America, LENSEC, Houston, Texas. “Boycotting or regulatory restrictions may come into play as an effort to dissuade this type of development and implementation.”
Regulatory restrictions are already at play in many parts of the world. “Many states and countries are creating policy around gathering and using personal information such as facial recognition, so it’s important that we are aware of the local restrictions before recommending solutions,” says Aaron Simpson of Stone Security. “No amount of legislation will ever ensure the ethical use of the remarkable technology we are seeing and using today. Each integrator and customer will need to determine what is considered ethical in their geography. Companies should create their own or adopt an ethics policy, like the Copenhagen Letter, so that they already have a response to ethical issues that will certainly arise.”
Many municipalities are adopting some form of legislation that regulates the use of face recognition, says Blaine Frederick of Alcatraz. “Much of this legislation is based on the Illinois Biometric Information Privacy Act (BIPA), which was the first major piece of biometric legislation passed in the U.S. Initial steps such as advising individuals about the details of the program and requiring participants to provide their consent in order to join the program are critical in order to maintain compliance with those regulations.”
As of right now, the messy patchwork of differing regulations can be incredibly hard to navigate.
“Analytics are an emerging technology, so the legal and regulatory landscape is dynamic and it can be a bit murky,” says Axis’ Ryan Gregory. “That said, there are laws, regulations and policies that affect analytics and resultant data collection and storage. For example, some state laws and local ordinances have banned or restricted the use of facial recognition technology and some have sought to put protections in place for citizens like the California Consumer Privacy Act. Currently, legislators at the federal, state and local levels have introduced legislation that impacts data analytics in a variety of manners — both positively and negatively.”
The positive, according to Gregory, is that bills have been introduced in congress that would create a national data privacy framework and preempt state and local data privacy laws, making the rules easier to understand and follow. The negative is that on the state and local level, there has been a litany of facial recognition bans or moratorium laws that prohibit both commercial and government personnel from using and deploying facial recognition systems.
Europe already has more comprehensive rules surrounding the use of analytics, and the collection and processing of personal information — the General Data Protection Regulation (GDPR). Some companies in the U.S. are already adhering to its guidelines. According to Burrows, that’s a good idea.
“U.S. organizations need to be aware that GDPR is relevant wherever EU citizens are present, such as healthcare settings, hotels, overseas tourist attractions and public space surveillance,” Burrows says. “GDPR encompasses the use, storage and processing of all personal data including video, but it is more rigid when it comes to biometric data, although it’s yet to be tried in court. But unsurprisingly, some corporate enterprises headquartered in the U.S. have adopted GDPR as a gold standard since they regularly employ and welcome EU guests and operate across the Eurozone.”
One important distinction to make, though, is the difference between facial recognition and facial authentication, explains James Segil, president and co-founder of Openpath Security, Los Angeles. “There’s a lot of concern as a result of confusion for consumers in terms of what facial authentication is, rather than facial recognition,” Segil says. “Facial authentication is similar to how we use our iPhones — it stores a photo of our face and matches what we’ve given them. Facial recognition is where they’re mapping faces they see against a broad database of users that may have not consented.”
Ultimately, all of this comes down to a trade-off — privacy versus convenience. “It’s important to understand that anything in our physical space can now be turned into a digital twin,” says Steve Washburn of Convergint. “From a business perspective, the data captured and monitored can enable businesses to enhance their customer experience and improve workflows. However, companies will have to carefully determine guidelines around what type of data is collected on employees, ensuring they remain transparent and ethical. There is a significant trade-off here for individuals, and we’ve seen it online — privacy versus convenience. It boils down to comfort and what people are comfortable giving up in return for convenience.”
Hopefully, as regulations become more comprehensive, security industry leaders will be a part of the discussion.
“The industry needs to have a seat at the table in defining the ethical use of analytics and participate in government discussions on this topic in various geographies,” says Ewa Pigna, chief technology officer at LenelS2, Pittsford, N.Y. “Development and support of analytic standards as applicable to our industry will ensure participation from multiple manufacturers and increase the focus on an ethical approach.”
The Many Use Cases of Analytics
Blaine Frederick, vice president of product at Alcatraz AI, Redwood City, Calif., likes to think of security analytics as belonging in two distinct categories: real-time analytics and trend-based or forensic analytics.
Real-time analytics encompass a variety of applications such as face authentication, instances when an item is left behind or a speeding vehicle, Frederick explains. “These types of analytics provide information to systems and users who are making decisions on the fly. They are a key component in any automated system or in security operations centers where there is simply too much data for an individual to process manually.”
Trend-based analytics provide system administrators with insights into what is happening in facilities over the course of time, Frederick continues. “This can be anything from heat mapping in a retail store to vehicle counting at a highway toll booth. Analyzing room occupancy has gained significant attention over the past year as companies plan their return to work after the global pandemic.”
Steve Washburn, executive director of business development for Convergint Technologies, Schaumburg, Ill., adds, “As businesses continue to prioritize digital transformation to attain real-time data and behavioral analytics, more use cases continue to come to fruition. For example, computer vision has unlocked data and analytics that businesses have leveraged to enforce workplace safety initiatives, such as PPE detection, hardhat usage, protective eyewear, gloves or safety vests. Additionally, computer vision technology is flexible and can be deployed across nearly every vertical market, allowing additional use cases to evolve over time.”
At their simplest, analytics make security personnel more effective, says Paul Garms, director of regional marketing, video systems, North America, Bosch Security Systems, Fairport, N.Y.
“When thinking purely about using video surveillance for the protection of people and assets, analytics make security personnel more effective by raising their awareness and ability to react to system events,” Garms explains. “In a manned system environment, this adds an extra layer of protection by providing alerts to potential security risks before or as they occur — such as detecting loitering in a parking lot or a perimeter breach after hours.”
Conditions that video analytics can be programmed to alert on include line crossing, illegal parking, loitering, speeding and color matching.
False alarm filtering is another hot analytic offering in the security industry right now. Calipsa, Ashburn, Va., provides one such solution.
“Our software platform identifies the live movement of humans and vehicles within the view of the surveillance cameras,” says Calipsa Chief Revenue Officer Brian Baker. “Nuisance factors, such as windblown foliage, are filtered out, significantly reducing false alarms and improving a video monitoring agent’s response times to genuine security threats while improving their overall efficiency.”
One of the factors that makes analytics so appealing is that it allows for a proactive response, rather than a reactive one. “When it comes to security applications, analytics can offer many advantages by serving as a force multiplier, enabling proactive response and providing operational efficiencies,” Axis Communications’ Gregory says. “For example, in the past, security personnel were required to monitor multiple live feeds and react to incidents or scour hours of footage from a forensic standpoint. Today, video analytics are helping end users quickly identify objects or sounds in real-time and respond in a proactive manner by dispatching live personnel or even triggering a sound recording to deter trespassers or undesirable behavior.”
“Customers want to know where people are within a building and to predict situations before they happen,” says Sheeladitya Karmakar, global offering leader for Honeywell Commercial Security, Atlanta “We can create reports on how many zones an individual has accessed in the last 24 hours and are developing the ability to predictively tell security teams where people will be in the next 20 minutes. This will help security teams get ahead of possible issues. For example, we can leverage data to know that many people exit the building around a certain time, so we can preemptively open or unlock doors to allow for a more efficient process.”
While cameras tell you a story, analytics help you write the story.
“Analytics can be used in multiple ways to help safeguard people, places and property,” says Devarshi Shah, CEO and founder of Lumeo, a custom video analytics platform based in Oakland, Calif. “Cameras can only tell a story, but analytics can help security react and respond to any given incident. For threat management and incident response, they can let first responders know where an individual is located and allow for proactive response at venues with respect to crowds or alert when a secured area has been breached. Beyond that, analytics can also be leveraged for compliance and better operational planning, like where and when to deploy security. There are countless ways in which analytics can be leveraged within the security industry.”
John Nemerofsky, chief operating officer at Sage Integration, Kent, Ohio, says the analytics application his client use the most is multi-camera search.
“When analytics first came out, it was about law enforcement being able to do investigations quicker with a single camera; but multi-camera search is the most popular analytic we’re using,” Nemerofsky says. “Another thing we’re using as people move back into corporate headquarters is watch list rules. It could be for bad guys, or it could be used for VIPs so when they arrive at the building, everyone can stand at attention and make a good impression — we’ve used that a lot.”
Ladd has also seen a number of applications using people tracking and the patterns that come from that. “Look at contact tracing — you’re using this capability with the camera. Analytics can be used to track people from one camera to another. In an emergency you can track someone as they walk through a building, such as an active shooter. The camera locks onto a person and tracks them until they reach the next camera, and so on. In the retail market, it’s been talked about for a long time. Cameras can track shoplifting, but also work with marketing departments to give them data on displays and how people move around the store. The camera helps the department decide where to place certain items in the store and helps them analyze how that impacts shoppers’ time and buying decisions.”
Analytics can even detect cyber and terrorist attacks, Baker says. “Law enforcement and homeland security officials use analytics to monitor social media, phone communications, travel records and many other data sources to spot potential crimes or terrorist attacks before they occur.”
Too many options to choose from? You could always mix and match.
“A la carte analytics seem to be gaining popularity,” Stone Security’s Simpson says. “As mentioned before, most analytic platforms have forced customers to deploy a single solution across their entire application and require costly on-premise resources to do so. Many analytics are only needed in a few locations, and most customers would benefit from a variety of analytics.”
Selling & Setting Up Analytics
If you’ve been in the industry for a while, analytics may seem like scary uncharted territory. But as more and more end users demand the convenience they bring, it’s important to become educated on how to properly sell and set up these new features. Here, leaders in the industry share their advice.
“When selecting a video analytics vendor, end users and integrators must consider their video analytics as part of a long-term relationship, rather than as a part of a one-time transaction. The vendor will need to evaluate the site where the analytics are being deployed, and assist with the site design, installation and calibration in order to be responsive in supporting the product after deployment. If the vendor is not willing or able to do these things, it’s unlikely that the analytics will perform as desired.” — Kai Moncino, Teledyne FLIR
“Start small and grow. Can you begin with incident management, access control and visitor management? What can you learn from pulling that information together and analyzing the outputs? Test your theories and create hypotheses. Once you’ve created a framework, determine other valuable data sources to bring into your analytics program.” — Jonathan Harris, vice president, Group337, a Bethesda, Md. company dedicated to helping integrators.
“Integrators need to become familiar with the various analytics available and select the appropriate technologies as part of an integrated solution based on the customer’s needs. It’s not a one-size-fits-all approach to analytics, and the deployment should be tailored to solving specific problems.” — Ewa Pigna, LenelS2
“There is so much intelligence that can be put into security systems without the need for bleeding edge analytics, and this is a largely ignored capability. Scripting, changing views, audible alerts and alarms and different modes of operation all make your security system that much more usable. Integrators need to do these basics instead of relying on a silver bullet. With that said, analytics can turn a generic security system into a highly capable security system within the same infrastructure. And it allows you to revisit existing installations and enhance them with intelligence.” — Rob Merchant, MTS Intelligent Security Solutions
“It helps to understand the technical possibility and limitation, as well as what will fit within a budget for the customer. Not every analytic is a right fit for every customer, but by understanding the spectrum of choices, integrators can serve their customers very well.” — Michael Trask, LENSEC
“Work directly with clients on realistic solutions for their needs rather than making fanciful promises that analytics will solve all of their problems.” — Christopher Gandy, chief technology officer, Unified Command, an integration company from Las Vegas
“Understand the pricing for analytics, the licensing costs per camera, how it’s going to sit on your current VMS or NVR, how that gets deployed, how long it gets deployed, etc. Understanding the power you need in the server that’s going to run the analytic — that’s one of the things integrators miss. Work closely with your clients to develop that budget, and understand the return on investment.” — John Nemerofsky, Sage Integration
“Integrators should think of providing analytics as a service to end customers. For instance, a customer may not have the resources to provide on-site security service to all of their remote locations, which means they might want to have a cloud-based connection to their main server at headquarters to analyze and fix problems remotely. This way, integrators can help customers solve problems before they happen, which also increases service revenues for themselves — so everyone benefits.” — Sheeladitya Karmakar, Honeywell
A Limitless Future
Right now, it seems we are only at the beginning of what analytics will ultimately be capable of.
“Artificial intelligence and its subsets, machine learning and deep learning, massively expand the capabilities of today’s analytics,” Baker says. “However, training these analytics is a time-consuming and costly process. For example, training a deep learning platform to accurately detect the differences in movement caused by a human or stray dog requires exposing neural-node networks to millions of photos of humans, making corrections each time the software makes a mistake. Over time, the platform will accurately predict the presence of humans or whatever else it has been trained to recognize.”
Supply chain shortages caused by the pandemic are stalling progress a bit for now, Hanwha Techwin’s Saks says.
“[In order for analytics to progress] the cost needs to go down, and right now with post-pandemic chip shortages and shipping delays prices aren’t going down; but generally they do,” he explains. “What we’re going to see is analytics becoming more commonplace. We’re going to get the camera optics better optimized and get the best wide dynamic range. Often AI would be included in the premium lineup of cameras, but as product lines mature we will see AI cameras becoming more of the feature sets you typically expect on all of your cameras without limitations.”
Since many companies require high compute power to run analytics on edge-level devices, according to Karmakar, the decreasing cost of compute power should be helpful. “This is happening gradually, not drastically, but it’s happening. This is why we have access to so many features on our smartphones today that came along in the early 2000’s. The same will happen for these analytical servers, which will make higher compute power more accessible.”
While lower prices are necessary for growth, they’re also inevitable.
“In order for analytics to reach their full potential in security applications, prices will need to drop, while technologies and their capabilities will need to increase,” Ladd says. “I’ve been in this industry for 46 years, and everything continues to grow and get better and faster. When cameras with analytics were first introduced in the security industry, they cost thousands of dollars, but now the price is a fraction of that. I’m always impressed with how things continue to develop and go further — it’s up to the processing capabilities and how you send out the information. Everything is wireless and yet there is so much further to go. Technology will continue to grow because the desire for smarter and faster technology will always be there.”
Lower costs and improved technologies go hand in hand, Merchant explains.
“I think advances in computing hardware have allowed these complex algorithms to be scaled to run on more commodity-based platforms,” Merchant says. “You are seeing these analytics actually work in more relevant conditions; aside from LPR and other analytics that have been around for a while, most of these [other applications] had very niche roles. They are becoming more affordable and more applicable across a greater user base.”
Analytic companies that don’t improve the cost and ease-of-use will likely not last much longer, Simpson says. “Many traditional analytic companies are currently working to reduce significant hardware resource demands and leverage the untapped edge capabilities. I believe we will see some of the more proactive companies adapt to the ever-changing environment while others will remain stagnant and fall by the wayside.”
As these technologies become more accessible, security integrators will need to become educated on how they work — or they could fall to the wayside, too.
“Increasing awareness of the full capabilities of analytics is essential,” Garms says. “Integrators need to get trained on the new possibilities for video analytics that are enabled by AI, and they should look to manufacturer training courses to assist with this. Understanding the various use cases where analytics can be applied is also important. This will help integrators identify situations where IoT video systems will be beneficial for their customers.”
And since analytics are ever-changing, education will have to be continuous — not just a one-time course.
“For analytics to reach their full potential, improved education and training is required,” Gregory says. “A well-informed value chain — from developers and systems integrators to end users and even the public — can properly set expectations and provide greater understanding. Also, while major strides have been made around AI training models, an even greater emphasis on continuous improvement will benefit existing applications and spark new applications.”
Nemerofksy says that new unique use cases will push the technology to advance. “It’s going to take the end user continuing to come up with challenges and problems they want to solve. … It’s going to take that security practitioner to keep challenging integrators.”
In reality a lot of the technology is already here, Interface Security’s Kelso says. “It’s ever-growing, but the selection of a right partner to guide you through the human element and also having that ease of set up and being able to take these data sets and make sure they’re actionable is key. And there is an opportunity with the technology in place to gain a lot of great insight.”
Quality guidance is especially important in an industry like security, points out Jonathan Moore, vice president of product management at AMAG Technology, Hawthorne, Calif. “The security industry is a tough market because a security failure can carry serious implications, opening the organization to unnecessary risk. Analytics solutions are improving; however, the security market is typically looking for perfect or near-perfect solutions. For analytic solutions to reach their full potential, they need to operate at a very low failure rate while keeping the cost of the solution (and the supporting servers and technology) at an acceptable level.”
Kai Moncino, global business development manager, security, Teledyne FLIR, Thousand Oaks, Calif., says, “The goal for any analytics software is a detection rate approaching 100 percent and a false alarm rate approaching 0 percent. While this is nearly impossible in real world scenarios, there are some analytics that are closer to these metrics than others. Looking forward, the strongest competitors in the video analytics space will be those that deliver consistent detection rates and are able to reduce false alarms.”
The Time Is Now
Like many other technologies, the adoption of analytics sped up during the pandemic, as businesses were forced to accelerate their digital transformation initiatives to survive, Convergint’s Washburn says. “Now, an abundance of data can be applied for business intelligence purposes, allowing them to thrive. We’ve reached an inflection point where now, the technology is so far advanced, it’s time to maximize use and unlock untapped potential. The limitless flexibility behind computer vision’s AI offers new opportunities, leveraging ground truth and sample data to train machine learning models to recognize specific elements or behaviors.”
Now, security professionals just have to get in on the action — they already have the assets.
“As an industry, we have a huge asset we are sitting on — there are over 1 billion security cameras deployed worldwide,” Lumeo’s Shah says. “Most of these cameras are just recording devices. But when you start thinking about the wide range of use cases you can solve for using these cameras and AI-driven computer vision, you break out of the silo that these cameras are in today. You start to realize their true potential: to solve business problems across various domains, improve customer experience and open up new revenue streams.”
