It’s impossible not to see video content analysis as being the future of video surveillance. There are ever more CCTV cameras with briefs grounded firmly in situational awareness and efficiency. At the same time, there’s downward pressure on the price of systems. Automation built around video analytics is the answer but exactly how that automation should be delivered remains difficult to say.
VIDEO analytics is CCTV’s R&D hotspot – hundreds of millions of dollars are being invested globally to create solutions that leverage the vast data input of video surveillance cameras that remain largely reactive devices which are only called upon in post incident investigations. What’s important here is that it’s not just manufacturers who are embracing the power and capability of analytics, it’s end users wanting to liberate the full operational potential of their expensive CCTV systems. That last is the key driver of video analytics.
According to Will Hasna of Bosch Security Systems, VCA and video analytics is becoming increasingly important.
“These days customers are looking towards video systems to provide more functionality than simply capturing a good quality image, being able to record these images and replaying them after an event occurs in a reactionary way,” Hasna explains.
“As video system technology improves, we see that a security camera can provide much more data about the scene within its field of view. An example of this is when a camera can detect, differentiate and often classify different objects – for example the difference between an upright person and a car, or a car and a bicycle or a truck.
“The Bosch range of cameras with built in Intelligent Video Analytics is able to track up to 32 different objects per frame and report on metrics such as object size, area, geolocation, speed of travel, direction of travel and many more for every object it is tracking. This metadata can then be captured in a separate stream and becomes useful in terms of being able to raise dynamic alerts to a security operator for different tasks, or simply be able to collect this data to optimise the efficiency of a site. This is where the future of video systems will lie; in utilising the data that is generated by an intelligent camera that is another sensor in an overall solution.”
According to Hasna, end users are showing a strong interest in video analytics.
“We have been speaking with end users regarding our most recent advancements in this space and find that we have quite a high level of interest and commitment,” he explains. “We use analytics with our cameras for an endless number of applications with end users across many different verticals such as traffic monitoring, healthcare, mining, retail, education, corrections, ports and airports and many more. The possibilities are enormous.
“Functionalities of interest for end users depend on application and how they want to use the data. We find some users want simple metrics, such as counting the number of people or cars etc. entering their establishment, or how long they have stopped in certain areas in a retail setting. Other users want to be able to identify if a conveyor belt in their distribution centre is blocked.
“We have users who are interested in our AVIOTEC video-based fire analytic camera that has the ability to detect rising smoke in certain applications far quicker than traditional methods, and others who use our cameras to determine if staff are wearing the correct safety vest in designated areas. Really, when it comes to the use of analytics and our ability to have 16 different rules and combinations together with a script engine within a Bosch camera. The applications are only limited by the user’s imagination.”
If you were adding analytics to an existing solution, what would the process and the ultimate network topology look like? Server-based or analytics at the edge?
“It really depends on each application, budget and the level of video content analysis required,” Hasna says. “An edge-based analytic system using intelligent cameras can help to reduce bandwidth and the overall network design as the processing is being conducted in the camera, and you only need to pass the metadata or events out to other systems and not the entire video stream. This can also be helpful when you have cameras deployed in remote areas or low bandwidth connections.
“The total cost of ownership in an edge-based design can be cheaper. Often with server-based systems you have a license fee per analytic rule or algorithm you apply to one video stream. This means if in one area you need to monitor the crowd density as well as whether a car enters a certain restricted area, this may require more than one license cost. Also, a server-based system can only support a certain number of cameras before you need to deploy additional hardware, which equals additional rack space, power and cooling requirements.
“Server-based analytics can also provide a single point of failure in a system, for example, if the server is managing streams for 50 cameras to do the analysis and that server was to go off-line, then the analysis would stop for those 50 cameras. Analytics on the camera itself helps to mitigate this risk.”
Hasna says that Bosch’s analytics are available license-free on the camera and feature up to 16 concurrent rules in different combinations, with several object filters that can be applied.
“This makes having an edge solution very powerful and can add to the operational efficiency of your overall video system deployed,” he explains. “You typically find that analytics algorithms such as facial recognition are very processing intensive and generally server-based in nature. With the improvements in GPU technology, this has helped to increase the amount of server-based analytic programs in the market.”
How important is VMS integration for implementing analytics with an operational focus?
“Again, it depends on the needs,” Hasna says. “The ability for a camera to stream metadata generated as its own individual stream means that you can tap into this data and use it in its own right, without the need of a VMS essentially. You can have a scenario where the camera may be recording the video and incorporated into a VMS system on site, but the metadata can be streamed to an organisation’s private cloud for statistical reporting of operational goals. Another example would be to stream the metadata as its own alarm event to a control room for video verification of an intruder on site.
“Alternatively, a VMS that has the ability to record the metadata stream along with the video stream means that you have the ability to use a forensic search and retrospectively apply a rule to make searching for the relevant event easier. As an example, an operator could search the metadata for an object that is classified as a car travelling in a certain direction, between a certain time frame, and is this colour tone – and then only the relevant footage that meets these characteristics we have captured would display. There are many ways to interact with the powerful analytics that are incorporated into a Bosch camera, and as such it really depends on the customer’s needs as to the best way to design the solution.”
Pelco’s Branon Painter says that thanks to their ability to record video in digital formats, future surveillance solutions would be able to take advantage of intelligent video management systems (VMS) that harnesses the latest innovations like machine learning and data analytics.
“This helps security professionals retrieve valuable forensic evidence from recorded video and analytics will also enable security operators to perform investigations quickly,” Painter explains.
“Analytic solutions are top priorities for anyone looking to secure their business with a proactive strategy. This is made possible because modern VMS solutions can analyze live video against large stores of data comprised of predictive markers, like physical attributes and anomalous behavior, making it far easier spot threats before they can happen. End users are hungry for the technology – functionalities of interest include facial recognition, people counting, vehicle counting, and employee compliance regulation are some of the functions that security professionals look for when finalizing on a surveillance strategy.”
If you were adding analytics to an existing solution, what would the process and the ultimate network topology look like? Server-based or analytics at the edge?
“It could look like a combination of both elements,” says Painter. “With analytics at the edge, security professionals must keep in mind that only newer camera models may have this capability. This approach takes info about what is going on in the scene, i.e. tracking people, and sends it to a server, which uses a rules engine to decide the best course of action to take.
“For best operational performance, implementing analytics in VMS integrations play a key role in increasing operational efficiency, not only because it shifts the surveillance strategy to one that is proactive rather than reactive, but also because it helps to ensure employee compliance, which increases business intelligence. A camera can tell you about what’s going on in a scene; when you integrate VMS, you can use the information to react to activity throughout the property.”
Installation, calibration, commissioning – how hard is the process of putting together coherent analytics-based solutions and how important is it? Is it beyond integrators and does it need to be handled in partnership with suppliers?
“While installation can often take care of itself, commissioning and calibration can also become a simpler process through analytics,” says Painter. “In any case, complex analysis becomes much more reliable as a result.”
What would the perfectly balanced analytic solution look like and how would it function, in your opinion – all other application specific things being equal?
“It might look like analytics at the edge and forensic search capabilities on a centralized server all combined into a VMS,” Painter explains. “The VMS is the core capability of the system, so the more we treat it as such, the more effective the solution will be.”
Over at Axis, Sargon Yousif argues video surveillance technology has experienced quantum leaps in the past decade.
“We now have superior resolution with HDTV, 4K and higher clarity. With better low light sensors, more light sensitive lenses, smart sensors and powerful CPU, and cameras have evolved from mere tools for deterrence to great too for business intelligence,” Yousif explains.
“We see analytics requests growing, and in some cases surveillance cameras are being deployed for primary use of analytics and surveillance is secondary. As more people realise the power of video analytics, it will keep growing and playing an important role. Interest is consistently growing. Say, 4-5 years ago, it was retail end users who were primarily more interested in analytics as a way in collecting purchasing behavioural data. However, we have seen this shift moving to other sectors such as health, education, government and commercial. Currently there is interest in face recognition software but the main driver for analytics still is loss prevention and business efficiency improvements.”
If you were adding analytics to an existing solution, what would the process and the ultimate network topology look like – server-based or analytics at the edge?
“This would depend on the existing cameras being used,” Yousif explains. “We make our own CPU designed specifically for video processing called ARTPEC (Axis Real Time Picture Encoding Chip) and have capacity to run analytics at the edge. We could deploy some or all analytics at the edge. But in cases of large database requirements such as face recognition, this may be a server-based application or hybrid.”
Yousif says VMS integration for implementing analytics with an operational focus can be important if it’s operator-driven and acting on real time or near real time.
“This applies if the surveillance operator is making a decision to produce an outcome based on what they see or what they have missed,” he explains. “For example, the operator may need to know if a person or vehicle of interest has entered their store or car park. Analytics like face recognition or ANPR can give this info to the operator quickly, and let the operator take it further. Another example is in shop where the length of queue or wait time is long, letting the store manager know to allow more staff rather than surveillance operator. For this reason, some applications can work independently of the VMS.”
When it comes to installation, calibration, commissioning – how hard is this process and how important is it? Is it beyond integrators and does it need to be handled in partnership with suppliers?
“This aspect of analytics is not easy and is sometimes underestimated, in my opinion,” Yousif says. “Every install is different and for this reason needs to be designed and calibrated onsite. In today’s face-paced world, typical surveillance installations are bench-programmed, tested and sent to site, then installed by sub-contractors quickly to save time. However, analytic applications require a lot of time on site to calibrate and commission correctly. If the integrator has not done analytics before, it’s recommended they do some training before and seek the vendor’s help in design for the first few installs. Some complex applications will always require vendor’s help, in my opinion.”
According to Florian Matusek, product group director – video analytics at Genetec, analytics is going to be very important
“Organizations of every type and size are always looking to make more informed decisions,” Matusek says. “In order to do better and plan for the changing world, they want to have access to as much information as possible. And, rather than implementing a new information gathering system, many are looking to mine the data already being collected by their physical security systems and augment video with metadata to help operators make better decisions, both during live monitoring and during investigations.
“But it’s not enough to simply access this data. You need to make sense of it, and that’s where video analytics comes in. In general, analytics tools take large amounts of unstructured data and structure it to allow you to unlock its value. When you can correlate and extract information, you can gain all manner of insight into your business and environment.
“Video analytics has been part of security systems for 10 to 15 years. Traditionally, security departments used video analytics as a reactive tool that could detect something in a live video stream and then create an event that operators could respond to. But organizations have come to realize that you can use video, as well as other related data from access control and license plate recognition, for more than physical security. Since then, video analytics, powered by the advances in computing power, has evolved considerably to extract very reliable and powerful data from our video streams.
“And the industry itself is getting better at presenting the mined data in more consumable ways. We’re seeing a more end-user-oriented approach towards the development of systems. Instead of just providing all the data that is available, solution providers are looking at what users need and how to present that information in the clearest possible way.”
Matusek says Genetec is seeing several major vertical sectors where video analytics is generating a lot of interest – including retail, airports, and traffic management.
“Retailers see video analytics as a key tool for gathering information about how many people come into their stores, what they do while there, and which products they are looking at,” he explains. “Using this information displayed in heat maps or in people counting applications, they can analyse merchandising browsing behaviours and calculate conversion rates: how many people go into their stores vs. how many people buy products. Then, with greater understanding of their customers’ behaviour, they can make informed business decisions, including item placement and staff optimization, to better serve shoppers.
“Meanwhile, airports are also looking to make better business decisions that will improve customer traffic and increase revenue – particularly ways to optimize the flow of people through their spaces to make the process of boarding and disembarking as efficient as possible. Video analytics is a great tool for understanding how long people stand in security lines, where roadblocks occur, and where people gather. With this information, airports can optimize their staffing, reduce known congestion sites, and inform passengers where they should go and how long they can expect to wait in security lines. This will allow people to move through lines and checkpoints as quickly as possible, which can result in a direct increase in the revenue generated by duty-free shopping.
“And we are seeing also seeing a big push in smart cities to measure traffic in a reliable and flexible way. The data collected and understood through video analytics can provide cities with valuable information about what’s happening on their roads, how many cars are on a given street or how many cars are going through a specific intersection. This data can be invaluable for city planning, particularly when it comes to traffic coordination. City planners and traffic engineers can use this information, for example, to re-route traffic through alternative routes during rush hour to avoid congestion and to optimize the flow of vehicles through their streets. This would not only make commuters happy but could also reduce car emissions as vehicles would travel at a more consistent speed.”
According to Matusek, the functionalities end users are they most interested in are real-time event detection.
“But ultimately, video analytics is a means of creating value in many different industries through a variety of solutions,” Matusek explains. “As acceptance grows video analytics is moving away from existing as a separate standalone application. Security platforms that come with built-in analytics are speeding up deployment and delivering accurate results.”
If Genetec was adding analytics to an existing solution, what would the process and the ultimate network topology look like? Server-based or analytics at the edge?
“Before we dive into the pros and cons of each option, let’s clarify the terms,” says Matusek. “When we refer to edge-based analytics, this means that the camera or encoder is processing the image and creating metadata. In a server-based analytics setup, video streams are sent to and processed on the server, independently from the cameras. Each option is viable and effective but choosing the best option will depend on your environment.
“When you process analytics on the camera, the main advantage is that you’re able to reduce bandwidth usage. You can set up a distributed system architecture and lower server costs too. That’s because the camera filters the information, so you avoid having to transfer all video data to the servers. However, this is only true if you are only storing the video which the analytics solution has classified as relevant. This means that if you want to keep all video for a certain period, or just keep all video with motion-detected events, you would not gain much value from this option.
“When you opt for server-based analytics, you’re free to choose any cameras you want. This can be particularly advantageous to anyone who is upgrading their security system and want to reduce costs by keeping existing edge devices. While it’s best to check with your analytics provider, server-based analytics usually work with most cameras, regardless of the camera vendor or model. Another main advantage is that you’ll get better performance. Servers have more processing power and can process more video and more analytics applications. In return, leveraging the processing power allows for the development of more advanced analytics.
“Server-based analytics can be easier to setup and use as well. If you’re using a security platform with unified analytics, for example, you’ll be able to configure the analytics from the same interface as your video management system. This is also true for various analytics applications, providing the same user experience across all types of video analytics which simplifies configuration and operation. In larger deployments with hundreds or thousands of cameras, this can be a huge time saver.
“Choosing the right analytics technology really comes down to your needs. If you’re only looking to do basic analytics or you have a small to medium-sized installation, an edge-based solution might be the right choice for you. If you require high-end analytics or have a larger enterprise system, server-based analytics is the way to go.”
How important is VMS integration for implementing analytics with an operational focus?
“Very important,” Matusek says. “One big current change is that customers are demanding more and more analytics tightly coupled with the VMS. In the end, video analytics should be something running in the background, providing additional information to usual VMS operations. Customer see it as part of the VMS and don’t want to use separate configuration interfaces and take completely separate training classes to learn a new user interface. And in the end, if there are issues with either VMS or analytics, customers want one team to talk to.”
How hard is setting up and commissioning analytics systems? Is it beyond integrators and does it need to be handled in partnership with suppliers? And what would an ideal analytics solution look like?
“Analytics still require specific know-how, so proper training for integrators is essential,” Matusek says. “However, any scalable video analytics solution should be deployable without supplier involvement after technicians have been properly trained. Of course, suppliers will always have their back in case something goes wrong, or a specific use case requires extraordinary analytics know-how.
“Modern analytics solution put the application for the user before the technology itself. In the end, analytics are a means to achieve a business outcome for the customer, so a perfect analytics solution focuses on outcomes rather than touting the latest and greatest technology it uses.”
Over at Milestone Systems, Mike Metcalfe sees analytics as a key dynamic to the future of video surveillance.
“Automated processes through video AI and metadata analysis can only strengthen business operations, not only for security but beyond security business operations into areas such as consumer heat mapping in retail settings and workplace optimisation,” Metcalfe explains. “Access to business intelligence enables businesses to become more efficient. Analytics and video analysis will certainly be an enabler.
“We find that requirements differ depending on the vertical. In smart cities for example, it may be shooter detection triangulation. Corrections? Perimeter breaches and thermal object tracking as a start. The mining industry look for business intelligence statistics about their vehicles and their route frequencies through GPS data.”
When it comes to the layout of the analytics solution, Metcalfe argues this depends on the exact requirements of the site and what the outcomes-based expectations are.
“A good combination of both can really give you a great solution,” he explains. “We work with some excellent, well-respected server-side analytical partners that can deliver a host of desirable outcomes. With edge devices being manufactured smarter and to run leaner by way of more efficient and faster chipsets, this allows for a wider choice of clever, AI edge-based analytics, which in turn enables solutions to scale more simply, based on the specified requirements. It’s important when choosing solutions to remember to consider an open technology that enables their analytics to be available on any platform and not just to work in a proprietary environment.”
For Metcalfe, VMS integration is integral to implementing analytics with an operational focus.
“When a system can manage various vendor analytics, both server and edge-based, and tie those to different connected technologies thereby creating the combined effect of automating what used to be human operational tasks, then the power of that technology simply cannot be ignored,” he says.
According to Metcalfe, installation, calibration and commissioning of well-designed analytics is not beyond integrators.
“Some analytics do require more calibration than others,” he explains. “Some require learning periods and then tweaking. The challenge for integrators, I think, is finding what really fits the needs of their customer’s application. You may consider selecting a core group of technologies which supports most requirements, knowing that anything outside of that core group can be investigated and tested before being deployed.”
What would the perfectly balanced analytic solution look like and how would it function, in Metcalfe’s opinion – all other application-specific things being equal?
“Wouldn’t it be great if the solution delivered outcomes for various facets of a business? Let’s consider an outside/inward approach through a building,” he says. “Let’s start with access control, where automated entry to the employee parking area based on license plate recognition, matched with a vehicle’s make, model and colour. Then upon entry to the premises, the system recognises the employee’s face at the card scanning entry point.
“In the lobby entry automated speed gates use facial recognition to allow access, as well as monitoring tailgating and wrong-way passing and for lift management biometric scanning defines which floor level an elevator should travel to for the employee. Next comes business intelligence which shows heatmapping for employee location frequency, while people counting can be used in the event of evacuation to ensure security teams know who is on the site and who is not.
“Then there are things like dwell time and loitering: Solutions will map how much time is spent in non-productive areas or areas less frequented, while wellness and work life balance analytics functions will tell workers they have spent too long at their desk and it’s time to move, eat a snack, or take a lunch break. All of this is intended to contribute to employee operational efficiency by delivering more value and driving positive outcomes to an organisation’s business delivery.”
#sen.news
