OVER the past 10
years or so the financial burden of digital storage on the overall surveillance
spend has increased exponentially while at the same time vastly increasing the
usefulness of CCTV systems as investigative tools.
In more recent
times improvements in compression and analytics and falls in the price of
storage are all contributing to a reduction in costs on the one hand while a
shift to HD and megapixel cameras makes greater demands on the other.
According to
Milestone’s Angelo Salvatore, solutions he is involved with show that while
costs are falling users seem to want longer retention times.
“While
compression is improving and storage costs are dropping customers are demanding
longer retention periods which is directly affecting the cost of storage as
part of the overall solution,” Salvatore explains.
“It is quite
common to have 30-60 days retention where only a few years ago it was 7-30
days. Added to this is the cost of redundancy which is also common place.
Milestone can leverage existing storage infrastructure due to the unique dual
database structure that allows multiple seamless storage locations which can be
SAN, NAS, DAS to mention a few.”
In terms of the
ideal storage solution for video surveillance Salvatore says that most
companies are using generic storage systems like those used for data while
others are moving to more security specific storage systems.
“These might
include systems like those offered by our Milestone Solution Partners (MSP)
partners Data Direct Networks and Pivot 3 which are more expensive but designed
specifically for this application,” he says.
“These systems have
higher throughput, greater redundancy like RAID 6e, not only across drives, but
across whole units. They can also self diagnose and repair HDD issues.”
A challenge for
integrators and security managers is that as camera technology pushes into HD
and megapixel there’s pressure on storage solutions to support the increased
demand. To combat this pressure, Salvatore says Milestone uses a unique high
speed video database structure (not SQL) that allows for a distributed storage
model with a mix of high speed drives for live data which is used for
unicasting or multicasting video and lower speed, lower cost SATA II drives for
archive data.
“This reduces the
overall cost of storage significantly because all of your storage does not need
to be high speed or enclosed in one box,” he says. “You can utilize existing
and low cost storage throughout the network.
“With recent
development in IP camera compression like H.264 it is obvious we have made
significant gains in compression. So much so that you can store a 720P image
which can be easily compressed to around 20KB per frame.
“If you put this
20KB into perspective it was only a few years ago a usable MPEG4 frame of only
4CIF was around 30-40KB/ What we have achieved as an industry is a higher image
quality without the storage cost penalty,” Salvatore explains.
The latest
generation of analytics is a vital development that makes use of intelligent
recording and searching in order to extend the storage potential of current HDD
technologies without a need for vast increase in disk space.
“Analytics and
storage are mutually exclusive. I believe with the use of smart Meta Data
search software like that offered by Agent Vi will greatly improve search
capability, but ultimately you still need “real video” and hence integration
into the VMS storage medium remains the same.
As far as
Salvatore is concerned, the debate about local and centralized storage is the
same argument had in the past between IBM Mainframe and the LAN.
“I don’t believe
one technology will suit all applications; it will be a combination of both
storage mediums. What is important is that the VMS you are choosing can support
both storage systems,” he explains.
This storage
support should include onboard solutions.
“Milestone is
working with IP camera vendors closely and I think the advent of onboard
storage is a final nail in the DVR argument ‘what happens if the network goes
down’,” Salvatore says.
“Milestone will
take advantage of this technology and it will play an important part of everyday
surveillance design in the very near future. The size of the application is
irrelevant here as it can be used in conjunction with an integrated Open
Platform VMS.”
When it comes to
greatest challenges faced by video storage solutions, Salvatore says the main
issues are speed, capacity and reliability.
“Unlike data,
video doesn’t lie,” he says. “Skipped frames and video loss cannot be masked.
The advent of solid state storage is a big step in the right direction.”
Over at Avigilon,
storage is a key issue and the company’s Matthew Fisher says it’s tough to call
the percentage of a total CCTV spend that would be consumed by the storage
solution.
“This is not an
easy question to answer given that end users have varied requirements for the
length of video storage required,” he says. “This could be as little as 7 days
and up to 90 days, with the average being 30 days.
“Image rates and
resolution of the cameras also play a factor in the amount of storage required
for any given system, and this has more impact with today’s technology of HD
and multi-megapixel cameras.”
“I think that on
average CCTV hardware makes up 40-50 per cent of a surveillance system
deployment cost, with storage making up to 15-30 per cent of the CCTV hardware.”
From Fisher’s
perspective, the ideal storage solution for video surveillance is off-the-shelf
as he says it offers excellent flexibility and cost advantages over most of
today’s proprietary offerings. And Fisher sees compression as an important
element of future video storage solutions.
“In the future
compression codec’s will develop to offer greater storage, however, one of the
key considerations for any surveillance system is image quality of the recorded
video,” he explains.
“The mainstream
use of lossy compression codec’s such as H.264 may save on storage but it often
compromises recorded image quality. We believe that a lossless compression
codec such as JPEG2000 offers more advantages to HD Megapixel Surveillance
Systems than that of a lossy compression.”
Fisher agrees
that analytics is another vital weapon in the storage system designer’s
arsenal.
“A video
management system with good motion detection analytics will extend storage
potential,” he says. “Also a VMS, such as Avigilon’s, with Data Aging features,
will extend storage even further. Data aging works on the philosophy that the
most recent recorded footage is the most valuable and then older footage is
archived over time to lower image rates thus increasing storage capacity.
“In my opinion,
centralised storage is still the most cost competitive for small to medium size
IP Surveillance Systems. Decentralising has key resilience and system design
benefits for larger IP surveillance systems,” Fisher explains.
“Some
considerations need to be addressed with storage local in switch cupboards
though, with emphasis on environmental conditions for IT hardware. When it
comes to decentralised storage I think SD card storage at the camera is best
suited to small or temporary systems and those that have a direct web interface
to the camera.”
For Fisher, the
single greatest challenge for video storage solutions designers in the future
is market pressure.
“The market wants
more storage for less cost, space requirement and carbon footprint.”
At Axis
Communications, Wai King Wong says that while centralised storage is an ideal
solution it is highly dependent on infrastructure available.
“If the
infrastructure is not available, centralised storage will be difficult to
implement,” Wong says. “What this means is that in today context, it is still
best to have decentralised solutions with remote storage cabinets around a
site.”
Another storage
option is onboard the camera using SDHC cards with capacity of up to 32GB at
present and more in the future and Wong says this on-board storage is useful
for backup purposes.
“Such backups
include those times when the network goes down when onboard storage can be
temporarily used.” He explains. “When the network comes back up, it can stream
the videos back to the video management software.
“At Axis we
utilise the SD slot for this purpose and one of the main reason for this is
that SD cards has a maximum read/write capacity. Video solution writes at 25
frames per second which requires a lot of writing to the SD card.
“Constant read/writing
of SD cards means the lifespan of the SD card will not be a long one.”
According to
Wong, SD cards aren’t the only storage device challenged by the demands of
video surveillance. He says that the core of virtually every storage solution
is the hard drive and he suggests these HDDs are a limiting factor.
“There are many
storage solutions in the market but the ultimate capability of all these
storage solutions boil down to the slowest hardware component in the process
which is the hard disk drive,” Wong explains.
“HDDs are
mechanical products which have latency so to get the optimum speed for
read/write, it will require solid state drives (SSDs).”
“With recent
development in IP camera compression like H.264 it is obvious we have made
significant gains in compression. So much so that you can store a 720P image
which can be easily compressed to around 20KB per frame”
Along with the
development of low cost SSDs, Wong says compression technology is another key
element any successful storage solution.
“Compression
technology is highly important as it determines storage and bandwidth
requirements,” he explains. “Standardisation of compression is also an
important factor as users are able to stream/playback the video in any form.
“The H.264
compression is the ideal compression to use. The main reason for this is that
multiple market segments such as telecommunications (Mobile phones), broadcast
movies (Blue-Ray)and IP video security are all adopting the same compression
technology.
“Analytics is
important, too, and in my opinion analytics must be at the edge – in the
network camera itself,” Wong says. “The reason for this is that only
information required to be analysed will be sent to the server.
“Obviously this
reduces storage and transmission requirements. At Axis, we have released an
Axis Camera Application Platform which enables developers to develop software
to be implemented into any Axis cameras. Axis cameras in future will be similar
to the iPhone which allows the enduser to implement different types of
applications into an Axis camera.”
From Wong’s
perspective, the single greatest challenge in video surveillance storage
solutions today is increasing the read/write speed of the HDD given it is the
slowest component in any video storage solution.
According to
VideoIQ’s CTO and co-founder, Doug Marman, the percentage of overall cost of a
surveillance system that storage consumes is as high as 40 per cent, depending
on the size of the system and the type of servers used, which he says is higher
than it needs to be.
“I think in large
scale systems, off-the-shelf hardware is desirable to make system management
and support easier. Using known hardware is preferred by many IT managers,”
Marman says. “Lower cost solutions are available, but in large systems these
incur an added management cost and risk, being unknown by the IT managers.”
“I think that on
average CCTV hardware makes up 40-50 per cent of a surveillance system
deployment cost, with storage making up to 15-30 per cent of the CCTV hardware”
As camera
technology pushes into HD and megapixel there’s obviously pressure on storage
solutions to support the increased demand on storage solutions. Marman says
that H.264 compression brings significant improvements in reducing the storage
and bandwidth demands of networked surveillance systems.
“When it comes to
megapixel video, H.264 compression is going to become almost a requirement in
most cases,” says Marman. “While there is continued effort at improving
compression and offering scalable compression, there are no big pending
improvements as significant as the H.264 compression standard.
“Scalable video
codecs, which are compatible with H.264, allow one video file to be able to
play back at multiple resolutions and frame rates. There are advantages to
this, but they are not so compelling as to drive rapid adoption. We can expect
this to gradually make its way into the market.”
Marman agrees
that video analytics are an important tool to extend the storage potential of
HDD with technologies without a need for vast increase in disk space?
“Video analytics
have the ability to reduce storage and bandwidth requirements even more than
the move to H.264 compression,” says Marman. “There is no need to record at the
highest quality and frame rates when nothing important is happening. On the
other hand, it is desirable to have high quality, high frame rate video when
there is a critical event taking place.
“Video analytics
can recognize the difference and adjust the quality, resolution and frame rates
accordingly, in which case storage and bandwidth requirements are drastically
reduced.”
Like others,
Marman believes the decentralised model is the way ahead when it comes to
networked video surveillance solutions.
“All sensor-based
networks are much more efficient when the sensor’s intelligence and memory is
kept closest to the sensors,” he says.
“This is just as
true for video surveillance systems. Traditional network video has followed a
data centric approach, which has created a series of unnecessary problems that
require added cost and equipment to resolve. More and more of the storage is
going to migrate to the edge of the network.
“Onboard storage
represents a significant cost savings, and at the same time reduces bandwidth
requirements dramatically and improves system reliability, with lower
maintenance costs. It also reduces energy requirements and rack space in data centers.
“For standard
resolution, SDHC and the new SDXC cards will become popular and represent a big
opportunity in the future. For megapixel cameras, or for long term storage
requirements, hard drives in the cameras offer significant advantages over the
existing data centric approach,” Marman explains.
“When combined
with intelligent video analytics in the camera and H.264 compression, even 32GB
cards can store a month’s worth of video. SD cards have the added advantage of
longer life expectancy than hard drives in storage servers.
“The other big
advantage of storage in the camera is that the system scales from one camera to
thousands without any forklift upgrades. To protect video stored in the
cameras, however, adding video analytics that can automatically send video
copies to a remote location whenever any important event takes place provides
storage redundancy without the need for significant added storage space.”
As far as Marman
is concerned, the single greatest challenge for video storage solutions
designers moving forward is the complexity of centralized storage systems,
especially in large scale solutions.
“There are so
many variables to consider for reliable storage that system designers have a
hard time covering all the bases, especially properly scaling it to accommodate
for future growth.”
“Systems should
always be designed to meet the needs and requirements of the site and the kind
of security that is being provided. Is the system going to only be recording
video for later review? Is it going to be used for live monitoring? Does the
system want to be used to prevent problems before they occur?
“Clearly, the
future is moving towards systems with more intelligence in cameras offering
proactive threat detection, not just recording of video,” Marman says.
“Obviously,
including storage in the camera reduces systems requirements, storage costs,
and complexity dramatically but each application is different, and having a
toolbox of options to adapt to the needs of the application is most important.”
At Pivot-3, CMO
Lee Caswell, sees the cost percentage of surveillance system storage as
significant, with storage and servers comprising roughly 50 per cent of the
cost of modern surveillance systems.
“I expect to see
storage to continue to grow as a percentage of installed systems primarily
because customers are demanding more cameras, higher resolution cameras and
longer retention times,” Caswell explains.
“Modern networked
storage systems preserve investments in analog cameras and established cabling
which has the net effect of increasing the percentage that customers should
expect to spend on storage. This is a good thing.”
For Caswell, the
ideal storage option of proprietary, off-the-shelf or a combination is an open
off-the-shelf solution.
“Open systems
always offer customers lower costs and more options than proprietary systems,”
he explains. “However, open systems can only thrive when open standards exist
to ensure interoperability and competition. The IT market moved to open systems
15 years ago once standards such as Ethernet, Intel x86 and Microsoft Windows
emerged.
“We’ll see both
centralised and distributed environments because of the tremendous limitation
that the WAN places on surveillance systems. Once you realize that a single
camera can fill a T1 line, it becomes imperative to locate video recording
close to the cameras”
“Fortunately,
these same IT standards are ideally suited for open systems video surveillance
systems. The adoption of mainstream standards means that vendors can concentrate
on their specific strengths, namely software, cameras, switches and
server/storage infrastructure. Naturally there is a responsibility for open
system vendor to make test products so that interoperability is proven in the
labs and not in the field.”
Caswell sees
compression as a key element of storage solutions moving forward.
“Compression is a
key technology that allows customers to optimize video quality and manage
storage needs,” he says. “We will continue to see improvements in compression,
such as H.264, but my expectation is that the growth in cameras, resolution and
retention times will dominate the overall market and drive increased storage
requirements.”
Interestingly,
and probably correctly, Caswell takes an opposite position to others when it
comes to analytics. He thinks analytics will not lead to a decrease is storage
space but an increase.
“Intelligent
video content analysis always benefits from higher resolution and higher frame
rates. All of these factors drive increased storage requirements, not less,”
Caswell explains.
And according to
Caswell, things will not be cut and dried when it comes to centralised or
distributed architecture.
“It won’t be an
either/or world,” he says. “We’ll see both centralised and distributed environments
because of the tremendous limitation that the WAN places on surveillance
systems. Once you realize that a single camera can fill a T1 line, it becomes
imperative to locate video recording close to the cameras.
“Centralized
systems are perfect for LAN-based systems typically found in large centralized
environments such as airports, prisons and casinos. Distributed systems are
often found in rail stations, schools and enterprise buildings where WAN
connections are the norm. This is why video as a service (VAAS) has not
succeeded so far – it is simply too difficult to ship video across a WAN with
any time sensitivity or resolution.
“I think onboard
camera storage is a great idea for distributed applications where expensive WAN
connections would be required for a centralized system to work,” Caswell
continues.
“It satisfies the
general rule that recording should occur near the cameras although it certainly
has some drawbacks, such as limited retention times, a single point of failure
design and the fact that any local disaster or vandalism would compromise the
system.”
From Caswell’s
point of view, the single greatest challenge of surveillance storage solutions
is to meet increased storage needs without exceeding customer budgetary
limitations for acquisition costs and management costs.
“Virtualization
is a key technology that can help bridge the gap,” he explains. “Storage
providers that incorporate virtualization technology can reduce acquisition
costs by up to 25 per cent and operational costs by up to 40 per cent. With
storage representing such a high cost component of installations,
virtualization represents an important tool for resellers and customers looking
to meet budgetary constraints.”
“If I was using
current technology to build a storage solution for a 100 camera installation
requiring 40TB of storage, then I’d use four Pivot3 CloudBank appliances that
provide high-availability storage with no single point of failure and deploy
virtualization to eliminate $15K of server hardware.
“This system is
completely standards-based and works with open system video management software
from leading companies such as Genetec, Milestone and OnSSI,” Caswell explains.
“Complete
failover protection is provided free for both server and storage components and
the self-healing nature of the product ensures that customers will not suffer
video recording or access interruptions and that reseller support costs are
minimized.”
Sony Asia
Pacific’s regional product manager, Mark Franklin, says the percentage of
overall cost of a storage system taken up by storage depends on how the systems
is being managed.
“Storage needs to
be managed to minimise the cost ratio and high storage costs can occur only if
intelligent cameras and storage management are not used in a video security
system,’ explains Franklin.
“If intelligent
motion detection (IMD) cameras are used, storage costs can be minimised. For
example, intelligent cameras only tell an NVR to record what is needed. The
emphasis should be in camera infrastructure, (camera) edge storage and
analytics technology.”
Franklin takes a
practical approach to the ideal storage solution for video surveillance
technology.
“Of proprietary,
off-the-shelf or a combination, the best will be whatever gives the best
outcome for an application,” says Franklin.
“Proprietary
storage management features in cameras and recorders can reduce storage costs.
Also, iSCSI storage is popular, performs well, and is not proprietary.”
And Franklin says
developments like H.264 have made a real difference, with a 40 per cent
improvement over MPEG-4 Part 2. Analytics too, have a major role to play.
“Yes – analytics
is important,” Franklin explains. “Sony’s IMD is recommended as it eliminates
random movement over 15 frames removing many annoying false alarms and reducing
storage space required. If IMD is used, then major reduction in HDD space is
achievable as 24/7 recording is not needed.”
From Franklin’s
perspective, the single greatest challenge for video storage solutions designers
moving forward is education.
“Educating system
designers to plan video security systems efficiently, avoiding high storage
costs, particularly if IMD and edge storage are incorporated – that’s the
challenge,” he explains.
For Franklin, a
current top-line solution installed using the latest technology would include
intelligent motion detection (IMD), a camera with edge storage for failover and
use of an NVR with local iSCSI external archiving.
“This
configuration gives maximum performance and minimizes storage costs,” Franklin
explains.
At IndigoVision,
product manager, Alex Swanson, explains that when it comes to cost of storage
as a percentage of total surveillance system cost there’s plenty of variation.
“It can vary a
lot but cost of storage can be up to 50 per cent of the overall cost,” Swanson
explains. “This is not too high a proportion. There is always latent demand for
more recorded data. This means better quality video, higher frame rates and
video being retained for longer periods. Network solutions are simply enabling
users demand for ever more recordings.
“If I was using
current technology to build a storage solution for a 100 camera installation
requiring 40TB of storage, then I’d use four Pivot3 CloudBank appliances that
provide high-availability storage with no single point of failure and deploy
virtualization to eliminate $15K of server hardware”
As far as use of
off-the-shelf gear in concerned, Swanson thinks proprietary hardware has the
edge.
“Managing
security recordings requires proprietary specialised software and hardware,” he
explains. “The volumes of data involved and the relentless nature of video data
means that standard hard disks and standard file management software are simply
not good enough to provide a robust surveillance solution.
“Choice of
compression technology is crucial. The only suitable compression standard for
HD video is H.264,” Swanson says. “The H.264 standard is very large and
contains many tools for specialist forms of compression. This standard should
meet the needs of the security market for several years to come.
“The use of
analytics to either trigger specific recordings rather than rely on 24 hour
recording is another important way of reducing overall storage. In addition,
analytics can be used to vary frame rate so that recordings use a low frame
rate when there is no motion and only jump to full frame rate when there is
motion.”
Swanson believes
that different operational requirements will lead to different video storage
solutions.
“At IndigoVision
we see distributed storage as the way forward. The advantages are that data is
not needlessly transported across the core of a network back to a central
storage facility,” he explains.
“This leads to
lower network costs – both in terms of the capital cost of core network
switches and in the network costs of any leased WAN network links.
“Around 99 per
cent of recorded video is never looked at. Why pay for this to be transported
across your network? The only possible reason would be physical security of
disks which might be easier to arrange in a central storage location.”
Swanson says he
sees onboard storage as only suited for smaller locations.
“By all means
distribute the storage out to the same network segment as the cameras in the
form of small capacity NVRs but the storage still needs to be a properly
managed recording solution rather than a flash card and some on-camera
software,” he says.
As far as the
major challenge moving forward, Swanson sees this as meeting the latent demand
for ever more recording performance from end users.
“In the same way
users are demanding higher resolutions and frame rates for live video, the
video storage part of the solution has to keep up as well. This is coupled with
ever larger numbers of cameras on a given project,” Swanson explains.
And Swanson’s
take on the ideal solution is practical. It includes H.264 IP cameras or codecs
for migrating existing analog cameras, dedicated NVRs with 24×7 surveillance
rated hard disks and specialised recording management software. There’s also a
simple to use video management software front-end that allows tight integration
of live and recorded video.
“All three
components can be sourced from different companies but to get a tightly
integrated set of features that meets the operational needs of the security market,
a single vendor solution is best,” he says.
Western Digital’s
regional marketing manager, enterprise & CE, Patrick Lo, says the overall
cost of storage as part of an overall solution really depends on what kind of
surveillance system applications (stand-alone system or big NVR-Enterprise
storage RAID system).
“It also depends
on the number of drives inside of the system – 1, 2, 4 or more HDDs,” Lo
explains. “In general, we probably can say storage consumes about 25-50 per
cent of the total cost of the system.
“This is not too
high a proportion due to the fact that HDDs now can provide better
systems/solutions than when VCRs were used. Applications and resolution have
been able to become more advanced due to the capability of the systems
utilizing HDDs. An analogy is the horse-drawn buggy vs the automobile. The
horse drawn buggy provides the same fundamental capabilities (faster
transportation than walking), but the automobile has taken over, leveraging
much of the same infrastructure that the buggies used – in this case, roads.
“The ideal
storage solution obviously depends on what is expected of the system,” says Lo.
“Typically, off the shelf solutions can provide a certain amount of capability
leveraging off the shelf commodity costs/prices.
“If the system is
supposed to do something unique that COTS (common off the shelf) components
can’t provide, then a proprietary solution will have to be developed. The other
experience is that for the big projects (such as airport, train-station) are
mostly using proprietary for some advanced features requests.”
Lo agrees there
is pressure on storage to continue growing in modern surveillance systems and
he says multiple variables will play a part.
“The
functionality of cameras (HD, NightVision, etc…) being used in a given solution
and the duration of video retention will determine how much storage capacity is
required. Compression of the video being stored is important, and drive
capacities (the maximum is currently 2TB).
“Then there’s the
compression technology used, mega-pixels in cameras, number of cameras in a
system, frame-rate and length archiving required. There are a lot of
variables.”
“It’s even more
important to be able to transport the captured video from the camera to the
system,” Lo explains. “Transporting raw video versus compressed video can/will
require bigger pipes to move the data, even before it gets to the HDD.
Lo agrees
analytics play a significant role.
“In order to
analyse video, you need to have it stored on the system’s HDDs,” Lo says.
“Depending on the type of analytics being done, you might actually need more
HDD space than usual. For example, what if you were capturing data on half of
the end caps in a retail store to determine the demographics of the persons
looking at the products? Does the user pick the product up, read the back of
the product, and then put it back? Do they pick it up and put in their carts?”
And when it comes
to local vs centralised storage, Lo says both configurations play a role.
“It depends,” he
explains. “Local storage can provide quicker access to the data from a
local location. Centralized storage can often provide economies of scale and
better utilization than localized storage.
“Many of the
large “search” type web companies and content delivery networks are trying to
determine this question as well and typically have both local and centralized
storage at this moment.”
Lo agrees with
some of the others that storage on the camera is only suited to smaller
solutions, and for him the major question would be how people access the data
on the SDHC cameras.
“You’ll need to
plug them in one at a time to a computer to access the data,” he says. “Or,
you’ll have to create a network to be able to access the data on each card,
which probably minimizes the value of the on-board storage, in larger
deployments.”
Not surprisingly,
Lo says price and performance are the key challenges of the future.
“Customers want
easier to use systems with greater capabilities at lower prices,” he explains.
