EARLY adopters don’t always get the best of new
technology. It’s usually smarter to hold off and make a measured selection when
time has scrutinized shortcomings and polished out flaws. SKYCITY Darwin held
out longer than most – just 12 months ago its surveillance system was all
analog and staff juggled VHS tapes recording video streams from more than 300
cameras.
In hindsight the delay in adopting digital technology was
fortuitous. Instead of wrestling with a major hybrid solution incorporating
elderly matrix switchers, SKYCITY Darwin chose Dallmeier’s SeMSy Video Management
system supplied by C.R.Kennedy and installed by ACE Cabling. The SeMSy solution
is supported by Dallmeier’s DIS-2 MPEG-2/MPEG-4 Dual Streaming encoders
(running on Linux), and a selection of quality off-the-shelf network hardware.
It’s a solution that’s as elegant as it is functional, a design that truly
keeps storage at the network edge.
Important to consider is that SKYCITY Darwin’s new
digital system needed to be more than just capable. Darwin is a long way from major distributors
and integrators in the South, and like all casinos, the solution’s design brief
demands always-on surveillance buttressed by an underlying capacity to perform
under virtually any circumstances.
Part of this system-toughening process is ensuring the
casino’s inhouse technicians know how to install and adjust cameras and other
components in the event of failure. While unusual, this training guarantees the
casino’s operational viability has signification protection in the event of
failure – that’s important because casinos in Australia can’t legally operate
without surveillance.
“I couldn’t put a figure on the improvement the switch to
digital has brought us. However from the point of view for staff productivity,
it’s huge. Digital means more hours of the staff observing the monitors and
protecting the assets of the casino”
Like most modern systems, the new SKYCITY Darwin solution
incorporates analog CCTV cameras, which are carried via coaxial cable to the
server room and then funneled into Dallmeier DIS-2 encoders. The moment video
arrives at these encoders, resemblance to other networked surveillance
solutions ends. The reason for this is that Dallmeier’s DIS-2 encoders sport a
pair of onboard 250GB hard drives between the encoder and the network interface
card.
Each DIS-2 supports a single input and 8 days of full
frame MPEG2, 25ips video is stored in an onboard RAID-1 mirror, with selected
incidents directed to an archive server for long-term RAID-5 storage. The
result of this architecture is profound. There’s more than 180 Terabytes of
hard drive storage at SKYCITY Darwin yet the surveillance system runs on a Fast
Ethernet LAN.
Analog past
The former analog system which incorporated a 1000
switcher and oodles of attention seeking VCRs was installed in 1995 and
according to SKYCITY Darwin’s security, surveillance and property services
manager, Bill Page, it was the best of
CCTV systems in its day. By 2007, however, its capabilities were distinctly
pedestrian and there was a real need to leverage the efficiencies and
functionalities of a digital solution.
Page, who began his career in the casino industry in Las
Vegas 30 years ago as a casino dealer, eventually spent 11 years on the casino
floor as a supervisor and pit manager
before moving on to surveillance director roles at Tropicana Hotel and Casino,
and MGM Grand Las Vegas – a major casino with a huge VCR surveillance solution.
n between jobs in big casinos in the U.S. Page came out
to Australia
for 12 months in 1995 and oversaw the installation of SKYCITY Darwin’s analog
surveillance solution as director of security and surveillance. After returning
to the U.S. to oversee the
installation of the CCTV system at Greektown Casino in Detroit and
at the Planet Hollywood Resort and Casino in Las Vegas Page came back to Australia
in February of this year.
Page’s deep-rooted casino experience gives him a vigorous
operational perspective when it comes to video surveillance and this is
important because it means he approaches his digital surveillance solution with
extreme clear-sightedness. While admiring the slick new technology it’s obvious
that for Page, operational efficiencies are the vital ingredient.
As Page explains, SKYCITY Darwin’s old CCTV system, like
all the surveillance solutions he’d worked with, was end-to-end analog and
supported by VCRs that needed to be constantly babied throughout the day.
According to Page, when digital first came out the cost
was too high from his perspective but he says that now prices have dropped the
technology is now viable for casinos.
“When I returned to SKYCITY Darwin the first question I
asked was: What are we doing with the surveillance system?” Page explains. “I
was told by management that they were planning to install a digital Dallmeier
solution. I didn’t know much about the Dallmeier system but I looked at
everything I could find on the internet and soon realized it was right for us.”
“A great thing about the Dallmeier system from my
operational perspective is the fact you don’t have to work with video tapes and
all the problems associated with them,” Page says.
“You don’t have to take operators away from their
workstations for 30 or 40 minutes at a time to change out tapes. You don’t have
to store and manage those tapes, you don’t have to incur the cost of purchase
of new tapes and new VCRs year after year.”
“I couldn’t put a figure on the improvement the switch to
digital has brought us. However from the point of view for staff productivity
it’s huge,” he says. “Digital means more hours for staff to observe the
monitors and protect the assets of the
casino. If you have 2 people at the monitors on a quiet day and one of them is
off changing tapes for 30-40 minutes that’s a set of eyes taken away.”
Page says the intuitive archive management system is the
best feature of the new digital Dallmeier solution in terms of day to day
operations.
“In the old days if an incident occurred you’d have dig
out the log books and locate the tapes
then some one would have to sit down and view the tapes,” Page says. “With this
system you can search by date by time, by location by name, by whatever –
there’s a drop down menu and the system will find the video for you.”
“And it’s redundant the way the system is designed – I
know there’s no possibility of losing coverage – that was always a problem with
tape”
Planning the solution
The decision to install a Dallmeier solution was driven
by SKYCITY Darwin consultant Craig Graham of Radford Consulting and the
casino’s surveillance manager, Greg Bretag. The system Craig and Greg selected
combined the Dallmeier SeMSy security management system with Dallmeier’s DIS-2
MPEG2/MPEG4 dual streaming encoders with onboard RAID-1 storage.
This was an interesting choice and an astute one. A
system with this structure not only gives a great deal of functionality, it’s
also loaded with layers of redundancy at multiple levels. According to C.R.
Kennedy’s Vlado Damjanovski, failure of one of the two drives in each encoder
will not compromise the integrity of the recording.
“Each DIS-2 is hot-swappable and each subrack hosting 10
DIS-2 encoders runs on dual redundant power supplies with dual UPS support,”
Vlado explains. “Even the Cisco network switchers are designed with redundancy
in mind so that system uptime at SKYCITY Darwin is projected to hit the same
numbers as Dallmeier’s Macau Casino-proven uptime with “six nines” at 99.99994
per cent – that’s higher than the best servers in the world.”
Vlado says that SKYCITY Darwin chose broadcast quality
MPEG2 rather than the more common MPEG4. This compression protocol offered
picture quality no different from the live image coming from the camera,
irrespective if this was live of playback.
What’s important in this decision is the fact the
architecture of the Dallmeier solution allowed the end user to choose a
compression protocol dedicated to image quality, not optioned for streaming
through skinny network pipes.
Integral to the installation and its commissioning was
Steve Maynard, general manager of C.R. Kennedy’s CCTV Division. Steve and Vlado
were heavily involved from start to finish with this job.
“When we won the job at the end of last year the first
thing Vlado and I did was to come up to Darwin with an analyzer and made a
record of the signals on every piece of surveillance cable in the casino,”
explains Steve.
“We audited the system completely and ultimately that was
the right approach. Surveillance in a casino is vital to the operation of
tables so this analysis was central to ensuring the casino would always have
the coverage it needed to legally operate during throughout the upgrade.”
According to Maynard, most the work conducted by C.R.
Kennedy and ACE Cabling took place behind the patch panel into which all the
system’s cameras are directed on their way to the server room.
“Cameras in the field come back by RG-59 to the patch
panel and then go onto thin coax to the server room and into the encoders,”
Steve says. “There are 360 inputs at the patch panel and 360 inputs with
expansion space for an additional 180 inputs down in the server room.”
“The server room is the heart of the SKYCITY Darwin
installation and a stunning piece it is. Along with the fundamental cleverness
of Dallmeier’s system design and a sweet general layout incorporating
under-floor cabling, dual air conditioning units and integrated fire detection,
the stand-out feature of the server room is the high quality of the cabling and
looming. It’s simply beautiful work”
“Signals go from the encoders to the hard drives within
the DIS-2 itself and they can then be accessed from anywhere on the network,
with incidents stored to the archive server’s RAID-5 storage system if they are
to be retained long term.”
“Live footage can be called up directly from the DIS-2
encoders from the control room when required but multiple live streams are not
supplied there unless they are called for,” explains Steve. “The control room’s
access to image streams is via a standard Fast Ethernet network. It’s a
dedicated surveillance network and not connected to the casino’s data network
at all.”
“From a network design point of view the system is
relatively simple,” Steve says. “It’s a dedicated Fast Ethernet LAN that runs
from the server room to the control room with no external ports – a fairly
typical configuration for high security CCTV LANs.”
“There’s an access layer controlled by core servers,”
Steve says. “These are all Cisco switchers – 2960 Series access switchers for
the access layer and 3560 series core switchers handling the core.”
“Each of these core switchers is connected to each access
layer switch. There are 8 access layer switches, each 48 port, with the core
switches being 24 port,” Steve explains. “With this installation you can lose a
whole core switch and still have plenty of workstations operational.”
“The idea is that no matter what happens up here the guys
have is something that’s fully redundant and very robust – under all circumstances
the system will keep delivering,” Steve explains.
“From a technical standpoint something that’s really
important is that the architecture of this system is specifically designed so
the surveillance network runs on layer 2 with SNOOP and without the need for a
gigabit LAN,” Steve explains.
And Steve says the Dallmeier system has multicasting to
reduce data traffic when multiple operators select the same live view.
“With many other digital surveillance solutions the
encoders transfer all the data from the cameras onto the network all the time
but in this system, the data is encoded to HDDs within the DIS-2 and does not
go directly onto the network unless it’s requested,” he explains.
“So the big difference here is that all the bandwidth heavy
work is being done off the network. If video was being transferred directly
onto the network then you’d need Layer 3 support – and at Layer 3 the cost in
network infrastructure becomes enormous.”
“Very important is that this overall system configuration
as installed at SKYCITY Darwin allows you to build very large digital solutions
– the largest digital system in the world uses Dallmeier’s system architecture
with more than 3500 cameras,” explains Steve.
Operational capability
No account of this solution is complete without
recognition of the capability of Dallmeier’s SeMSy video management software.
The control platform is extremely hard to disassociate from the Dallmeier’s
digital matrix and in some ways it’s best think of them as a single entity.
The digital matrix delivers full frame resolution and
real time storage for every channel, with images able to be recorded from any
camera and simultaneously switched to a linked monitor using SeMSy. Operators
use SeMSy to view images in live and playback mode, while recording continues
simultaneously.
“From an operational point of view the reason this
Dallmeier system uses MPEG-2 as opposed to MPEG-4 is that MPEG-2 offers better
quality,” says Steve. “MPEG-2 is also a broadcast standard and able to be
played on almost any device.”
“In a casino you simply have to see fine detail – a ball
falling into a hole, a card, a dice,” he explains. “MPEG-4 won’t give you the
clarity that means such a lot to a casino.”
All the cameras are being recorded in full frame MPEG2 at
25 frames per second and the SKYCITY Darwin guys have chosen 3 megabits/s per
channel.
“Regardless of this, Ethernet requirements are still
minimal because the cameras are being recorded off the network – that’s very
important,” Steve explains.
“The workstations only need 100Base but a camera still
has the potential to be viewed 9 x 4 x 6 x 3 times, plus whatever multiscreens
are being deployed.”
Allowing the operators to display multiple screens in 9
or 16-way spilts on large screen formats, Dallmeier Vario-Decoders at SKYCITY
Darwin utilise a Gigabit portion of the CCTV LAN.
“The Vario-Decoders handle multiscreen decoding in full
frame MPEG2 real time for 4-way, 9-way splits, and do 16-way splits in nearly
real time.
“This is in addition to the 6 operators operator each one
having three true composite monitors in real time” Steve says. “These big
configurations allows all operators to keep an eye on key areas and call up
incidents and review playbacks on their own workstations or system keyboards.”
Another interesting characteristic of this solution is
the fact there are some CRT monitors in the control room along with the plasma
screens and LCD screens.
“It was felt in the selection process that good quality
CRT gave superior viewing quality compared to LCD at these full frame MPEG2
resolutions, simple as that,” says Bill. “Overall picture quality was just
better and CRT has better reliability.”
Meanwhile, supporting mouse operation of the GUI is an
external keyboard with tons of onboard processing power that’s able to switch
the matrix, review, archive, control PTZs and activate specific functions. A
jog/shuttle is integral part of the Dallmeier VMC keyboard and it offers preset
keys to switch between live and playback instantly. Keyboard functions include
one button to search, another to archive, one button to burn to DVD – then
there’s jog shuttle, frame by frame search and easy manipulation.
According to Steve the system keyboard has a lot of
program capability and this makes it ideal for casinos.
“The system can operate differently using keyboards with
onboard processing that take shortcuts to give operators faster reaction time,
or it can be done via graphical interfaces on workstations using a mouse,”
Steve explains.
“There are maps, there’s a lot of feedback to the
operator and there’s a warning system integrated into the system that allows it
to report things like HDD failure.”
“All the maps are original AutoCAD maps and can be one
touch loaded to the server and then downloaded from there to the workstations,”
he says. “You can select a camera directly from a map same as one touch
selection on the keyboard. Maps will follow the camera selection. You can edit maps, add cameras and numbers
directly into the AutoCAD drawing and the system will recognize these
automatically. This has never been done in CCTV before.”
Bill agrees the Dallmeier system with its SeMSy video
management system is extremely easy for staff to operate.
“It’s simple to operate as well as having all the bells
and whistles you could possibly ever want,” Bill says. “It allows surveillance
staff much greater access to the video footage, allowing them to not only
monitor an area more effectively but to better support security staff on the floor.
”
“That’s what you get with digital. We can track a wide
range of situations across the casino floor, we can have point of sale
interfacing allowing us to track data from that source as well. There are all
sorts of analytical tools available. The Dallmeier system is great.”
Bill says that from his operational point of view, the
archive management system is the best element of the SyMSy video management
solution.
“In the old days if an incident occurred you’d have dig
out the log books and locate the tapes then some one would have to sit down and
view the tapes. With this system you can search by date by time, by location by
name, by whatever – there’s a drop down menu and the system will find the video
for you,” Bill says.
“And because the system is designed to be redundant at
multiple levels I know there’s no possibility of losing the coverage.” Some of
the operators were nervous making the change from analog and thought they might
have trouble operating the system but it’s so easy they’ve adapted.”
Bill says that while the decision to go with Dalmeier was
made before he came on board, it was based on the fact this solution was being
used in major casinos around the world.
“Importantly, with the Dallmeier system we could send
people to see the same system working perfectly in applications much larger
than our own like the large casinos in Macau.”
Bill explains. “We were obviously interested in latency but we found it was
very minimal operationally.”
Steve says latency is the big question that arises with
digital solutions but he says latency with the Dallmeier solution is a non
issue.
“The system is very user friendly in that regard and
obviously that was one of the key performance criteria with the SKYCITY Darwin
system,” he says. “With viewing of single cameras from single encoders there’s
virtually no latency – maybe a hundred milliseconds.”
“The latency and the control issues were question marks
and they have been well and truly answered,” Steve explains. “And the quality
of this system versus the hybrid solutions available made this a much better
option.”
“I think a lot of people have a false sense of security
in relation to their analog hardware – it’s a big primary piece of gear to go
down – you lose a switcher and that’s the end of that,” Steve says. “The matrix
switcher at SKYCITY Darwin was 12 years old and becoming hard to maintain – the
level of redundancy is greater now, and at all levels.”
Steve says other strengths of the Dallmeier system
include the fact digital has a lot a capability for integration including
support for things like Micros cash management systems, as well as facial
recognition and numberplate recognition.
“Another great strength of this solution is a mobile
disaster recovery module that can be handled at a single workstation via a
laptop,” he says. “It’s possible for the casino to retain control of the system
under the most adverse and unlikely conditions. Mind you, recording never stops
irrespective of how you control the system.”
“I can’t imagine a normal situation this solution is not
equipped to handle with ease,” Steve says. “But if anything comes up, the
control room will still be in charge of surveillance.”
According to Bill Page, while digital has a higher up
front cost than analog solutions he says the maintenance is significantly less
and the functionality vastly greater.
“From an operational point of view the key thing about
this Dallmeier system is the user friendly and powerful archive management –
it’s perfect for operators viewing and recalling incidents,” Page says.
“There are the bells and whistles but that’s the heart of
the system for me. You could save a tape for 5 years and then view it again and
possibly have a VCR eat the tape. When an incident occurred you had to save the
original tape then you had to make a copy of the incident, then you had to put
it all on a master tape so there were 3 VHS tapes for each incident. In the
event of an incident you quadrupled the number of tapes you had to manage.”
Bill says the modularity of the system makes easy to
maintain, flexible, easy to expand and easy to upgrade, too.
“The system has the potential to integrate with software
analytics that look at card counting and so on. We’ve got no plans for this yet
but it’s nice to know the system is capable of supporting that software in the
future. It is also capable of integration with the access control system.”
Steve agrees.
“The access system here is Concept and InnerRange
people have made a great interface with Dallmeier so it would be very easy to
integrate Insight with this solution at some point in the future,” he explains.
There are a bunch of casinos in Australia using
Dallmeier products – some have done so for more than a decade with most adding
digital components to their existing analogue matrices. Such casinos include
Burswood in Perth, Crown in Melbourne,
Reef in Cairns, Lasseters in Alice
Springs. But this new solution at SKYCITY Darwin takes Dallmeier’s
engineering to a new level.
The installation
The Dallmeier installation at SKYCITY Darwin was
characterized by SKYCITY Darwin’s request to make the switch to digital with no
downtime. According to Vlado, this was a vital request. No casino in Australia can
operate without a CCTV system running and that means surveillance is integral
to casino operation – no cameras, no business.
Vlado says that to the satisfaction of the customer, C.R.
Kennedy and Ace Cabling delivered this.
“The project was completed in record time just as the
consultant and customer required, making this project a show piece of what can
be done with good planning and organisation.”
Same as most large installations, the SKYCITY Darwin installation
called for a layered approach with multiple stages. Firstly, a temporary new
Dallmeier system was installed in half of the SKYCITY Darwin human resources
training room and then operators were trained in its use. At this stage, both,
analogue and digital matrices, were running in parallel.
“Once the operators were up to speed, they moved to the
temporary system and the old system was shut down and the equipment and cabling
pulled out of the old spaces,” says Steve. “The new control room was then built
and when it was completed, the team moved from the temporary control room to
the new one – all this was well planned and went by the numbers.”
Steve says that overall there were a couple of little
hiccups relating to equipment delivery, often a problem fro Darwin, but that’s about all.
“During the early stages of the project we were delayed
because the racks for the server room were bent while in transit and we had to
address that problem,” he explains.
“Strongabuilt manufactured some new parts quickly and we
were able to resuscitate other parts.”
“You see some of the key characteristics of the Dallmeier
solution in the server room at this casino,” Steve says.
“Racks and rack
cable management systems were designed specifically for this application
by Kevin Wilson from Strongabuilt.
“Racks are built to be serviceable front and back and the
system displays a high level of redundancy. It’s a very reliable solution and
that comes from experience with big casino installations – big ones like Macau.”
The server room is the heart of the SKYCITY Darwin installation
and a stunning piece it is. Along with the fundamental cleverness of
Dallmeier’s system design and a sweet general layout incorporating under-floor
cabling, dual air conditioning units and integrated fire detection, the
stand-out feature of the server room is the high quality of the cabling and
looming. It’s simply beautiful work.
What throws the excellence of these looms into such sharp
relief is the rudimentary standards evident in older parts of the cable plant.
Like any elderly building, SKYCITY Darwin has been worked on by a large number
of contractors over a very long period of time and the result is a sprawling
cabling infrastructure of variable quality and unusual configuration.
“Handling the legacy cabling was a major task,” explains
Bill. “It was literally spaghetti. Each individual cable had to be isolated and
tested to establish its content and once that was done it was either tagged or
chopped, depending on what the testing found.”
To some extent replacing and upgrading the existing cable
plant became a defining feature of the installation, with a large amount of
testing and tagging required to ensure the system was robust enough to meet
operational standards as various key times throughout changeover process.
According to Steve, even though the site is not a large
one, ACE Cabling still put one-and-a-half tons of RG-59 in a cable management
system in the ceiling void above the casino floor with a combined distance of
about 20km.
Bill says a particular issue during the installation was
the discovery that the old style PTZs which use a DTMF tone for data comms were
not compatible with the new system.
“We had to remove dozens of these cameras and replace
them – then power supplies became an issue because multiple of these old PTZs
were daisy chained to a single power supply,” he explains. “That had to be
cleaned up, too.”
Bill says that an integral part of the job was meeting
with upper management to keep them informed of how the work was progressing.
“Surveillance manager Greg Bretag and I would meet weekly
with upper management to inform them where we were at and to outline any risks
and requirements,” explains Bill.
“It was important that management knew exactly where we
were throughout the upgrade,” he says. “SKYCITY Darwin management was very
involved and supportive – that was important to the success of this
integration.”
Bill says an integral part of the installation and
commissioning process involved training operators to manage the new system.
“The training undertaken during installation went very
well,” Bill says. “We broke it down into technicians and operators and then
went through a schedule – the system was not really all up and running at that
point but it went well.”
“As part of that process we created our own training
manual in addition to the Dallmeier training manual – we gave that manual to
new operators when they begin. As we add functionality we add that in to the
manual,” he says.
Steve agrees.
“Taking 20 operators and training them all on an entirely
new system, while the old system is still running – it’s harder than you
think,” he says.
Another important element of this installation was
customization of the video management software.
“The archive management system is also customized during
installation,” says Bill. “It’s a matter of sitting down with the system
designers and integrators and letting them know what functions are needed and
if this can be accomplished within the
guidelines of the system.”
Typically all SeMSy video management solutions get some
customization, according to Steve.
“There’s a process in the installation where elements of
customization can be included in the system design – frame rate, resolution and
so on,” Steve explains.
“Some casinos like more latency and higher image quality
– it depends on the applications. Then there are things like Micros
point-of-sale integration or fine tuning of the Vario-Encoders.”
According to Bill, when the new control room was
finished, the temporary control room was switched over.
“We switched over at about 4.30 in the morning – we had 3
teams of 2 technicians and they went around and switched the data for the PTZs
to make ensrue that the controls of the
cameras were operational – that went really well,” Bill explains.
“We had worked out spreadsheets that assessed every camera
and considered what day they could be switched over right down to the time of
day. Our temporary control room was a case in point. We had to be out of there
by a certain date and we met that date – it was very pleasing.”
Steve agrees.
“We hit all the right targets and the system even came in
a little bit under budget – that was popular,” Steve says.
“The final commissioning and programming was done by two
experienced engineers from Dallmeier; Umberto
Vicenzotto and Philip Rissettio; assisted by Trevor Joseph from CRK
Queensland office, and Paul Cafeo from CRK Melbourne office,” Steve explains.
“Those engineers came all the way from Germany and
they put in some long hours – I thought that was fantastic,” Bill says. And
Bill clearly felt Dallmeier’s engineering was enhanced by the support of C.R.
Kennedy’s dedicated people.
“An important part of this system is maintenance and even
though the system is now installed and the contract completed, C.R. Kennedy
still comes and views the system,” Bill says.
“With a casino it’s important to keep the system working
perfectly – that’s what CR Kennedy gives us. Other vendors were not as
forthcoming in terms of system support but these guys made extra effort during
the installation and after it.”
Steve says C.R. Kennedy is delighted with the system,
too.
“From C.R. Kennedy’s perspective we’re obviously very
proud of this installation,” he says. “We think SKYCITY Darwin is the first
casino in Australia,
and probably in the world, that had an existing analogue matrix system
converted to complete digital matrix, without any down time during the
conversion.”
“Converting large analogue CCTV systems into digital
solutions is not only possible but easy to do – that’s what we have proven with
the SKYCITY Darwin project,” Steve says.
