Convergint Technologies, ConnectEast (CE), Broadspectrum (BRS) and Barco have integrated Genetec Security Center VMS and Axis IP cameras with Citilog analytics, UGL OMCS and Barco OpSpace to a design by security consultancy, ACAD Services. The solution balances complex topology, an exceedingly sharp operational focus, many interest groups and the stringent requirements of EastLink operators.
ANY piece of road infrastructure is a challenge for its managers – upgrades must be handled with meticulous planning and careful process control to ensure traffic flows are never impeded and that safety is maintained at all times. These priorities colour many aspects of the EastLink CCTV and network upgrade, from product choice and the involvement of multiple interest groups, through to incremental upgrades after the system went live.
There are currently 230 cameras across the freeway, tunnels and buildings that support the EastLink freeway, but this modest camera count belies the intensity of this application and the complexity of the solution integrated here. On a camera-by-camera basis, this is the most complex video surveillance application we’ve ever seen. It’s tempting to say the bulk of that complexity lies in the layered network topology but there are too many other things going on at EastLink to be certain.
The monster lead time with its painstaking process of selecting hardware and software, testing before production, testing after production, and ensuring buy-in from multiple internal and external stakeholders, speaks of the importance of its video surveillance system to the safe and efficient operation of EastLink. This is a serious system designed, built and installed for a serious operator.
In terms of hardware and software at EastLink the new IP CCTV system is built around an Allied Telesis network, with a DELL VMWare virtual server environment, Genetec VMS, Barco OpSpace operator workspace solution and third-party integrations with Citilog video analytics and UGL’s SCADA (OMCS). Analogue cameras that have been brought into the IP solution via encoders and are now being replaced by Axis IP cameras through a process of progressive upgrade.
Bob Firth, Stuart Lindsay and Neil Bernabe at a Barco OpSpace Workstation.
Visiting the site, we go into the bright green EastLink building and down the Emerald staircase into the magical heart of the EastLink traffic control room. Stuart Lindsay, engineering/project delivery manager at ConnectEast (EastLink), meets us – he’s matter of fact and operationally on-point. The traffic control room is large and well set up with plenty of adjacent workspaces. In the room are system designer Bob Firth of ACAD and Neil Bernabe of integrator, Convergint Technologies.
According to Lindsay, the new CCTV system went live at the end of February 2019, but the story began years before.
“We needed to replace a legacy Honeywell MAX-1000 analogue switcher, which supported a GE management system and Panasonic analogue cameras, with an IP solution,” Lindsay explains. “But this IP solution needed to incorporate many of the analogue cameras, as well as new IP cameras, and it needed to integrate with the OMCS Scada system, the VicRoads Genetec VMS and much more besides. We started looking at upgrading our system about 18 months before we went out to tender in mid-2018 – we looked at cameras, VMS suppliers, switch manufacturers, the lot.
“Convergint Technologies Australia was selected as the integrator of the head end of the CCTV solution. We had to get the IP backend integrated first and we converted all analogue cameras to IP via encoders so they could be integrated with the new Genetec VMS. Over the next 2 years we have a roll-out of new Axis IP cameras with an installation crew going along the freeway and tunnels changing cameras – about 30-35 per cent of cameras have been upgraded on the road out of 208, with extras being added, including those covering depots and other assets. We have designed the system to support 300 cameras, so we have room to expand.”
It goes without saying that there’s an operational requirement for the system.
“We are required as managers of EastLink to have gapless CCTV coverage of the 39km of open freeway and the two 1.6km road tunnels,” Lindsay explains. “The purpose of the system is to maintain traffic flow, maintain health and safety of staff and our customers, and to monitor the safety of maintenance works.”
System design was undertaken by principal consultant Bob Firth of ACAD Services who says that early on the ACAD team engaged with all the user interest groups and then designed a solution around their requirements.
“EastLink and all the stakeholders had done a lot of thinking – they were one of the most educated clients we’ve ever worked with,” Firth explains. “They originally thought about delivering the project themselves, so had done a lot of research on VMS solutions, cameras – it was different coming into that environment compared to a customer wanting to be told what to do. But despite the challenges of the application, it was an enjoyable process, and everyone has worked very well together.”
The Network Side
The layered nature of the system makes the over-arching scope of works especially hard to nail down. And while Convergint Technologies has undertaken the testing and installation of the head end, there was related work being done on adjacent sub systems and networks by Broadspectrum, Eastlink’s engineering dept, EastLink’s corporate IT department and VicRoads IT department that feeds into the functionality of the EastLink CCTV system.
The network side is Allied Telesis with redundant cores in the main traffic control building and the administration building. The cores are split between these 2 locations, but they behave as a single virtual stack. What this means is that if this the main building goes down the other core seamlessly takes over. We look at the network schematic showing the Allied Telesis network monitoring management software. There aren’t too many solutions around that sprawl across 40km but when you take the federated network into account, the system gets much larger again.
Structurally, there’s a dedicated single mode fibre network supporting field CCTV cameras in a dual redundant ring out in the field, while the internal network is built using a VMWare virtual network with multiple redundancies across multiple servers.
According to Lindsay, moving from an analogue solution to an IT solution was a big shift. It was not only about the VMS – the SCADA system with which the operators were very familiar needed to be functionally integrated. When I suggest to the room this overall process is easier to say than do, there’s an intense burst of laughter.
“As a business we have always worked on the lines of IT and OT – IT is in one building and outside technology is in another building,” Lindsay explains. “IT tends to deal with the commercial LANs – the business operations. Everything on the OT side runs the road – that has included this control room, the PLC networks to the tunnel, the comms to the tunnel, the CCTV system – the 2 have never been entwined.
“Now OT has come into the 21st century with networked virtual environments and there are challenges. These are systems vital to the operation of the freeway. With some commercial applications you could outsource IT functionalities, but we cannot do that – this is a critical life safety solution. Our systems cannot fail or be at risk of failure, and this means we need a highly redundant solution located and managed inhouse in close liaison with our IT department. Adding complexity, we also need access to the internet, and we get that connectivity behind layers of firewalls.
“At all times redundancy and support is vital,” explains Lindsay. “For example, the maintenance team in another building will soon have a monitor which allows them to be alerted to any issues with the system in real time – network, connectivity, switch usage. This is also displayed in the ops centre – we never had that capability before, which made maintenance harder. Another big improvement is that the path from here to the roadside used to have many steps, with multiple instances of encoding and decoding causing signal degradation and latency. The new design has improved that – we are getting far better performance from the new system.”
From his position behind the Barco OpSpace workstation, Neil Bernabe, IT project manager, Convergint Technologies Australia, points out the salient features of the system.
“You can see the core represented here, you can see the open road area of the freeway here – this represents the tunnel, and these are the buildings,” he explains, pointing out each key piece of infrastructure.
“There’s redundancy for the tunnels within the network – if a master switch goes down the node in the cabinet at the other end will take over. As well as CCTV, there are radio repeaters, the OMCS infrastructure and multiple Barco OpSpace workstations incorporated into the overall network – this increases situational awareness by allowing sub system events to be displayed on the video wall.
“On the freeway there’s an EPSR ring (Allied Telesis) which daisy chains from switch to switch via 10Gb fibre through the whole ring. Here in the main building, each switch stack (some in ops room, some in the server room which support virtual workstations) has a redundant path to the 2 buildings on site here – this is what facilitates continuity, allowing every other switch a path to the core even if the head end goes down,” Bernabe explains.
“It’s the same in the tunnel with the ME02 and ME01 going back one to each side – it means that if you lose a path you retain connectivity all the way down the tunnel. A lot of the devices have redundancy built into them as well – things like dual power supplies with solenoids which are monitored here.
“Moving into the virtual side of the network, this is virtual centre (V Sphere) with 4 hosts, 2 in each building designed so that if you lose a host the virtual machines will be migrated across to the other location with no break in service. There are redundancies everywhere – the VM are duplicated in the box so can shift between storage locations. The system has active directory, solarwinds, housing the Barco OpSpace system, the V Centre environment and Veeam backup.
“We installed all the head end gear for the network but there was a lot of input from the IT department (as well as from VicRoads), as to how they wanted their system to be set up, as they would be administrating it,” Bernabe explains. “It was a challenging and time-consuming process getting the network right.”
According to Firth, building a network that functioned perfectly was the key to ensuring the overall solution worked properly and met the expectations of the stakeholders.
“This network is complex – we have our own network here then there’s the corporate network, then the VicRoads network,” Firth explains. “There’s a connection between the VicRoads network and the corporate network and we are inside their firewall and inside our own firewall. If we talk about who had to be involved in the network config it was the Convergint headend integration team, EastLink corporate IT, Corporate IT’s outsourced IT provider, Eastlink engineering, VicRoads IT, VicRoads IT’s outsourced IT provider, and the VicRoads CCTV contractor. So, 6 interest groups just to manage network configuration.
“There were many phone calls and a lot of troubleshooting was required because we needed to negotiate multiple firewalls. It was hard work just coordinating it, let alone trying to work out where blockages might be. There were different systems with different processes – it all took time. We even got Genetec engineering involved to establish whether both systems were configured properly in order to get persistent issues sorted. Finally, this project was put through a cyber security review which was being undertaken concurrently across the entire organisation – that was just another moving target. The reviewers recommended changes to our system, which we incorporated.”
Bernabe says communication was key to resolving complexities.
“Resolving a lot of issues came down to one big meeting with the system explained to everyone in detail – we ran through what everyone needed to be doing and then established what everyone was actually doing,” he explains. “A single phone call ended up solving the federation issues we had when we were trying to get the network layer and the application layers working.”
Something else that fed into the nature of the head end was the choice of video analytics solution. The old system had a bank of analysers crunching data to allow automated alerts to be brought to the attention of operators. But EastLink decided it would take analytic technology to the edge with the upgrade.
“The analytics solution used is Citilog, which is a suite of functionalities inside Axis cameras,” Lindsay explains. “This means the analytics processing is done at the edge, not in metal boxes in racks in the network room. It allowed us to rationalise 8 servers when we installed the Axis P1367E bullet cameras and the new system is running and works well – in fact, we are putting in another 19 of these Axis cameras on the weekend.
“Something else is that each one of the old video analysers supported 16 cameras and if an analyser went down you lost all those cameras. With Citilog running on the Axis cameras if you lose 1 camera you lose 1 analytic, not 16 as previously. We went through a long process when trying to decide which cameras we would use – the amount of work that went into that decision was phenomenal. Ultimately, there’s a significant financial saving – we’ve removed the operational expense of replacing those 8 dedicated servers every 5 years.”
There must be multiple sub systems that needed to be integrated into the management system during the installation process, I ask. How does that work?
“Everything here runs on UGL’s Operational Maintenance and Control System (OMCS) – it’s proprietary infrastructure used in many tunnels across Australia,” Lindsay explains. “That’s what you see there on the video wall – it’s the variable speed signs, variable message signs – that’s the heart and the brains of the whole network. The old analogue CCTV was part of OMCS.
“When we brought in Genetec, which is a third-party software solution, we had to integrate it into that existing core and brain of OMCS. One example of the OMCS CCTV integration would be if someone picked up a roadside help phone – operationally, OMCS would tell the operator a help phone been picked up, would call the help phone contact number and would control the matrix to swing cameras around to provide video of the help phone. With this upgrade we were moving away from the analogue matrix but we wanted to retain functionality and continue to integrate with the existing OMCS.
“The appeal of Genetec was that it is also used in other road systems – the M7 in Sydney – so the drivers to make it work with OMCS were written and an SDK existed. We wanted Genetec because it had additional functionality and is off the shelf, which means it’s more cost-effective than a proprietary solution of the type we had before.
“Contributing to our decision, VicRoads uses Genetec and we must supply VicRoads with access to certain cameras. And not only does VicRoads use Genetec, Victoria Police uses Genetec and the 2 freeways we interface with also use Genetec. In order to integrate with all these interest groups, it made sense to go with Genetec ourselves. It was so much easier and cost-effective to federate our own Genetec system with the Genetec systems of VicRoads and the others. It also means we can see some of their cameras when required.”
The EastLink Installation
Given the complexity of this solution, installation was a real process. Convergint Technologies handled the head end, while Broadspectrum took care of installing cameras and network components in the field. Concurrently with the CCTV upgrade, Barco delivered the OpSpace solution, while UGL updated OMCS, VicRoads configured its systems and EastLink corporate IT worked on the EastLink network.
“Each of these contractors had its own role during the installation,” Firth explains. “Broadspectrum did the cabling on the freeway – going up in a bucket truck during quiet periods at night, pulling out the coax, installing the CAT-6 to put in the pre-configured cameras given to them by Convergint Technologies.
“The EastLink maintenance team then came in next day to finalise the handover from old camera to new camera. They put the camera on the map, added it into the viewing task and configured visual tracking.”
According to Bernabe, Convergint Technologies started work on the project in the first half of 2018 and planning was a key element of the installation.
“The initial thought was that we would duplicate the system in parallel, connecting the analogue cameras to the encoders and allowing the operators to continue working via the analogue solution,” he explains. “As we qualified the system we would cut over to the IP-based solution.
“There were 3 phases to the installation for Convergint – the head end, roadside network and the cameras. The head end was everything related to the network including Genetec, the second stage was having everything out on the road in the cabinets. Stage 3 was PTZs being installed on road itself. Another aspect of the process was decommissioning legacy components of the analogue system.”
Lindsay believes the most important part of the installation took place before the physical installation began.
“Prior to production of the system we did full-on factory acceptance testing at Convergint Technologies in Port Melbourne and during that process we identified some issues with hardware that needed addressing before we could proceed,” he explains. “There were issues with servers that needed to be resolved with the manufacturer and once that difficult process was completed, we got the core installed and then the network infrastructure in place. We needed all that finished before we could go ahead with the rest of the application.”
According to Firth, complicating the installation was the fact the importance of surveillance to operations meant there could be no break in service – the cameras had to keep working no matter what.
“A key element of the process was running the old and new systems in parallel with a T-piece through all the analogue feeds to the matrix,” Firth explains. “The integrators added encoders to give us a second digital stream and the whole Genetec system was built to a fully operational level alongside the analogue system.”
Firth says every part of the installation process was painstaking.
“We went 3 rounds with the tunnel cameras – we needed to look at the lenses required and the brackets – we wanted to make it as easy as possible to retrofit the new IP cameras where the old analogue cameras had been in their Strongabuilt housings, which open from the bottom. While we could not find cameras that worked with the existing housings, we also couldn’t drop cameras further into the tunnel because trucks needed the airspace. We ended up designing a custom bracket.”
Also driving camera choice on the freeway was the fact the new cameras needed to be installed on the same poles as the old, with the installation process being as similar as possible, in order to streamline the upgrade.
“When you look at the schematic that network is 39km long – each of the nodes you can see is a comms node cabinet (CNC) and the CNC feeds multimode fibre out to a local camera location,” Firth explains. “There’s also power fed out for pole-mounted cabinets. What happens with replacement is that the maintenance team removes the camera, then puts a media converter in place. They use the existing fibres that went to the local fibre modem (which is removed) and the camera becomes PoE between converter and camera for power and video data.”
According to Bernabe, mirroring of the functionality of the old SCADA-based solution ran deep.
“The existing media converter had an input for the tamper switch on the cabinet which fed through to the OMCS system – when the cabinet was opened it would trigger the OMCS to swing a camera to a position,” he explains. “This posed a challenge because this equipment was being de-commissioned and replaced with an IP camera without a tamper.
“Instead the techs used the camera input/output as a tamper, giving us the ability to drive the camera to a preset position whenever the cabinet doors opened, and to display this video stream in the control room. There have been a few things like this – things the new system has taken from the old system in terms of field functionality, including re-using existing multi-mode fibre going directly into the switch so there are less parts.”
Lindsay says retaining existing components extended to the network cabinets by the roadside, though this posed challenges during the process.
“Something we needed to consider was that all the switches in the roadside cabinets needed to fit into existing shallow cabinets that reach 60C during summer month,” he explains. “Only 2 of the numerous switch brands we looked at would fit into these cabinets and this meant Allied Telesis virtually chose itself as the field switch because the other options either would not fit our space or could not run the core at 40Gbps.”
Challenges of the EastLink Application
Something that stands out when visiting the site is the complex nature of every aspect of the system and the overwhelming intensity of the application itself. There’s very little that’s not challenging about a life safety application like this one. Not surprisingly, for Convergint Technologies, the greatest pressure was felt once the new system was commissioned.
“For us the pressure really came on once we hit the live button – when the system was live even the smallest tweaks became deadly serious,” Bernabe explains. “There’s a time window between 10am and 2pm for doing major works and if there’s an incident during that period all work stops. Then there’s different management teams and onsite maintenance teams that need to be involved in any works when the system is live – coordinating all this isn’t easy.
“During the earlier part of the integration we found that as soon as we cutover into the production environment just flicking a switch on the network side required a lot of buy-in from multiple internal and external interest groups. Then as soon as we had our system working, we added the Barco OpSpace and the federated network – then we really started finding wider issues that needed resolving.”
The vital nature of the system made troubleshooting more difficult, too.
“When integrating a highly secure networked solution you need support from the supplier, but the gear is now installed on site in a secure environment with restricted connection to the outside world,” Bernabe explains. “It meant we needed people to come on site, to diagnose issues, get results, come back to site with the results and only then could we have R&D test and analyse the system and the findings to resolve issues.”
For Firth, salient challenges revolved around ensuring the system could perform to its specification during early testing of system components. This process was much more difficult and time consuming than you’d imagine it could be.
“Challenges during the setup phase included a number of different manufacturer issues – faults with switches, hardware configurations,” he explains. “We would uncover an issue during performance or failover testing – things worked during normal operation but if you switched a device off there was a problem. Each issue took weeks to resolve and involved manufacturers updating hardware and firmware, and Convergint laboriously re-testing.
“We extended the testing phase by months in order to work through and resolve all the issues and re-test to EastLink’s satisfaction. We would be ready to cutover and would run through all the pre-cutover testing and we’d find an issue with NICs refusing to come live after a failover – in one case there was a different current draw from specification. We would try to resolve the issues but would have to fail the testing and go back to supplier support, working all the way through the issue with new hardware.
“Next, we would re-do the testing and find a firmware update that impacted on the custom software engineering. We’d go back to R&D, they’d fix this issue, then we’d go through it all again. That process was torturous. A lot of sites you would not go to this level of testing but because of the life safety requirements of EastLink we failed every component that could possibly fail – one power supply, both power supplies, the fibre, network components, the lot – we wanted to know whether we got an alert on failure, we wanted to know if the automated processes designed into the system ensured that it continued running – EastLink needed absolute certainty of performance.
“When the project began, we said it was an engineering project not a typical commercial project cutover – there were layers of commissioning checks,” Firth explains. “We did 3 solid days of testing, failing over, shutting things down making sure we got the alerts before we allowed the system to be produced, let alone installed and integrated. It was heavy duty from start to finish. Other complications included commissioning the Citilog analytic and the federation to VicRoads – all these aspects had to be worked through.”
EastLink was highly engaged with the process of resolving issues.
“We were heavily involved in sorting issues out and escalating issues as high as we needed to in order to resolve them – process control was a big part of this upgrade,” Lindsay explains. “It was through process control we found issues in hardware and software and it’s good we did – you can’t fault-find a system after it’s gone into production.
“We needed to ensure suppliers knew how serious our application was – things like power supplies needed to be utterly dependable. The level planning and process control were taken was frustrating for suppliers who may not have dealt with a client like us before but when you have serious life safety responsibilities there is simply no room for complacency.”
Driving the New System
EastLink’s operations centre is more complex than a typical commercial CCTV control room. The video wall is extensive, and the Barco OpSpace gives operators highly sensitive situational awareness across multiple integrated systems located on multiple networks.
“The traffic control room is on that side and we are standing in the incident room,” Lindsay explains, pointing. “In the event of an incident – an accident, a fire in a tunnel – fire brigade and police will be in this room with access to their radio communication systems. Our duty manager will interface with them and the teams will work seamlessly to ensure the scene is made safe and we return traffic flow to the EastLink freeway as efficiently and safely as possible.”
While there are similarities in function and appearance, a traffic control room is unlike a typical CCTV control room.
“A key differential in this application is that it’s primarily a live operation – for EastLink operators it’s about managing flow in real time, while for most sites there’s little real time monitoring and recordings are investigated post-event,” explains Firth.
“This need for live monitoring impacted on system design – we’ve put more effort into accessing cameras than into recordings, although those functions are there. This whole workspace here – it’s a Barco OpSpace – is bringing together workstations across multiple networks. Barco OpSpace means operators can pull up any of their authorised workstations as a window within an OpSpace workstation – it’s very powerful.
“Looking at this workstation OpSpace you can see the Genetec workstation, the OMCS SCADA workstation and here are the video wall feeds,” Firth explains. “On the other side of the same monitor you see servers in case we have maintenance tasks to do. The strength of this is that operators can bring up anything they need to view or manage, no matter where it is in any network. And because it’s linked DVI to monitor, access is possible even if a network is air-gapped – we are directly encoding remote streams into the Barco Ops Space.”
According to Firth, this allows a huge amount of flexibility in terms of what’s possible.
“One operator might have cameras on that side, another has them on this side – to set the OpSpace up operators just open the source they want and lay the space out as they want – it looks like one workstation but it’s multiple workstations cross-network,” he explains.
“Creating a control room with this functionality is not easy to do – it was a tough process and it really should have been a separate project on its own, not something we were trying to manage at the same time as everything else. Just to highlight the complexity, there are 8-9 virtual servers supporting Barco OpSpace. Operationally, there are 4 workstations running the system via Barco OpSpace, each with 4 displays. There are another 3-4 dedicated Genetec workstations, as well as the workstations that run the video walls.”
A key part of the traffic control room’s real time monitoring functionality is built around analytics inside tunnels.
“We use the Citilog automatic video incident recognition in conjunction with new Axis cameras to alert staff in the operations centre to incidents in real time,” Lindsay explains. “The rules-based algorithms operate end-to-end in both directions of both tunnels and approaches. They will detect a stopped car, debris on the road, smoke in the tunnels, wrong way traffic, pedestrians on the road and slow vehicles. You get to know about vehicles behaving outside what’s normal instantly – staff are alerted, and video displays will automatically switch to incident locations.
“The Citilog cameras are fixed Axis cameras which have masks and detection zones programmed into them. How the system works is that when a camera picks up an incident in a zone upstream, downstream PTZs will turn to look at that zone and feed it to specific screens so operators know where to look. Once operators have assessed an incident, they can notify incident response teams, fire, law enforcement.”
That’s legacy analogue on the right showing some hum bars. The comparison with the new Axis cameras is startling.
Looking at the video wall in the traffic control centre we can see 1080p Axis cameras and 4CIF analogue cameras side by side. The analogue cameras take a circuitous route to the ops centre so the quality of the image streams is weakened. Watching the monitors, I can see there’s a struggle to identify license plates or faces from the analogue cameras. But the Axis HD cameras are having no trouble getting plates and high levels of detail, with everything rendered in that very specific Axis-looking video stream – tending to cool temps, with relatively low contrast, subtle colour saturations yet strong definition.
To demonstrate the way the system works, Bernabe gives us a run-through.
“If you’re an operator and you need to access the system there are a number of ways to manage the process,” he explains. “First, there’s a full map of the freeway system including roadway and tunnels, and you can zoom into an icon and click on a camera to access a view. Operators can also track an incident sequentially by clicking cameras using keyboard shortcuts and they can also access cameras related to specific incidents that take place on the freeway or tunnels.
“If an incident occurs, they look to the incident camera view on the video wall– this mirrors the way the analogue system worked. Importantly, there are shortcuts hardcoded into the system and the operators, most of whom have worked at EastLink for many years, know the camera numbers by heart. To take advantage of this familiarity, we mirrored shortcuts when building the new system.
“The operators know exactly which camera is viewing what and they will just hit the shortcut to pull it up, making managing the system much faster for them,” Bernabe says. “It’s just one of the ways in which the operational aspects of the old system have been translated into the new IP system, which in a real sense mirrors the analogue matrix.
“Operators can also get access to cameras views using directory trees for tunnel inbound, tunnel outbound, all fixed cameras and all PTZ cameras. A really neat feature of the new system is a functionality called area of incident – operators can rewind multiple cameras associated with an area of incident to get a complete snapshot of an event from start to finish.”
According to Lindsay, the old system could only record what the operator was looking at on the video wall.
“What was being viewed on the video wall was being recorded by 16-channels DVRs,” he says. “But now, regardless of what’s being displayed, everything is being recorded all the time. Further, when the OMCS triggers a camera to move to a preset position, that is recorded as well – in the past operators needed to be viewing an event or it would be missed.
“Operators love the fact they can get to recordings quickly and easily – before they had to go through a protracted process – they were working live off the matrix using a keyboard and to view recorded footage they had to log into DVR software and go and view it. This gives them everything all the time as drag and drop.”
Lindsay says this part of the system was designed in partnership with the operators.
“Operators were integral to the design of the interface,” he explains. “We needed stakeholder engagement with them and on this project, we had the operators on board from shop floor level. They were also part of the factory acceptance testing process and as the system progressed, they were feeding suggestions back in. They certainly got a lot of what they wanted.”
Firth explains that the system was designed to mirror the analogue matrix in terms of some functionalities.
“That’s why the operators have a keypad and a joystick with the new solution,” he explains. “Some operators are using more mouse when driving the system but in the first instance we needed to have a completely seamless transition in that early cutover period. If there was a life-threatening incident in the tunnel and the operators’ first instinct was to type a camera number, we wanted to accommodate that instinct.”
The UGL OMCS we see on the workstation seems to be the core of the system in many ways – is that how it is? I ask.
“OMCS is like a SCADA system for the freeway,” says Firth. “When it comes to CCTV, all the cameras have the ability to trigger a preset to one of these monitors so alarms might trigger an event to the video wall – an over-height vehicle approaching the tunnel for instance – 3 cameras will pop up and show the event. With the analogue system, this process was serial but now it’s a driver in the back end of Genetec. That functionality is essentially identical, so it remains familiar to the operators.
“Getting this right was a process with a lot of testing and a custom driver. UGL wrote the driver to Genetec’s SDK but Genetec was involved in the process – they reviewed the design before procurement. Importantly, this meant we had a sign-off from them that this was the right design and later we had a sign-off from them on the actual implementation before we cut over. Zane from Genetec flew down from Brisbane and did a full health check.”
Is there anything unique about the management system functionality?
“There’s custom integration between the OMCS and the Genetec SDK and the rest is mapping which was customised from the functionalities within Genetec Security Center,” Firth explains. “There are hot actions, moving multiple cameras to a view an event. Probably the biggest deal is the level of configuration that has gone into it. It’s not just install Genetec Security Center, add some cameras and walk away. It’s the naming, the links, the vehicle tracking – that’s what sets it apart.”
Bernabe agrees.
“If you were to install Genetec Security Center in a greenfield application you would never use it like this – there’s been so much customisation to meet the demands of the application and the demands of the operators – it’s a special integration and certainly the most complex I’ve been involved with,” he says.
Conclusion
When we walk the site our first port of call is the network room complete with 3 racks of gear dedicated to the system. There are the Genetec Streamvault boxes, the servers, the fibre switch and plenty of related gear. We look at the core, the firewalls, switches for the workstations, the encoders for the Barco system. It’s all very tidy and nicely done, as you’d expect.
Visiting a cabinet beside the freeway highlights further installation challenges of the system – the huge distances, the relentless traffic flow and the combination of analogue and IP componentry. There’s a lot of infrastructure supporting a road system and I come away from EastLink with a strong sense that road systems are a serious business – ensuring life safety and monitoring traffic flows is vital and nothing can be left to chance.
Something else that’s telling is just how difficult it is for the narrative to flow without spilling into the challenges of process control, the challenges of the layered network, the complexity of wrangling interest group requirements. No matter where a topic starts, it ends up in that place where all challenges meet. Listening to the chat in an office beside the traffic control room it’s impossible not to get a strong sense of what an epic undertaking this EastLink upgrade turned out to be.
According to Firth, the EastLink solution is one of the most complete end-to-end upgrades in Australia.
“The attention to detail in the engineering, setup and testing of the system has been intensive due to the high expectations of this customer,” Firth explains. “As a result, it’s a nicely polished solution. EastLink is a complex environment with lots of redundancy so getting it right was a process for everyone involved, the hurdles we had to jump through to meet the requirements of the customer – engineering, operations, training, configuration, managing the buy-in of interest groups – it was a real process.”
Meanwhile, Lindsay says the challenges of the application have been worth it and the upgraded system meets EastLink’s requirements.
“Upgrading the CCTV solution has been a process to put it mildly,” he says. “We needed to understand what we wanted, how to achieve it and how to deploy it and we had to address major challenges along the way. If we had a greenfield site, things would have been easier but being brownfield made the upgrade more complicated and being a road/tunnel system with a life safety focus added pressure to many aspects of the work. But we needed to look at the bigger picture of EastLink’s converged future and I believe we have achieved that with our new solution.”
#sen.news
