Note: This blog is a transcription of the Q&A session from our recent webinar with STL Partners, “5G as a Growth Engine: How do Telco IT Systems Need to Evolve?” You can access the full webinar recording HERE.
- Andrew Keene, Senior Director of Product Management, VoltDB
- Dalia Adib, Director and Edge Practice Lead, STL Partners
- Reah Jamnadass, Senior Consultant, STL Partners
- Patrick Montague-Jones, Senior Consultant, STL Partners
What is the biggest challenge when it comes to evolving telco IT systems? (poll: organisational (52%), technological (10%), business (38%)
Reah Jamnadass, STL Partners: Even with 4G we’re hearing that a lot of operators are bringing in some level of cross-functionality, particularly between the network and IT teams. It is even more necessary to have them work as a unified team with 5G. One of the challenges that remain is upskilling and ensuring that telcos have the right in-house skills to support some of these changes. Furthermore, telcos should know how to outsource when needed. One internal challenge is justifying the broad 5G systems business case to stakeholders not deeply connected with the technology, like the finance organisation. They need to understand the monetization opportunities before they are willing to invest.
How can operators start to overcome these challenges, particularly the organisational ones?
Andrew Keene, VoltDB: The poll results are very interesting because, from my experience, I worked in an operator many years ago and one of the biggest challenges absolutely was organisation. With siloed networks and fragmented IT parts of the business, it made the big transformational change very hard, and ultimately a lot of programs failed. These were mostly organisational failures rather than technical failures. It is a key area that CIOs are going to have to address as they seek to bring network and IT teams closer together. It is critical that they do this by either combining the organisations and mixing the teams up or somehow increasing collaboration so that the teams work better together. I’ve seen examples where network and IT teams have combined and been mixed up, so they are forced to join. If you don’t solve this problem, you can end up in a lot of trouble.
Patrick Montague-Jones, STL Partners: I would echo Andrew’s point that one thing telcos need to be aware of is that organisational change is not going to be an overnight change. So, from a structural perspective, you may bring the teams together but in building effective relationships and ways of working there will be growing pains. Therefore, it is important to enact change now so that telcos are prepared to adequately serve 5G at scale in the future.
Dalia Adib, STL Partners: I would add that it is one thing to merge teams together, and another thing to equip teams with the required underlying skills and knowledge. Operators, as they are moving to a more software-based network, need those who are running and operating the network to have software-based skills. To some extent, this is happening organically and there are big training programs and recruitment drives going on, but this still remains a challenge. We’ve heard from some operators that there are practical challenges in recruiting people with software skills to a telecoms operator as people with those skills don’t think of telecoms companies as software-based as much as a cloud provider or an internet provider or Facebook, for example.
What is changing with B2B business models from an IT system perspective?
Dalia Adib, STL Partners: The reason why there is such a focus on B2B is because that is where the industry sees growth in contrast to the B2C space, which is to some extent plateauing. There are some innovative operators who have trained their markets to buy data based on speeds and therefore may also be able to charge by latency in the future as well. Therefore, you have all-you-can-eat bundles but pay a different price depending on what speed you receive. This will have implications on billing especially and how you can guarantee that level of speed and quality of service. So that is one area where there is maybe an impact in B2C but generally, the largest changes are happening in B2B. Billing and charging will be impacted in two different ways. One thing we’ve heard is that there is a desire from telecoms operators to become more general ICT providers, providing services that go higher up the value chain. This will depend on stitching together services from partners as the telecoms operator can’t do everything themselves. So, you might get cloud infrastructure from a cloud provider, applications from independent software vendors, etc. That will make a much more complicated solution to take to market and monitor if you provide a managed service so that has implications for your IT and service insurance. There is also the billing aspect that is more complicated as it’s not as simple as just a connectivity product. Another area we talked about is a move towards network slicing or network as a service (NaaS) and that’s an area where there is a big impact on charging. This is where you want a real-time view of the service which you can then charge by as soon as an event happens. This will be particularly challenging when we move onwards to dynamic network slicing where telecoms operators may have to provision and set up a specific slice for a specific workload or application in real time. They will then have to monitor that slice and charge for that immediately. That’s where there will be a big change in how charging works as there is a move to this new B2B business model.
Would you be able to explain what ‘charging based on outcome’ means and from a slicing perspective how is that going to impact charging?
Patrick Montague-Jones, STL Partners: In terms of outcome-based charging, it really depends on customer priorities and some of these metrics are yet to be thought of or even implemented. But if you take network slicing, for example, you could charge based on quality of service, guaranteed limits of packet loss, or on the number of successful API calls. The sky is the limit in terms of outcomes you could base charging on and, because of this world of possibilities, there is a lack of clarity for operators for what they should charge on. Operators should align charging models with what their customers want and need.
Andrew Keene, VoltDB: It’s a good question because in Europe even post-paid is often managed in real time so that there is no bill shock, for example. But there are parts of the world where there is no real-time post-paid charging. In a 5G world, this will not work so well as you will be able to charge by different metrics, and charging models will be more dynamic, if you’re charging by the quality of service you would have no idea how much you will get charged until the end of the month. Therefore, especially in the enterprise space, being able to be transparent about billing in real time throughout the month is important. In addition, there are the technical elements and the standards expectations that the charging platform must respond to the network in real time anyway. So, if you need to be able to manage even offline events to the 5G core in real time it makes sense that you are transparent with your customers and show them what is happening throughout the month in real time as well. This gives a much better customer experience and the opportunity to give better offerings, you might be able to trial a pre-paid month with a post-paid customer, for example, to tease the service. You can offer all these things if you are measuring in real time, it gives a much bigger scope of offerings for operators.
To what extent are telcos using a multi-vendor approach in evolving their IT systems?
Dalia Adib, STL Partners: There is a lot of desire and ambition to open up the vendor ecosystem and there have been a lot of changes in the telco ecosystem in the last two years with new players joining and some of the efforts around open RAN and 5G core accelerating this openness. Also, now we’re seeing the cloudification of OSS and BSS, which is also changing the vendor landscape. The reality is that these things take time to be seen in practice. For a few of the operators we spoke to in our interview program, they want to have a multi-vendor approach in the future. However, in the short term, for cultural reasons or the risk factor of moving from a single vendor to the multi-vendor solution, this is sometimes slow. As a telecoms operator, if your new architecture comes from a single vendor it will likely be modular so there is some level of future-proofing and the ability to open up and use other vendors. But in the short term, there will be a fair few in the industry who stick with a more traditional partner approach.
Andrew Keene, VoltDB: You’re dead right. The operators would love the nirvana of not being held to ransom by one vendor and being able to pick and choose and certainly you see it in their infrastructure decisions that they are mostly going with multi-cloud strategies and not just picking one and then being stuck with them. But, as you say, for many of the core functions, in reality, they are picking vendors that they are comfortable with to deliver these solutions. Maybe on the periphery, you’ll see them doing stuff in the IT space with new and different vendors or even building stuff themselves. Some of the bigger and group operators are increasingly showing an appetite for building parts themselves. But, for a lot of the core systems, certainly, for the next while, it will be the known vendors in the mix and providing the functionalities.
Dalia Adib, STL Partners: I agree but I think that an interesting thing that is going on is with private and dedicated networks, they’re almost like a testbed for these multi-vendor scenarios. They’re quite a nice way of looking at alternate methods of building a network and the surrounding support systems and testing that out with enterprise customers and, as that starts to work, or if there are lessons learned, then implementing that in the public network.
What would you say would be the next steps for the CIO team in a telco organisation, how can they prepare for this evolution?
Patrick Montague-Jones, STL Partners: Start thinking about driving convergence between your IT teams and traditional network teams and help the adoption of those DevOps processes that your IT team is perhaps more comfortable with, drive that into the network teams. Think about the current skill sets you have in the business and then think about organisation of these skills in the business. Finally, get in a room with the product teams to help understand customer needs to make sure you are building a horizontal, flexible architecture that meets the needs of customers.
Andrew Keene, VoltDB: The operators need to really grasp straight away that the 4G to 5G change is a lot different than the 3G to 4G upgrade and the standards and skill sets needed are drastically changing. Instead of telco protocols, standards are now being adopted that have been in the IT space for a long time so there will be a natural merging of IT and network teams due to skill sets that will be demanded. But this will take planning. Firstly, you need to acknowledge that this is happening and then also realize that it is not an incremental change to 5G but that if you really want to benefit from 5G, move up the value chain, and see a significant ROI, then you really must do it right. As Patrick said, the organisation of your teams and the flexibility of your infrastructure are both important.
Dalia Adib, STL Partners: I agree and would emphasize the importance of beginning to think about the commercial side of the implications of the 5G core. A lot of innovation is happening in the network teams and that is great, there is always a need for that. However, at the same time, if you are to take offerings to market to enterprise customers you must have clarity on what you are providing. Therefore, you really need this collaboration between technical and commercial teams, it’s not just IT and network convergence. There needs to be two-way innovation between technical and commercial teams so they can understand what can be done with these new functionalities and move away from just a testbed or trials and get closer to real B2B services that they can take to market.
What level of E2E latency is required for the control and user planes to successfully run 5G systems? Is the range in the hundreds or tens of milliseconds, or even single-digit milliseconds?
Andrew Keene, VoltDB: The user plane (also known as the data plane) is the network user traffic — the actual data traffic, while the control plane carries all the signaling traffic required to authenticate, establish, and manage the data plane. One key aim of 5G is to bring down user plane latency from the hundreds of milliseconds experience of 4G down to single-digit ms. 3GPP standards state figures as low as <0.5ms for uplink and downlink user plane latency without high-reliability requirements and 1ms where higher reliability is needed. With that in mind, the control plane has to authenticate and continuously manage ongoing policy and charging decisions such that there are no interruptions to the data plane traffic. So, end-to-end control plane latency should be in the tens of milliseconds, and for this to be possible, individual functions, such as charging, need to respond ideally in single-digit milliseconds, low 10s at most.
Traditionally IT has been underfunded for years, is this changing, and who are the leading telcos investing in IT? Has under-investment left some operators doomed?
Dalia Adib, STL Partners: Most telcos we interviewed acknowledged the importance of investing in IT in parallel to infrastructure. To some degree, this is already happening as part of general digital transformation programs, however, the opportunities 5G presents does put additional pressure to ensure the backend systems in place are flexible and agile to meet customers’ needs. The opening of the market, for example from private 5G networks, means that telcos will compete with new players(e.g. cloud providers, systems integrators, managed service providers, etc.), which will increase the pressure to invest in IT and meet best-in-class standards.
How does 5G enable traffic control? Does this include Wi-Fi or alternative wired solutions?
Patrick Montague-Jones, STL Partners: 5G can be considered as complementary to existing connectivity solutions, not necessarily as only a replacement. There are use cases where 5G might work better than some other types of connectivity, for example connecting large campus locations with hundreds, or thousands, of sensors and devices. Traffic control for WiFi, wired solutions (e.g. MPLS), and 4G today is enabled by SD-WAN, where individual QoS requirements are applied to data shared on one or more access lines. 5G brings network slicing to the table and takes this to the next level—SD-WAN and network slicing can be combined, too. Network slicing creates multiple virtual access connections, facilitating total data separation from point of origin and further enhancing business and network performance outcomes.
If operators need to improve IT, why don’t they just use a hyperscaler cloud provider?
Dalia Adib, STL Partners: Telecoms operators are certainly considering hyperscale cloud providers for their IT, for example, AT&T’s announcement to move all its IT to Microsoft Azure. There are some challenges, however. The telecoms industry is heavily regulated which makes it difficult to move to the public cloud, even for applications that are not mission-critical. Plus, it takes time to rationalize infrastructure and migrate workloads.
For software deployments, which deployment method do you believe is better: virtual machine or bare metal containers?
Andrew Keene, VoltDB: For software deployment, I would always recommend containers running on bare metal, rather than deploying directly onto a virtual machine (VM). Containers are much lighter weight than VMs, and so provide the level of agility cloud native promises that isn’t possible with VMs. For example, containers enable much faster startup (and shut down) times, along with rapid scaling and auto-healing. Containers also allow a greatly optimized use of the underlying hardware. This is because you can more easily fit multiple containers onto worker nodes than you can many VMs. You can, of course, deploy containers onto virtual machines, which is common for cloud-native deployments. You still get the agility benefits of containers, but the drawback that an additional virtualization layer consumes a portion of the compute resources.
Do you think that 5G telco cloud charging can be based on a subscription-based model based on active connections served at any given time?
Andrew Keene, VoltDB: 5G provides many new charging triggers and attributes, many of which are aimed at better monetizing massive-scale IoT, private networks, and enterprise. For example, 3GPP Release 16 introduces new capabilities around connection and mobility management, allowing charging based on a number of connections and the location from where devices connect to the network. This will allow for exactly this kind of subscription-based model with the active connections and potentially their location (depending on the use case) to drive any variable component of the subscription charging model.
Please could you explore how the control plane and data plane can enable 5G in a multi-cloud environment?
Dalia Adib, STL Partners: Distributed data planes and user plane functions are paramount in enabling edge computing environments, to ensure traffic can be routed, aggregated and processed at the edge. In terms of multi-cloud, most telecoms operators are keeping edge cloud environments from different vendors separate, plus the associated network functions, however, we would assume these converge over time.
How can operators achieve a fast ROI on the large CAPEX investments associated with 5G programs?
Patrick Montague-Jones, STL Partners: Payback periods on 5G stacks will not be instant. Minimizing this period will be a combination of a few factors: well-negotiated technology contracts; efficient teams; high value/innovative 5G services; rapid market launch. While high investment will be required to build and launch 5G non-standalone and standalone cores, a great starting point is to ensure operator procurement negotiates strong agreements with suppliers. Chances of this can be maximized by ensuring the right procurement incentives are in place and proving to suppliers the full extent of volume that the operator will drive. Well-trained, dynamic and multidisciplinary teams (IT, networks, product) are also crucial to success in offering clear, compelling, and scalable propositions for customers and increasing operating efficiency, reducing development cycle time. Within these teams, understanding customer needs and what can be offered above and beyond basic connectivity can drive revenue and high margins, which will contribute to payback. Finally, if credible 5G services can be launched first in the operator’s given market, then that first-mover advantage could facilitate a sharp increase in market share and a greater chance of quicker payback timelines on 5G investments. Of course, being first does not guarantee future success, so operators must not be complacent and look to constantly improve and broaden their 5G offering to businesses and consumers.
How can operators achieve strong 5G revenues despite regulatory pressure on pricing?
Patrick Montague-Jones, STL Partners: In markets where regulation exists to monitor connectivity pricing, it is true that increasing revenues on the base connectivity can be difficult, or impossible. Operators should focus on the value-added services that 5G enables, which will most likely not fall in the regulators’ purview. Examples of these services are: monetizing core programmability, industry-specific bundles (connectivity, hardware, and software) to enable 5G use cases, and managed services.
What are some examples of B2B plans that would require real-time billing and/or charging?
Reah Jamnadass, STL Partners: The need for real-time (i.e. sub-second latency) is more important for charging, but there are some low latency needs for billing as well. With billing, there is an increasing desire from enterprises to have full visibility and a “real-time” picture over their usage and costs. This becomes more complex when there are multiple partners providing different parts of the solution—however, the latency requirements here are not as stringent as for charging. 5G-enabled B2B use cases will often bring much higher levels of network traffic (e.g. IoT), and new solutions, such as network slicing, will evolve over the next 2-3 years giving CSPs the opportunity to charge for network usage in new ways. In order to exploit these network monetization opportunities (e.g. charging for QoS delivered) and charge for every event, despite there being more data, charging systems will require millisecond latency processing.