When SAP unveiled Joule in 2023, it promised to serve as a generative AI copilot, using natural language to deliver insights from across an enterprise’s tech stack, including SAP and third-party systems.
In the two years since Joule was announced, 20 agents have launched across SAP S/4HANA Cloud, SAP Concur, and more. At the inaugural SAP Connect conference Oct. 6–8 at the Fontainebleau and Resorts World Las Vegas, SAP announced 15 additional Joule agents for finance, HR, procurement, and supply chain, designed to bring role-aware assistants to these functions and help augment repetitive tasks. Even more Joule agents are in the works.
“As an end user, you shouldn’t need to care about how many Joule skills and agents are out there,” said Ted Way, PhD, VP and Chief Product Officer, Business AI Product Engineering at SAP America. “You should care about whether Joule can solve your problem or not.”
SAP is working to create these agents, while its customers begin adopting the agentic AI tools, gaining traction and trust. Over time, Joule will become a go-to resource for solving business problems, using data from across an enterprise’s technology landscape as a single source of truth.
Way recently sat down with ASUG to talk about how Joule is solving business problems, how to harmonize data, and the continued importance of humans in the current AI boom.
This interview has been edited and condensed.
ASUG: To start, what can you say about AI changing the way organizations are running — not just automating tasks, but rethinking roles and processes around AI? How would you assess recent progress from SAP?
Way: I only joined SAP almost two years ago, and so I’m starting to appreciate more and more SAP’s expertise and understanding in these roles, in the sense that, like every single company, whether you’re a manufacturer, a medical office, or in oil and gas, you need to have an HR function, you need to have a finance function, you have a procure-to-pay process. This is just common across every single company. Nobody understands that better than SAP.
We’re getting into a world with so much complexity. Everybody’s talking about the bajillion agents that they are creating. In that complexity, how do we simplify that back to the user? What do we have available that can help you get that job done and solve that problem?
ASUG: I’m curious if you could discuss your vision of the agentic AI ecosystem. Why is this the direction that SAP is moving in?
Way: As an SAP employee, I would love for you as a customer to only use SAP. And we acknowledge that’s not the reality. We are aware that our customers utilize tools from many, many vendors. And now, the question is, how do we get into a world where you can use the expertise, where it makes sense in the right ecosystem? For example, if all of your ERP is done with SAP, then it makes sense to keep the data in there. Sometimes other vendors will ask you to get that data into their central data lake. And the promise is, then you can reason over that data in one place.
The issue with that is now you’re making five different copies of that data. Your permissions are getting changed up, and you don’t know what to trust anymore. Is the data getting there at the right time? This is where SAP Business Data Cloud and the zero copy announcements came out, where now you have the ability to do things with Delta Share and Delta Tables, to have that data and reason in one place.
It will make sense at times to keep stuff in SAP, because that’s where your data gravity is. That’s where all your processes are, and that’s where it’s managed. So we have tools for you to build out your agents. We have tools to enable you to build out the AI that you need. But we also acknowledge that you also have other centers of gravity. Within the other centers of gravity, you can still build out the agents, and then these agents can talk to each other in a common protocol. And what we want to offer at the end of the day is that our customers come first. How do we help them be successful in managing their landscape and enable that interoperability with the different centers of gravity?
ASUG: It makes a lot of sense when you’re thinking about where data exists within this orchestrated organization. When you are talking with customers currently about where their data exists in their organizations, how close or how far are most customers from being able to operationalize SAP Business Data Cloud?
Way: I think customers are in different phases of that journey. And there are different points of that journey. AI is important, and in order to have good AI, you need to have good data. And in order to have good data, you have to have a modern data estate. Some customers are recognizing that, and have made great strides there. Other customers recognize it, but then definitely have challenges in getting there, and they’re frantically making sure that they can get to that point. What we want to offer them is this platform where they can do that.
ASUG: For those who are dealing with a lot of disjointed, fragmented, siloed data across their organizations at that initial stage, what have you seen to be most helpful?
Way: Aligning on the use cases. That’s super critical, because I think it’s so easy to get caught up in these big initiatives and transformation movements, and you lose sight of what you are doing this for in the first place. I definitely would not recommend trying to boil the ocean, but to understand what you are trying to solve. It will have a tremendous impact on your organization.
I like the two-by-two framing. Think of two low-hanging fruit use cases, where we generally have an idea of what we need to do, and let’s just get that done, and show proof of value. And then two of the huge-impact—but taking longer—use cases. You don’t want to be stuck in each one. If you get stuck in the first one, then you may not be able to solve the really big problems. You’re just dabbling and solving some point solutions. If you only focus on the second one, you might not show anything for six months or a year, and then people lose steam. And so two-by-two is to solve some first, make progress towards the other one, and then that can help inform how you should architecturally redesign your data state.
We’re a big organization of people too, and we’re thinking about who owns what data and what kind of controls they have, and what are some common goals and priorities that can excite them to know that doing this together will actually benefit them? It’s about framing it in that way, and then sharing that vision so that they can see that they all win.
It’s also a people transformation and change management process, and I think we shouldn’t lose sight of that. The technology is there. It can be done. But it’s really the people who should be bought in and coming along to do that.
ASUG: To the point of the people part of the equation, when you think about advocacy for that initiative within a company, and you’re looking at specific roles within a business that can actually take that challenge on, who do you find those people to be, generally?
Way: It’s definitely an exploration process. We’re seeing kind of this flattening of roles. It’s not necessarily about this job title and this role, but it’s more about what needs to be done. Who are the people aligned with the strategy and who understand where this company is going? Who’s in tune with what the executives want, their vision, and where they want to take this company? Once you understand that strategic road map, then determine: Who are the people who have the technical expertise of the architecture? Who are the people who are the influencers within an organization?
The social network within a company is also very important. Who are the influencers and who are the people who can drive that? Having the strategy, the technical layer, and the people who can implement that and move it forward are the ingredients that can help make this more successful.
ASUG: What’s exciting to you about the way AI is being assessed, both within lines of business, but as more of this holistic organizing principle that brings them together in a common goal? What would you say about AI’s role there?
Way: I think you hit the nail on the head in AI being the glue that can hold it all together. With agentic AI, for example, we all have our specific roles. The HR leader is not going to be the one that’s debugging a supply plan. We all have areas of expertise, but we’re all interdependent too. AI can help glue that together in the sense that you have specific data that you need for a specific role, but then that data can be integrated with data from the other roles. And then decisions can be made. And then the experts can collaborate with each other to come up with recommendations.
ASUG: One theme at SAP Connect has been this term of AI-native solutions. Could you share a little bit more about the phrasing of “AI native” and a clarification of what makes something AI-native?
Way: If you think about making a cake, one way to go about it is seeing AI as a sprinkle on the cake. I got my cake, I got my flour, I got my water, I got my sugar, and I got my eggs, and, oh, by the way, I need to sprinkle AI on top of that. That’s the wrong way of looking at it. And we’ve seen it a lot. I have my way of doing this, but I need to slap the AI label on it, because that’s the only way to get any traction with anything nowadays. But it’s still the mentality of this cake is a cake, and the sprinkle is superfluous.
AI native is really starting with AI and thinking about: What are the jobs to be done and what are the problems that need to be solved? And then, what do I know about what AI can do? So, starting with AI, and building around that. That, to me, is where AI native comes in. So, it’s a core ingredient, and not just a sprinkle that could be left out.
ASUG: Thinking about the capabilities of AI, what specific capabilities are you referring to in the context of a solution like SAP Supply Chain Orchestration? What makes the foundation of that? I’m thinking specifically about SAP Knowledge Graphs here, in the context of supply chains.
Way: I’m not an SAP Supply Chain Orchestration expert. But I can take a step back in terms of the supply chain scenario. There’s a Knowledge Graph, and you can think of it like a LinkedIn of suppliers. As a manufacturer, I have my tier-one suppliers, but each of my tier ones have their tier twos and have their tier threes and tier fours. And so the question is, if something impacts a tier-four supplier, how is that going to impact my tier one? Because I don’t have visibility into tier four. I only have visibility into tier one.
The idea here is leveraging the information from that Knowledge Graph and being able to build into it, then to be able to get those insights. AI could be reading the news or identifying what’s going on, or it could be just picking up signals in public domains to know that something’s impacting tier four. And then with the Knowledge Graph, if tier four is impacted, that means tier three is going to be impacted with tier two and tier one, which will ultimately impact me. What is my fallback? How do I predict what to do next? Do I need another supplier? Do I need to increase my manufacturing times? What do I need to do to be able to account for this?
AI native would mean starting from that perspective of: What am I able to understand and what am I able to predict, so that I can prepare? We’re not able to predict the future, but we can prepare for it. How do we best prepare so that we’re in the best position to face whatever happens?
ASUG: You’ve talked about multi-agent orchestration, you’ve talked about chain-of-thought reasoning as being like one of these core aspects of agentic AI. When you think about SAP’s implementation of those features, how mature is it today? And what excites you about the progress of SAP’s continuing evolution of thinking about those aspects of agentic AI?
Way: I’m super excited about the pace of progress that’s been coming in, and just how reasoning is being built into these large language models as a core capability. From our perspective, what we want to do is just leverage the state-of-the-art AI models. We decided early on that we’re not interested in creating our own general large language model. We have our tabular SAP foundation model. We are experts in tabular AI, and no one else is building out a table-native foundation model. So that’s where we’re going to leverage our strengths and build up a native foundation model. But for the other models, we’re going to leverage the state-of-the-art technology out there, and do the same when it comes to agentic reasoning.
What we can do is leverage the ability to make a plan. For example, a customer sends an angry email to your company, and the agent sees that email, and then it needs to make a plan of what to do. It needs to read the email, classify it, and look for the historical information. It needs to look at previous interactions with customers. And so we’re leveraging the reasoning capabilities based on our knowledge of these processes.
So you can definitely prompt a model, and you can ask it to make a plan, but I think where we differentiate is our understanding of these processes, and where we can add to that prompt, where we then prompt the model in a more intelligent way to come up with a better plan, for example. So these are the things that we’re using from a reasoning perspective. We’re also leveraging, as you mentioned, the A2A protocol when agents can talk to other agents, and then also model context protocol (MCP) servers, where agents can leverage a lot of tools.
But even when it comes to MCP, there’s a lot of different problems there. So you think about doing a web search, or you think about going to an app store. And if you want to go to an app store and you want to look for a calculator, there are five bajillion calculators out there. How do you know which one to use? Or if you go to the internet and you do a search, there’s 20,000 websites. How do you know which one to use or trust?
The same goes for SAP APIs. We differentiate in that we’re actually building out our knowledge of all of SAP APIs and our SAP Knowledge Graph to then be able to help Joule be more intelligent than leveraging all the different tools out there.
So we are using state-of-the-art capabilities to help our customers. As a product person, I can’t lose sight of that. We can’t lose sight of what we’re trying to do in the end, which is to solve our customers’ problems and not get too caught up in the technology. Technology is our tool; customers’ problems are what we focus on first.
ASUG: I’m just thinking of our readers here who might not be as kind of enmeshed in AI development. When you’re referring to SAP’s expertise in the tabular AI space, why is it that you don’t see table-based AI emerging from other competitors who are pushing AI, and what makes SAP unique in that position?
Way: I think there are a few reasons. One is, just as humans, we’re naturally more attuned to text and images. Most of the data on the internet is text and images. And so if you needed to build a foundation model and you needed a massive amount of training data, the internet’s out there. That’s why a lot of the work was in text. And it’s really easy for humans to understand the output of a model.
With tabular models, we as humans are not naturally inclined towards tables, and there is a lack of training data. We have an advantage at SAP. We’re not in the business of taking customer data and selling it. (We never do. I just want to make that as clear as possible.) The data that we have permission to use to improve our products, is the data that we are able to leverage in order to build up a foundation model. The idea here is, just like you take the text from the internet into a text model, you can take all these tables and create a foundation model that understands tables natively. (Since this interview took place, SAP launched its table-native foundation model, RPT-1.)
ASUG: In returning to that idea of the customer centricity of this movement with the role-based AI, do you see adoption of AI copilots proceeding similarly to adoption of other SAP solutions or being more gradual?
Way: We definitely want to make the learning curve as easy as possible. And I think the nice thing about large language models and these assistants is that you can interface with them naturally as a human. Coming from a computer science background, I spent nights in the computer lab, because I would be coding, and I would forget a semicolon at the end of a line. It would not compile, and it was my fault as a human. We are in a much better world today, where I can express myself naturally as a human in natural language, rather than in the language of a computer. I think that’s where the adoption, I hope, is faster now, because now you can interface with Joule and other copilots naturally as a human, and that’s the ease-of-use part.
But then the adoption part, the trust part that we have to earn. Just like SAP over the past 50 years has earned the trust of customers in that you can use SAP to run your business-critical processes, we need to earn that trust so that users can know they can interface with Joule to give them the right information and actually help you in your job. Once we have earned that trust, then we can accelerate the productivity of people, because they know that Joule is a trusted copilot.
ASUG: What goes into earning trust with something like this that is so dependent on both the validity of a customer’s own data, and things that are outside of the SAP reach?
Way: The core component is data privacy, compliance, and AI ethics. In the sense that you as an end user should only have access through Joule to the tables, and the data that you have access to. Joule should not be able to do any more or any less than the permissions you have.
There’s also the explainability, and there’s the understanding of the different harms that may be open to bias. Having the ability to provide citations so the users can actually go back and verify is, to me, just table stakes.
And then, lastly, is Joule doing what you actually ask it to do? Can it understand what you’re saying, and can it help adapt over time based on what you’re doing? Just like if I asked my child to take out the garbage, the first time I am going to walk next to him to make sure that he takes the bag, he’s not dragging it on the ground and ripping the bag and having the trash coming out of it. Verify that he knows how to carry it, he knows how to put it in the garbage bin, he knows how to take it out to the curb. Then after a while, I can start to be more hands off, because I can trust that he’s able to take out the garbage.
In the same way, I think that’s what will be built up in Joule. You will trust Joule to do a few simple things, and then you ask Joule to do some more complex things. And then with these Joule AI assistants, more and more complex things.
ASUG: Referring back to your session earlier today, "From competition to collaboration: agent-to-agent ecosystems," tell us about the work you’re doing with Microsoft.
Way: The Microsoft supply chain team needs to be able to understand, in this stratospheric demand for data center buildouts and servers and all the stuff that they need to do to power AI, how can they leverage what they have with SAP in supply chain planning and understanding and inventory balancing? What is the expertise that SAP has, and then what are the things that they’re building out in their own AI platform? And how does that come together?
It became very compelling, because you see in the news all the time, hyperscalers are having a hard time keeping up and just building out all of these data centers. It’s really cool to see that they’re building out their own AI platform, which is great, but they’re also seeing SAP as a critical partner. That’s where having our Microsoft and SAP collaboration, where our agents can talk to each other using the A2A protocol, is so exciting, and I hope this is a pattern of the synergy that’s possible, because it’s not just Joule at Microsoft, for example — it could be Joule with any agent with an A2A protocol.
ASUG: Do you see the kinds of capabilities and the direction that AI is developing in potentially being able to help avoid this data center climb? Do you see the solution to that problem potentially coming through applications of AI to make this more efficient and less power-consuming?
Way: Absolutely. I think there are a few different dimensions in which you can help. One is, like you said, AI in terms of helping understand capacity, numbers of servers and racks, and components. I know which suppliers can provide that, and I can predict which ones will come in at what time. Or if there’s a natural disaster, how do I rebalance or how do I move my inventory to make sure that everything happens on time?
The other part of it is like two competing forces. One force is to make the model smaller and faster, because the models are massive. They take up a ton of compute power. If we can make the model smaller and we can get the same performance, that would be great. But on the other hand, the use of AI is also higher. So where, in the past, you might have 10 calls to a model. Now I have a smaller model, but I need to call the model 100 times. It might be the same amount of computing, the same amount of electricity I need to use.
As AI is being used more and more, the buildout is still going to be there. Where we’re having data centers with hundreds of megawatts of capacity to train a model, that human brain uses 20 watts. We talked about humans being at the center of all of this, which is still super critical. Human ingenuity is what really is there at the end of the day to take us to the next level.
Lauren Dixon contributed to this article.