In his posi­tion as Glob­al Head of Arti­fi­cial Intel­li­gence (AI) at SAP, Dr. Wal­ter Sun is work­ing to devel­op and deploy enter­prise AI tech­nol­o­gy to solve for cus­tomers’ most press­ing busi­ness issues.

Hav­ing spent 18 years pre­vi­ous­ly work­ing at Microsoft, where he held senior sci­en­tif­ic and prod­uct devel­op­ment posi­tions, Sun is respon­si­ble for lead­ing the AI strat­e­gy for SAP and has been an influ­en­tial voice in clos­ing the dis­tance between the gen­er­a­tive-AI hype cycles and real busi­ness results for the SAP glob­al user base. 

Ahead of ASUG Tech Con­nect, where Sun will dis­cuss explor­ing arti­fi­cial intel­li­gence for real-world results and help to deliv­er the conference’s day-three keynote focused on unit­ing cus­tomers, part­ners, and SAP through AI and strate­gic col­lab­o­ra­tion, Sun sat down with ASUG to dis­cuss the SAP-Microsoft AI part­ner­ship, the impor­tance of deep and bi-direc­tion­al inte­gra­tion between dig­i­tal copi­lot Joule and Microsoft Copi­lot, how WalkMe will aug­ment Joule’s gov­er­nance capa­bil­i­ties, and what role SAP will play in upskilling cus­tomers to get the most val­ue out of AI technologies.

This inter­view has been edit­ed and condensed.

At a high lev­el, how would you artic­u­late the SAP Busi­ness AI strat­e­gy to our readers?

At SAP, our strat­e­gy for deliv­er­ing busi­ness AI breaks down into three core pillars.

The first is Joule, our dig­i­tal copi­lot, which con­nects SAP appli­ca­tions and also enables exten­sion by cus­tomer-built apps — so across every­thing for busi­ness users’ and devel­op­ers’ apps. Sec­ond­ly, we embed AI direct­ly in the appli­ca­tion lay­er. For exam­ple, in SAP Suc­cess­Fac­tors, users can use gen­er­a­tive AI to auto­mate the cre­ation of job descrip­tions when some­one is look­ing to fill a role. And third­ly, we’re focused on cre­at­ing capa­bil­i­ties for devel­op­ers. We offer a wealth of devel­op­er tools on the SAP Busi­ness Tech­nol­o­gy Plat­form. A key exam­ple is our Gen­er­a­tive AI Hub, which allows both inter­nal and exter­nal devel­op­ers to access a vari­ety of large lan­guage models. 

We cur­rent­ly sup­port 25 of them. We have an ​“abstrac­tion lay­er,” where we have cus­tomers give us a use case, and we can find the right large lan­guage mod­el for them, which is the one that gives them the best response — the best bang for the buck, if you will. 

In oth­er words, if you told me you want­ed to cre­ate a QA-bot engine, and I told you these five large lan­guage mod­els will all will give you the same per­for­mance, you can say, ​“Give me the cheap­est one,” which is both less cost for you and your com­pa­ny, as well as bet­ter in terms of the sus­tain­abil­i­ty foot­print, with the least amount of com­pute nec­es­sary. That’s how we think about our mul­ti-ven­dor strat­e­gy. We want to help the busi­ness user find the best large lan­guage mod­el for them.

Giv­en your back­ground at Microsoft, what excites you about the part­ner­ship announced at SAP Sap­phire, through which Joule will be inte­grat­ed with Microsoft Copi­lot to estab­lish a more open-end­ed, part­ner-cen­tric evo­lu­tion of busi­ness AI? 

At SAP, we believe in part­ner­ships to help our end users ben­e­fit the most. SAP pow­ers the vast major­i­ty of For­tune 500 com­pa­nies; 99 of the 100 biggest com­pa­nies use SAP. We’re the leader in busi­ness appli­ca­tions. At the same time, many of these cus­tomers use Microsoft­’s Office Pro­duc­tiv­i­ty Suite. As you can see, and as you observed, there are imme­di­ate syn­er­gies there. 

Instead of a user work­ing through two dif­fer­ent copi­lots, which don’t speak to one anoth­er, we have a part­ner­ship where, if you have Microsoft Copi­lot and you have a busi­ness appli­ca­tion-spe­cif­ic ques­tion that SAP can answer, it can invoke Joule and say, ​“Hey, Joule, can you han­dle this one?” Like­wise, in Joule, if you have ques­tions about Microsoft Graph, or oth­er top­ics relat­ed to the Microsoft Copi­lot, we can ask the Microsoft Copi­lot for that infor­ma­tion. Giv­en that we have a lot of mutu­al cus­tomers, this makes it eas­i­er for peo­ple who run both not to have to jump from one copi­lot to the other. 

With­out per­son­i­fy­ing copi­lots, if you con­sid­er them as dig­i­tal assis­tants, with­out hav­ing to go to two dif­fer­ent assis­tants and ask them ques­tions sep­a­rate­ly, or go to eight or 20 dif­fer­ent assis­tants in the long run, our vision is that you have part­ner­ships — start­ing with Microsoft, but also with many oth­er copi­lots that exist, so that we can make it easy for our busi­ness users. We can tell them, ​“Go to Joule, and every­thing will be there, from con­nec­tion to all oth­er busi­ness appli­ca­tions that SAP has to exten­si­bil­i­ty across oth­er busi­ness appli­ca­tions in your space.”

In terms of the deep, bi-direc­tion­al inte­gra­tion that you’re dis­cussing, and in terms of embed­ding Joule in SAP S/4HANA Cloud, SAP Build, SAP Inte­gra­tion Suite, SAP Ari­ba, SAP Ana­lyt­ics Cloud, and more, what excites you the most about what this will enable for busi­ness users?

What’s excit­ing to me is that we can help users go across dif­fer­ent SAP appli­ca­tions. I’ve demon­strat­ed a Joule copi­lot engage­ment in SAP Con­cur, where­in you can ask it about book­ing a flight for a busi­ness trip, ask it also about extend­ing that trip and using vaca­tion days to do so, have it engage SAP Suc­cess­Fac­tors to deter­mine if I have vaca­tion days left, then come back into SAP Con­cur and add two vaca­tion days to the trip. That’s one example.

The next lev­el involves a col­lab­o­ra­tive, mul­ti-agent frame­work. If you’re refur­bish­ing a build­ing, how do you approach that inno­va­tion? Nor­mal­ly, you’d con­tact plan­ning agents and pric­ing agents, but you can alert Joule to your plan, and var­i­ous agents will act on your behalf, pulling the rel­e­vant infor­ma­tion from your data­bas­es and nego­ti­at­ing with one anoth­er. Imag­ine an agent for SAP S/4HANA look­ing at the sup­ply chain, an agent for finances, and an agent for accounts receiv­able, all coor­di­nat­ing behind the scenes. Today, in a Microsoft Teams meet­ing with mul­ti­ple col­leagues, you might ask some­body to find you the SOW tem­plate; tomor­row, an AI agent with pric­ing skills will deter­mine the cost to buy 100 lap­tops, and then a sep­a­rate SOW agent will obtain your approval and write the SOW for you. 

Mak­ing life eas­i­er for busi­ness users, we’ll have these dig­i­tal agents in the back­ground doing that type of work, and if what you need doesn’t fit with­in the SAP ecosys­tem, we have these copi­lot con­nec­tions to Microsoft, where you can get addi­tion­al infor­ma­tion from your Microsoft Office Graph and extend the space. That exten­si­bil­i­ty, both through a copi­lot like Joule at the top lev­el and the exten­si­bil­i­ty of oth­er peo­ple build­ing mini-agents, can allow us to do much more with what we have.

SAP recent­ly com­plet­ed its acqui­si­tion of WalkMe, which was announced at SAP Sap­phire ear­li­er this year. Tell me more about the moti­va­tion behind that and where you see WalkMe inte­grat­ing with Joule and this over­ar­ch­ing vision for busi­ness AI. 

WalkMe is a lead­ing dig­i­tal adop­tion plat­form provider; acquir­ing them helps SAP increase its focus on the suc­cess of the busi­ness soft­ware user. Their tech­nol­o­gy gives users enhanced guid­ance and automa­tion fea­tures that allow them to per­form work­flows smooth­ly across all dif­fer­ent appli­ca­tions, includ­ing third-par­ty applications. 

Users are able to mas­ter very com­plex dig­i­tal process­es with sim­ple on-screen guid­ance and ana­lyt­ics. It increas­es usage of those appli­ca­tions and dri­ves val­ue cre­ation to our cus­tomers. We feel like the acqui­si­tion com­ple­ments our SAP Busi­ness Trans­for­ma­tion Man­age­ment port­fo­lio. It adds a strong peo­ple com­po­nent to our exist­ing busi­ness trans­for­ma­tion, which has thus far cov­ered the process, appli­ca­tions, and data dimen­sions. There are those three dimen­sions that we had; this peo­ple com­po­nent com­pletes the picture. 

Hav­ing SAP Sig­navio, SAP LeanIX, SAP Busi­ness Tech­nol­o­gy Plat­form, and now WalkMe work­ing togeth­er, we can help our cus­tomers per­form a dig­i­tal trans­for­ma­tion jour­ney even faster and more effec­tive­ly. Of course, WalkMe will help to enhance the pro­duc­tiv­i­ty of our dig­i­tal AI assis­tant; it can over­lay web, mobile, and desk­top apps, includ­ing SAP and non-SAP sys­tems, with­out inte­grat­ing online soft­ware. By com­bin­ing WalkMe’s adop­tion capa­bil­i­ties with SAP’s Joule copi­lot, SAP will be able to offer bet­ter AI assis­tance in the user experience.

Can you expand on those adop­tion capa­bil­i­ties that SAP unlocks through com­bin­ing WalkMe with Joule? From a UX per­spec­tive, this could pro­vide an intu­itive path into sys­tems for SAP users, which ben­e­fits onboard­ing process­es and also relates to the reskilling and upskilling ini­tia­tives that SAP is empha­siz­ing both inter­nal­ly and for customers. 

It’s a good ques­tion you’re ask­ing about adop­tion. The gen­er­a­tive AI space is mov­ing so quick­ly, and appli­ca­tions add new fea­tures so quick­ly, that the aver­age busi­ness user can be over­whelmed by what’s avail­able. In Joule, native­ly, we real­ly want to make it as easy as pos­si­ble to actu­al­ly com­mu­ni­cate with appli­ca­tions with­out hav­ing to learn their manuals, 

A year ago, when I joined SAP, Suc­cess­Fac­tors was new to me; to fig­ure out where I could open a job descrip­tion took work. Going into SAP S/4HANA and look­ing at ERP tools, it took work. Now, a new hire can use nat­ur­al-lan­guage prompts and say, ​“I want to know how to how many vaca­tion days I get a year. What’s the com­pa­ny pol­i­cy on busi­ness trav­el? Who are our ten biggest sup­ply chain providers?” That infor­ma­tion can be gath­ered with nat­ur­al language. 

WalkMe goes even fur­ther where it actu­al­ly pro­vides the auto­mat­ed fea­tures and work­flows upfront; you have guid­ance. In addi­tion to hav­ing the exist­ing appli­ca­tion soft­ware, a dig­i­tal adop­tion plat­form helps peo­ple see how you use dif­fer­ent process­es. There’s on-screen guid­ance and ana­lyt­ics say­ing, ​“This is what you need to do.” Almost like a human guide, it says, ​“This a new appli­ca­tion you nev­er use. These are some tools in terms of how you can do bet­ter.” Based on peo­ple’s activ­i­ties in the SAP or non-SAP appli­ca­tions, WalkMe can over­lay guid­ance and pro­vide fur­ther assis­tance, on top of what Joule does, in terms of mak­ing it eas­i­er for any busi­ness user to use technology.

So much of the promise of busi­ness AI is sim­pli­fi­ca­tion of process­es for busi­ness users. At the same time, as an emerg­ing tech­nol­o­gy, AI presents oppor­tu­ni­ties for peo­ple to reskill and upskill to effec­tive­ly nav­i­gate AI-embed­ded appli­ca­tions. With SAP in the midst of its own reskilling ini­tia­tives, what capa­bil­i­ties can SAP users build in their orga­ni­za­tions to be able to most effec­tive­ly har­ness busi­ness AI? 

SAP has long invest­ed in train­ing mea­sures, based on the direc­tion of the tech indus­try, to stim­u­late our growth and keep up with mar­ket demand. We host­ed a bunch of AI Days in 2023 inter­nal­ly, for our employ­ees to quick­ly learn and upskill them­selves in gen­er­a­tive AI. As the num­ber of skills expect­ed for each of our roles increas­es, and as skills turn over and these roles accel­er­ate, SAP is on its way to becom­ing a skills-led organization.

Today, we’re enhanc­ing, build­ing, and shift­ing our learn­ing pro­grams to fol­low the trends of gen­er­a­tive AI; look­ing at our enter­prise cloud ser­vices; and look­ing at all the dif­fer­ent nec­es­sary skills that we think are nec­es­sary for the AI space. Now, we have AI train­ing cours­es, not only on machine learn­ing and AI but gen­er­a­tive AI tech­nol­o­gy as well. 

Across indus­tries, the tech­nol­o­gy out there democ­ra­tizes the abil­i­ty for our cus­tomers to do this as well. SAP is build­ing ​“ABAP AI tools,” or devel­op­ment tools for our ABAP domain-spe­cif­ic lan­guage. Think about that as democ­ra­tiz­ing the abil­i­ty to write code, because there’s a sub­set of users in the world that know ABAP; now, in nat­ur­al lan­guage, you can open up the door to how many peo­ple can actu­al­ly write ABAP code with machine assis­tance. The same is true of the Joule copilot’s tools for using nat­ur­al-lan­guage com­mands to com­plete actions. It’s not just pro-code; now, you’ve got low-code capa­bil­i­ties for writ­ing scripts or flow charts, and no-code capa­bil­i­ties via nat­ur­al lan­guage. Peo­ple are able to do more. 

In terms of upskilling, what we need to under­stand now is how to speak to machines. 

With our gen­er­a­tive AI hub, SAP is includ­ing ​“prompt man­age­ment” tools. That’s a fan­cy way of say­ing ​“ways to speak to machines bet­ter.” You have horse whis­per­ers that can work with hors­es, you have nan­nies that work well with chil­dren, and you can have com­put­er whis­per­ers with these prompt engi­neers. How do you actu­al­ly ask for some­thing most appro­pri­ate­ly to help get the best answer? 

I’ve used an anal­o­gy that these large lan­guage mod­els are almost like chil­dren that know a lot of infor­ma­tion but don’t have a lot of extra rea­son­ing, so you actu­al­ly pro­vide spe­cif­ic instruc­tions: ​“Hey, can you tell me what the hours of this restau­rant are, and can I make it to the restau­rant if I dri­ve from this loca­tion to this loca­tion at this time peri­od?” And if the answer is ​“yes,” that’s what you want, rather than the machine say­ing, ​“this restau­rant is cur­rent­ly open.” 

If you ask if it will still be open when you get there, the machine knows that you’re ask­ing a more spe­cif­ic ques­tion: not just, ​“is it open at this cur­rent moment,” but ​“is it going to be open when I arrive, giv­en that I am 60 min­utes away?” All that infor­ma­tion is the upskilling: how do you com­mu­ni­cate with machines better? 

That lev­el of intu­ition will be a par­tic­u­lar­ly inter­est­ing chal­lenge in heav­i­ly reg­u­lat­ed or high­ly tech­ni­cal indus­tries, where users will need high­ly spe­cif­ic ques­tions to hone AI tools to assist them. What role will SAP play in help­ing its cus­tomers to learn those skills?

We’ll help in a few ways. We offer retrieval-aug­ment­ed gen­er­a­tion (RAG) tech­nol­o­gy, which is basi­cal­ly an index that the cus­tomer brings in them­selves. If ASUG has a set of HR pol­i­cy guide­lines, you can upload those doc­u­ments into your ten­ant, pri­vate­ly. And when­ev­er you want to use a lan­guage mod­el, that mod­el can first pull infor­ma­tion from the ten­ant to get infor­ma­tion. That’s valu­able, for three reasons. 

1. Speci­fici­ty: If you ask a ques­tion about ASUG, the inter­net has infor­ma­tion, so the large lan­guage mod­el knows some­thing about ASUG, but not a lot. You can actu­al­ly get spe­cif­ic information.
2. Pri­va­cy: Your com­pa­ny’s vaca­tion pol­i­cy is not going to be avail­able on the internet.
3. Tem­po­ral nature: If I ask whether the Chica­go Cubs won last night, the large lan­guage mod­els won’t know, but if it can pull from a doc­u­ment data­base, it can find the right answer.

We’re help­ing peo­ple index infor­ma­tion. We’re pro­vid­ing prompt man­age­ment tools to help peo­ple do the best they can with each lan­guage mod­el. And each lan­guage mod­el is dif­fer­ent; it’s almost like speak­ing to dif­fer­ent peo­ple, if you will. In each case, we want to make sure the prompts are set appropriately. 

And we also have promi­nent checks on the out­put side to help dou­ble-check, to make sure the infor­ma­tion is cor­rect. How do we min­i­mize hal­lu­ci­na­tions? If we give you an input that says, ​“We’re talk­ing to ASUG, based in Chica­go,” and then a large lan­guage mod­el writes me a sto­ry about ASUG, and it says it’s in Wis­con­sin, we can check the fact is incor­rect, with basic fact-check­ing capabilities.

We can also lever­age oth­er lan­guage mod­els and call lan­guage mod­els again to dou­ble-check; each instance of a lan­guage mod­el call is almost like ask­ing a dif­fer­ent mod­el. And so, mod­els them­selves can help to check infor­ma­tion. In reg­u­lat­ed indus­tries, in many busi­ness cas­es, you can’t afford to make any errors. Mak­ing mis­takes dur­ing a din­ner con­ver­sa­tion is not going to cost any­body a bil­lion dol­lars, but mak­ing a mis­take in a press meet­ing or a quar­ter­ly release could cost a pub­lic com­pa­ny a lot of mon­ey. We need to make sure that we have the tools to give our cus­tomers the high­est con­fi­dence pos­si­ble, in terms of what we’re pro­vid­ing them.

There’s a say­ing: ​“garbage in, garbage out.” Maybe the pos­i­tive spin is: ​“good infor­ma­tion in, good infor­ma­tion out.” Peo­ple some­times have neg­a­tive expe­ri­ences with large lan­guage mod­els, because they’re not giv­ing it the right infor­ma­tion. They might either pro­vide wrong con­text or actu­al­ly ask some­thing that’s out­dat­ed: ​“Who’s the leader in sus­tain­abil­i­ty?” If you ask that ques­tion to a lan­guage mod­el that was trained a year ago; per­haps the answer has changed since then. 

One need­ed to actu­al­ly pro­vide a new con­text to help to get the right answer. It’s no fault of the user, because it’s a very com­pli­cat­ed space, help­ing cus­tomers under­stand when the lan­guage mod­els we’re using were last trained, let­ting them know we need to pro­vide new­er infor­ma­tion before ask­ing time-sen­si­tive ques­tions. The goal, we hope, is that we can help edu­cate our users on how to best use these mod­els to their benefit.

In infus­ing Joule with mul­ti­ple autonomous AI agents, what data struc­ture will be required to ensure that these agents can be gen­uine­ly col­lab­o­ra­tive, and what impor­tance does the ​“human in the loop” approach play?

For dif­fer­ent solu­tions, the struc­ture could dif­fer per busi­ness func­tion, such as HR or Finance. But the good news is that we recent­ly announced an SAP Knowl­edge Graph which helps bridge the gap across these indus­tries and solu­tions, like man­u­fac­tur­ing ver­sus ener­gy ver­sus retail. 

Regard­less of our SAP Knowl­edge Graph’s abil­i­ty to parse dif­fer­ent ter­mi­nol­o­gy across dif­fer­ent func­tions, hav­ing a human con­firm and review the out­comes will always be of great impor­tance to ensure the best pos­si­ble out­come of an AI agent’s response.

With the intro­duc­tion of SAP Knowl­edge Graph to ground AI in spe­cif­ic SAP busi­ness seman­tics and inter­re­la­tion­ships, how is SAP ensur­ing that these capa­bil­i­ties tru­ly ensure reli­able AI outputs?

Across our solu­tions, we often have fields which might con­tain acronyms; to make it more com­plex, these could even be acronyms from dif­fer­ent lan­guages. For exam­ple, Arti­fi­cial Intel­li­gence would be AI in Eng­lish, but it’s KI in Ger­man. As a result, cer­tain fields could be ref­er­enced in many dif­fer­ent ways. Our SAP Knowl­edge Graph allows us to both indi­cate when two seem­ing­ly dif­fer­ent enti­ties are the same as well as pro­duce con­nec­tive rela­tion­ships, which increas­es the com­pre­hen­sion of the mod­els to a cus­tomer’s request.

You can imag­ine if you ask a per­son a ques­tion about dis­parate top­ics that, if he or she hap­pened to know both top­ics well, they could inter­nal­ly bridge between them and pro­duce a more knowl­edge­able answer. This is what hav­ing the SAP Knowl­edge Graph can do to ensure more reli­able gen­er­a­tive AI out­puts, with less hallucinations.

For more from Wal­ter Sun, don’t miss ASUG Tech Con­nect, where Sun will dis­cuss explor­ing arti­fi­cial intel­li­gence for real-world results and help to deliv­er the conference’s day-three keynote focused on unit­ing cus­tomers, part­ners, and SAP through AI and strate­gic col­lab­o­ra­tion.