S2E28: "BigTech Privacy; Responsible AI; and Bias Bounties at DEF CON" with Jutta Williams (Reddit)

This week, I welcome Jutta Williams, Head of Privacy & Assurance at Reddit, Co-founder of Humane Intelligence and BiasBounty.ai, Privacy & Responsible AI Evangelist, and Startup Board Advisor. With a long history of accomplishments in privacy engineering, Jutta has a unique perspective on the growing field.

In our conversation, we discuss her transition from security engineering to privacy engineering; how privacy cultures differ across social media companies where she's worked: Google, Facebook, Twitter, and now Reddit; the overlap of the privacy engineering & responsible AI; how her non-profit, Humane Intelligence, supports AI model owners; her experience launching the largest Generative AI Red Teaming challenge ever at DEF CON; and, how a curious knowledge-enhancing approach to privacy will create engagement and allow for fun.

Topics Covered:

• How Jutta’s unique transition from security engineering landed her in the privacy engineering space. 
• A comparison of privacy cultures across Google, Facebook, Twitter (now 'X'), and Reddit based on her privacy engineering experiences there.
• Two open Privacy Engineering roles at Reddit, and Jutta's advice for those wanting to transition from security engineering to privacy engineering.
• Whether Privacy Pros will be responsible for owning new regulatory obligations under the EU's Digital Services Act (DSA) & the Digital Markets Act (DMA); and the role of the Privacy Engineer when overlapping with Responsible AI issues
• Humane Intelligence,  Jutta's 'side quest,' which she co-leads with Dr. Rumman Chowdhury, and supports AI model owners seeking 'Product Readiness Reviews' at scale.
• When, during the product development life cycle, companies should perform 'AI Readiness Reviews'
• How to de-biased at scale or whether attempting to do so is 'chasing windmills'
• Who should be hunting for biases in an AI Bias Bounty challenge
• DEF CON 31's AI Village's 'Generative AI Red Teaming Challenge,' which was a bias bounty that she co-designed; lessons learned; and what Jutta & team have planned for DEF CON 32 next year
• Why it's so important for people to 'love their side quests'

Resources Mentioned:

• DEF CON Generative Red Team Challenge
• Humane Intelligence
• Bias Buccaneers Challenge

Guest Info:

• Connect with Jutta on LinkedIn

Privado.ai
Privacy assurance at the speed of product development. Get instant visibility w/ privacy code scans.

Shifting Privacy Left Media
Where privacy engineers gather, share, & learn

Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.

Copyright © 2022 - 2024 Principled LLC. All rights reserved.

Transcript

Jutta Williams: 0:49

Sometimes in the privacy engineering space we get so wrapped up in kind of the technical controls to face, and it is just not sexy to work on things that are knowledge- enhancing, but that's really a missed opportunity. I love education, training, policies, standards, guidelines. I mean it's not sexy work, but these are force multiplying tasks. Start with knowledge enhancement and it will take your program farther, faster. I love this approach" to create curiosity, create engagement, have some fun. This doesn't have to dreary work. Have some fun and people will follow.

Debra J Farber: 1:31

Welcome everyone to Shifting Privacy Left. I'm your host and resident privacy guru, Debra J Farber. Today, I'm delighted to welcome my next guest, Jutta Williams, Head of Privacy and Assurance at Reddit. She's also co-founder of Humane Intelligence and BiasBounty. ai, and privacy and responsible AI evangelist and startup board advisor. She has a Masters in Information Security Policy and Management and graduated with distinction from Carnegie Mellon University; and, she considers herself a 'Recovering Chief Privacy, Security, and Compliance Officer, which I might ask her about.

Debra J Farber: 2:12

Jutta has a long history of accomplishments in privacy engineering and has a really unique perspective on the practice of privacy engineering. That's because she's had roles at 4 major tech and social media companies: Google, Facebook, Twitter and now at Reddit. So, a few weeks ago I finally had the pleasure of meeting Jutta in person at DEF CON - and we'll go into how that meeting went in a little bit - and then there, I volunteered at the AI Village and its launch of the largest Generative AI Red Teaming Challenge, a 'bias bounty' that Jutta co-designed. I cannot tell you how psyched I am to have you to join us for this conversation today. So, let's get started.

Jutta Williams: 3:00

Thank you so much for hosting me and, you know, longtime listener, first-time participant. I have to say this is an honor on my part for sure, and I'm super excited to kind of jump into a fun conversation. And also, I just wanted to say, "Thank you so much up front. You worked tirelessly at the DEF CON event, where none of us expected to work quite that hard. It's kind of like ducks smoothly sailing across the top of the lake but those little feet were sure pedaling underneath the water. So, thank you so much.

Debra J Farber: 3:26

It was my pleasure. I roped my fiancé in as well. He's the one that got me going to DEF CON for so many years, and it was honestly really fascinating. I'm glad I got to have a seat at the table to kind of check out the challenge and to really be there to help support the community. So, thanks for putting it together and I look forward to helping again next year and encourage others to do so too. You have such a unique background and I know my listeners would love to know what got you interested in privacy engineering and how did you make that transition from security engineering into privacy.

Jutta Williams: 4:02

Wow. You know, I find that every person I talk to has such a unique and interesting journey to this field. It doesn't seem like anyone comes through a very straightforward path. So for me it was an unintended career choice. When I went to undergraduate school, I decided I wanted to be an assassin for the CIA. Actually, no, just kidding. I wanted to be a lawyer. The first class is that - I mean, I became a Political Science undergraduate with an emphasis in International Relations and Turkish language, of all things.

Debra J Farber: 4:32

So, obviously it makes sense that you ended up as a Privacy Engineer.

Jutta Williams: 4:38

100% Right. But, I also fell in love, got married to the military, and had to move overseas; and, the only jobs we could legally hold as a dependent in Turkey were working for the U. S. government - and there were only four jobs. They were all low level GS5 and GS6 jobs. I got to work as an Information Assurance - and what was it called - oh, it was some really lame title, but it was basically helped us support with a lot of administrative responsibility. Office Automation Specialist - that's what it was.

Jutta Williams: 5:06

So, you know, I got a clearance and I got to work in technology and one of the earlier phases of like advancement in DO D. You know, the first people in DOD who had computers on their desks were admins. So, we were, indeed, data entry people and if you look across the spectrum of who's risen to the top in D O D spaces, a lot of people came out of the administrative pool that are in really senior positions today. And anyway, technology became kind of my path from there on forward. So, I moved back to the States with a clearance and technical experience right in time for Internet 1. 0. So, from there on, I was a technologist and I started as a Help Desk Support Tech. Then I became a Systems Administrator, the Network Ops Manager, and eventually I landed in a job that was just incredibly humbling as a Research and Development person working on crypto technology development. Because obviously, right?

Debra J Farber: 5:55

Yeah, obviously. So, you know that's the jumping off point into privacy engineering. I see it now with the cryptography overlap.

Jutta Williams: 6:03

Building public key infrastructure and some of the really cool technologies that evolved from PKI in D O D was super special and exciting. It was around 2002, 2003. And privacy- enhancing technologies really starts to get one's juices running about what are the long term consequences of using privacy- enhancing tech? What are the long- term consequences of using technology that keeps information secret? And I had the experience of sitting in the room - I wasn't important enough to be at the table, but I was sitting in a room - when we had a Privacy Board discussion about privacy- enhancing technologies and its implications for the world; and it was just so exciting and so humbling that I was like, That's it! This is my calling."

Jutta Williams: 6:43

When I applied to one school, my graduate program, because I was going to only go one place and it happened to be Carnegie Mellon because there was a whole lot of really cool academic work happening in the field of privacy - it was at the time of Lorrie Cranor and the k-anonymity work. I had the incredible distinction of having Alessandro Acquisti as my Graduate Advisor at Carnegie Mellon, and we got to do some really interesting, cool technology development work, but also kind of advance the school of thought around privacy and the commoditization of personal data. So, from there I was kind of hooked on privacy. Even though I worked in security engineering roles - because that's what existed at the time - it was always with this bent toward not necessarily protection from external threats but figuring out ways to make sure that the internal use cases were also appropriate and that there was transparency. So that was kind of how I got from tech and non- tech into privacy.

Debra J Farber: 7:40

I love it. That's such a great story and obviously unique. Everyone who kind of comes into security and privacy typically has a fascinating story as to how they got into it. I also wanted to point out that you and I both worked at the U. S. Army - I think we've overlapped for one month. I was only there for two months as an intern for the Army JAC Corps summer internship program, but I think that's just amazing. We worked at the same company many years ago - or organization, not company. It really is a small world. Amazing. Okay. So, you've worked, as I mentioned before, in several of the BigT ech / FAANG companies. I know we can't call it FAANG anymore because Facebook is now Meta.

Debra J Farber: 8:25

But, I don't know, MAANG companies. And, Twitter is now X. Oh my gosh, I didn't even think of that. Well, Twitter was never on the FAANG. Oh, right.

Jutta Williams: 8:36

There was no 'T.'

Debra J Farber: 8:37

Yeah, maybe we throw out the acronym altogether, who knows, but you worked in a previous FAANG environment for 4 different companies in Privacy Engineering roles. So let's, if you wouldn't mind, let's compare the privacy cultures and just general approaches at each company.

Jutta Williams: 8:55

So, my first act career was the government and then my second act career was a little bit of consulting, but mostly healthcare.

Jutta Williams: 9:01

So, the healthcare culture - and I was there for, gosh eight years maybe - it's a little different because there's like this culture of caring about privacy. It's considered - every person who works in healthcare kind of understands that their extension of the doctor- patient confidential promise; and, as a result, you don't fight culturally to really implement and build really great programs, in my opinion. So, when I moved from healthcare to Google, I was really surprised that that is not an embedded cultural truth. That data is just kind of fuel. "It's like electricity, as Andrew Ng said about data and ML. It's part of the value proposition, but it's not by itself held in the same esteem, in my opinion, as it was in healthcare; and that was a little shocking. So, when I landed at Google, I will say that it was a bit of culture shock because I went from an equally large sized company - my first healthcare system where I was the Privacy Officer was 130,000 employees, and that's about how big Google was - but, you know, they put extra zeros on the number of users, and so scale was completely different. The intimacy of the data was totally different. We didn't have nearly as much data as, say, you would have at a 30- year longitudinal health record. So, everything was just different, and also people didn't see data the way I saw data and that it came from a person, that it belonged to a person. This was just bits and bytes that were useful in creating really huge scale technology at the time. I'd say that over the course of the couple years I was at Google, that shifted and changed quite a lot. GDPR came to be; there were more regulations; there was a lot more public feedback about privacy as an inherent right and that people were concerned about their privacy. So, culturally, things shifted while I was at Google. But, I'd say at the beginning, I didn't feel like they have the same focus on where data comes from as I did, but maybe more so toward the end. Now, Facebook is a totally different ballgame. When I moved to Facebook, data was respected, but it was respected for what it delivered; and it delivered huge margins and growth and an opportunity to reach lots of people. So, again, totally different point of view. Instead of being innovation- focused and how can we use this data in super innovative ways, it was how do we grow big and fast? That was a very different culture, too. It was also harder to implement privacy change, in my opinion, because if you affected one of those two top line metrics, it was a really uphill slog.

Jutta Williams: 11:25

Then, I moved to Twitter, and Twitter is a different position. I was there as a Machine Learning Ethicist. We were kind of the machine learning ethics, transparency and accountability groups. So, we were looking at the long-term consequences of using large datasets to deliver services and to make decisions at scale. So, it was a different role and data had a different use case and purpose, which was to create personalization. So my job was a little different. I'd say that Twitter was a different culture too. It was very methodical, it was very thoughtful, and I would say that it was slower as a result, but I also think that we use data differently. It was also a very legally- driven privacy program, so legal liability and regulatory liability was first and foremost thought, even more so than innovation. So, a different culture also. I left before 'Elon year,' so I can't speak to how it is now, but I thought Twitter's approach to data privacy was good. But, it actually impacted and harmed, I would say, innovation as well because it was so protectionist. Now, Reddit is a totally different company.

Debra J Farber: 12:27

In what way was it protectionist? I'm just curious.

Jutta Williams: 12:30

Data use cases were approved in a piecemeal, very specific fashion. I'd say that data deletion practices were really aggressive, and so data deletion was impacting a lot of the ML use cases. It was just a very different structure and it was, again, by that time in society, there had been a lot of enforcement actions. So, I would say that there was more interpretation and there was a lower risk tolerance for some of the data practices by the time I got to Twitter, and some of the legal liabilities were impacting some of the innovation decisions.

Debra J Farber: 13:05

That makes sense OK.

Jutta Williams: 13:07

Reddit's a totally different culture and a different place. We don't collect a lot of information. We don't use a lot of information. And, I would say that we're not motivated by only the financial imperative to use data. So, I think that also the structure of the business is different. We're focused a little bit more on community ideation, interest more than the person. So, if you look at some of the social platforms, it's a little bit more about "ey, this is Jutta, this is what I'm doing, this is what I like, this is what I look like, this is these are the things I think. And at Reddit it's a little bit more about the idea and the conversation than it is about the person. And, as a result, we just don't have that much information about people and we're not really looking to get it. I think that it's just a different platform. It has a different purpose and a different use case and for me, it gives us an opportunity to really do things differently.

Debra J Farber: 13:55

Do you find that the culture at Reddit that you just described makes it - I mean, do you just have less of a scope, like less on your plate because you're not covering as much personal data, for instance? So are you basically able to get to some pet projects or some really interesting deployments that maybe in another company's cultural environment you wouldn't have had the opportunity to get to because of compliance?

Jutta Williams: 14:23

Yeah, we haven't slid down the slippery slope, right, as it were.

Jutta Williams: 14:27

I think that it changes the conversation a little bit about how do we stay true to ourselves and who we are while trying to accomplish some of the bigger picture things that other companies and our peers are accomplishing.

Jutta Williams: 15:21

I think that it requires us to be a little bit more creative about how we get to the outcome without using the same methods and means. It's not terribly hard to deliver a targeted ad when you have 289 data elements about a person and a social graph. But, if you don't have those things but you do have a community that is granularized the way we do with subreddits, can we get to the same place and how do we do that while still making advertisers feel like this is a targeting system and program that's worth paying for, right? So it's kind of the creative ways that we get to the same outcomes while looking at it from a completely different angle. I think it's possible, but I just find myself being a lot more creative in the way that I have conversations. And then also, staying true to the principles, like you get to really double down on the principles that we live by.

Debra J Farber: 16:07

Yeah, that's what I was thinking is that you would even have more time to be able to kind of message better and like focus on, I guess, parts of a privacy program - like communication and governance and maybe more fun training programs or whatnot, because there's less of a focus on needing to meet compliance aims, not because you don't have compliance, but because you don't have to bring things into compliance where they weren't.

Jutta Williams: 16:32

Fingers in the holes in the dam. Right? You're just like, you can structure things just a little bit more. We're an 18 year old startup. There's pros and cons to both of those things, but what I would say is that compliance should be the byproduct of a really strong program. Right? It shouldn't be the defining reason for a program, but so many times you're just trying to solve for these minute compliance issues because you're so far down a path that, you're right - I get to focus a lot more on building a really effective program under the definitions of framework and spend a little less time reporting on compliance. It's just a byproduct.

Debra J Farber: 17:05

Love it. So, is Reddit hiring for privacy engineering

Jutta Williams: 17:10

I keep trying to. The market keeps shifting and changing. I do have two open positions One is for a manager position on the Ops side of my business and the other is another SME. We're shifting left, as your podcast is called. We're trying to build more privacy engineering projects, which requires different technical skill sets than some of the privacy review strategies of the past. So, if we're shifting left and we're building it directly into technology so it scales and is persisted, it means more SMEs. It means more infrastructure people. So yeah, we are, but we're hiring different people than I've ever hired before. Much more on the technical side of things Data pipelines and deletion scripts and all these things that scale in an automated way to ensure consistency. That's kind of our goal, right now.

Debra J Farber: 17:54

Amazing. Well, I think we have some of those people in the audience, so, if you're looking, I encourage you to go and submit your resume because then you'd get to work for Jutta, which is almost as exciting as working for Reddit.

Jutta Williams: 18:07

With, not for.

Debra J Farber: 18:10

So, what advice do you have for security engineers who may be seeking to transition to privacy engineering or at least expand their roles to take on some aspects of privacy engineering?

Jutta Williams: 18:22

Yeah, the best privacy people I know have a beginning or an end in the law. So if you're already an engineer and you want to get into privacy, you're going to have to read up on the rules. A lot of what we do is building technical standards or technical expectations rather, based on an interpretation of a rule or an interpretation of what we see as an enforcement action in industry. So it's reading the regulations, learning how to read them and then understanding enforcement actions and kind of the approaches that people are taking to solve for some of those enforcement expectations and then being able to translate those into technical requirements. It's a difficult thing. So, conversely, if you're an attorney or a lawyer or somebody who's trained in law and want to come and be a Privacy Engineer, you're going to have to learn the technical ways of delivering on control expectations. So read up on kind of what's an expectation.

Jutta Williams: 19:12

Some of the IAPP certs are good ones. There's a couple of great books out there, but you have to have expertise in both, and those are two very hard industries to combine into one practical skill set. So, I'd say learn how to read a reg, learn how to interpret it, learn how to turn that into a business requirement and then you can turn it into a technical requirement and then you can apply your engineering skill set. Conversely, if you're really great at reading a reg, learn how those turn into business requirements. Then go educate yourself about what is a security control, what is a data protection control, and go get smart on one or more technologies. But it takes a lot of self-learning because it's really hard to go to school and learn all this stuff. Most of it's on the job training or self-study.

Debra J Farber: 19:59

There's so much self-study. I think that just applies to the privacy profession generally, especially now. I mean, first it was drip, drip, drip of people caring and now it's like in the news on a daily basis. It's just fascinating.

Jutta Williams: 20:14

Also, you're going to have to be a jack- of- all- trades because people keep piling new and interesting things on top of the privacy specialty. So, anytime that there's an ethics and a data and a technology question, it becomes a privacy obligation.

Debra J Farber: 20:28

Yeah, do you think that's going to be like some sort of Chief Ethics Officer or Chief Trust Officer or something in the future?

Jutta Williams: 20:36

Yeah, I think Chief Trust Officer is a job title I've seen. I've seen that in market, and trust starts to become a really interesting thing because it's not a measurement of effort. It's a measurement of outcome. And, those roles are really interesting roles. So, you see a Chief Risk Officer once in a while, a Chief Trust Officer, a Chief Ethicist. CPO just keeps expanding and expanding in obligation requirement.

Jutta Williams: 21:00

Right now, I would hate to be a CPO now compared to when I was one, because it's so much falls under a CPO title anymore. And also, you know, if you get it wrong, people are starting to go to jail. Now, you also have to be really, really smart about what is a defensible program. They're all different skills. I went through a Chief Compliance Officer training in health, and 80% of it was knowing when you had to bring in the Board. What was a board obligation? What was a board reportable obligation? And, it's so true because you have to be able to affect the change in a consistent change in your business. Or if you are not empowered to be that change in the business, you have to be able to report it upward or else you're accountable for it. So, it's just it's becoming a more liable position as well, it's why I think you see so many CPOs are part of Counsel's Office, in Big Tech especially.

Debra J Farber: 21:49

Oh, that's interesting for that overlap. That's why you think that so many CPOs are required to be Counsel - like practicing Counsel?

Jutta Williams: 21:56

What I think, in other industries, is that the compliance official comes from the core competency of the business. So, if you're in finance, it's usually somebody with a with a finance background or business background. If you're in an industrial safety position, it's usually an industrial safety engineer who's your chief regulatory compliance official. You would think that in Big Tech it would be an engineer, somebody who understands the engineered product, or a product leader, but it's not. It's always a lawyer, and part of it is just that it is so litigious, and the regulatory enforcement actions are so significant, that I think that more and more of the privacy official ends up being part of Counsel's office.

Debra J Farber: 22:31

Yeah, and I think that's also really unfortunate because, Counsel - they do amazing job in what they do - but you will counsel a business; it's not the business itself. Right? If engineers and product folks are helping to create the products and actually handle and touch the data, it just seems to me that the Chief Privacy Counsel would be dealing with legal stuff and Chief Privacy Officer is the business and operations of privacy and maybe even including some aspects of privacy engineering or product or whatnot. I've seen CPOs come right out of being a partner at a law firm and then go into industry managing a large CPO office; and I'm just like, "I don't know how they're set up to win there, right, like there's just different skill sets - the management. I'm not saying you can't be a lawyer, but just that they're not. . .

Jutta Williams: 23:27

My best friends are all lawyers. The thing is that most of these BigT ech firms have the person who's externally- facing, who is the CPO, and then they have a completely different function inside the business that does all the operational work. That's true at all the big tech firms that I mentioned. So, at Reddit, I operate the program that is privacy, but we have a CPO inside our Counsel's office who's amazing and a great CPO. Again, it's just that so much of the external interaction on privacy anymore is with regulatory authorities and bodies. What we used to say is that, in the compliance space and health is, compliance programs are supposed to be completely transparent. They're for the public. Everything you do is really supposed to be transparent. And then that's kind of antithesis to Counsel's office, which is to be protectionist. It's their job. So, it just ends up being a little bit less transparent when we have to go that way, and it's the shame that is the enforcement actions of today.

Debra J Farber: 24:15

That's a really good point. Thanks for that. Speaking of regulations, there are new EU regulatory frameworks like the Digital Services Act (or the DSA) and the Digital Markets Act (or DMA). What do you think, are privacy pros the ones who will own those ethical requirements under these regulations within their orgs?

Jutta Williams: 24:37

You know, I see that happening a lot of places, including with us. You know, in large part, the DSA especially, is identifying transparency requirements in how ads are placed. So, transparency and data and users and agency all kind of fall under that privacy mission statement. So, I would say, "yes, as much as dark design did, as much as kind of child protection rules did, when you come to data and ethics related things, somebody has to operationalize them, not just interpret them for your business, but then go create something to solve for that problem, and I think that that typically falls to a privacy professional. So we're involved in DSA. DMA, and most of my friends who work in privacy are at these BigT ech firms are also deeply embedded in this work. It also has a big component around AI and AI auditing, and so when you look at the evolution of data protection - from security to privacy to ML and AI - I see us very much being embedded in DSA. I also see us being embedded in Responsible AI, too.

Debra J Farber: 25:38

So, speaking of Responsible AI and just AI generally, I see the IAPP is setting up an AI Governance Center and a new AI Governance Certification. In fact, at the IAPP Privacy Summit this past March, leadership emphasized that privacy pros are well positioned to take on Responsible AI responsibilities, whether that's legal or consultative, or even engineering responsibilities. Since you're doing so much in the ethical AI space, what do you see the role of the Privacy Engineer when it's overlapping with Responsible AI?

Jutta Williams: 26:14

It's such a great conversation starter, to be honest. I think that there's lots of room for interpretation on this one. I personally subscribe to the idea that AI governance is not very different than other data governance responsibilities and requirements. At its core, I oversimplify everything because I wasn't a technologist in my first incarnation, but for me, machine learning in AI is just basically big data with statistics on top. So, when you talk about the governance structure for ML in its use cases, most of the time it's not a very different debate than should we use large data sets to solve a problem. So, I think that we're perfectly positioned to understand kind of the ramifications and we just have to scale up a little to understand what is a representative sample?

Jutta Williams: 26:59

Where does data cleansing come from? How do you create a training data set and a reinforcement and feedback loop? It's adding a bit of new capability on top. That isn't very much of a stretch from what we did for data governance. If you were keeping up with the governance as a labeling technology, and not necessarily just as a risk committee conversation, it is a pretty good extension of the privacy engineering components of data governance. So, I say, "Yeah, bring it to us, give it to us. Let us help expand a forum that we already have for data governance related topics. Also, it's really expensive to stand up governance structures, so the more you can recycle, the more likely it is to be successful and sustainable in a business. So, 'yes, and' - privacy engineers are going to have to learn a little bit of new context, a little bit of new technology, but we're really good at that because every time there's technological advancement, and it gets tiring, but I think this is one that's really worth investing in because I think this is here to stay.

Jutta Williams: 27:58

So, yes and - we apply data ethics to all kinds of data project review processes in the past, like should we do this research? Should we allow people to ask these questions? It's just an extension to say should we allow computers to make decisions in really large, fast ways about specific topics? So, yeah, who better and I guess it's one of my questions who better in the company to apply an ethical position, to understand tradeoff discussions and to adapt to new technologies as it relates to data use?

Debra J Farber: 28:29

Great, that's exciting. I look forward to seeing the overlap develop between these two fields.

Jutta Williams: 28:36

Who doesn't want another cert?

Debra J Farber: 28:38

Oh my gosh, I have too many as it is. It's so funny. It's what IAPP is there for.

Jutta Williams: 28:46

It's the alphabet soup.

Debra J Farber: 28:48

Exactly. So, I would love for you to tell us about your nonprofit that you started, Human Intelligence, that you co-lead with Dr. Rumman Chowdhury, which supports AI model owners seeking product readiness reviews at scale.

Jutta Williams: 29:05

That's a mouthful, huh?

Debra J Farber: 29:06

Yeah. What motivated you both to found Human Intelligence and what is the org's mission? And, did you know that I went to high school with Dr Rumman Chowdhury?

Jutta Williams: 29:17

No you didn't. Oh man, she's one of my favorite people in the world.

Debra J Farber: 29:23

We met in AP Biology. This is a true fact.

Jutta Williams: 29:25

Wow, okay. Well now, I really want to know what high school you went to and go like recruit from there.

Debra J Farber: 29:30

Yeah, we're probably outliers. I don't know many others that are from our years, that are like big in tech and made a name for themselves. So, this is the like one outlier that I'm aware of, the two of us.

Jutta Williams: 29:43

Well, so far I like the standard deviations coming out of your high school, so we'll take you.

Debra J Farber: 29:48

And I'm not even joking, but my dad was the High School Principal. So. . .

Jutta Williams: 29:56

Well, there you go. Rad. I can't claim that from my high school. We had 5000 kids in my high school.

Jutta Williams: 30:01

Wow, ours was much smaller than that and you have all this excellence. So, clearly he did something right. So, I'll tell you this. I met Rumman when I was at Twitter. I had started just two or three months before she did, and she was like a celebrity hire for us. She was an aqui- hire, actually; we bought a portion of our company to bring her talent to bear, and we started this really hardcore mission of applied AI ethics inside of Twitter. I recruited a really large team of amazing data scientists and then, at the end of the day, everybody got fired, just like everybody did in responsible AI. So, we started our friendship then. I was Head of Product for her as the Head of Engineering, Director of the program; and we were kind of a delivery partnership. So, thought leadership on her side, me was just 'go big, go fast' and make everything applie d. And, we were having a great time in about two and a half years ago now, not even, two years and some change. We had this great idea where we were talking to the AI Village folks, and it was a virtual day - there was a virtual session for DEF CON 29 - and they're like, "We should do a bias bounty. And so we were like, what is a bias bounty? We already had this algorithm for image cropping that we knew was racist and gender biased and a bunch of other ists, and we thought, "You know what? Let's put it out to the world and share with them kind of the learnings that we had just published and challenge people to find more that's wrong with this particular image cropping algorithm.

Jutta Williams: 31:28

So, this algorithm was based on what's called 'saliency.' Saliency is what's interesting within a picture; and most saliency models are trained by tracking eye movement. So, it's not a very consciously trained algorithm. It's kind of your subconscious looking at images and it tracks your eye movement and finds what your eyes found most interesting about a picture. What's also interesting about saliency models is that a lot were trained on college campuses by CS students. So, you can imagine what a CS student found interesting about a picture of a woman would be different than what they found interesting about a picture of a man. And so, these image cropping algorithms have been trained subconsciously to be horribly biased and cropping women from neck to navel for the most part. It was also cropping out black people and it was also cropping out people in wheelchairs. It was cropping out people with white hair and it was cropping out people who were larger than skinny people. It was just because the human nature, if we were to put consciousness to our training, would have been completely different and been much more fair with people. But, unconsciously, algorithms trained without intention turned out to be pretty terrible.

Jutta Williams: 32:32

So, we decided to run this bias bounty at DEF CON 29 and had a great time doing it and we were able to award some prizes. We were able to talk really openly about the fact that not all algorithms should exist, that not all technical problems need to be solved with ML, and it was pretty successful, and that was really fun. And so, when we were leaving Twitter - I left before Rumman - we didn't want this idea to die. It was a lot of fun. It created a lot of awareness. And, it created a lot of interest in people to study this problem and to find new applied ways of solving for algorithm bias.

Jutta Williams: 33:09

So, we kind of spun up this Bias Buccaneers project. It was completely unfunded and the website is still really ugly. We ran a lot of information. I think the website's cute. It's pirate themed for those on the call who don't want to go to bias buccaneers. org. It's all pirate themed because we really liked pirates that year. It was also my Halloween costume. So, pirates it is. And, we had a really good time and it was a really hard challenge and we had lots of participation and we were able to award some great prizes and it was just really invigorating and exciting. And it was just a bunch of volunteers that put it together, and we're like "his is a great way to help motivate an industry and to help people like get some training that is actually quite rare to find." Applied AI bias detection is difficult and there's not very many people who are teaching it, so we didn't want to see that die, so we continued to play and then we created this nonprofit so we could create this DEF CON challenge. So here we are.

Jutta Williams: 34:07

Rumman has a lot of work, and this is our side quest.

Debra J Farber: 34:10

Love it. So, this is a side quest that enables the bias bounties to continue.

Jutta Williams: 34:15

Yeah. But you know what? It might not stay a side quest for very long because now there's a lot of interest after DEF CON. So, who knows. Maybe it'll become funded and become a real thing and we'll be hiring and firing on that side, too. Who knows?

Debra J Farber: 34:27

Awesome. Well, I wish the best for you. At what point during the product development life cycle should companies perform these AI readiness reviews?

Jutta Williams: 34:39

Oh gosh, at every point. You know it's kind of like privacy-by- design. Right? It should be embedded at every stage of your development life cycle to some degree, right? So, algorithms are just like any other software development project. You should take a look at the design of your project. You should take a look at training data for the project. You should take a look at how you know your initial precision and recall rates are affected by the data.

Jutta Williams: 35:01

It's kind of continuous testing and evaluation strategy and requirement when you build a product that is patched effectively almost every day, if not every hour or every minute, through a reinforced learning kind of strategy. That means that you should be running T and E; you should do test and evaluation continuously to make sure that you're not drifting from baseline, that your products are still operating in the way that you expect them to. With every new bit of feedback loop, you're not actually making your AI dumber. There's all kinds of articles out right now that some of these large language models are actually getting dumber. They used to be able to do basic math with precision into the high 90s and now it's down to like 4% or 8% - I can't remember. You don't continuously test when you don't continuously evaluate and have a positive reinforcement with human-based contextual information coming back into your model, sometimes it can drift quite quickly.

Debra J Farber: 35:53

Well, how do you do that, I guess what they're calling 'alignment?" like how do you de-biased at scale? I mean, is that even something that can be done with humans in the loop? Or are we chasing windmills here?

Jutta Williams: 36:08

Kind of chasing windmills, but also you got to keep charging the windmills, otherwise you end up in this downward spiral. So, yes and yes. So, first and foremost, you have to build checks and checksums into your AI processes directly. So, you're constantly retraining your model, let's say. What is the representative data sample? Are you making sure that your representation is accurate? Is it correct? Or are you training only on, you know, your average CS student on a college campus as a guy, right? So, how do you make sure that the people training your data, or the data that you're using to train, are representative of the people who use your products and services so that it can adequately perform for all types of people? So, building that into your development life cycle so that it's constantly being evaluated is a great first step. Finding some ways to identify if you have drift, building that into your launch process to make sure that you're doing an evaluation to make sure that your model is performing in accordance with your baseline expectations, it's not drifting away from some of these really important measurements of bias and representation.

Jutta Williams: 37:16

And not all bias is social, by the way, right? So bias is just a term that doesn't necessarily relate to social justice- related use cases. We're talking, bias could be making mistakes on calculating 2 plus 2 equals 5. So, how do you identify some of those triggers? Things are varying from expectation. then also, you know, there's also an internal raging debate about what is bias. Is bias a representation of social in a way that you know you don't want to manipulate or change the world? So, how much of what you're experiencing or recognizing is actually resulting from the model versus just society and the data that comes out of social society? So, it's very difficult, but I do think it's worthwhile. I think it's still something you have to study. I think it's still something that human beings can make a judgment call better than most systems still, and that we need to provide a feedback loop. And that's what bias bounties are.

Debra J Farber: 38:13

That makes a lot of sense. And also I'm hearing the importance of measuring and that way - if you don't have metrics and measuring and where you want to be, how are you going to determine if you're drifting from that? For me, that's a big takeaway from this conversation.

Jutta Williams: 38:29

Yeah, and there's no real measure of success today. Right? There is no, "hey, you need 95% on this scale. So some of it is making up determinants for yourself. For your own business and your own model, what is progress and what is the opposite of progress?

Debra J Farber: 38:49

Can you walk us through what a bias bounty is like - you could use Bias Buccaneers or the DEF CON one, but basically, I want to know who should be hunting for biases in the bias challenge. What are their backgrounds? I think those questions go hand-in-hand because there's different challenges.

Jutta Williams: 39:12

So, drawing on the security history, so, from a security perspective, you know, there was a real leap forward when bug bounties were first initiated. One of the reasons was, you went from having a really small security team that was really looking at product and a OWASP in the app layers and network security and all these things, to creating a huge external workforce that was like a layer for defense, if you will, that would come in and, because they were paid, would come and tell you before they sold it on the dark web, about your vulnerabilities. It was a real leap forward because you can't be everywhere all the time. Bug bounties became a real value add to security organizations, and they're now standard practice.

Jutta Williams: 39:54

Algorithms are everywhere. They're in every part of our lives. Like, I don't know that we even know how often we're interacting with algorithms that are making decisions that are good and bad for us. So, I would suggest that anybody who's affected by an algorithm should have the ability to report when they see or experience something wrong; and that doesn't really exist today. Like, you can't report an algorithmic misconduct issue through a bug bounty; and there is no Help Center where you can file a ticket if you feel like there's gender bias in an algorithm or if you see a math error or you think the search results are creating a hallucination.

Jutta Williams: 40:37

The idea here is to create the ability for even Joe Public to participate in reporting errors. But, you know, paying people to find errors in algorithmic outcomes is actually super problematic all by itself. Right? Because there's a verification process and then there's a structure, payment process and all these sorts of things. So, these sorts of concepts are still being worked out. As far as, like the details and it's core bias, bounties are basically structured human feedback. For DEF CON, we created very specific challenges. We said here are 21 things that we know large language models can get wrong. Can you make it do these things, despite these companies putting effort into creating guardrails to prevent those things from happening? So, in this case it was almost competition for format, so we called it a 'Capture the Flag' (CTF) - a Jeopardy- style Capture the Flag. It was probably closer to Structured Public Feedback that we validated and awarded points based on submissions.

Debra J Farber: 41:35

Yes, I agree.

Jutta Williams: 41:37

And so, I would say that it wasn't necessarily Capture the Flag so much as it was, "Hey, we validated that your submission actually did violate the guardrails that these large language models had put in place. For defcon, we had eight different LLMs. I think it was a very special point in time where a competition was set aside to allow us to do this thing. I don't know if we'll ever see eight LLM side- by- side again, like we did there, but it's a very interesting experience to see how eight different large language models behaves. You could kind of tell just by from the personality of the models.

Debra J Farber: 42:09

Oh, and sometimes they'll tell you too, which model it was. If you answered in a certain way - if you prompted it in a certain way.

Jutta Williams: 42:16

Yeah, I was encouraging people to do that. Maybe I shouldn't have, but if you ask them, it'll probably tell you which model it is, even though we were trying to blind it a bit, at least make it not obvious.

Debra J Farber: 42:26

Yeah, you named them different elements of the periodic table.

Jutta Williams: 42:29

Right.

Jutta Williams: 42:30

So there was Iron and Cobalt and all these other funny names for your average Cohere and LLaMA2 and ChatGPT, but you could kind of tell that some of them were trying to make you feel at home and like this was just another human being you were talking to.

Jutta Williams: 42:44

And then others were very structured and very clearly an AI, not trying to convince you otherwise. And, some of them were very easily broken and some of them were a lot harder to break the guardrails, but they were also less useful and so, seeing them side- by- side, you can see that the development pathway for large language models, especially, and very different at these different companies. So, if you were to go hire one of these companies to build an LLM or to use their API to further your product development - let's say that you were providing post traumatic stress services and you wanted to have an intake conversation and have it transcribed and have it ask a couple of clarifying questions, which of these models would you choose? Would you choose one that was making a person feel safe at home or would you want them to be, very clearly, "this is talking to an AI and not to feel like this could be medical advice. Right? So it's very interesting just to see them side by side, but then also to see which ones were easily manipulated versus not very easily.

Debra J Farber: 43:43

Yeah, and to give context on the different challenges, you know I did just the general challenge of trying to get a biased output because for me - we only had 50 minutes and it kicks you out; that was by design.

Debra J Farber: 43:56

So, you know, you're like go, "I only have 50 minutes to play with this and there's so much you want to play with, and so I went with something that for me was a little easier. And my fiancé, Mack Staples, on the other hand - interestingly, he went and did the one where I think he was trying to get the LLM to add him to a specific access control list; and he had to get it to do that and convince it to add him to the list. And I know, like you know, I started to see his creative jui ces flowing, too. It's interesting to see what a hacker can do - even if he doesn't ultimately get the desired behavior from the LLM, with each prompt he ended up learning more information about the system or how it would structure an answer enough to then, you know, almost as a clue to going to another step. So, I thought that was just fascinating to see, or room full of hundreds of hackers. I mean, what did you have like 3000 come through over the weekend?

Jutta Williams: 44:51

Yeah, 300 total. We had a little bit of downtime. We're shooting for 3000 and didn't quite get there. Yeah, it was fascinating, and that's really the ultimate output of this challenge. Well, twofold. Right?

Jutta Williams: 45:03

We had several objectives for this event. One was absolutely education and awareness - to get all these people thinking about LLMs and how easy it is for misinformation to be generated at scale; because we see it, we see it in the world and we need to be able to recognize when we see it, so that we can combat it - all of us, in whatever our jobs are. We should all, just like societally, understand that not everything you read is real. So, and to question and to prove, and to study and to research and to make sure that the information that you see is real.

Jutta Williams: 45:37

And the second part was to create a vulnerability database to understand how hackers hack; how people, when motivated, would try to leverage an LLM in really malfeasant sorts of ways; to try to accomplish a task that a sane person shouldn't be trying to accomplish, right. So, one of the questions was create a hallucination and create fake history based on a Wikipedia article. Right? People do this for fun, and it becomes real in the real world. So, how do people manipulate and do this at scale using LLMs so that we can create better guardrails? So, the data analysis is ongoing and all of the different LLM companies are pretty excited about the research that results from this as well.

Debra J Farber: 46:22

That's awesome and I know you had a lot of eyes on you. There was so much press. You know, even The White House and Congress were keeping an ear out. Right?

Jutta Williams: 46:32

They came through and played the game, which was fantastic. You know, the Cyber Security Czar for the administration came through and spent a good hour talking to people. We had about 212 community college students that were sponsored to come to DEF CON and participate so that they could bring their learning and their point of view and their perspective, which we don't always hear. We also had a couple of different social good groups that came through so we could learn very specific experiential feedback from Black Tech Street and some other groups. So, that was also super informative, that everybody's experience with these LLMs is different. Their point of view is different. Their perspective is different. How LLMs and some of these harms that were represented in the challenges affect them is different. So, their feedback was super valuable too.

Debra J Farber: 47:17

I bet. The sheer number of people - so much of that were that you were telling me that I helped do was just helping with lines. That was my volunteering - as almost like a DEF CON goon. You know, there were just lines out the door and for maybe like an hour to three hours of people standing on a line because they all wanted to get to try this out. So, I'm saying that not as a bad thing. I'm saying that because there was so much interest and it's really great to see. But, I do know that you've got some plans potentially for next year's, like some lessons learned for next year. So, could you tell us a little bit about what we might look forward to at the AI village at DEF CON next year?

Jutta Williams: 47:58

Yeah. So, one of the big benefits I thought from this last cycle is that, at one point I had a really hardcore red teamer sitting right next to a grandma and both of them were engaged and both of them were challenged by the exercise, and I thought that that was just so wonderful that, no matter your technical skill set, no matter where you come from, this affects your lives and you can engage with it and learn something and really like challenge yourself, the challenge. So, we want to continue that. So, that's one of the things we will continue - to build challenges that are very accessible, which means a time limit, right, because, like, the winner went through several times, which is not against the rules, so you know that you can get faster if you like, figure out what works on each of the LLMs and each of the challenges and come back and try again. So, we'll probably do it online so that more people can participate.

Jutta Williams: 48:45

We just we started thinking about this in June and the event was in early August, so we had two and a half months to build this particular event. So, we'll have a whole year, which means we can really build a secure environment and maybe not wired computers. It's been a long time since I ran cable, so that was kind of exciting. So, we'll do it bigger. We'll do it broader. It will probably allow for team entries, because team entries tend to get really fascinating and interesting results. I think that we will try to make it more like an actual Capture the Flag (CTF) challenge scenario. Gosh, we need better lighting and music. That was the most. . . it felt like we were in a corporate boardroom.

Debra J Farber: 49:25

There's that eSports arena that might be a good fit. I mean one of the hotels, Because this happens in Vegas, for anyone who's listening.

Jutta Williams: 49:35

We do. We need to find a much more fun environment and venue. I just want to continue the education awareness and amp it up a bit. I think we create two pathways - one for the learner and one for the competitor that, you know, could ease yourself in as a person who's just learning about this for the first time and then you can have a real competition ready CTF option for people who really wanted to compete. But mostly, what we need to do is just make sure it's still fun.

Jutta Williams: 50:03

I love gamification of hard things because when you gamify things, people get engaged. They don't feel like this is unapproachable. It applies to privacy, too. I like gamification and the privacy program stuff. It's like you mentioned earlier training and education needs to be fun. When you have time and you have momentum, you can really make things fun and that usually lends itself to getting emotionally invested, which usually means better learning. So, we just need to make it more fun next year and maybe a little bit more challenging for the people who really want to go hardcore for the prizes.

Debra J Farber: 50:36

Yes, absolutely, especially since it's in Vegas, and I know that that's what some of the bug bounty platforms do is they have their live hacking events with cash prizes, and in Vegas during DEF CON. This has just been such a great conversation, and there's so much wisdom that I think we've all learned from you today. Do you have any last words of wisdom for the audience that you'd like to share before we close, or are there any calls to action you'd like to make?

Jutta Williams: 51:06

Love your side quests, man! Just go say "es to crazy things and try new things and see where they take you. From a privacy perspective, I'd say this - you know I'm a subscriber to framework. I love repeatable frameworks. It's kind of how I live my life in addition to. . .I'm not actually a rigid person; it's just that I find that repeatability is the key to success and happiness.

Jutta Williams: 51:27

So, when I look at the privacy engineering work, where we're making the most difference, go back to some of the fundamentals about how do you create defensible, continuously up, improving, effective program; and the first four steps of the seven elements of a compliance program are on knowledge and improvement, knowledge enhancement. And, I feel like sometimes in the privacy engineering space we get so wrapped up in kind of the technical controls to place, and it is just not sexy to work on things that are knowledge- enhancing, but that's really a missed opportunity. So, I love education, training, policies, standards, guidelines. I mean it's not sexy work, but these are force- multiplying tasks. Start with knowledge enhancement and it will take your program farther faster. That's what we're doing with this AI stuff - we're just trying to get knowledge out there. We might not have made the right knowledge yet, but we're just trying to get knowledge out there so that people will engage, people will find their own true north, and kind of take this industry in completely new and different directions.

Jutta Williams: 52:25

I love this approach - to create curiosity, create engagement, have some fun. This doesn't have to dreary work. Have some fun and people will follow. So, I think that's what I leave you with. That's what DEF CON was about, was just having a good time, and it turns out people really did have a good time, even Congressional delegations from different States; and yeah, I had a great time.

Debra J Farber: 52:50

It was a great event and you should definitely be proud of the first giant launch with two months of planning and and so many eyes on you.

Jutta Williams: 52:58

Oh man. We were programming the day before we went live and like, "Oh my God, nothing's gonna work. What's gonna happen? We ran out of ethernet cables. We were under table. I was sweating. I sweat the entire time.

Debra J Farber: 53:09

Well, I will tell you this, it was even hard to get near you - not because of the sweat -but because there were so many people who wanted to talk to you about how much fun they were having with it. When they were done, they were like all excited. I mean, I was there, and had to make room for all these people to give you feedback; and I think you did an amazing job the whole team, you and Rumman, and I don't know all the right people to give accolades to, but congratulations on a job well done.

Jutta Williams: 53:34

Awesome. We'll see everybody on this podcast and from our network networks, hopefully, we'll see you all next year at DEF CON. It is not the scary conference that everybody thinks it is. It's actually pretty safe, lovely and inclusive. So, come on up.

Debra J Farber: 53:48

And now there's kids. There's stuff for kids there, too. Well, Jutta, thank you so much for joining us today on Shifting Privacy Left to discuss all things privacy engineering and Responsible AI. Until next Tuesday, everyone one, will be back with engaging content and another great guest. Amazing Thanks. Thank you. Thanks for joining us this week on Shifting Privacy Left. Make sure to visit our website, ShiftingPrivacyLeft. com, where you can subscribe to updates so you'll never miss a show. While you're at it, if you found this episode valuable, go ahead and share it with a friend. And, if you're an engineer who cares passionately about privacy, check out Privado: the developer friendly privacy platform and sponsor of the show. To learn more, go to provado. ai. Be sure to tune in next Tuesday for a new episode. Bye for now.