Podcast: Play in new window | Download
Subscribe: Apple Podcasts | Spotify | Android | TuneIn | RSS
Jeffrey MacIntyreScalable content personalization systems create huge value for businesses. Like most valuable endeavors, they’re really hard to do well.
Jeffrey MacIntyre orchestrates the activities – terminology and taxonomy work, metadata strategy, information architecture, and more – that help businesses build content operations that deliver the customer-focused experiences that consumers expect.
We talked about:
- two trends he sees that are driving content strategy for customer experience: product thinking and the need for better IA in large enterprises
- the opportunity that better IA affords to build a competitive moat around your business
- how structured, semantically meaningful content and enriched metadata enable better personalization
- his take on the “personalization gap”
- his prediction that something like the Hippocratic oath will emerge in the near future to address ethical issues around AI
- the importance of a cross-functionally and collaboratively generated controlled vocabulary
- his advocacy for up-skilling content design professionals with better taxonomy and metadata skills
- how “cowboy taxonomy” work fits in his practice
- his assertion that the best way to advance collaboration is a robust test and learn program
- how information structure provides a backstop for AI
- a shout-out to Marcia Bates for her insights about “berry picking”
- how good metadata can prevent the accumulation of “experience debt”
- the benefits of Leidy Klotz’s notion of “subtraction”
Jeffrey’s bio
A personalization optimist and information retrieval obsessive, Jeffrey MacIntyre is an independent consultant focused on “shovel-ready” solutions to personalized, automated, and simplified customer experiences. He writes Bucket List, a newsletter of tales from the trenches of his consultancy, Bucket Studio. He speaks widely on the IA for AI — the role of information science in shaping connected experiences — and runs Bucket Brigade, the only community for those who design such experiences.
Connect with Jeffrey online
Video
Here’s the video version of our conversation:
Podcast intro transcript
This is the Content Strategy Insights podcast, episode number 189. Crafting the personalized content experiences that consumers expect nowadays is not an easy job. To assemble those made-to-order interactions, you need to first align your internal teams on the terminology you use and organize it around a clear metadata strategy, and then you need to structure and categorize your content so that you can create engaging experiences at scale. Jeffrey MacIntyre specializes in orchestrating content operations like these.
Interview transcript
Larry:
Hi, everyone. Welcome to episode number 189 of the Content Strategy Insights Podcast. I am really happy today to welcome to the show Jeffrey MacIntyre. Jeffrey is an independent personalization consultant. He’s well known in the industry as “the personalization optimist.” He’s the principal and founder at a studio, Bucket Studio, which is his agency. He also runs the Bucket Brigade community, which we might talk about a little bit in here. He’s also a founding member of the Consortium for Personalization Professionals. So you’re catching the theme of personalization here, but tell the folks a little bit more about what’s going on these days, Jeffrey.
Jeffrey:
Yeah, absolutely, and thanks, Larry, for having me on – a big fan of the pod. So what’s happening is we’ve got two interesting developments in my mind. We have the ascendancy and design circles of product thinking, so a lot of people wanting to really skillfully design and deliver really discreet user flows within a customer journey and do growth hacking along it and really measure and understand success, sense, and respond. That’s trend one. And then trend two, particularly in the enterprise, but not alone, we have very large sprawling sites that are ripe for rationalization, particularly now with AI whereby people are taking stock right now of, how do I want to steward this gargantuan site of mine and what would an essential customer experience? If I took it down to the bones, what would this house look like? And that’s where IA really comes in. Obviously, we know AI is going to be a delivery mechanism for this, but we need to structure the data, and that’s why sometimes I talk in bumper stickers about IA before AI.
Larry:
I would love to elaborate on that a little bit because I think I get what you mean, but I want to hear your take on why is information architecture crucial to AI?
Jeffrey:
I think that there’s a few reasons. Obviously, the one that’s well known in the practitioner community is the rather large and clear present danger or risk to your brand and to your legal office and legal counsel around hallucinations and other mis-firings or over fittings that AI can produce. I think that this is well known and well diagnosed, so I don’t want to spend too much time on it. Let’s talk about the upside of structuring all of the things. The upside is that if you are a company or a brand that’s been producing content over a long period of time, you actually have a new moat. If you can rationalize your site and make it digestible and machine consumable enough, you can have a moat around your products and places. Because we’re looking at the real time evolution of information retrieval. It’s happening for us right now where we’re seeing search alongside a co-pilot UI alongside navigation.
Jeffrey:
And I think the Walmart iOS app is a great example of this where all these things are coming together and it’s not too hard to see in five years time that you’re probably going to go down to just a conversational input query, query and response mechanism for traversing large sites and products and experiences. Now, where do the semantics come in? The semantics come in where you want that AI to traverse your information product and data successfully and effectively, maybe consistently. I put an asterisk on that, but nevertheless, you want to be able to baseline what quality is. And as a personalization consultant, I feel like I’ve got a little bit of a competitive advantage in that we’ve been working under existing legislation for years, GDPR being only among the better known pieces of that legislation, which requires data destructibility, requires traceability, requires working backwards from algorithmically generated decisions so that I can pull my data out of this service if I’m a California resident or if I’m a European resident.
Jeffrey:
And those rights and those legal standards are going to follow in AI. So I think that there’s a lot of upside both from a brand perspective of having that competitive moat, because imagine you’ve got a co-pilot on your site that knows your content better than Google, better than Bing. This is a very different way of thinking about the power of your content when it’s effectively structured.
Larry:
And can you talk a little bit, just dive a little bit deeper on that? Because I get what you’re saying. Because you’re going to have this unique corpus that you’re drawing from, whereas the big search engine, they’re looking at the whole internet. You have this unique focus on your own thing, plus you know what you want to do. It’s like content strategy. You know who you are and what you’re about, what you have to say and have crafted that. Tell me about how the semantic tooling can amplify that and enable it.
Jeffrey:
I think that there’s just the existing structuring of content that you want the AI to respect and acknowledge so that it’s not pulling in false positives. If you query, “I want to see all the white papers,” which is a broad and unrealistic query, “I want to see all the white papers on computing,” or, “I want to see all the white papers on cloud computing in the last quarter about Kubernetes,” now you’re starting to get narrower, narrower, narrower. That result really should be using essentially TF-IDF or nearest neighbor or some very sophisticated algorithms so that we’re pulling really relevant results. But if we’re also dealing with an ontology, or if we got a knowledge graph or a knowledge base in the backdrop, even just we have a taxonomy, an effective working taxonomy or polyhierarchy, we know that results that are put in these branches are relevant and others are going to be of diminished or lesser importance.
Jeffrey:
So that’s just the ranking and stuff, because, again, with AI, a lot of the queries are coming down to, how can you synthesize knowledge for me very quickly? And we run into these problems of synthetic data and misleading results. But it doesn’t change the human craving for, hey, can you just summarize a whole bunch of stuff for me and give me the one true answer? That’s going to be a garbage in, garbage out thing, but it’s not going to stop us humans from doing that. And so we’ve got to use structure to reinforce and have those guardrails.
Larry:
As you say that I’m reminded that computers behave differently now than we’re used to them behaving, this hallucination and creativity on the part of computers. They’ve always been reliable computing machines just doing a lot of fancy stuff. And back to what you said about semantics, it sounds like semantics is the thing that can add to the number-crunch-ey part of these things, which the LLMs are just predicting what people are going to say. And you mentioned ontologies and knowledge graphs, even a simple knowledge base with a taxonomy so you’re ascribing with that metadata, you’re ascribing characteristics to this content that you want to make sure people get. Stitch that together for me how content people can use that to amplify this treasure trove that they’re hiding behind that they now have behind a moat.
Jeffrey:
Yeah, and I think a lot of work that I’ve been pulled into involves metadata enrichment. Now, why would somebody want to enrich their metadata? Because they’re trying to go live with a feature or a function that’s going to allow you to have a more nuanced information retrieval experience. And so topping up metadata, you might have a bunch of implicit metadata in terms of date stamp, like last published on, last updated, that stuff that you might not even be expressing on the glass, on the presentation layer, but it might be vitally important that the AIs not bring up results prior to X date or they may not be able to bring results post another date because your training model hasn’t been trained on the new content in your corpus. All of these things are going to be very important guardrails.
Jeffrey:
I often talk about bowling with my kids, and when your kids are younger, you bowl. In American bowling alleys, there’s these bumpers you can put up. I don’t know if this is an international convention or if this is just Yankee sloth at work, but I do know that they have a good time ricocheting balls around. The real world involves gutters, and we as information management professionals and content professionals know that we need to get good quality results because regardless of how good the content is, it’s the experience that people will be judging that content on.
Larry:
Interesting. And so the content, obviously, contributes to that experience, but it’s the ability to build that experience that addresses the customer’s needs. That’s important. I love the metaphor of the bowling alley because another thing that you’re known for is the perso-fails, personalization fails hashtag, and that’s the gutter. That’s when you’re operating without the bumpers. That’s where a lot of your personalization efforts will end up.
Jeffrey:
And one of the things, and I warned you that I talk in bumper stickers, consultants, we’re terrible at this, or great, pick your poison, there’s a personalization gap. There is a lot of received wisdom out there of practitioners who have tried and failed, organizations that have invested and spun out, lots of technologies and vendors that promised a blue sky and delivered nothing but pain. And so there is a lot of “conventional wisdom” that this is too hard or not worth it, or we don’t have the talent or the budget or the gumption to get this done. I call that the personalization gap. For me, identifying perso-fails and I very specifically focus on Darling’s poster children of automation and personalization like Amazon, like Spotify, like Netflix, is, if these guys can get it wrong, it’s okay at the level that you know are going to have some failures, you’re going to have what I call progressive personalization, that gradation of maturity.
Jeffrey:
And the instructive part of the perso-fails is we talk about them and we say, “Why is this person getting re-targeted for toilet paper after buying toilet paper?” Well, the reductio ad absurdum is because they can. It’s like notifications in the old days. It was just like, let’s bombard the user because we can, and then the user feedback gradually starts to come back first in the B2C products, but also in B2B experiences that this is overwhelmed and the users are turning off or tuning it out or even uninstalling applications and products because of that predilection. Now that we’re in this second or third wave around personalization practice, it’s important to recognize that you don’t just re-target people on anything that they see on the internet.
Jeffrey:
And particularly, that toilet paper issue to me comes down to metadata. If we have a replenishment cycle for a household perishable, you darn better believe it’s going to be different than a household durable, which is what a toilet seat would be. If you’re targeting them on toilet tissue paper, that makes more sense than a toilet seat, which is exactly what the Amazon example that I talk about is. So it’s little things like this that I think they can be instructive if we work backwards and try and think, how did this happen or why are people resorting to this hack so that their recommendations don’t get contaminated on the Amazon or the Spotify family account? Those kinds of things I think can lessen the gap and help us understand ways that we can start thinking of syntax that would make sense for us to automate interactions in our own product or place.
Larry:
As you talk about that, I’m reminded of – like the replenishing the toilet paper. I guarantee you there’s 10 startups in Silicon Valley right now developing IoT toilet paper dispensers that have built-in shopping e-commerce capabilities, but there’s probably other information you could discern and turn into metadata in your CRM or something that would help you do that. Is that correct?
Jeffrey:
Yes. Well, obviously, there’s inferences. You can sample from all kinds of data inputs to run a decision. You can black box those decisions with machine learning. But again, I tend to be a pretty loud and obnoxious voice in the room that I feel design professionals will have a Hippocratic oath in five to 10 years. We will have “do no harm” vows like doctors do, because there’s going to be a level of ethical regard that’s important in the work of digital products in places that is going to be incredibly important and the law will be heavily involved. People’s personal data will be more valuable tomorrow than it is today. And so I think that there’s going to be a real premium on being able to have explainable AI, have traceability, observability. And the way we get to this is understanding the syntax of what makes the decision.
Jeffrey:
Now, in terms of your question, what can we pull in terms of data, that depends on the software you’re talking about. For myself, I work with a lot of companies that have personalization engines, and that’s a suite of software that does recommendations and experience optimization on the front end, and it might do notifications or other sorts of things. Lots of organizations roll their own versions of a notifications tool or other microservices. It doesn’t really matter at the end of the day. What does matter is that they start to distill down consistent practices so that their users can actually habituate to some level of trust that, okay, this is what HubSpot is going to nurture me on because they think that I’m in this industry and I’m this professional with these kinds of needs. Great, or not so great. But this is where we get into the perso-fails.
Jeffrey:
You will have a different regard for that brand if they misfire on you. And so to me, the card game that my good colleague, Colin A. Eagan, and myself have come up with is really around, what would it look like if we take this down to the bare bones and say, “What are the nouns and verbs in every automated or personalized interaction you have all the features you want to build?” And then bring all of the different departments, the functional groups, marketing, product, engineering, everyone together around the same table and have that controlled vocabulary in the backdrop. You’re not coming forward and making it a purity test, but you are saying, “Well, a cohort is this, it’s not a segment, and we’ve got a card here or we’ve drawn up our own cards.” And I think that that’s where we’re at, Larry, is we do need to get back to some first principles about literally the syntax of what we’re doing.
Larry:
And as you’re saying that, that gets back to the structure. There’s both the semantic, the meaning of what you’re working with, that’s an important part of it, but that, as you sort out the nouns and verbs and then start to connect them all together, that gets into your domain model and your content model that drive this. And especially because this is content strategy podcast, I’m particularly curious about how you think that content strategists and content designers can stay attuned to this and influence better decision making that contributes to building that trust that’s so necessary to this.
Jeffrey:
You can’t tell, but I’m actually on a milk carton right now about to tell you that there needs to be up-skilling for content design professionals. I have a nascent training program I call Just Enough Taxonomy in all due homage to Erika and Just Enough Research, phenomenal book, and it really is a boot camp about, what are the brass knuckles IA skills that a content designer can benefit from having and working in patterned motifs, working with greater facility and comfort around vocabulary management and other things that we know really earns a seat at the table with product or with engineering because they see the value? They’re like, “Oh, that’s like a regex, right?” And you’re like, “Yeah, yeah, that’s exactly it. We’re white listing and blacklisting,” to use bad terminology that I shouldn’t use.
Jeffrey:
But nevertheless, I think your content professional needs to know more about information management and your information management professional needs to know more deeper semantics, arguably, and I think we can get there and we can expand the impact that both of these domains have across content discovery. That’s the big opportunity that I see. Search, navigation, recommendations, notifications. Everything about content discovery is going to be driven by metadata. It already is.
Larry:
Right. And as you say that, I’m reminded back to just to revisit just what you said a minute ago about, yeah, that’s just like regex. We most recently saw each other at the Information Architecture Conference a couple of weeks ago, and I did a workshop there with Ren Pope on ontological thinking, and one of the things he showed in there was this brilliant, he has this great for that cross-functional team alignment that you’re talking about, he has just a table that shows like, “Okay, here’s the concept we’re talking about. Here’s what they call it in design. Here’s what they call it in engineering. Here’s what they call it over there.” Do you have tricks or tips or tools like that that you use?
Jeffrey:
Yes. Literally, I have done this work and created synonym tables just like this, but then in actual projects there’s been sometimes multiple repositories, and I’ll link those with a crosswalk. Now, this is known as cowboy taxonomy work. This is not best practice. This is proof of concept type practice, because a crosswalk is going to be very brittle. It’s static. You need to update it. It’s not a true mapping table or a transform layer. It’s a once and done, okay, we proved we can link these two repositories. Now the executive can give us more budget to do a proper migration or normalization.
Jeffrey:
But yeah, so much of the undercarriage of the products and places we’re building depends on terminology. And the terminology doesn’t need to be one-to-one, but as Ren clearly illustrates, you do need to have at least a map so that people have some loose agreement and understanding like, oh, when they talk about that, that’s what they mean. Because ultimately this is collaborative work.
Larry:
Yeah. And apart from the language around it, part of that language reflects fundamentally different… I think about the notion in decoupled architectures of separation of concerns. I’ve been thinking about that a lot lately. And each discipline justifiably and correctly has its own concerns, but among those concerns for any of them should be collaboration. So I guess do you have many success stories about how to transcend that? That’s the next layer down it feels like in terms of getting that aligned.
Jeffrey:
Yeah. Well, I think the gateway drug to better collaboration is to develop and nurture a really robust test and learn program. Because in my experience that is often when I get the call from an organization, is when they are trying to move from, “We run dozens and dozens of tests a month,” or, “We run X amount of tests a year,” but they’re trying to actually productize or turn on always on automations and personalizations. That’s a great time for a wider array of stakeholders to be involved and invested in, this is how we’re going to actually automate this product or place.
Jeffrey:
Because having run experiments over time proves that there’s a muscle memory and we know how to do this and we have a science that we can back into, but when you’re doing proper personalization, you’re really taking a stair step into this is going to be an always on automation. Every time you register a user, they’re going to be subject to one of these three nurture flows or onboarding flows. And I think that’s the point at which you can really engage other functional groups because they will know we’re not just testing here now. We’re actually going to change the surface and texture of this product experience.
Larry:
As you say that, I’m curious now that test and learn is an inroad for your engagements as a consultant, but I’m wondering if there’s almost like a big, not T&L, but everybody’s trying stuff with AI now. Does that present more opportunities for all of us, for any content person?
Jeffrey:
I think so. And I think a lot of organizations are starting their AI or gen AI or assistive AI experiments with the employee experience. They’re starting internally to figure out what they can do, what the risks are, trade-offs, and that kind of thing. But perso-fails, I think it’s telling when we see what is hitting end consumers by way of assistive AI. And again, I would point to the Walmart iOS app as this fascinating example where they’re actually using pre-fills in their copilot like UI for their assistive artificial intelligence input. And it’s almost like the old days of the carousel, like, oh, they’re just merchandising here.
Jeffrey:
This is an IA thing. There’s a trending thing. There’s most sold in my zip code thing perhaps, and you can start to see the ways that structure is providing a backstop for AI. So they have micro copy that says, “Your results will definitely vary. You’re not going to get a canonical result when you query this,” which is great. And I think that’s fascinating too, but I think it’s going to create for end consumers more of an appetite for, what are the durable forms of information retrieval my nav, my search, and so on? And I’m not holding those up as better, but I am saying that I think people are going to want to be able to toggle pretty quickly between those two things.
Larry:
As you’re saying that, you’re reminding me, because all of our interfaces the last 20, 30 years have been gooey of some kind or another, and now all of a sudden we have these conversational interfaces, and it sounds like that’s not only providing another big option by itself, but also causing reflection of the other ways to get at this information. Does that make sense?
Jeffrey:
Yeah. I think this all goes back to berry picking. Shout out to Jenny Benevento who was my verbal inroads to Marcia Bates and berry picking. And that is something that I have tried in my wee way to update and modernize how those information retrieval behaviors are now part of the work that we do as stewards of content data and information. And it’s still the same things. It’s browse. It’s search. It’s nudge me, I’m off channel. It’s, I need to have a deep immersive experience. It’s really only a handful of interactions that we do with content data and information when you bring it down to its very basic essence.
Larry:
Now you’re reminding me too of, I first heard Jared Spool talk about this idea 20 years ago, the notion of designing for the scent of information. It sounds like that family.
Jeffrey:
A colleague of mine, the great Andy Fitz, brought this up in a project that we were involved in, and it was some distant thing in the back of my head. I was like, “Oh, yes, I remember this word.” And I had never really deployed that nomenclature before, but it makes a lot of sense not to, oh God, real dad cringe, pun there. Another thing that Jared Spool popularized is the idea of experience debt. And I think this brings us full circle. So if we know that AI and conversational interfaces will be the prevailing mode of interaction sometime, and until then there’s going to be a bevy of different interaction points, but ultimately users are going to do the things that don’t make me think.
Jeffrey:
We are going to do the things that are the simplest modes of interaction. We’re going to need that base level of metadata or you’re going to continue to have lots of experience debt in your applications, in your products and places. Because you can’t quite do the things, you can’t quite customize or personalize or automate this string of interactions in the way that you otherwise could with better data.
Larry:
Interesting. I love that idea of metadata debt. Because I think about that constantly. If it was a job, I would cross train into metadata strategist, but you don’t see that on LinkedIn very often. But hey, Jeffrey, I can’t believe it. We’re coming up close to time. Before we wrap up, is there anything last, anything you want to revisit from the conversation, or just make sure that we share before we wrap up?
Jeffrey:
It’s just like the Uncle Sam enlistment poster, which is you can do really high quality information management work in many systems of record, a marketing automation platform, a product information management tool, a personalization engine, a CMS, a DAM, a MAN, on and on, the list goes. But in the contemporary era, most customer experiences rely on a single experience that threads a series of those systems of record. Content data and information professionals are uniquely positioned to become effective stewards of that customer experience. I call these connected experiences, and I think that this is really the ticket for a lot of our colleagues to provide a lot of sanity and sense to what’s going to be an unruly future for those who don’t steward their products in places effectively.
Larry:
I love that. To be the stewards of getting us through the chaos with connected content, that’s my fondest dream.
Jeffrey:
Yeah. Can I do a quick plug?
Larry:
Sure. Yeah, of course.
Jeffrey:
It’s not even for myself. A lot of my thinking about simplification is directly indebted to a book that came out maybe two years ago called Subtract by Leidy Klotz, L-E-I-D-Y K-L-O-T-Z, or Zed as us Canadians will say. It’s a fine book. He doesn’t take up design in quite the ways I want him to, and he and I have had some fun interactions since the publication, but I think that we’re all very familiar with content overwhelm and information fatigue and those kinds of things. Subtraction is a very powerful activity if you can pull it off in design. Dieter Rams told us. “Make this thing the simplest expression it can be and nothing less.” And I think it’s up to us information management professionals to figure out, what is that Goldilocks level of detail that is going to provide as much user benefit or commercial advantage and on and on that we can?
Larry:
Great. Thanks. I’ll include links to that book in the show notes. And I’m going to order it myself as soon as we hang up. Well, thank you so much, Jeffrey. Really fun conversation.
Jeffrey:
Awesome. This has been great, Larry. And onwards and upwards.
Leave a Reply