Generative AI strategy, user experience, and consulting. San Francisco, California

Design and AI

Does generative AI create design opportunities or does it threaten design? Your answer to this question depends on your views of AI. The AI we know from large language models and the base models of OpenAI, Anthropic, Meta, and Google continue to make progress. Progress fast enough that base model experiences, integrations, customizations, and productizations alone are a rapidly-changing and dynamic field. Design can be applied to speech and conversation, to image and video creation and editing, to music generation, text research and summarization, tool orchestration and automation, and more. Some experiences work and others fail. The methods and techniques developed around base models are often unwieldy and produce inconsistent results, as solutions to fundamental architectures have not been settled on. When commercial data, user data, application integrations and other architectural components are added in, model behaviors and outputs become more experimental. Designing user experiences in this space clearly requires some flexibility.

Large language models "read" and generate text and documents, generating writing and documents in response. Language models also converse, using styles that are chat conversation or voice-based conversation. From an interaction desigbn persepctive, these are different modes of communication. The conventions common to speech and talk are distinct from those that shape discourse, even while both use language (and language is the foundation of LLMs).

In human social activity, "doing things with words" is already unstable. It is even moreso with generative AI. But the promise of using natural language to instruct, command, and request responses from technical systems is too enticing to leave design just to model engineering.

Conversational AI: LLMs expand the conversational capabilities and applications of AI, from open dialog to task-oriented dialog. Engaging and effective conversation are a matter for design.

Design Theory: Design needs new concepts for the plastic and adaptive nature of generative AI. Speech and language as an interaction system, real-time generation as a technical capability, and many agentic architectures and integrations require us to update design concepts if not also methods.

Domain-Specific AI: Domain-specificity is needed to make generative AI applications capable of handling the requirements of vertical commercial use cases. Design can contribute to assessing these needs within a domain and to shaping the performance of AI-based applications within it.

Emotional AI: Emotions are added to voice mode conversational AI, but in order to make AI seem more human. The use of emotions creates novel design issues and considerations.

Interestingness: In replacing search and creating experiences based more on discovery and exploration, language models can be designed to engage user interests more widely than with focused search. Design concepts for interest, over use, could help to shape these applications.

Organizational Design: AI integrated into software and organizational applications will need to be designed, from task and job workflows to front-end interfaces.

Personalization: AI will greatly expand its usefulness for end users when it is personalized to user preferences and data. Personalization is a design topic.

Prompts: Prompts are an obvious application of design but are often considered engineering. Given the impact of prompting on LLM response generation, I think design theory needs concepts specific to prompting.

I have a list of the whitepapers I have read here.