Frans Willems will give the 2026 Shannon Lecture The Context-Tree Weighting Algorithm as a Language Modeler.
The ISIT 2026 plenary speakers are:
- Wen Tong, Huawei Wireless: On the Capacity Scaling in Wireless Communications-Past and Future 30 Years
- Tara Javidi, University of California San Diego, USA: From Source Coding to Active World Models: Information Theory of Physical Intelligence
- Emanuele Viterbo, Monash University, Australia: Nanopores, Bases, and Bits: An Information Theorist's Guide to DNA Storage
- Giuseppe Caire, Technical University of Berlin, Germany: The Information Theory of ISAC
Shannon Lecture
Frans Willems (Technische Universiteit Eindhoven, The Netherlands)
Date: Thursday (July 2)
Title: The Context-Tree Weighting Algorithm as a Language Modeler
Abstract: First an introduction is given to the Context-Tree Weighting algorithm [W., Shtarkov, and Tjalkens (1995)], focussing on binary sources. Then we show how the algorithm can be adapted to text-sources. We use the complete works of E.A. Poe, J. Austen, and M. Twain as corpusses. We discuss the Huffman Decomposition (see Volf [2002]), and investigate what compression it achieves on the corpusses. Then we use Context-Tree Maximizing to extract models from the corpusses. The storage of these models will be considered in the lecture. The generation of text following a prompt by a model will be demonstrated. Although good compression can be achieved with letters, we can also scale up from letters to tokens. The effect on the overall compression (that includes tree-model and parameter descriptions) turns out to be small, but we observed that the extracted token-model achieves a significantly better compression than a letter-model. We finish the lecture by discussing the computational complexity of the weighting and maximizing method. The major drawback of the context-tree weighting/maximizing algorithm is its small context-length. Therefore it does not produce what we call a large language model (LLM) now.
Bio: Frans M. J. Willems (Life Fellow, IEEE) was born in Stein, The Netherlands, in 1954. He received the M.Sc. degree in electrical engineering from the Technische Universiteit Eindhoven, Eindhoven, The Netherlands, and the Ph.D. degree from the Katholieke Universiteit Leuven, Leuven, Belgium, in 1979 and 1982, respectively. From 1979 to 1982, he was a Research Assistant at the Katholieke Universiteit Leuven. In 1982, he joined the Electrical Engineering Department, Technische Universiteit Eindhoven, where he is currently a Full Professor. He has contributed more than two hundred journal and conference papers and holds several patents. His research contributions are in the areas of multi-user information theory, noiseless source coding, data-embedding, and biometrics. From 1999 to 2016, he was an advisor for Philips Research Laboratories, Eindhoven, for topics related to information theory.
From 1998 to 2000, he was a member of the Board of Governors of the IEEE Information Theory Society. From 2014 to 2015, he was a Distinguished Lecturer of the IEEE Information Theory Society. He was a Counselor of the IEEE Student Branch Eindhoven from 2007 to 2014 and was the Chairman of the IEEE Benelux Chapter on Information Theory from 2007 to 2017. More recently, he received a 2011 Best Paper Award from the IEEE Signal Processing Society for the paper “Biometric Systems: Privacy and Security Aspects.” He received the Marconi Young Scientist Award in 1982. He is a co-recipient of the 1996 IEEE Information Theory Society Paper Award for a paper in which the Context-Tree Weighting Algorithm was proposed. From 1988 to 1990, he has served as an Associate Editor for Shannon Theory for the IEEE Transactions on Information Theory. From 2002 to 2006, he was Associate Editor for the Information Theory for the European Transactions on Telecommunications.
Plenary Talks
Wen Tong (Huawei Wireless)
Date: Monday (June 29)
Title: On the Capacity Scaling in Wireless Communications-Past and Future 30 Years
Abstract: In this talk, we investigate the capacity scaling for wireless communications. The breakthrough of MIMO technology, 30 years ago, enabled both theoretical and practical success for scaling wireless communications capacity. Starting from 3G to 4G and from 4G to 5G, each generation design has achieved 10X gains in spectral efficiency, particularly in downlink multi-user access scenarios.
The classical MIMO framework is constructed under the assumption of modeling the radio propagation as an isotropic and homogeneous stochastic process, a valid approximation for sub-6GHz spectrum deployment. However, as we explore the centimeter-wave spectrum, due to 10X shorter wavelength and 10X wider bandwidth and the hardening-effect of radio propagation, the channel model became more site-specific, sparse, and directionally structured; as a result, the channel is better represented by a quasi-deterministic wireless-twin. Under these heterogeneous and highly non-unformal directional conditions, the classical MIMO capacity scaling with the conventional Fourier-domain MIMO precoding becomes saturated.
In this talk, we propose a Sobolev-space framework to achieve 10X MIMO capacity increase. We use integrated sensing and communications (ISAC) to construct a quasi-deterministic wireless-twin. First, we allocate Johnson–Lindenstrauss-density MIMO pilots in a pivot space, to reduce the scalability barrier of the channel-side-information (CSI) based MIMO channel estimation. Second, we employ Sobolev non-Gaussian reproducing kernels to optimize high-resolution anisotropic spatial dimensions, enabling capacity scaling by increasing the dimensionality of MIMO transmit/receive antenna arrays. Real-world field trial results will be presented as well.
Bio: Wen Tong is the CTO, Huawei Wireless, he is the chief scientist for Huawei 5G/6G. He is a Huawei Fellow and an IEEE Fellow. Prior to joining Huawei in 2009, Dr. Tong was the Nortel Fellow and head of the Network Technology Labs at Nortel. He joined the Wireless Technology Labs at Bell Northern Research in 1995 in Canada. For the past three decades, he had pioneered fundamental technologies from 1G to 6G wireless and WiFi. His current research focus is AI-Wireless. He is a Fellow of Canadian Academy of Engineering, and a Fellow Royal Society of Canada.
Tara Javidi (University of California San Diego, USA)
Date: Tuesday (June 30)
Title: From Source Coding to Active World Models: Information Theory of Physical Intelligence
Abstract: The fundamental problem of physical intelligence is that of reproducing either exactly or approximately the physical reality — in time and space — from sensor readings, samples, and images that are subject to the stochastic laws of physics. Viewed in this light, physical intelligence is at its core an information theoretic problem, one where Shannon's original framework — source, channel, redundancy, and rate-distortion — proves not only relevant but foundational.
In this talk, we trace a path from classical source coding through active hypothesis testing and Bayesian optimization, all the way to active inference, active learning, and generative world modeling, arriving at a unifying framework that naturally integrates multiple modalities from multiple sensors. Furthermore, we present recent theoretical and algorithmic results showing that this information theoretic perspective unlocks dramatic gains in efficiency, robustness, and generalization for 3D perception, novel view synthesis, video compression, and active world models.
In the last part of the talk, we discuss various models for measurement and data acquisition processes — scenarios where the observer need not passively wait for data but can actively decide where, when, and what to sense. Finally, we show that machine curiosity, far from being a metaphor, can be precisely defined and quantified.
Bio: Tara Javidi is the inaugural Jerzy (George) Lewak Chair and a Professor of Electrical and Computer Engineering with a joint appointment in Halicioglu Data Science Institute at the University of California, San Diego (UCSD). Dr. Javidi received her BS in electrical engineering at Sharif University of Technology, Tehran, Iran. She received her MS and Ph.D. in electrical engineering and computer science from the University of Michigan, Ann Arbor, in 1999 and 2002 under a Barbour Graduate Fellowship. Dr. Javidi also has an MS in applied mathematics from the University of Michigan. She is an IEEE Fellow and the recipient of the following awards: 2021 IEEE Communications Society & Information Theory Society Joint Paper Award, the Qualcomm Faculty award (twice), the 2021 University of Michigan ECE’s Distinguished Educator Award , the 2020 Halicioglu Data Science Institute Fellowship and a US National Science Foundation (NSF) CAREER Award. Dr. Javidi is the founding CSO/CTO of KavAI, a startup that provides active physical sensing, data, and intelligence for large-scale industrial applications. She is founding co-director of the UCSD Center for Machine-Intelligence, Computing and Security, and a co-PI of the NSF Institute for Learning-enabled Optimization at Scale (TILOS). Dr. Javidi has served two terms on the IEEE Information Theory Society’s Board of Governors and was the Editor in Chief of IEEE Journal on Selected Areas in Information Theory. She has been a Distinguished Lecturer for the IEEE Information Theory and Communications Societies. Dr. Javidi has served as the Chair and Vice-Chair of the UCSD division of the University of California Academic Senate. She has received the 2021University of California Academic Council Chairs Award for Mid-Career Leadership.
Dr. Javidi is a faculty member of the Centers of Information Theory and Applications (ITA), Wireless Communications (CWC), Contextual Robotics Institute (CRI) and Networked Systems (CNS). She is also a founding faculty member of Halicioglu Data Science Institute (HDSI)and an affiliated faculty member in the departments of Computer Science and Engineering as well as Ethnic Studies. She helped the UCSD Jacobs School of Engineering establish the IDEA Engineering Student Center, served on the Jacobs School of Engineering Dean's Diversity Advisory Council, chaired the UCSD Faculty Senate Committee on Equity and Diversity and was awarded a 2015 UCSD Equal Opportunity/Affirmative Action and Diversity Award. Her research is in active machine learning, feedback and network information theory, and stochastic control and optimization with applications in the design of wireless and multi-agent networks.
Emanuele Viterbo (Monash University, Australia)
Date: Wednesday (July 1)
Title: Nanopores, Bases, and Bits: An Information Theorist's Guide to DNA Storage
Abstract: DNA has emerged as one of the most promising media for long-term digital information storage, offering extraordinary density, durability, and the capacity to preserve data across centuries. As the global volume of digital information continues to expand at an unprecedented rate, DNA storage presents a compelling and timely solution — one that sits squarely at the intersection of molecular biology, bioinformatics, and information theory.
This talk presents an information-theoretic treatment of the DNA storage pipeline, from encoding digital data into synthetic DNA sequences to retrieval via sequencing technologies. Central to this discussion is nanopore sequencing — a rapidly evolving platform that reads DNA directly from ionic current signals and gives rise to a rich class of channel modelling and coding problems that are largely open. We examine the fundamental limits of this channel, the inference challenges it poses, and the coding strategies that can bring its performance closer to capacity.
Beyond the technical foundations, this talk will highlight the most compelling open problems in the field — in channel modelling, sequence reconstruction, and end-to-end system design — and argue that the information theory community is uniquely positioned to make decisive contributions to their resolution.
Bio: Emanuele Viterbo is a Professor in the Department of Electrical and Computer Systems Engineering at Monash University, where he leads the Software Defined Telecommunications Laboratory. He received his Laurea and Ph.D. degrees in Electrical Engineering from the Politecnico di Torino, Italy.
Prior to his academic career, he worked as a patent examiner at the European Patent Office in The Hague, Netherlands (1990-1992), specializing in dynamic recording and error-control coding. This experience provided him with unique insights into the practical applications and intellectual property landscape of coding technologies.
Professor Viterbo's research focuses on coding and information theory with applications to modern communication systems. His contributions span lattice codes for wireless channels, algebraic space-time coding, and advanced coding techniques for emerging technologies. More recently, his work has expanded to include information-theoretic approaches to DNA data storage, where he applies coding theory principles to address the unique challenges of storing and retrieving digital information in DNA molecules. His pioneering work in wireless communications has established him as a leading figure in the field, with research that bridges theoretical foundations and practical implementations.
He was recognized as an ISI Highly Cited Researcher in 2009, placing his work in the top 1% by citations in his field. In 2011, he was elevated to IEEE Fellow for his contributions to coding and decoding for wireless digital communications. Professor Viterbo has received numerous prestigious fellowships, including the NATO Advanced Fellowship, the Australia-India Fellowship from the Australian Academy of Science, and an Invitation Fellowship for Research in Japan from the Japan Society for the Promotion of Science
Professor Viterbo's expertise in "codes" extends beyond telecommunications. In 1997, he successfully deciphered a unique 250-symbol cipher created by his great-great uncle, an amateur Egyptologist, who had used it to write a 355-page memoir of late 19th-century life. After nearly a century of failed attempts by family members and experts, Professor Viterbo spent three months cracking the code and translating the manuscript
Professor Viterbo has served the IEEE Information Theory Society in multiple leadership roles, including as a member of the Board of Governors, Conference Committee Chair, and General Co-chair of the 2021 IEEE International Symposium on Information Theory. He has also served as Associate Editor for the IEEE Transactions on Information Theory and other leading journals in the field.
Giuseppe Caire (Technical University of Berlin, Germany)
Date: Friday (July 3)
Title: The Information Theory of ISAC
Abstract: In the forthcoming 6th generation (6G) of wireless systems, integrated sensing and communications (ISAC) is expected to be one of the major technology innovation. The motivation for ISAC is compelling: wireless networks are the most pervasive systems in human history, in terms of connected people and geographic coverage. Endowing wireless networks with sensing capability will allow the systematic collection of real-world quasi-real time massive data, which will serve to train the next generation of “Physical AI”. In this talk, I will review a number of information theoretic models and results for ISAC that have appeared in the literature in the past few years. In some cases, the optimal tradeoff between communication rate and estimation error can be precisely characterized. In other cases, the problem is more difficult and only inner and outer bounds have been provided. Then, I diverge from the straightforward (yet sometimes challenging) strict information theoretic models and discuss a broader class of problems for which even the problem formulation itself is not so clear, with the purpose of stimulating new research ideas for the information theory community.
Bio: Giuseppe Caire (S '92 -- M '94 -- SM '03 -- F '05) was born in Torino in 1965. He received a B.Sc. in Electrical Engineering from Politecnico di Torino in 1990, an M.Sc. in Electrical Engineering from Princeton University in 1992, and a Ph.D. from Politecnico di Torino in 1994. He has been a post-doctoral research fellow with the European Space Agency (ESTEC, Noordwijk, The Netherlands) in 1994-1995, Assistant Professor in Telecommunications at the Politecnico di Torino, Associate Professor at the University of Parma, Italy, Professor with the Department of Mobile Communications at the Eurecom Institute, Sophia-Antipolis, France, a Professor of Electrical Engineering with the Viterbi School of Engineering, University of Southern California, Los Angeles, and he is currently an Alexander von Humboldt Professor with the Faculty of Electrical Engineering and Computer Science at the Technical University of Berlin, Germany.
He received the Jack Neubauer Best System Paper Award from the IEEE Vehicular Technology Society in 2003, the IEEE Communications Society and Information Theory Society Joint Paper Award in 2004, in 2011, and in 2025, the Okawa Research Award in 2006, the Alexander von Humboldt Professorship in 2014, the Vodafone Innovation Prize in 2015, an ERC Advanced Grant in 2018, the Leonard G. Abraham Prize for best IEEE JSAC paper in 2019, the IEEE Communications Society Edwin Howard Armstrong Achievement Award in 2020, the 2021 Leibniz Prize of the German National Science Foundation (DFG), and the CTTC Technical Achievement Award of the IEEE Communications Society in 2023. Giuseppe Caire is a Fellow of IEEE since 2005. He has served in the Board of Governors of the IEEE Information Theory Society from 2004 to 2007, and as officer from 2008 to 2013. He was President of the IEEE Information Theory Society in 2011. His main research interests are in the field of communications theory, information theory, channel and source coding with particular focus on wireless communications.