Dacheng Tao
University of Sydney, Australia
Dacheng Tao is an Australian Laureate Fellow, a Professor of Computer Science, and Peter Nicol Russell Chair in the School of Computer Science, and an advisor and chief scientist of the digital sciences initiative and the founding director of the Sydney AI Centre in the Faculty of Engineering at The University of Sydney. His research is detailed in one monograph and over 200 publications in prestigious journals and proceedings at prominent conferences such as IEEE TPAMI, TIP, TNNLS, IJCV, JMLR, NeurIPS, ICML, ICLR, CVPR, ICCV, ECCV, AAAI, IJCAI, ICDM and KDD, with several best paper awards. He received the 2015 and 2020 Australian Eureka Prize, the 2018 IEEE ICDM Research Contributions Award, and the 2021 IEEE Computer Society McCluskey Technical Achievement Award. He is a Fellow of the Australian Academy of Science, the Royal Society of NSW, TWAS, AAAS, ACM and IEEE.
Title of Talk: More Is Different - Beyond Wittgenstein's Philosophy
Abstract: Unleashing the hidden wisdom within broad data has become a captivating pursuit for the community. Among the myriad of possibilities, one solution stands out: foundation models. These behemoth architectures, powered by transformers, possess the ability to extract and harness the enigmatic dark knowledge that resides within broad data. Parameters, computations, and data combine in a symphony of potential, demonstrating that in the world of transformers, "more is different", and reigniting our dreams for Artificial General Intelligence. In this presentation, we embark on a thrilling journey into the world of foundation models. We begin by introducing the ground-breaking LLMs ChatGPT and the wave of innovation they have set in motion. Along the way, we discuss concerns about the singularity of these techniques and offer our insights into this emerging trend. We then delve into theoretical foundations, example designs in NLP and CV, efficient decentralized optimization algorithms, and useful applications that flourish under the influence of foundation models. Yet, this adventure also highlights the challenges and opportunities that lie ahead in the era of these models. As we conclude, we do so with unwavering optimism: foundation models will play a pivotal role in shaping artificial intelligence. Join us on this remarkable expedition into the seamless integration of data, computational power and algorithms, where the future unveils itself in unprecedented ways.