Research library

Search papers on equipment connectivity, quality inspection, production analysis, and manufacturing work.

Search and filter

Paper list

T01. LLM agent orchestration and harness design

External research on how multiple language-model agents are planned, routed, and evaluated.

Show this topic only
  1. T01-13Similar problemSince 2025

    Why Do Multi-Agent LLM Systems Fail?

    Authors
    Mert Cemri, Melissa Z. Pan, Shuyi Yang, Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, Ion Stoica
    Year
    2025
    Venue
    NeurIPS 2025 Datasets and Benchmarks Track spotlight

    Korean review of T01-13

  2. T01-14Candidate approachSince 2025

    Multi-Agent Collaboration via Evolving Orchestration

    Authors
    Yufan Dang, Chen Qian, Xueheng Luo, Jingru Fan, Zihao Xie, Ruijie Shi, Weize Chen, Cheng Yang, Xiaoyin Che, Ye Tian, Xuantang Xiong, Lei Han, Zhiyuan Liu, Maosong Sun
    Year
    2025
    Venue
    NeurIPS 2025

    Korean review of T01-14

  3. T01-19Similar problemSince 2025

    tau^2-Bench: Evaluating Conversational Agents in a Dual-Control Environment

    Authors
    Victor Barres, Honghua Dong, Soham Ray, Xujie Si, Karthik Narasimhan
    Year
    2026
    Venue
    arXiv 2025-06-09

    Korean review of T01-19

  4. T01-12Similar problemSince 2025

    tau-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains

    Authors
    Shunyu Yao, Noah Shinn, Pedram Razavi, Karthik Narasimhan
    Year
    2025
    Venue
    ICLR 2025 Poster

    Korean review of T01-12

  5. T01-1Structural parallel

    ReAct: Synergizing Reasoning and Acting in Language Models

    Authors
    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao
    Year
    2023
    Venue
    ICLR 2023

    Korean review of T01-1

  6. T01-2Structural parallel

    Toolformer: Language Models Can Teach Themselves to Use Tools

    Authors
    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, Thomas Scialom
    Year
    2023
    Venue
    NeurIPS 2023

    Korean review of T01-2

  7. T01-3Structural parallel

    HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face

    Authors
    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, Yueting Zhuang
    Year
    2023
    Venue
    NeurIPS 2023

    Korean review of T01-3

  8. T01-4Candidate approach

    Generative Agents: Interactive Simulacra of Human Behavior

    Authors
    Joon Sung Park, Joseph C. O'Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, Michael S. Bernstein
    Year
    2023
    Venue
    UIST 2023

    Korean review of T01-4

  9. T01-5Candidate approach

    Reflexion: Language Agents with Verbal Reinforcement Learning

    Authors
    Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao
    Year
    2023
    Venue
    NeurIPS 2023

    Korean review of T01-5

  10. T01-6Candidate approach

    CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society

    Authors
    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, Bernard Ghanem
    Year
    2023
    Venue
    NeurIPS 2023

    Korean review of T01-6

  11. T01-7Similar problem

    WebArena: A Realistic Web Environment for Building Autonomous Agents

    Authors
    Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, Graham Neubig
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T01-7

  12. T01-8Similar problem

    SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

    Authors
    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T01-8

  13. T01-9Structural parallel

    AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversations

    Authors
    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W. White, Doug Burger, Chi Wang
    Year
    2024
    Venue
    COLM 2024

    Korean review of T01-9

  14. T01-10Structural parallel

    MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework

    Authors
    Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, Jürgen Schmidhuber
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T01-10

  15. T01-11Structural parallel

    SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

    Authors
    John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press
    Year
    2024
    Venue
    NeurIPS 2024

    Korean review of T01-11

  16. T01-15Structural parallel

    Tree of Thoughts: Deliberate Problem Solving with Large Language Models

    Authors
    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, Karthik Narasimhan
    Year
    2023
    Venue
    NeurIPS 2023

    Korean review of T01-15

  17. T01-16Similar problem

    AgentBench: Evaluating LLMs as Agents

    Authors
    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, Jie Tang
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T01-16

  18. T01-17Similar problem

    GAIA: a benchmark for General AI Assistants

    Authors
    Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, Thomas Scialom
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T01-17

  19. T01-18Similar problem

    OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

    Authors
    Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, Tao Yu
    Year
    2024
    Venue
    NeurIPS 2024

    Korean review of T01-18

T02. Recursive language models and long-context handling

Research on reading documents that do not fit in one context window.

Show this topic only
  1. T02-1Structural parallelSince 2025

    Recursive Language Models

    Authors
    Alex L. Zhang, Tim Kraska, Omar Khattab
    Year
    2026
    Venue
    arXiv (MIT CSAIL)

    Korean review of T02-1

  2. T02-2Structural parallel

    RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval

    Authors
    Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, Christopher D. Manning
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T02-2

  3. T02-3Similar problem

    Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading (MemWalker)

    Authors
    Howard Chen, Ramakanth Pasunuru, Jason Weston, Asli Celikyilmaz
    Year
    2023
    Venue
    arXiv

    Korean review of T02-3

  4. T02-4Candidate approach

    MemGPT: Towards LLMs as Operating Systems

    Authors
    Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica, Joseph E. Gonzalez
    Year
    2023
    Venue
    arXiv (UC Berkeley)

    Korean review of T02-4

  5. T02-5Structural parallel

    Chain of Agents: Large Language Models Collaborating on Long-Context Tasks

    Authors
    Yusen Zhang, Ruoxi Sun, Yanfei Chen, Tomas Pfister, Rui Zhang, Sercan O. Arik
    Year
    2024
    Venue
    NeurIPS 2024

    Korean review of T02-5

  6. T02-6Similar problem

    Lost in the Middle: How Language Models Use Long Contexts

    Authors
    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang
    Year
    2023
    Venue
    Transactions of the ACL (TACL)

    Korean review of T02-6

  7. T02-7Similar problem

    RULER: What's the Real Context Size of Your Long-Context Language Models?

    Authors
    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, Boris Ginsburg
    Year
    2024
    Venue
    COLM 2024

    Korean review of T02-7

  8. T02-8Candidate approach

    LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression

    Authors
    Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu
    Year
    2024
    Venue
    ACL 2024

    Korean review of T02-8

  9. T02-9Structural parallel

    Recursively Summarizing Books with Human Feedback

    Authors
    Jeff Wu, Long Ouyang, Daniel M. Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike, Paul Christiano
    Year
    2021
    Venue
    arXiv (OpenAI)

    Korean review of T02-9

  10. T02-10Candidate approach

    Efficient Streaming Language Models with Attention Sinks (StreamingLLM)

    Authors
    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, Mike Lewis
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T02-10

  11. T02-11Similar problem

    LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

    Authors
    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li
    Year
    2024
    Venue
    ACL 2024

    Korean review of T02-11

  12. T02-12Candidate approach

    Ring Attention with Blockwise Transformers for Near-Infinite Context

    Authors
    Hao Liu, Matei Zaharia, Pieter Abbeel
    Year
    2023
    Venue
    arXiv (UC Berkeley)

    Korean review of T02-12

  13. T02-13Candidate approach

    Extending Context Window of Large Language Models via Positional Interpolation

    Authors
    Shouyuan Chen, Sherman Wong, Liangjian Chen, Yuandong Tian
    Year
    2023
    Venue
    arXiv (Meta)

    Korean review of T02-13

  14. T02-14Candidate approach

    Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

    Authors
    Tsendsuren Munkhdalai, Manaal Faruqui, Siddharth Gopal
    Year
    2024
    Venue
    arXiv (Google)

    Korean review of T02-14

  15. T02-15Candidate approach

    H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models

    Authors
    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Re, Clark Barrett, Zhangyang Wang, Beidi Chen
    Year
    2023
    Venue
    NeurIPS 2023

    Korean review of T02-15

T03. Tool use, function calling, and the Model Context Protocol

External research and specifications for letting a model call outside tools.

Show this topic only
  1. T03-13Similar problemSince 2025

    Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions

    Authors
    Xinyi Hou, Yanjie Zhao, Shenao Wang, Haoyu Wang
    Year
    2026
    Venue
    ACM Transactions on Software Engineering and Methodology (TOSEM), 3796519

    Korean review of T03-13

  2. T03-12Similar problemSince 2025

    The Berkeley Function Calling Leaderboard (BFCL): From Tool Use to Agentic Evaluation of Large Language Models

    Authors
    Shishir G. Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, Joseph E. Gonzalez
    Year
    2025
    Venue
    ICML 2025

    Korean review of T03-12

  3. T03-9Similar problemSince 2025

    Tool Learning with Foundation Models

    Authors
    Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding and 41 others in total (corresponding: Zhiyuan Liu, Maosong Sun)
    Year
    2025
    Venue
    ACM Computing Surveys vol. 57 no. 4, 101, pp. 40

    Korean review of T03-9

  4. T03-1Structural parallel

    WebGPT: Browser-assisted question-answering with human feedback

    Authors
    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, John Schulman
    Year
    2022

    Korean review of T03-1

  5. T03-2Structural parallel

    MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning

    Authors
    Ehud Karpas, Omri Abend, Yonatan Belinkov, Barak Lenz, Opher Lieber, Nir Ratner, Yoav Shoham, Hofit Bata, Yoav Levine, Kevin Leyton-Brown, Dor Muhlgay, Noam Rozen, Erez Schwartz, Gal Shachaf, Shai Shalev-Shwartz, Amnon Shashua, Moshe Tenenholtz
    Year
    2022

    Korean review of T03-2

  6. T03-6Similar problem

    API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs

    Authors
    Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, Yongbin Li
    Year
    2023
    Venue
    EMNLP 2023, 3102~pp. 3116

    Korean review of T03-6

  7. T03-7Structural parallel

    Gorilla: Large Language Model Connected with Massive APIs

    Authors
    Shishir G. Patil, Tianjun Zhang, Xin Wang, Joseph E. Gonzalez
    Year
    2024
    Venue
    NeurIPS 2024

    Korean review of T03-7

  8. T03-8Candidate approach

    ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

    Authors
    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, Maosong Sun
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T03-8

T04. Planning, reflection, judging, and self-improving agents

Research on step-by-step reasoning, self-critique, and model-as-judge evaluation.

Show this topic only
  1. T04-1Structural parallel

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

    Authors
    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, Denny Zhou
    Year
    2022
    Venue
    NeurIPS 2022

    Korean review of T04-1

  2. T04-4Structural parallel

    Self-Refine: Iterative Refinement with Self-Feedback

    Authors
    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, Peter Clark
    Year
    2023
    Venue
    NeurIPS 2023

    Korean review of T04-4

  3. T04-7Structural parallel

    Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

    Authors
    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, Ion Stoica
    Year
    2023
    Venue
    NeurIPS 2023 Datasets and Benchmarks Track

    Korean review of T04-7

  4. T04-8Candidate approach

    G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment

    Authors
    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu
    Year
    2023
    Venue
    EMNLP 2023

    Korean review of T04-8

  5. T04-9Similar problem

    Let's Verify Step by Step

    Authors
    Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, Karl Cobbe
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T04-9

  6. T04-10Candidate approach

    Agent-as-a-Judge: Evaluate Agents with Agents

    Authors
    Mingchen Zhuge, Changsheng Zhao, Dylan Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, Yangyang Shi, Vikas Chandra, Jürgen Schmidhuber
    Year
    2024
    Venue
    arXiv

    Korean review of T04-10

  7. T04-11Similar problem

    Large Language Models Cannot Self-Correct Reasoning Yet

    Authors
    Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, Denny Zhou
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T04-11

  8. T04-12Candidate approach

    Self-Rewarding Language Models

    Authors
    Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, Jason Weston
    Year
    2024
    Venue
    ICML 2024

    Korean review of T04-12

  9. T04-13Candidate approach

    Voyager: An Open-Ended Embodied Agent with Large Language Models

    Authors
    Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, Anima Anandkumar
    Year
    2024

    Korean review of T04-13

  10. T04-14Candidate approach

    STaR: Bootstrapping Reasoning With Reasoning

    Authors
    Eric Zelikman, Yuhuai Wu, Jesse Mu, Noah D. Goodman
    Year
    2022
    Venue
    NeurIPS 2022

    Korean review of T04-14

  11. T04-15Structural parallel

    Constitutional AI: Harmlessness from AI Feedback

    Authors
    Yuntao Bai, Saurav Kadavath, Sandipan Kundu and others, 51 authors in total (Anthropic)
    Year
    2022
    Venue
    arXiv

    Korean review of T04-15

  12. T04-16Similar problem

    CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing

    Authors
    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, Weizhu Chen
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T04-16

T05. Human review and approval workflows

Human-in-the-loop and approval research, including automation-induced complacency.

Show this topic only
  1. T05-22Candidate approachSince 2025

    What You Approve Is What Executes: Consent Integrity for Black-Box LLM Agents

    Authors
    Xiaoqi Weng
    Year
    2026
    Venue
    arXiv 2026-06 1

    Korean review of T05-22

  2. T05-23Candidate approachSince 2025

    Oversight Has a Capacity: Calibrating Agent Guards to a Subjective, Fatiguing Human

    Authors
    Emre Turan
    Year
    2026
    Venue
    arXiv 2026-06 8

    Korean review of T05-23

  3. T05-21Similar problemSince 2025

    Human-In-the-Loop Software Development Agents (HULA)

    Authors
    Wannita Takerngsaksiri and 9 others (Atlassian, Monash University)
    Year
    2025
    Venue
    ICSE-SEIP 2025

    Korean review of T05-21

  4. T05-1Similar problem

    Ironies of Automation

    Authors
    Lisanne Bainbridge
    Year
    1983
    Venue
    Automatica vol. 19 no. 6 775-779

    Korean review of T05-1

  5. T05-2Similar problem

    The Out-of-the-Loop Performance Problem and Level of Control in Automation

    Authors
    Mica R. Endsley, Esin O. Kiris
    Year
    1995
    Venue
    Human Factors vol. 37 no. 2 381-394

    Korean review of T05-2

  6. T05-3Similar problem

    Humans and Automation: Use, Misuse, Disuse, Abuse

    Authors
    Raja Parasuraman, Victor Riley
    Year
    1997
    Venue
    Human Factors vol. 39 no. 2 230-253

    Korean review of T05-3

  7. T05-4Structural parallel

    A Model for Types and Levels of Human Interaction with Automation

    Authors
    Raja Parasuraman, Thomas B. Sheridan, Christopher D. Wickens
    Year
    2000
    Venue
    IEEE Transactions on Systems, Man, and Cybernetics Part A vol. 30 no. 3 286-297

    Korean review of T05-4

  8. T05-5Similar problem

    Complacency and Bias in Human Use of Automation: An Attentional Integration

    Authors
    Raja Parasuraman, Dietrich H. Manzey
    Year
    2010
    Venue
    Human Factors vol. 52 no. 3 381-410

    Korean review of T05-5

  9. T05-6Structural parallel

    On Optimum Recognition Error and Reject Tradeoff

    Authors
    C. K. Chow
    Year
    1970
    Venue
    IEEE Transactions on Information Theory vol. 16 no. 1 41-46

    Korean review of T05-6

  10. T05-7Structural parallel

    Learning with Rejection

    Authors
    Corinna Cortes, Giulia DeSalvo, Mehryar Mohri
    Year
    2016
    Venue
    Algorithmic Learning Theory (ALT) 2016, Springer LNCS 67-82

    Korean review of T05-7

  11. T05-8Structural parallel

    Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer

    Authors
    David Madras, Toniann Pitassi, Richard Zemel
    Year
    2018
    Venue
    NeurIPS 2018

    Korean review of T05-8

  12. T05-9Structural parallel

    SelectiveNet: A Deep Neural Network with an Integrated Reject Option

    Authors
    Yonatan Geifman, Ran El-Yaniv
    Year
    2019
    Venue
    ICML 2019

    Korean review of T05-9

  13. T05-10Structural parallel

    Consistent Estimators for Learning to Defer to an Expert

    Authors
    Hussein Mozannar, David Sontag
    Year
    2020
    Venue
    ICML 2020

    Korean review of T05-10

  14. T05-11Structural parallel

    Learning to Complement Humans

    Authors
    Bryan Wilder, Eric Horvitz, Ece Kamar
    Year
    2020
    Venue
    IJCAI 2020

    Korean review of T05-11

  15. T05-12Structural parallel

    Machine Learning with a Reject Option: A survey

    Authors
    Kilian Hendrickx, Lorenzo Perini, Dries Van der Plas, Wannes Meert, Jesse Davis
    Year
    2024
    Venue
    Machine Learning (Springer) 113 3073-3110

    Korean review of T05-12

  16. T05-13Similar problem

    Trust in Automation: Designing for Appropriate Reliance

    Authors
    John D. Lee, Katrina A. See
    Year
    2004
    Venue
    Human Factors vol. 46 no. 1 50-80

    Korean review of T05-13

  17. T05-14Similar problem

    Guidelines for Human-AI Interaction

    Authors
    Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, Eric Horvitz
    Year
    2019
    Venue
    CHI 2019 (Honorable Mention)

    Korean review of T05-14

  18. T05-15Similar problem

    Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making

    Authors
    Yunfeng Zhang, Q. Vera Liao, Rachel K. E. Bellamy
    Year
    2020
    Venue
    ACM FAT* 2020

    Korean review of T05-15

  19. T05-16Similar problem

    Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance

    Authors
    Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, Daniel S. Weld
    Year
    2021
    Venue
    CHI 2021

    Korean review of T05-16

  20. T05-17Similar problem

    To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-assisted Decision-making

    Authors
    Zana Bucinca, Maja Barbara Malaya, Krzysztof Z. Gajos
    Year
    2021
    Venue
    Proceedings of the ACM on Human-Computer Interaction vol. 5 CSCW1 1-21

    Korean review of T05-17

  21. T05-18Similar problem

    The flaws of policies requiring human oversight of government algorithms

    Authors
    Ben Green
    Year
    2022
    Venue
    Computer Law & Security Review 45 105681

    Korean review of T05-18

  22. T05-19Structural parallel

    Identifying the Risks of LM Agents with an LM-Emulated Sandbox (ToolEmu)

    Authors
    Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J. Maddison, Tatsunori Hashimoto
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T05-19

  23. T05-20Structural parallel

    AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents

    Authors
    Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, Florian Tramer
    Year
    2024
    Venue
    NeurIPS 2024 Datasets and Benchmarks

    Korean review of T05-20

  24. T05-24Candidate approach

    Deep reinforcement learning from human preferences

    Authors
    Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, Dario Amodei
    Year
    2017
    Venue
    NIPS 2017 (Advances in Neural Information Processing Systems 30)

    Korean review of T05-24

  25. T05-25Candidate approach

    Human-in-the-loop machine learning: a state of the art

    Authors
    Eduardo Mosqueira-Rey, Elena Hernandez-Pereira, David Alonso-Rios, Jose Bobes-Bascaran, Angel Fernandez-Leal
    Year
    2023
    Venue
    Artificial Intelligence Review vol. 56 no. 4 3005-3054

    Korean review of T05-25

T06. RAG, knowledge graphs, ontologies, and provenance

Retrieval-augmented generation and structured knowledge with traceable sources.

Show this topic only
  1. T06-14Similar problemSince 2025

    Enhancing retrieval-augmented generation for interoperable industrial knowledge representation and inference toward cognitive digital twins

    Authors
    Dachuan Shi, Jianzhang Li, Olga Meyer, Thomas Bauernhansl
    Year
    2025
    Venue
    Computers in Industry vol. 171, 2025, 104330

    Korean review of T06-14

  2. T06-1Structural parallel

    Dense Passage Retrieval for Open-Domain Question Answering

    Authors
    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, Wen-tau Yih
    Year
    2020
    Venue
    EMNLP 2020, pp. 6769-6781

    Korean review of T06-1

  3. T06-2Similar problem

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

    Authors
    Patrick Lewis and 11 others
    Year
    2020
    Venue
    NeurIPS 2020

    Korean review of T06-2

  4. T06-3Structural parallel

    Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering

    Authors
    Gautier Izacard, Edouard Grave
    Year
    2021
    Venue
    EACL 2021, pp. 874-880

    Korean review of T06-3

  5. T06-4Similar problem

    Retrieval-Augmented Generation for Large Language Models: A Survey

    Authors
    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, Haofen Wang
    Year
    2024
    Venue
    arXiv (v1 2023-12-18, v5 2024-03-27)

    Korean review of T06-4

  6. T06-5Structural parallel

    From Local to Global: A Graph RAG Approach to Query-Focused Summarization

    Authors
    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, Jonathan Larson
    Year
    2025
    Venue
    arXiv, Microsoft Research (v1 2024-04-24, v2 2025-02-19)

    Korean review of T06-5

  7. T06-6Candidate approach

    HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

    Authors
    Bernal Jimenez Gutierrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, Yu Su
    Year
    2024
    Venue
    NeurIPS 2024

    Korean review of T06-6

  8. T06-7Structural parallel

    Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection

    Authors
    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, Hannaneh Hajishirzi
    Year
    2024
    Venue
    ICLR 2024

    Korean review of T06-7

  9. T06-8Structural parallel

    Enabling Large Language Models to Generate Text with Citations

    Authors
    Tianyu Gao, Howard Yen, Jiatong Yu, Danqi Chen
    Year
    2023
    Venue
    EMNLP 2023, pp. 6465-6488

    Korean review of T06-8

  10. T06-9Similar problem

    Measuring Attribution in Natural Language Generation Models

    Authors
    Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, David Reitter
    Year
    2023
    Venue
    Computational Linguistics vol. 49 no. 4, 2023, pp. 777-840

    Korean review of T06-9

  11. T06-10Structural parallel

    Provenance Semirings

    Authors
    Todd J. Green, Grigoris Karvounarakis, Val Tannen
    Year
    2007
    Venue
    PODS 2007 (ACM SIGMOD-SIGACT-SIGART Principles of Database Systems), pp. 31-40

    Korean review of T06-10

  12. T06-11Structural parallel

    Provenance in Databases: Why, How, and Where

    Authors
    James Cheney, Laura Chiticariu, Wang-Chiew Tan
    Year
    2009
    Venue
    Foundations and Trends in Databases vol. 1 no. 4, 2009, pp. 379-474

    Korean review of T06-11

  13. T06-12Similar problem

    Knowledge Graphs

    Authors
    Aidan Hogan and 17 others
    Year
    2022
    Venue
    ACM Computing Surveys vol. 54 no. 4, 71, pp. 1-37

    Korean review of T06-12

  14. T06-13Similar problem

    A benchmark dataset with Knowledge Graph generation for Industry 4.0 production lines

    Authors
    Muhammad Yahya, Aabid Ali, Qaiser Mehmood, Lan Yang, John G. Breslin, Muhammad Intizar Ali
    Year
    2024
    Venue
    Semantic Web vol. 15 no. 2, 2024, pp. 461-479

    Korean review of T06-13

T07. Structured output, schema validation, and document understanding

Research on forcing machine-checkable output and reading business documents.

Show this topic only
  1. T07-10Structural parallelSince 2025

    XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models

    Authors
    Yixin Dong, Charlie F. Ruan, Yaxing Cai, Ruihang Lai, Ziyi Xu, Yilong Zhao, Tianqi Chen
    Year
    2025
    Venue
    MLSys 2025

    Korean review of T07-10

  2. T07-11Similar problemSince 2025

    JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Models

    Authors
    Saibo Geng, Hudson Cooper, Michał Moskal, Samuel Jenkins, Julian Berman, Nathan Ranchin, Robert West, Eric Horvitz, Harsha Nori
    Year
    2025
    Venue
    arXiv

    Korean review of T07-11

  3. T07-1Structural parallel

    LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking

    Authors
    Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei
    Year
    2022
    Venue
    ACM Multimedia 2022 (ACM MM)

    Korean review of T07-1

  4. T07-2Structural parallel

    OCR-free Document Understanding Transformer

    Authors
    Geewook Kim, Teakgyu Hong, Moonbin Yim, Jeongyeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, Seunghyun Park
    Year
    2022
    Venue
    ECCV 2022

    Korean review of T07-2

  5. T07-3Similar problem

    TableFormer: Table Structure Understanding with Transformers

    Authors
    Ahmed Nassar, Nikolaos Livathinos, Maksym Lysak, Peter Staar
    Year
    2022
    Venue
    CVPR 2022

    Korean review of T07-3

  6. T07-4Candidate approach

    Nougat: Neural Optical Understanding for Academic Documents

    Authors
    Lukas Blecher, Guillem Cucurull, Thomas Scialom, Robert Stojnic
    Year
    2023
    Venue
    arXiv (Meta AI)

    Korean review of T07-4

  7. T07-5Similar problem

    FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents

    Authors
    Guillaume Jaume, Hazim Kemal Ekenel, Jean-Philippe Thiran
    Year
    2019
    Venue
    ICDAR 2019 OST (Workshop on Open Services and Tools for Document Analysis)

    Korean review of T07-5

  8. T07-6Similar problem

    DocVQA: A Dataset for VQA on Document Images

    Authors
    Minesh Mathew, Dimosthenis Karatzas, C.V. Jawahar
    Year
    2021
    Venue
    WACV 2021

    Korean review of T07-6

  9. T07-7Structural parallel

    PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models

    Authors
    Torsten Scholak, Nathan Schucher, Dzmitry Bahdanau
    Year
    2021
    Venue
    EMNLP 2021

    Korean review of T07-7

  10. T07-8Structural parallel

    Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning

    Authors
    Saibo Geng, Martin Josifoski, Maxime Peyrard, Robert West
    Year
    2023
    Venue
    EMNLP 2023

    Korean review of T07-8

  11. T07-9Structural parallel

    Efficient Guided Generation for Large Language Models

    Authors
    Brandon T. Willard, Rémi Louf
    Year
    2023
    Venue
    arXiv, Outlines

    Korean review of T07-9

  12. T07-12Similar problem

    Let Me Speak Freely? A Study On The Impact Of Format Restrictions On Large Language Model Performance

    Authors
    Zhi Rui Tam, Cheng-Kuang Wu, Yi-Lin Tsai, Chieh-Yen Lin, Hung-yi Lee, Yun-Nung Chen
    Year
    2024
    Venue
    EMNLP 2024 Industry Track

    Korean review of T07-12

  13. T07-13Similar problem

    Grammar-Aligned Decoding

    Authors
    Kanghee Park, Jiayu Wang, Taylor Berg-Kirkpatrick, Nadia Polikarpova, Loris D'Antoni
    Year
    2024
    Venue
    NeurIPS 2024

    Korean review of T07-13

T08. OCR, vision-language models, and industrial display reading

Research on reading meters, indicators, and shop-floor displays from images.

Show this topic only
  1. T08-1Similar problemSince 2025

    Do Vision-Language Models Measure Up? Benchmarking Visual Measurement Reading with MeasureBench

    Authors
    Fenfen Lin, Yesheng Liu, Haiyu Xu, Chen Yue, Zheqi He, Mingxuan Zhao, Miguel Hu Chen, Jiakang Liu, JG Yao, Xi Yang
    Year
    2026
    Venue
    arXiv (cs.CV, cs.AI)

    Korean review of T08-1

  2. T08-2Structural parallelSince 2025

    DialBench: Towards Accurate Reading Recognition of Pointer Meter using Large Foundation Models

    Authors
    Futian Wang, Chaoliu Weng, Xiao Wang, Zhen Chen, Zhicheng Zhao, Jin Tang
    Year
    2025
    Venue
    arXiv (cs.CV, cs.AI)

    Korean review of T08-2

  3. T08-14Candidate approachSince 2025

    Qwen2.5-VL Technical Report

    Authors
    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang (27 authors in total)
    Year
    2025
    Venue
    arXiv (cs.CV, cs.CL)

    Korean review of T08-14

  4. T08-3Similar problem

    Convolutional Neural Networks for Automatic Meter Reading

    Authors
    Rayson Laroca, Victor Barroso, Matheus A. Diniz, Gabriel R. Goncalves, William Robson Schwartz, David Menotti
    Year
    2019
    Venue
    Journal of Electronic Imaging 28(1), 013023

    Korean review of T08-3

  5. T08-4Similar problem

    Utilizing Smartphone-Based Machine Learning in Medical Monitor Data Collection: Seven Segment Digit Recognition

    Authors
    Varun N. Shenoy, Oliver O. Aalami
    Year
    2018
    Venue
    AMIA Annual Symposium Proceedings 2017;2017:1564-1570

    Korean review of T08-4

  6. T08-5Structural parallel

    An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition

    Authors
    Baoguang Shi, Xiang Bai, Cong Yao
    Year
    2017
    Venue
    IEEE Transactions on Pattern Analysis and Machine Intelligence 39(11), 2298-2304 (2017)

    Korean review of T08-5

  7. T08-6Structural parallel

    Real-time Scene Text Detection with Differentiable Binarization

    Authors
    Minghui Liao, Zhaoyi Wan, Cong Yao, Kai Chen, Xiang Bai
    Year
    2020
    Venue
    AAAI 2020

    Korean review of T08-6

  8. T08-7Candidate approach

    TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models

    Authors
    Minghao Li, Tengchao Lv, Jingye Chen, Lei Cui, Yijuan Lu, Dinei Florencio, Cha Zhang, Zhoujun Li, Furu Wei
    Year
    2023
    Venue
    Proceedings of the AAAI Conference on Artificial Intelligence 37(11), 13094-13102 (2023)

    Korean review of T08-7

  9. T08-9Structural parallel

    General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model

    Authors
    Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, Chunrui Han, Xiangyu Zhang
    Year
    2024
    Venue
    arXiv (cs.CV)

    Korean review of T08-9

  10. T08-11Candidate approach

    OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models

    Authors
    Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xucheng Yin, Cheng-lin Liu, Lianwen Jin, Xiang Bai
    Year
    2024
    Venue
    Science China Information Sciences (2024)

    Korean review of T08-11

  11. T08-12Structural parallel

    ScreenAI: A Vision-Language Model for UI and Infographics Understanding

    Authors
    Gilles Baechler, Srinivas Sunkara, Maria Wang, Fedir Zubach, Hassan Mansoor, Vincent Etter, Victor Carbune, Jason Lin, Jindong Chen, Abhanshu Sharma
    Year
    2024
    Venue
    IJCAI 2024

    Korean review of T08-12

  12. T08-13Candidate approach

    Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution

    Authors
    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen (19 authors in total)
    Year
    2024
    Venue
    arXiv (cs.CV)

    Korean review of T08-13

T09. Industrial time series, anomaly detection, and predictive maintenance

Sensor-driven fault detection, remaining useful life, and condition monitoring research.

Show this topic only
  1. T09-1Similar problem

    Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress

    Authors
    Renjie Wu, Eamonn J. Keogh
    Year
    2023
    Venue
    IEEE Transactions on Knowledge and Data Engineering vol. 35 no. 3 pp. 2421 pp. 2429

    Korean review of T09-1

  2. T09-2Similar problem

    Towards a Rigorous Evaluation of Time-series Anomaly Detection

    Authors
    Siwon Kim, Kukjin Choi, Hyun-Soo Choi, Byunghan Lee, Sungroh Yoon
    Year
    2022
    Venue
    AAAI 2022

    Korean review of T09-2

  3. T09-3Similar problem

    The Elephant in the Room: Towards A Reliable Time-Series Anomaly Detection Benchmark

    Authors
    Qinghua Liu, John Paparrizos
    Year
    2024
    Venue
    NeurIPS 2024 Datasets and Benchmarks Track

    Korean review of T09-3

  4. T09-4Similar problem

    Volume Under the Surface: A New Accuracy Evaluation Measure for Time-Series Anomaly Detection

    Authors
    John Paparrizos, Paul Boniol, Themis Palpanas, Ruey S. Tsay, Aaron Elmore, Michael J. Franklin
    Year
    2022
    Venue
    Proceedings of the VLDB Endowment vol. 15 no. 11 pp. 2774 pp. 2787

    Korean review of T09-4

  5. T09-5Similar problem

    A Review on Outlier/Anomaly Detection in Time Series Data

    Authors
    Ane Blazquez-Garcia, Angel Conde, Usue Mori, Jose A. Lozano
    Year
    2021
    Venue
    ACM Computing Surveys vol. 54 no. 3 56, pp. 1 pp. 33

    Korean review of T09-5

  6. T09-6Structural parallel

    Anomaly Detection in Time Series: A Comprehensive Evaluation

    Authors
    Sebastian Schmidl, Phillip Wenig, Thorsten Papenbrock
    Year
    2022
    Venue
    Proceedings of the VLDB Endowment vol. 15 no. 9 pp. 1779 pp. 1797

    Korean review of T09-6

  7. T09-7Similar problem

    Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding

    Authors
    Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, Tom Soderstrom
    Year
    2018
    Venue
    KDD 2018

    Korean review of T09-7

  8. T09-8Structural parallel

    Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network (OmniAnomaly)

    Authors
    Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, Dan Pei
    Year
    2019
    Venue
    KDD 2019

    Korean review of T09-8

  9. T09-9Similar problem

    A Dataset to Support Research in the Design of Secure Water Treatment Systems (SWaT)

    Authors
    Jonathan Goh, Sridhar Adepu, Khurum Nazir Junejo, Aditya Mathur
    Year
    2017
    Venue
    CRITIS 2016

    Korean review of T09-9

  10. T09-10Structural parallel

    Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy

    Authors
    Jiehui Xu, Haixu Wu, Jianmin Wang, Mingsheng Long
    Year
    2022
    Venue
    ICLR 2022 (Spotlight)

    Korean review of T09-10

  11. T09-11Structural parallel

    TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data

    Authors
    Shreshth Tuli, Giuliano Casale, Nicholas R. Jennings
    Year
    2022
    Venue
    Proceedings of the VLDB Endowment vol. 15 no. 6 pp. 1201 pp. 1214

    Korean review of T09-11

  12. T09-12Structural parallel

    Time-Series Anomaly Detection Service at Microsoft

    Authors
    Hansheng Ren, Bixiong Xu, Yujing Wang, Chao Yi, Congrui Huang, Xiaoyu Kou, Tony Xing, Mao Yang, Jie Tong, Qi Zhang
    Year
    2019
    Venue
    KDD 2019

    Korean review of T09-12

  13. T09-13Similar problem

    Damage Propagation Modeling for Aircraft Engine Run-to-Failure Simulation

    Authors
    Abhinav Saxena, Kai Goebel, Don Simon, Neil Eklund
    Year
    2008
    Venue
    2008 International Conference on Prognostics and Health Management (IEEE)

    Korean review of T09-13

  14. T09-14Similar problem

    A review on machinery diagnostics and prognostics implementing condition-based maintenance

    Authors
    Andrew K. S. Jardine, Daming Lin, Dragan Banjevic
    Year
    2006
    Venue
    Mechanical Systems and Signal Processing vol. 20 no. 7 pp. 1483 pp. 1510

    Korean review of T09-14

  15. T09-15Similar problem

    Machinery health prognostics: A systematic review from data acquisition to RUL prediction

    Authors
    Yaguo Lei, Naipeng Li, Liang Guo, Ningbo Li, Tao Yan, Jing Lin
    Year
    2018
    Venue
    Mechanical Systems and Signal Processing vol. 104 pp. 799 pp. 834

    Korean review of T09-15

  16. T09-16Candidate approach

    Deep learning models for predictive maintenance: a survey, comparison, challenges and prospects

    Authors
    Oscar Serradilla, Ekhi Zugasti, Jon Rodriguez, Urko Zurutuza
    Year
    2022
    Venue
    Applied Intelligence vol. 52 no. 10 pp. 10934 pp. 10964

    Korean review of T09-16

T10. AutoML, model selection, evaluation design, and experiment tracking

Research on choosing and validating models instead of shipping a single fitted model.

Show this topic only
  1. T10-1Similar problem

    Random Search for Hyper-Parameter Optimization

    Authors
    James Bergstra, Yoshua Bengio
    Year
    2012
    Venue
    Journal of Machine Learning Research vol. 13 pp. 281-305

    Korean review of T10-1

  2. T10-2Structural parallel

    Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms

    Authors
    Chris Thornton, Frank Hutter, Holger H. Hoos, Kevin Leyton-Brown
    Year
    2013
    Venue
    KDD 2013 (Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining) pp. 847-855

    Korean review of T10-2

  3. T10-3Structural parallel

    Efficient and Robust Automated Machine Learning (auto-sklearn)

    Authors
    Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, Frank Hutter
    Year
    2015
    Venue
    NIPS 2015 (Advances in Neural Information Processing Systems 28)

    Korean review of T10-3

  4. T10-4Candidate approach

    Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization

    Authors
    Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, Ameet Talwalkar
    Year
    2018
    Venue
    Journal of Machine Learning Research 18 185, pp. 1-52

    Korean review of T10-4

  5. T10-5Candidate approach

    BOHB: Robust and Efficient Hyperparameter Optimization at Scale

    Authors
    Stefan Falkner, Aaron Klein, Frank Hutter
    Year
    2018
    Venue
    ICML 2018

    Korean review of T10-5

  6. T10-6Candidate approach

    Optuna: A Next-generation Hyperparameter Optimization Framework

    Authors
    Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, Masanori Koyama
    Year
    2019
    Venue
    KDD 2019 Applied Data Science Track

    Korean review of T10-6

  7. T10-7Structural parallel

    AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data

    Authors
    Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, Alexander Smola
    Year
    2020
    Venue
    arXiv

    Korean review of T10-7

  8. T10-8Structural parallel

    Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning

    Authors
    Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, Frank Hutter
    Year
    2022
    Venue
    Journal of Machine Learning Research 23 261

    Korean review of T10-8

  9. T10-9Similar problem

    AMLB: an AutoML Benchmark

    Authors
    Pieter Gijsbers, Marcos L. P. Bueno, Stefan Coors, Erin LeDell, Sebastien Poirier, Janek Thomas, Bernd Bischl, Joaquin Vanschoren
    Year
    2024
    Venue
    Journal of Machine Learning Research 25 101, pp. 1-65

    Korean review of T10-9

  10. T10-10Similar problem

    Statistical Comparisons of Classifiers over Multiple Data Sets

    Authors
    Janez Demsar
    Year
    2006
    Venue
    Journal of Machine Learning Research vol. 7 pp. 1-30

    Korean review of T10-10

  11. T10-11Similar problem

    On Over-fitting in Model Selection and Subsequent Selection Bias in Performance Evaluation

    Authors
    Gavin C. Cawley, Nicola L. C. Talbot
    Year
    2079
    Venue
    Journal of Machine Learning Research vol. 11 pp. 2079-2107

    Korean review of T10-11

  12. T10-12Similar problem

    Evaluating time series forecasting models: An empirical study on performance estimation methods

    Authors
    Vitor Cerqueira, Luis Torgo, Igor Mozetic
    Year
    2028
    Venue
    Machine Learning vol. 109 pp. 1997-2028

    Korean review of T10-12

  13. T10-13Structural parallel

    OpenML: Networked Science in Machine Learning

    Authors
    Joaquin Vanschoren, Jan N. van Rijn, Bernd Bischl, Luis Torgo
    Year
    2014
    Venue
    ACM SIGKDD Explorations Newsletter vol. 15 no. 2 pp. 49-60

    Korean review of T10-13

  14. T10-14Structural parallel

    Developments in MLflow: A System to Accelerate the Machine Learning Lifecycle

    Authors
    Andrew Chen, Andy Chow, Aaron Davidson, Arjun DCunha, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Clemens Mewald, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe, Avesh Singh, Fen Xie, Matei Zaharia, Richard Zang, Juntai Zheng, Corey Zumar
    Year
    2020
    Venue
    DEEM 2020

    Korean review of T10-14

  15. T10-15Candidate approach

    Model Cards for Model Reporting

    Authors
    Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru
    Year
    2019
    Venue
    ACM FAT* 2019

    Korean review of T10-15

  16. T10-16Candidate approach

    Datasheets for Datasets

    Authors
    Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daume III, Kate Crawford
    Year
    2021
    Venue
    Communications of the ACM vol. 64 no. 12 pp. 86-92

    Korean review of T10-16

T11. Object detection, multi-object tracking, and video understanding

Detection and tracking backbones behind camera-based safety and inspection work.

Show this topic only
  1. T11-6.3Candidate approachSince 2025

    SAM 2: Segment Anything in Images and Videos

    Authors
    Nikhila Ravi and 17 others (Meta FAIR)
    Year
    2025

    Korean review of T11-6.3

  2. T11-1.1Structural parallel

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT)

    Authors
    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov and 9 others (12 authors in total)
    Year
    2021

    Korean review of T11-1.1

  3. T11-2.1Structural parallel

    Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

    Authors
    Shaoqing Ren, Kaiming He, Ross Girshick, Jian Sun
    Year
    2017

    Korean review of T11-2.1

  4. T11-2.2Structural parallel

    You Only Look Once: Unified, Real-Time Object Detection (YOLO)

    Authors
    Joseph Redmon, Santosh Divvala, Ross Girshick, Ali Farhadi
    Year
    2016

    Korean review of T11-2.2

  5. T11-2.3Structural parallel

    Focal Loss for Dense Object Detection (RetinaNet)

    Authors
    Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollar
    Year
    2017

    Korean review of T11-2.3

  6. T11-2.4Structural parallel

    End-to-End Object Detection with Transformers (DETR)

    Authors
    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko
    Year
    2020

    Korean review of T11-2.4

  7. T11-2.5Structural parallel

    Deformable DETR: Deformable Transformers for End-to-End Object Detection

    Authors
    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, Jifeng Dai
    Year
    2021

    Korean review of T11-2.5

  8. T11-2.6Structural parallel

    DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection

    Authors
    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M. Ni, Heung-Yeung Shum
    Year
    2023

    Korean review of T11-2.6

  9. T11-2.7Candidate approach

    DETRs Beat YOLOs on Real-time Object Detection (RT-DETR)

    Authors
    Yian Zhao, Wenyu Lv, Shangliang Xu, Jinman Wei, Guanzhong Wang, Qingqing Dang, Yi Liu, Jie Chen
    Year
    2024

    Korean review of T11-2.7

  10. T11-2.8Candidate approach

    Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

    Authors
    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, Lei Zhang
    Year
    2024

    Korean review of T11-2.8

  11. T11-2.9Candidate approach

    YOLOv10: Real-Time End-to-End Object Detection

    Authors
    Ao Wang, Hui Chen, Lihao Liu, Kai Chen, Zijia Lin, Jungong Han, Guiguang Ding
    Year
    2024

    Korean review of T11-2.9

  12. T11-3.1Structural parallel

    Simple Online and Realtime Tracking (SORT)

    Authors
    Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, Ben Upcroft
    Year
    2016

    Korean review of T11-3.1

  13. T11-3.2Similar problem

    Simple Online and Realtime Tracking with a Deep Association Metric (DeepSORT)

    Authors
    Nicolai Wojke, Alex Bewley, Dietrich Paulus
    Year
    2017

    Korean review of T11-3.2

  14. T11-3.3Structural parallel

    ByteTrack: Multi-Object Tracking by Associating Every Detection Box

    Authors
    Yifu Zhang, Peize Sun, Yi Jiang, Dongdong Yu, Fucheng Weng, Zehuan Yuan, Ping Luo, Wenyu Liu, Xinggang Wang
    Year
    2022

    Korean review of T11-3.3

  15. T11-3.4Candidate approach

    Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking (OC-SORT)

    Authors
    Jinkun Cao, Jiangmiao Pang, Xinshuo Weng, Rawal Khirodkar, Kris Kitani
    Year
    2023

    Korean review of T11-3.4

  16. T11-4.1Structural parallel

    Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics (MOTA, MOTP)

    Authors
    Keni Bernardin, Rainer Stiefelhagen
    Year
    2008

    Korean review of T11-4.1

  17. T11-4.2Structural parallel

    Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking (IDF1)

    Authors
    Ergys Ristani, Francesco Solera, Roger S. Zou, Rita Cucchiara, Carlo Tomasi
    Year
    2016

    Korean review of T11-4.2

  18. T11-4.3Structural parallel

    HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking

    Authors
    Jonathon Luiten, Aljosa Osep, Patrick Dendorfer, Philip Torr, Andreas Geiger, Laura Leal-Taixe, Bastian Leibe
    Year
    2021

    Korean review of T11-4.3

  19. T11-4.4Structural parallel

    MOT16: A Benchmark for Multi-Object Tracking

    Authors
    Anton Milan, Laura Leal-Taixe, Ian Reid, Stefan Roth, Konrad Schindler
    Year
    2016

    Korean review of T11-4.4

  20. T11-5.1Structural parallel

    Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset (I3D)

    Authors
    Joao Carreira, Andrew Zisserman
    Year
    2017

    Korean review of T11-5.1

  21. T11-5.2Similar problem

    SlowFast Networks for Video Recognition

    Authors
    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, Kaiming He
    Year
    2019

    Korean review of T11-5.2

  22. T11-5.3Candidate approach

    VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

    Authors
    Zhan Tong, Yibing Song, Jue Wang, Limin Wang
    Year
    2022

    Korean review of T11-5.3

  23. T11-6.1Structural parallel

    Learning Transferable Visual Models From Natural Language Supervision (CLIP)

    Authors
    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh and 7 others
    Year
    2021

    Korean review of T11-6.1

  24. T11-6.2Structural parallel

    Segment Anything (SAM)

    Authors
    Alexander Kirillov, Eric Mintun, Nikhila Ravi and 9 others (12 authors in total, Meta AI Research)
    Year
    2023

    Korean review of T11-6.2

T12. Video understanding and evidence selection with vision-language models

Research on explaining what a camera saw and pointing back to the evidence frame.

Show this topic only
  1. T12-2Structural parallelSince 2025

    Adaptive Keyframe Sampling for Long Video Understanding

    Authors
    Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, Qixiang Ye
    Year
    2025
    Venue
    CVPR 2025

    Korean review of T12-2

  2. T12-1Structural parallelSince 2025

    Frame-Voyager: Learning to Query Frames for Video Large Language Models

    Authors
    Sicheng Yu, Chengkai Jin, Huanyu Wang, Zhenghao Chen, Sheng Jin, Zhongrong Zuo, Xiaolei Xu, Zhenbang Sun, Bingni Zhang, Jiawei Wu, Hao Zhang, Qianru Sun
    Year
    2025
    Venue
    ICLR 2025

    Korean review of T12-1

  3. T12-13Similar problemSince 2025

    Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis

    Authors
    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen
    Year
    2025
    Venue
    CVPR 2025, pp. 24108-24118

    Korean review of T12-13

  4. T12-3Structural parallel

    Self-Chained Image-Language Model for Video Localization and Question Answering (SeViLA)

    Authors
    Shoubin Yu, Jaemin Cho, Prateek Yadav, Mohit Bansal
    Year
    2023
    Venue
    NeurIPS 2023

    Korean review of T12-3

  5. T12-4Structural parallel

    VideoAgent: Long-form Video Understanding with Large Language Model as Agent

    Authors
    Xiaohan Wang, Yuhui Zhang, Orr Zohar, Serena Yeung-Levy
    Year
    2024
    Venue
    ECCV 2024. Computer Vision - ECCV 2024, LNCS, pp. 58-76

    Korean review of T12-4

  6. T12-5Candidate approach

    A Simple LLM Framework for Long-Range Video Question-Answering (LLoVi)

    Authors
    Ce Zhang, Taixi Lu, Md Mohaiminul Islam, Ziyang Wang, Shoubin Yu, Mohit Bansal, Gedas Bertasius
    Year
    2024
    Venue
    EMNLP 2024 main

    Korean review of T12-5

  7. T12-6Similar problem

    Can I Trust Your Answer? Visually Grounded Video Question Answering (NExT-GQA)

    Authors
    Junbin Xiao, Angela Yao, Yicong Li, Tat-Seng Chua
    Year
    2024
    Venue
    CVPR 2024 (Highlight)

    Korean review of T12-6

  8. T12-7Structural parallel

    TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding

    Authors
    Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, Lu Hou
    Year
    2024
    Venue
    CVPR 2024

    Korean review of T12-7

  9. T12-8Structural parallel

    VTimeLLM: Empower LLM to Grasp Video Moments

    Authors
    Bin Huang, Xin Wang, Hong Chen, Zihan Song, Wenwu Zhu
    Year
    2024
    Venue
    CVPR 2024, pp. 14271-14280

    Korean review of T12-8

  10. T12-9Similar problem

    QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries (Moment-DETR)

    Authors
    Jie Lei, Tamara L. Berg, Mohit Bansal
    Year
    2021
    Venue
    NeurIPS 2021

    Korean review of T12-9

  11. T12-11Structural parallel

    BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models

    Authors
    Junnan Li, Dongxu Li, Silvio Savarese, Steven Hoi
    Year
    2023
    Venue
    ICML 2023. PMLR vol. 202 pp. 19730-19742

    Korean review of T12-11

  12. T12-12Similar problem

    EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding

    Authors
    Karttikeya Mangalam, Raiymbek Akshulakov, Jitendra Malik
    Year
    2023
    Venue
    NeurIPS 2023, Datasets and Benchmarks Track

    Korean review of T12-12

  13. T12-14Similar problem

    Evaluating Object Hallucination in Large Vision-Language Models (POPE)

    Authors
    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, Ji-Rong Wen
    Year
    2023
    Venue
    EMNLP 2023

    Korean review of T12-14

  14. T12-15Similar problem

    VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models

    Authors
    Yuxuan Wang, Yueqian Wang, Dongyan Zhao, Cihang Xie, Zilong Zheng
    Year
    2024
    Venue
    arXiv

    Korean review of T12-15

T13. Edge AI, streaming inference, and resource scheduling

Running models near the machine under limited compute and latency budgets.

Show this topic only
  1. T13-1Structural parallel

    Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge

    Authors
    Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, Lingjia Tang
    Year
    2017
    Venue
    ASPLOS 2017, pp. 615-629

    Korean review of T13-1

  2. T13-2Similar problem

    Live Video Analytics at Scale with Approximation and Delay-Tolerance

    Authors
    Haoyu Zhang, Ganesh Ananthanarayanan, Peter Bodik, Matthai Philipose, Paramvir Bahl, Michael J. Freedman
    Year
    2017
    Venue
    USENIX NSDI 2017, pp. 377-392

    Korean review of T13-2

  3. T13-3Similar problem

    AWStream: Adaptive Wide-Area Streaming Analytics

    Authors
    Ben Zhang, Xin Jin, Sylvia Ratnasamy, John Wawrzynek, Edward A. Lee
    Year
    2018
    Venue
    ACM SIGCOMM 2018, pp. 236-252

    Korean review of T13-3

  4. T13-4Similar problem

    Chameleon: Scalable Adaptation of Video Analytics

    Authors
    Junchen Jiang, Ganesh Ananthanarayanan, Peter Bodik, Siddhartha Sen, Ion Stoica
    Year
    2018
    Venue
    ACM SIGCOMM 2018, pp. 253-266

    Korean review of T13-4

  5. T13-5Similar problem

    Reducto: On-Camera Filtering for Resource-Efficient Real-Time Video Analytics

    Authors
    Yuanqi Li, Arthi Padmanabhan, Pengzhan Zhao, Yufei Wang, Guoqing Harry Xu, Ravi Netravali
    Year
    2020
    Venue
    ACM SIGCOMM 2020, pp. 359-376

    Korean review of T13-5

  6. T13-6Structural parallel

    Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers

    Authors
    Romil Bhardwaj, Zhengxu Xia, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, Nikolaos Karianakis, Kevin Hsieh, Paramvir Bahl, Ion Stoica
    Year
    2022
    Venue
    USENIX NSDI 2022, pp. 119-135

    Korean review of T13-6

  7. T13-7Structural parallel

    Clipper: A Low-Latency Online Prediction Serving System

    Authors
    Daniel Crankshaw, Xin Wang, Giulio Zhou, Michael J. Franklin, Joseph E. Gonzalez, Ion Stoica
    Year
    2017
    Venue
    USENIX NSDI 2017, pp. 613-627

    Korean review of T13-7

  8. T13-8Structural parallel

    Serving DNNs like Clockwork: Performance Predictability from the Bottom Up

    Authors
    Arpan Gujarati, Reza Karimi, Safya Alzayat, Wei Hao, Antoine Kaufmann, Ymir Vigfusson, Jonathan Mace
    Year
    2020
    Venue
    USENIX OSDI 2020, pp. 443-462

    Korean review of T13-8

  9. T13-9Candidate approach

    INFaaS

    Authors
    Francisco Romero, Qian Li, Neeraja J. Yadwadkar, Christos Kozyrakis
    Year
    2021
    Venue
    USENIX ATC 2021, pp. 397-411

    Korean review of T13-9

  10. T13-10Candidate approach

    Orca: A Distributed Serving System for Transformer-Based Generative Models

    Authors
    Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, Byung-Gon Chun
    Year
    2022
    Venue
    USENIX OSDI 2022, pp. 521-538

    Korean review of T13-10

  11. T13-11Candidate approach

    Efficient Memory Management for Large Language Model Serving with PagedAttention

    Authors
    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, Ion Stoica
    Year
    2023
    Venue
    ACM SOSP 2023, pp. 611-626

    Korean review of T13-11

  12. T13-12Candidate approach

    Gandiva: Introspective Cluster Scheduling for Deep Learning

    Authors
    Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, Lidong Zhou
    Year
    2018
    Venue
    USENIX OSDI 2018, pp. 595-610

    Korean review of T13-12

  13. T13-13Candidate approach

    Tiresias: A GPU Cluster Manager for Distributed Deep Learning

    Authors
    Juncheng Gu, Mosharaf Chowdhury, Kang G. Shin, Yibo Zhu, Myeongjae Jeon, Junjie Qian, Hongqiang Liu, Chuanxiong Guo
    Year
    2019
    Venue
    USENIX NSDI 2019, pp. 485-500

    Korean review of T13-13

  14. T13-14Candidate approach

    AntMan: Dynamic Scaling on GPU Clusters for Deep Learning

    Authors
    Wencong Xiao, Shiru Ren, Yong Li, Yang Zhang, Pengyang Hou, Zhi Li, Yihui Feng, Wei Lin, Yangqing Jia
    Year
    2020
    Venue
    USENIX OSDI 2020, pp. 533-548

    Korean review of T13-14

  15. T13-15Candidate approach

    Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads

    Authors
    Deepak Narayanan, Keshav Santhanam, Fiodar Kazhamiaka, Amar Phanishayee, Matei Zaharia
    Year
    2020
    Venue
    USENIX OSDI 2020, pp. 481-498

    Korean review of T13-15

  16. T13-16Candidate approach

    Pollux: Co-adaptive Cluster Scheduling for Goodput-Optimized Deep Learning

    Authors
    Aurick Qiao, Sang Keun Choe, Suhas Jayaram Subramanya, Willie Neiswanger, Qirong Ho, Hao Zhang, Gregory R. Ganger, Eric P. Xing
    Year
    2021
    Venue
    USENIX OSDI 2021, pp. 1-18

    Korean review of T13-16

  17. T13-17Structural parallel

    MLPerf Inference Benchmark

    Authors
    Vijay Janapa Reddi, Christine Cheng, David Kanter, Peter Mattson, Guenther Schmuelling, Carole-Jean Wu (47 authors in total)
    Year
    2020
    Venue
    ACM/IEEE ISCA 2020, pp. 446-459

    Korean review of T13-17

  18. T13-18Similar problem

    MillWheel: Fault-Tolerant Stream Processing at Internet Scale

    Authors
    Tyler Akidau, Alex Balikov, Kaya Bekiroğlu, Slava Chernyak, Josh Haberman, Reuven Lax, Sam McVeety, Daniel Mills, Paul Nordstrom, Sam Whittle
    Year
    2013
    Venue
    Proceedings of the VLDB Endowment, Vol. 6, No. 11, 2013, pp. 1033-1044

    Korean review of T13-18

  19. T13-19Similar problem

    The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing

    Authors
    Tyler Akidau, Robert Bradshaw, Craig Chambers, Slava Chernyak, Rafael J. Fernandez-Moctezuma, Reuven Lax, Sam McVeety, Daniel Mills, Frances Perry, Eric Schmidt, Sam Whittle
    Year
    2015
    Venue
    Proceedings of the VLDB Endowment, Vol. 8, No. 12, 2015, pp. 1792-1803

    Korean review of T13-19

T14. Industrial protocol translation, code generation, and program synthesis

Research on generating and checking the code that talks to plant equipment.

Show this topic only
  1. T14-5Structural parallelSince 2025

    Training LLMs for Generating IEC 61131-3 Structured Text with Online Feedback

    Authors
    Aaron Haag, Bertram Fuchs, Altay Kacan, Oliver Lohse
    Year
    2025
    Venue
    LLM4Code Workshop @ ICSE 2025

    Korean review of T14-5

  2. T14-1Similar problem

    Evaluating Large Language Models Trained on Code

    Authors
    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan (58 authors in total, OpenAI)
    Year
    2021

    Korean review of T14-1

  3. T14-2Similar problem

    ChatGPT for PLC/DCS Control Logic Generation

    Authors
    Heiko Koziolek, Sten Gruener, Virendra Ashiwal (ABB)
    Year
    2023
    Venue
    arXiv, pp. 8 6

    Korean review of T14-2

  4. T14-3Structural parallel

    LLM4PLC: Harnessing Large Language Models for Verifiable Programming of PLCs in Industrial Control Systems

    Authors
    Mohamad Fakih, Rahul Dharmaji, Yasamin Moghaddas, Gustavo Quiros Araya, Oluwatosin Ogundare, Mohammad Abdullah Al Faruque
    Year
    2024
    Venue
    ICSE 2024 SEIP

    Korean review of T14-3

  5. T14-4Candidate approach

    Agents4PLC: Automating Closed-loop PLC Code Generation and Verification in Industrial Control Systems using LLM-based Agents

    Authors
    Zihan Liu, Ruinan Zeng, Dongxia Wang, Gengyun Peng, Jingyi Wang, Qiang Liu, Peiyu Liu, Wenhai Wang
    Year
    2024
    Venue
    arXiv, pp. 12

    Korean review of T14-4

  6. T14-6Similar problem

    Automated Control Logic Test Case Generation using Large Language Models

    Authors
    Heiko Koziolek, Virendra Ashiwal, Soumyadip Bandyopadhyay, Chandrika K R
    Year
    2024
    Venue
    IEEE ETFA 2024 (29th IEEE International Conference on Emerging Technologies and Factory Automation), pp. 1-8

    Korean review of T14-6

  7. T14-7Similar problem

    Automated generation of OPC UA information models - A review and outlook

    Authors
    Axel Busboom
    Year
    2024
    Venue
    Journal of Industrial Information Integration (Elsevier), vol. 39, 100602

    Korean review of T14-7

  8. T14-8Candidate approach

    Discoverer: Automatic Protocol Reverse Engineering from Network Traces

    Authors
    Weidong Cui (Microsoft Research), Jayanthkumar Kannan (UC Berkeley), Helen J. Wang (Microsoft Research)
    Year
    2007
    Venue
    16th USENIX Security Symposium, 2007

    Korean review of T14-8

  9. T14-9Candidate approach

    NetPlier: Probabilistic Network Protocol Reverse Engineering from Message Traces

    Authors
    Yapeng Ye, Zhuo Zhang, Fei Wang, Xiangyu Zhang, Dongyan Xu (Purdue University)
    Year
    2021
    Venue
    NDSS 2021

    Korean review of T14-9

  10. T14-10Structural parallel

    Automated Attack Synthesis by Extracting Finite State Machines from Protocol Specification Documents

    Authors
    Maria Leonor Pacheco, Max von Hippel, Ben Weintraub, Dan Goldwasser, Cristina Nita-Rotaru
    Year
    2022
    Venue
    IEEE Symposium on Security and Privacy (S&P) 2022

    Korean review of T14-10

  11. T14-11Structural parallel

    Syntax-Guided Synthesis

    Authors
    Rajeev Alur, Rastislav Bodik, Garvit Juniwal, Milo M. K. Martin, Mukund Raghothaman, Sanjit A. Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, Abhishek Udupa
    Year
    2013
    Venue
    FMCAD 2013 (Formal Methods in Computer-Aided Design), pp. 1-8

    Korean review of T14-11

  12. T14-12Structural parallel

    Automating string processing in spreadsheets using input-output examples

    Authors
    Sumit Gulwani (Microsoft Research)
    Year
    2011
    Venue
    POPL 2011 (38th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages), pp. 317-330

    Korean review of T14-12

  13. T14-13Structural parallel

    Combinatorial Sketching for Finite Programs

    Authors
    Armando Solar-Lezama, Liviu Tancau, Rastislav Bodik, Vijay Saraswat, Sanjit Seshia
    Year
    2006
    Venue
    ASPLOS 2006 (12th International Conference on Architectural Support for Programming Languages and Operating Systems), pp. 404-415

    Korean review of T14-13

  14. T14-14Candidate approach

    Polyglot: Automatic Extraction of Protocol Message Format using Dynamic Binary Analysis

    Authors
    Juan Caballero, Heng Yin, Zhenkai Liang, Dawn Song
    Year
    2007
    Venue
    ACM CCS 2007 (14th ACM Conference on Computer and Communications Security), pp. 317-329

    Korean review of T14-14

  15. T14-15Similar problem

    Program Synthesis with Large Language Models

    Authors
    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, Charles Sutton (11 authors in total, Google Research)
    Year
    2021

    Korean review of T14-15

T15. OPC UA, Asset Administration Shell, MQTT, and manufacturing interoperability

Specifications and research for describing equipment and moving its data.

Show this topic only
  1. T15-2.10Structural parallelSince 2025

    ESP32-Based Sparkplug B Gateway Framework for Brownfield PLC Integration into an IIoT Unified Namespace: A Data Foundation for Intelligent Industrial Systems

    Authors
    Ivana Šenk, Srđan Tegeltija, Laslo Tarjan
    Year
    2026
    Venue
    Electronics 15(15), 3441, 2026

    Korean review of T15-2.10

  2. T15-2.1Similar problem

    The Future of Industrial Communication: Automation Networks in the Era of the Internet of Things and Industry 4.0

    Authors
    Martin Wollschlaeger, Thilo Sauter, Jürgen Jasperneite
    Year
    2017
    Venue
    IEEE Industrial Electronics Magazine 11(1), pp.17-27, 2017-03

    Korean review of T15-2.1

  3. T15-2.2Similar problem

    Insights into Mapping Solutions Based on OPC UA Information Model Applied to the Industry 4.0 Asset Administration Shell

    Authors
    Salvatore Cavalieri, Marco Giuseppe Salafia
    Year
    2020
    Venue
    Computers 9(2), 28, 2020

    Korean review of T15-2.2

  4. T15-2.3Structural parallel

    Reusing OPC UA information models in the Asset Administration Shell

    Authors
    Arno Weiss, Dirk Reichelt
    Year
    2023
    Venue
    2023 IEEE 21st International Conference on Industrial Informatics (INDIN)

    Korean review of T15-2.3

  5. T15-2.4Similar problem

    OPC UA versus ROS, DDS, and MQTT: Performance Evaluation of Industry 4.0 Protocols

    Authors
    Stefan Profanter, Ayhun Tekat, Kirill Dorofeev, Markus Rickert, Alois Knoll
    Year
    2019
    Venue
    2019 IEEE International Conference on Industrial Technology (ICIT), pp.955-962

    Korean review of T15-2.4

  6. T15-2.5Structural parallel

    Evaluation and Extension of OPC UA Publish/Subscribe MQTT Binding

    Authors
    Hannes Raddatz, Eman Mahmoud, Fabian Hölzke, Peter Danielis, Dirk Timmermann, Frank Golatowski
    Year
    2020
    Venue
    2020 IEEE Conference on Industrial Cyberphysical Systems (ICPS), pp.543-548, 2020-06

    Korean review of T15-2.5

  7. T15-2.6Candidate approach

    Open-Source Implementations of the Reactive Asset Administration Shell: A Survey

    Authors
    Michael Jacoby, Michael Baumann, Tino Bischoff, Hans Mees, Jens Müller, Ljiljana Stojanovic, Friedrich Volz
    Year
    2023
    Venue
    Sensors 23(11), 5229, 2023

    Korean review of T15-2.6

  8. T15-2.7Candidate approach

    Generation of Asset Administration Shell With Large Language Model Agents: Toward Semantic Interoperability in Digital Twins in the Context of Industry 4.0

    Authors
    Yuchen Xia, Zhewen Xiao, Nasser Jazdi, Michael Weyrich
    Year
    2024
    Venue
    IEEE Access, vol.12, pp.84863-84877, 2024. DOI 10.1109/ACCESS.2024.3415470

    Korean review of T15-2.7

  9. T15-2.8Structural parallel

    File- and API-based interoperability of digital twins by model transformation: An IIoT case study using asset administration shell

    Authors
    Marie Platenius-Mohr, Somayeh Malakuti, Sten Grüner, Johannes Schmitt, Thomas Goldschmidt
    Year
    2020
    Venue
    Future Generation Computer Systems, vol.113, pp.94-105, 2020-12

    Korean review of T15-2.8

  10. T15-2.9Similar problem

    Streaming Machine Generated Data via the MQTT Sparkplug B Protocol for Smart Factory Operations

    Authors
    Pavel Koprov, Ashwin Ramachandran, Yuan-Shin Lee, Paul Cohen, Binil Starly
    Year
    2022
    Venue
    Manufacturing Letters 33, pp.66-73, 2022

    Korean review of T15-2.9

  11. T15-2.11Structural parallel

    A Cyber-Physical Machine Tools Platform using OPC UA and MTConnect

    Authors
    Chao Liu, Hrishikesh Vengayil, Yuqian Lu, Xun Xu
    Year
    2019
    Venue
    Journal of Manufacturing Systems, vol.51, pp.61-74, 2019-04

    Korean review of T15-2.11

T16. Manufacturing knowledge graphs and semantic layers

Research on giving plant data a shared meaning across systems.

Show this topic only
  1. T16-8Candidate approachSince 2025

    Intent-Driven Smart Manufacturing Integrating Knowledge Graphs and Large Language Models

    Authors
    Takoua Jradi, John Violos, Dimitrios Spatharakis, Lydia Mavraidi, Ioannis Dimolitsas, Aris Leivadeas, Symeon Papavassiliou
    Year
    2026
    Venue
    arXiv (cs.AI)

    Korean review of T16-8

  2. T16-6Structural parallelSince 2025

    Knowledge Graphs as the Missing Data Layer for LLM-Based Industrial Asset Operations

    Authors
    Madhulatha Mandarapu, Sandeep Kunkunuru
    Year
    2026
    Venue
    Agents+Graph (AG2026) workshop, VLDB 2026

    Korean review of T16-6

  3. T16-7Similar problemSince 2025

    Fault Cause Identification across Manufacturing Lines through Ontology-Guided and Process-Aware FMEA Graph Learning with LLMs

    Authors
    Sho Okazaki, Kohei Kaminishi, Takuma Fujiu, Yusheng Wang, Manu Sasidharan, Jun Ota
    Year
    2026
    Venue
    arXiv (cs.IR)

    Korean review of T16-7

  4. T16-1Structural parallel

    The Industrial Ontologies Foundry (IOF) Core Ontology

    Authors
    Boonserm Kulvatunyou, Milos Drobnjakovic, Farhad Ameri, Chris Will, Barry Smith
    Year
    2022
    Venue
    Formal Ontologies Meet Industry (FOMI) 2022,, 2022-09-12~15

    Korean review of T16-1

  5. T16-2Structural parallel

    Semantic Integration of Bosch Manufacturing Data Using Virtual Knowledge Graphs

    Authors
    Elem Guzel Kalayci, Irlan Grangel Gonzalez, Felix Losch, Guohui Xiao, Anees ul Mehdi, Evgeny Kharlamov, Diego Calvanese
    Year
    2020
    Venue
    ISWC 2020 (19th International Semantic Web Conference), LNCS 12507, 464~pp. 481

    Korean review of T16-2

  6. T16-3Similar problem

    Knowledge Graphs in Manufacturing and Production: A Systematic Literature Review

    Authors
    Georg Buchgeher, David Gabauer, Jorge Martinez-Gil, Lisa Ehrlinger
    Year
    2021
    Venue
    IEEE Access, vol. 9, 55537~pp. 55554

    Korean review of T16-3

  7. T16-4Candidate approach

    Literal-Aware Knowledge Graph Embedding for Welding Quality Monitoring: A Bosch Case

    Authors
    Zhipeng Tan, Baifan Zhou, Zhuoxun Zheng, Ognjen Savkovic, Ziqi Huang, Irlan Grangel Gonzalez, Ahmet Soylu, Evgeny Kharlamov
    Year
    2023
    Venue
    ISWC 2023 In-Use Track

    Korean review of T16-4

  8. T16-5Structural parallel

    Generation of Asset Administration Shell with Large Language Model Agents

    Authors
    Yuchen Xia, Zhewen Xiao, Nasser Jazdi, Michael Weyrich
    Year
    2024
    Venue
    IEEE Access

    Korean review of T16-5

  9. T16-9Structural parallel

    Ontop: Answering SPARQL queries over relational databases

    Authors
    Diego Calvanese, Benjamin Cogrel, Sarah Komla-Ebri, Roman Kontchakov, Davide Lanti, Martin Rezk, Mariano Rodriguez-Muro, Guohui Xiao
    Year
    2017
    Venue
    Semantic Web Journal, vol. 8 no. 3, 471~pp. 487

    Korean review of T16-9

  10. T16-10Structural parallel

    RML: A Generic Language for Integrated RDF Mappings of Heterogeneous Data

    Authors
    Anastasia Dimou, Miel Vander Sande, Pieter Colpaert, Ruben Verborgh, Erik Mannens, Rik Van de Walle
    Year
    2014
    Venue
    Linked Data on the Web (LDOW 2014),, 2014-04-08, CEUR-WS Vol-1184

    Korean review of T16-10

  11. T16-12Similar problem

    The Industry 4.0 Standards Landscape from a Semantic Integration Perspective

    Authors
    Irlan Grangel-Gonzalez, Paul Baptista, Lavdim Halilaj, Steffen Lohmann, Maria-Esther Vidal, Christian Mader, Soren Auer
    Year
    2017
    Venue
    2017 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 1~pp. 8

    Korean review of T16-12

T17. Natural language to SQL and grounded report generation

Research on turning a question into a checked query and a sourced report.

Show this topic only
  1. T17-5Similar problemSince 2025

    Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows

    Authors
    Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao
    Year
    2025
    Venue
    ICLR 2025 Oral

    Korean review of T17-5

  2. T17-9Structural parallelSince 2025

    MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL

    Authors
    Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, LinZheng Chai, Zhao Yan, Qian-Wen Zhang, Di Yin, Xing Sun, Zhoujun Li
    Year
    2025
    Venue
    COLING 2025

    Korean review of T17-9

  3. T17-1Similar problem

    Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning

    Authors
    Victor Zhong, Caiming Xiong, Richard Socher
    Year
    2017
    Venue
    arXiv

    Korean review of T17-1

  4. T17-2Similar problem

    Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task

    Authors
    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga and 8 others
    Year
    2018
    Venue
    EMNLP 2018

    Korean review of T17-2

  5. T17-3Structural parallel

    Semantic Evaluation for Text-to-SQL with Distilled Test Suites

    Authors
    Ruiqi Zhong, Tao Yu, Dan Klein
    Year
    2020
    Venue
    EMNLP 2020 (Long Paper)

    Korean review of T17-3

  6. T17-4Similar problem

    Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs

    Authors
    Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang
    Year
    2023
    Venue
    NeurIPS 2023

    Korean review of T17-4

  7. T17-6Structural parallel

    RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers

    Authors
    Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, Matthew Richardson
    Year
    2020
    Venue
    ACL 2020 (Long Paper), pp. 7567-7578

    Korean review of T17-6

  8. T17-7Structural parallel

    DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction

    Authors
    Mohammadreza Pourreza, Davood Rafiei
    Year
    2023
    Venue
    NeurIPS 2023

    Korean review of T17-7

  9. T17-8Structural parallel

    Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation

    Authors
    Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, Jingren Zhou
    Year
    2024
    Venue
    Proceedings of the VLDB Endowment vol. 17 no. 5 (2024) pp. 1132-1145

    Korean review of T17-8

  10. T17-10Structural parallel

    CHESS: Contextual Harnessing for Efficient SQL Synthesis

    Authors
    Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, Amin Saberi
    Year
    2024
    Venue
    arXiv

    Korean review of T17-10

  11. T17-13Structural parallel

    Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models

    Authors
    Bernd Bohnet, Vinh Q
    Year
    2023
    Venue
    arXiv

    Korean review of T17-13

  12. T17-14Structural parallel

    RARR: Researching and Revising What Language Models Say, Using Language Models

    Authors
    Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, Kelvin Guu
    Year
    2023
    Venue
    ACL 2023 (Long Paper), pp. 16477-16508

    Korean review of T17-14

  13. T17-16Structural parallel

    FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation

    Authors
    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, Hannaneh Hajishirzi
    Year
    2023
    Venue
    EMNLP 2023

    Korean review of T17-16

  14. T17-17Similar problem

    ToTTo: A Controlled Table-To-Text Generation Dataset

    Authors
    Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, Dipanjan Das
    Year
    2020
    Venue
    EMNLP 2020

    Korean review of T17-17

  15. T17-18Similar problem

    QTSumm: Query-Focused Summarization over Tabular Data

    Authors
    Yilun Zhao, Zhenting Qi, Linyong Nan, Boyu Mi, Yixin Liu, Weijin Zou, Simeng Han, Ruizhe Chen, Xiangru Tang, Yumo Xu, Dragomir Radev, Arman Cohan
    Year
    2023
    Venue
    EMNLP 2023

    Korean review of T17-18

  16. T17-19Candidate approach

    InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks

    Authors
    Xueyu Hu, Ziyu Zhao, Shuang Wei
    Year
    2024
    Venue
    ICML 2024, PMLR vol. 235 pp. 19544-19572

    Korean review of T17-19

T18. AI quality management, model monitoring, and audit trails

Research and standards for keeping a deployed model accountable over time.

Show this topic only
  1. T18-12Structural parallelSince 2025

    Provenance Tracking in Large-Scale Machine Learning Systems

    Authors
    Gabriele Padovani, Valentine Anantharaj, Sandro Fiore
    Year
    2025
    Venue
    ICPP Workshops 2025

    Korean review of T18-12

  2. T18-11Candidate approachSince 2025

    Time to Retrain? Detecting Concept Drifts in Machine Learning Systems

    Authors
    Tri Minh Triet Pham, Karthikeyan Premkumar, Mohamed Naili, Jinqiu Yang
    Year
    2025
    Venue
    ICSE 2025

    Korean review of T18-11

  3. T18-1Similar problem

    Hidden Technical Debt in Machine Learning Systems

    Authors
    D. Sculley and 9 others (Google)
    Year
    2015
    Venue
    NeurIPS(NIPS) 28

    Korean review of T18-1

  4. T18-2Structural parallel

    The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction

    Authors
    Eric Breck, Shanqing Cai, Eric Nielsen, Michael Salib, D. Sculley
    Year
    2017
    Venue
    IEEE Big Data

    Korean review of T18-2

  5. T18-3Structural parallel

    Data Validation for Machine Learning

    Authors
    Neoklis Polyzotis, Martin Zinkevich, Sudip Roy, Eric Breck, Steven Whang
    Year
    2019
    Venue
    MLSys(Proceedings of Machine Learning and Systems) 1

    Korean review of T18-3

  6. T18-6Structural parallel

    Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing

    Authors
    Inioluwa Deborah Raji, Andrew Smart, Rebecca N. White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, Parker Barnes
    Year
    2020
    Venue
    ACM FAT* 2020

    Korean review of T18-6

  7. T18-7Similar problem

    A Survey on Concept Drift Adaptation

    Authors
    João Gama, Indrė Žliobaitė, Albert Bifet, Mykola Pechenizkiy, Abdelhamid Bouchachia
    Year
    2014
    Venue
    ACM Computing Surveys 46(4), 1-37

    Korean review of T18-7

  8. T18-8Similar problem

    Learning under Concept Drift: A Review

    Authors
    Jie Lu, Anjin Liu, Fan Dong, Feng Gu, João Gama, Guangquan Zhang
    Year
    2019
    Venue
    IEEE Transactions on Knowledge and Data Engineering 31(12), 2346-2363

    Korean review of T18-8

  9. T18-9Structural parallel

    Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift

    Authors
    Stephan Rabanser, Stephan Günnemann, Zachary C. Lipton
    Year
    2019
    Venue
    NeurIPS 2019 (Advances in Neural Information Processing Systems 32)

    Korean review of T18-9

  10. T18-10Similar problem

    Operationalizing Machine Learning: An Interview Study

    Authors
    Shreya Shankar, Rolando Garcia, Joseph M. Hellerstein, Aditya G. Parameswaran
    Year
    2022
    Venue
    arXiv (UC Berkeley)

    Korean review of T18-10

From research to product use

Operating capabilities, pilots, and technologies in development are identified separately.

View technology