publications | Zlatan Ajanović

2026

From Language Priors to Field Adaptation: Preference Learning for Traversability Estimation

Simon Schwaiger, David Seyser, Alessandro Scherl, Zlatan Ajanović, Wilfried Wöber, and Gerald Steinbauer-Wagner

arXiv preprint arXiv:2610.02974, Oct 2026

Abs arXiv Bib PDF Code Website

Image-based traversability estimation is inherently dependent on the robot platform, deployment domain, and mission preferences, which limits the applicability of purpose-trained models. To facilitate domain adaptation, this work aims to reduce the number of required annotations in the target domain using sample-efficient preference learning. Our method represents traversability through von Mises-Fisher mixture prototypes in a frozen vision-language feature space. Relative natural-language rules provide a commonsense prior, while sparse relative image annotations adapt the prototype directions and utilities to a target domain through computationally and sample-efficient fine-tuning. Experiments on WayFAST demonstrate accuracy competitive with end-to-end trained estimators while enabling sample-efficient image-based adaptation. Qualitative experiments further demonstrate the language prior’s zero shot applicability and the fine-tuned estimator’s improved dense prediction on semantic maps. Evaluation is complemented via semantic interpretation of learned prototypes by dissecting semantically close natural language prompts. Code and trained estimators available at https://resireg.github.io.
@article{schwaiger2026language, title = {From Language Priors to Field Adaptation: Preference Learning for Traversability Estimation}, author = {Schwaiger, Simon and Seyser, David and Scherl, Alessandro and Ajanović, Zlatan and Wöber, Wilfried and Steinbauer-Wagner, Gerald}, journal = {arXiv preprint arXiv:2610.02974}, year = {2026}, month = oct, url = {https://arxiv.org/abs/2610.02974} }
LLM-Guided Task- and Affordance-Level Exploration in Reinforcement Learning

Jelle Luijkx, Runyu Ma, Zlatan Ajanović, and Jens Kober

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Jun 2026

Abs DOI arXiv Bib PDF Code Website

Reinforcement learning (RL) is a promising approach for robotic manipulation, but it can suffer from low sample efficiency and requires extensive exploration of large state-action spaces. Recent methods leverage the commonsense knowledge and reasoning abilities of large language models (LLMs) to guide exploration toward more meaningful states. However, LLMs can produce plans that are semantically plausible yet physically infeasible, yielding unreliable behavior. We introduce LLM-TALE, a framework that uses LLMs’ planning to directly steer RL exploration. LLM-TALE integrates planning at both the task level and the affordance level, improving learning efficiency by directing agents toward semantically meaningful actions. Unlike prior approaches that assume optimal LLM-generated plans or rewards, LLM-TALE corrects suboptimality online and explores multimodal affordance-level plans without human supervision.
@inproceedings{luijkx2026llm, doi = {10.1109/ICRA57385.2026.11695912}, title = {LLM-Guided Task- and Affordance-Level Exploration in Reinforcement Learning}, author = {Luijkx, Jelle and Ma, Runyu and Ajanović, Zlatan and Kober, Jens}, booktitle = {Proceedings of the IEEE International Conference on Robotics and Automation (ICRA)}, year = {2026}, month = jun, url = {https://arxiv.org/abs/2509.16615}, }
Novelty-based Tree-of-Thought Search for LLM Reasoning and Planning

Leon Hamm and Zlatan Ajanović

arXiv preprint arXiv:2605.06040, May 2026

Abs arXiv Bib PDF

Although advances such as chain-of-thought, tree-of-thought or reinforcement learning have improved the performance of LLMs in reasoning and planning tasks, they are still brittle and have not achieved human-level performance in many domains, and often suffer from high time and token costs. Inspired by the success of width-based search in planning, we explore how the concept of novelty can be transferred to language domains and how it can improve tree-of-thought reasoning. A tree of thoughts relies on building possible "paths" of consecutive ideas or thoughts. These are generated by repeatedly prompting an LLM. In our paper, a measurable concept of novelty is proposed that describes the uniqueness of a new node (thought) in comparison to nodes previously seen in the search tree. Novelty is estimated by prompting an LLM and making use of embedded general knowledge from pretraining. This metric can then be used to prune branches and reduce the scope of the search. Although this method introduces more prompts per state, the overall token cost can be reduced by pruning and reducing the overall tree size. This procedure is tested and compared using several benchmarks in language-based planning and general reasoning.
@article{hamm2026novelty, title = {Novelty-based Tree-of-Thought Search for LLM Reasoning and Planning}, author = {Hamm, Leon and Ajanović, Zlatan}, journal = {arXiv preprint arXiv:2605.06040}, year = {2026}, month = may, url = {https://arxiv.org/abs/2605.06040} }
Search Inspired Exploration in Reinforcement Learning

Georgios Sotirchos, Zlatan Ajanović, and Jens Kober

arXiv preprint arXiv:2602.00460, Jan 2026

Abs arXiv Bib PDF

Exploration in environments with sparse rewards remains a fundamental challenge in reinforcement learning (RL). Existing approaches such as curriculum learning and Go-Explore often rely on hand-crafted heuristics, while curiosity-driven methods risk converging to suboptimal policies. We propose Search-Inspired Exploration in Reinforcement Learning (SIERL), a novel method that actively guides exploration by setting sub-goals based on the agent’s learning progress. At the beginning of each episode, SIERL chooses a sub-goal from the frontier (the boundary of the agent’s known state space), before the agent continues exploring toward the main task objective. The key contribution of our method is the sub-goal selection mechanism, which provides state-action pairs that are neither overly familiar nor completely novel, assuring that the frontier is expanded systematically and that the agent is capable of reaching any state within it.
@article{sotirchos2026search, title = {Search Inspired Exploration in Reinforcement Learning}, author = {Sotirchos, Georgios and Ajanović, Zlatan and Kober, Jens}, journal = {arXiv preprint arXiv:2602.00460}, year = {2026}, month = jan, url = {https://arxiv.org/abs/2602.00460}, }

SYMBOLIZER: Symbolic Model-free Task Planning with VLMs

Sami Azirar, Zlatan Ajanović, and Hermann Blum

IEEE Robotics and Automation Letters, 2026

@article{azirar2026symbolizer,
  doi = {10.1109/LRA.2026.3726377},
  title = {SYMBOLIZER: Symbolic Model-free Task Planning with VLMs},
  author = {Azirar, Sami and Ajanović, Zlatan and Blum, Hermann},
  journal = {IEEE Robotics and Automation Letters},
  year = {2026},
  url = {https://ieeexplore.ieee.org/document/11662377}
}

Sequentially Teaching Sequential Tasks (ST)^2: Teaching Robots Long-Horizon Manipulation Skills

Zlatan Ajanović, Ravi Prakash, Leandro de Souza Rosa, and Jens Kober

IEEE Robotics & Automation Magazine, 2026

Abs DOI arXiv Bib PDF Website

Learning from demonstration has proved itself useful for teaching robots complex skills with high sample efficiency. However, teaching long-horizon tasks with multiple skills is challenging as deviations tend to accumulate, the distributional shift becomes more evident, and human teachers become fatigued over time, thereby increasing the likelihood of failure. To address these challenges, we introduce (ST)^2, a sequential method for learning long-horizon manipulation tasks that allows users to control teaching flow by specifying keypoints, enabling structured and incremental demonstrations. Using this framework, we study how users respond to two teaching paradigms: a traditional monolithic approach in which users demonstrate the entire task trajectory at once, and a sequential approach in which the task is segmented and demonstrated step by step.
@article{ajanovic2026sequentially, doi = {10.1109/MRA.2026.3650853}, title = {Sequentially Teaching Sequential Tasks $(ST)^2$: Teaching Robots Long-Horizon Manipulation Skills}, author = {Ajanović, Zlatan and Prakash, Ravi and Rosa, Leandro de Souza and Kober, Jens}, journal = {IEEE Robotics \& Automation Magazine}, year = {2026}, pages = {130--141}, volume = {33}, number = {1}, url = {https://ieeexplore.ieee.org/document/11369949/}, }

2025

Scenario-Based Hierarchical Reinforcement Learning for Automated Driving Decision Making

M. Youssef Abdelhamid, Lennart Vater, and Zlatan Ajanović

In 2025 XXX International Conference on Information, Communication and Automation Technologies (ICAT), Sep 2025

Abs DOI arXiv Bib PDF

Developing decision-making algorithms for highly automated driving systems remains challenging, since these systems have to operate safely in open and complex environments. We present Scenario-based Automated Driving Reinforcement Learning (SAD-RL), the first framework that integrates hierarchical reinforcement learning in a scenario-based environment, achieving safe and efficient decision policies.
@inproceedings{abdelhamid2025scenario, title = {Scenario-Based Hierarchical Reinforcement Learning for Automated Driving Decision Making}, booktitle = {2025 XXX International Conference on Information, Communication and Automation Technologies (ICAT)}, author = {Abdelhamid, M. Youssef and Vater, Lennart and Ajanović, Zlatan}, year = {2025}, month = sep, pages = {1--6}, issn = {2643-1858}, doi = {10.1109/ICAT66432.2025.11189264}, url = {https://ieeexplore.ieee.org/document/11189264}, keywords = {Automated Driving, Hierarchical Reinforcement Learning, Scenario-based Learning} }

Search-Based Robot Motion Planning With Distance-Based Adaptive Motion Primitives

Benjamin Kraljušić, Zlatan Ajanović, Nermin Čović, and Bakir Lačević

In 2025 XXX International Conference on Information, Communication and Automation Technologies (ICAT), Sep 2025

Abs DOI arXiv Bib PDF

@inproceedings{kraljusic2025search,
  title = {Search-Based Robot Motion Planning With Distance-Based Adaptive Motion Primitives},
  booktitle = {2025 XXX International Conference on Information, Communication and Automation Technologies (ICAT)},
  author = {Kraljušić, Benjamin and Ajanović, Zlatan and Čović, Nermin and Lačević, Bakir},
  year = {2025},
  month = sep,
  pages = {1--6},
  issn = {2643-1858},
  doi = {10.1109/ICAT66432.2025.11189289},
  url = {https://ieeexplore.ieee.org/abstract/document/11189289}
}

PARTNR: Pick and place Ambiguity Resolving by Trustworthy Interactive Learning

Jelle Luijkx, Zlatan Ajanović, Laura Ferranti, and Jens Kober

In 2025 XXX International Conference on Information, Communication and Automation Technologies (ICAT), Sep 2025

Abs DOI arXiv Bib PDF Video Website

@inproceedings{luijkx2025partnr,
  title = {PARTNR: Pick and place Ambiguity Resolving by Trustworthy Interactive Learning},
  booktitle = {2025 XXX International Conference on Information, Communication and Automation Technologies (ICAT)},
  author = {Luijkx, Jelle and Ajanović, Zlatan and Ferranti, Laura and Kober, Jens},
  year = {2025},
  month = sep,
  pages = {1--6},
  issn = {2643-1858},
  doi = {10.1109/ICAT66432.2025.11189274},
  url = {https://ieeexplore.ieee.org/abstract/document/11189274}
}

ASkDAgger: Active Skill-level Data Aggregation for Interactive Imitation Learning

Jelle Luijkx, Zlatan Ajanović, Laura Ferranti, and Jens Kober

Transactions on Machine Learning Research, Aug 2025

Abs arXiv Bib HTML PDF Code Website

We propose ASkDAgger, a framework for active skill-level data aggregation in imitation learning that balances human query effort and learning efficiency through adaptive gating and prioritized replay. Demonstrated on language-conditioned robotic tasks.
@article{luijkx2025askdagger, title = {ASkDAgger: Active Skill-level Data Aggregation for Interactive Imitation Learning}, author = {Luijkx, Jelle and Ajanović, Zlatan and Ferranti, Laura and Kober, Jens}, journal = {Transactions on Machine Learning Research}, year = {2025}, month = aug, url = {https://openreview.net/forum?id=987Az9f8fT}, }

Mode Collapse Happens: Evaluating Critical Interactions in Joint Trajectory Prediction Models

Maarten Hugenholtz, Anna Meszaros, Jens Kober, and Zlatan Ajanović

TechRxiv, 2025

Abs DOI arXiv Bib PDF

@article{hugenholtz2025mode,
  title = {Mode Collapse Happens: Evaluating Critical Interactions in Joint Trajectory Prediction Models},
  author = {Hugenholtz, Maarten and Meszaros, Anna and Kober, Jens and Ajanović, Zlatan},
  journal = {TechRxiv},
  year = {2025},
  doi = {10.36227/techrxiv.173611438.86372015/v1},
  url = {https://www.authorea.com/users/870022/articles/1255510-mode-collapse-happens-evaluating-critical-interactions-in-joint-trajectory-prediction-models},
}

ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models

Runyu Ma, Jelle Luijkx, Zlatan Ajanović, and Jens Kober

In 2025 IEEE International Conference on Robotics and Automation (ICRA), 2025

Abs DOI arXiv Bib PDF Website

We introduce ExploRLLM, a method that guides reinforcement learning exploration using large language model priors. It accelerates convergence and improves performance in robot manipulation tasks with limited data.
@inproceedings{ma2025explorllm, doi = {10.1109/ICRA55743.2025.11127622}, title = {ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models}, booktitle = {2025 IEEE International Conference on Robotics and Automation (ICRA)}, author = {Ma, Runyu and Luijkx, Jelle and Ajanović, Zlatan and Kober, Jens}, year = {2025}, url = {https://arxiv.org/abs/2403.09583}, }

2024

SLOPE: Search with Learned Optimal Pruning-based Expansion

Davor Bokan, Zlatan Ajanović, and Bakir Lačević

In ICAPS 2024 Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL), Jun 2024

Abs arXiv Bib PDF Code

We propose SLOPE, a heuristic search algorithm that prunes suboptimal nodes using learned optimal pruning-based expansion. It reduces memory and computational costs while maintaining near-optimal performance in complex motion planning problems.
@inproceedings{bokan2024slope, title = {SLOPE: Search with Learned Optimal Pruning-based Expansion}, author = {Bokan, Davor and Ajanović, Zlatan and Lačević, Bakir}, booktitle = {ICAPS 2024 Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL)}, year = {2024}, month = jun, url = {https://arxiv.org/abs/2406.04935} }
Search-Based versus Sampling-based Robot Motion Planning: A Comparative Study

Georgios Sotirchos and Zlatan Ajanović

In ICAPS 2024 Workshop on Planning and Robotics (PlanRob), Jun 2024

Abs arXiv Bib PDF Code

We benchmark search-based and sampling-based motion planning methods under unified conditions. Results show ARA* performs competitively in structured spaces, while RRT-Connect scales better in higher dimensions.
@inproceedings{sotirchos2024search, title = {Search-Based versus Sampling-based Robot Motion Planning: A Comparative Study}, author = {Sotirchos, Georgios and Ajanović, Zlatan}, booktitle = {ICAPS 2024 Workshop on Planning and Robotics (PlanRob)}, year = {2024}, month = jun, url = {https://arxiv.org/abs/2406.09623} }
Good Practices for Institutional Organization of Research Institutes: Excellence in Research and Positive Impact on Society

Zlatan Ajanović, Hamza Merzić, Suad Krilasević, Eldar Kurtić, Bakir Kudić, Rialda Spahić, Emina Aličković, Aida Branković, Kenan Šehić, Mirsad Ćosović, Admir Greljo, Sead Delalić, and Adnan Mehonić

In Scientific-Professional Conference “Artificial Intelligence in Bosnia and Herzegovina”- Research, Application and Development Perspectives, 2024

Abs arXiv PDF

In this paper, we analyze examples of research institutes that stand out in scientific excellence and social impact. We define key practices for evaluating research results, economic conditions, and the selection of specific research topics. Special focus is placed on small countries and the field of artificial intelligence. The aim is to identify components that enable institutes to achieve a high level of innovation, self-sustainability, and social benefits.

2023

Search-Based Task and Motion Planning for Hybrid Systems: Agile Autonomous Vehicles

Zlatan Ajanović, Enrico Regolin, Barys Shyrokau, Hana Ćatić, Martin Horn, and Antonella Ferrara

Engineering Applications of Artificial Intelligence, 2023

Abs DOI arXiv Bib PDF Code

To achieve optimal robot behavior in dynamic scenarios we need to consider complex dynamics in a predictive manner. In the vehicle dynamics community, it is well know that to achieve time-optimal driving on low friction surface, the vehicle should utilize drifting. Hence, many authors have devised rules to split circuits and employ drifting on some segments. These rules are suboptimal and do not generalize to arbitrary circuit shapes (e.g., S-like curves). So, the question “When to go into which mode and how to drive in it?” remains unanswered. To choose the suitable mode (discrete decision), the algorithm needs information about the feasibility of different modes (continuous motion). This makes it a class of Task and Motion Planning (TAMP) problems, which are known to be hard to solve optimally in real-time. In the AI planning community, search methods are commonly used. However, they cannot be directly applied to TAMP problems due to the continuous component. Here, we present a search-based method that effectively solves this problem and efficiently searches in a highly dimensional state space with nonlinear and unstable dynamics. The space of the possible trajectories is explored by sampling different combinations of motion primitives guided by the search. Our approach allows to use multiple locally approximated models to generate motion primitives (e.g., learned models of drifting) and effectively simplify the problem without losing accuracy. The algorithm performance is evaluated in simulated driving on a mixed-track with segments of different curvatures (right and left). Our code is available at https://git.io/JenvB.
@article{ajanovic2023search, title = {Search-Based Task and Motion Planning for Hybrid Systems: {{Agile}} Autonomous Vehicles}, shorttitle = {Search-Based Task and Motion Planning for Hybrid Systems}, author = {Ajanovi\'c, Zlatan and Regolin, Enrico and Shyrokau, Barys and \'Cati\'c, Hana and Horn, Martin and Ferrara, Antonella}, year = {2023}, journal = {Engineering Applications of Artificial Intelligence}, shortjournal = {Engineering Applications of Artificial Intelligence}, volume = {121}, pages = {105893}, issn = {0952-1976}, doi = {10.1016/j.engappai.2023.105893}, keywords = {Autonomous vehicles,Behavior planning,Hybrid systems,Motion planning,Search,Task and motion planning}, }
Robotic Packaging Optimization with Reinforcement Learning

Eveline Drijver, Rodrigo Pérez-Dattari, Jens Kober, Cosimo Della Santina, and Zlatan Ajanović

In 2023 IEEE International Conference on Automation Science and Engineering, 2023

Abs DOI arXiv Bib PDF Video

Intelligent manufacturing is becoming increasingly important due to the growing demand for maximizing productivity and flexibility while minimizing waste and lead times. This work investigates automated secondary robotic food packaging solutions that transfer food products from the conveyor belt into containers. A major problem in these solutions is varying product supply which can cause drastic productivity drops. Conventional rule-based approaches, used to address this issue, are often inadequate, leading to violation of the industry’s requirements. Reinforcement learning, on the other hand, has the potential of solving this problem by learning responsive and predictive policy, based on experience. However, it is challenging to utilize it in highly complex control schemes. In this paper, we propose a reinforcement learning framework, designed to optimize the conveyor belt speed while minimizing interference with the rest of the control system. When tested on real-world data, the framework exceeds the performance requirements (99.8% packed products) and maintains quality (100% filled boxes). Compared to the existing solution, our proposed framework improves productivity, has smoother control, and reduces computation time.
@inproceedings{drijver2023robotic, doi = {10.1109/CASE56687.2023.10260406}, title = {Robotic Packaging Optimization with Reinforcement Learning}, author = {Drijver, Eveline and Pérez-Dattari, Rodrigo and Kober, Jens and Santina, Cosimo Della and Ajanović, Zlatan}, year = {2023}, booktitle = {2023 IEEE International Conference on Automation Science and Engineering}, archiveprefix = {arXiv}, primaryclass = {cs.RO}, url = {https://arxiv.org/abs/2303.14693}, }

2022

Vision for Bosnia and Herzegovina in Artificial Intelligence Age: Global Trends, Potential Opportunities, Selected Use-cases and Realistic Goals

Zlatan Ajanović, Emina Aličković, Aida Branković, Sead Delalić, Eldar Kurtić, Salem Malikić, Adnan Mehonić, Hamza Merzić, Kenan Šehić, and Bahrudin Trbalić

In Scientific-Professional Conference “Artificial Intelligence in Bosnia and Herzegovina”- Research, Application and Development Perspectives, 2022

Abs arXiv Bib PDF

Artificial Intelligence (AI) is one of the most promising technologies of the 21. century, with an already noticeable impact on society and the economy. With this work, we provide a short overview of global trends, applications in industry and selected use-cases from our international experience and work in industry and academia. The goal is to present global and regional positive practices and provide an informed opinion on the realistic goals and opportunities for positioning B&H on the global AI scene.

@inproceedings{ajanovic2022vision,
  title = {Vision for {{Bosnia}} and {{Herzegovina}} in {{Artificial Intelligence Age}}: {{Global Trends}}, {{Potential Opportunities}}, {{Selected Use-cases}} and {{Realistic Goals}}},
  shorttitle = {Vision for {{Bosnia}} and {{Herzegovina}} in {{Artificial Intelligence Age}}},
  booktitle = {Scientific-{{Professional Conference}} ``{{Artificial Intelligence}} in {{Bosnia}} and {{Herzegovina}}''- Research, Application and Development Perspectives},
  author = {Ajanovi\'c, Zlatan and Ali\v{c}kovi\'c, Emina and Brankovi\'c, Aida and Delali\'c, Sead and Kurti\'c, Eldar and Maliki\'c, Salem and Mehoni\'c, Adnan and Merzi\'c, Hamza and \v{S}ehi\'c, Kenan and Trbali\'c, Bahrudin},
  year = {2022},
  eprinttype = {arxiv},
  eprintclass = {cs},
  pages = {13--46},
  publisher = {{Federalno ministarstvo obrazovanja i nauke/znanosti : Fondacija za inovacijski i tehnolo\v{s}ki razvoj}},
  keywords = {Computer Science - Artificial Intelligence}
}

Interactive Imitation Learning in Robotics: A Survey

Carlos Celemin, Rodrigo Pérez-Dattari, Eugenio Chisari, Giovanni Franzese, Leandro Souza Rosa, Ravi Prakash, Zlatan Ajanović, Marta Ferraz, Abhinav Valada, and Jens Kober

Foundations and Trends in Robotics, 2022

Abs DOI arXiv Bib PDF

Interactive Imitation Learning (IIL) is a branch of Imitation Learning (IL) where human feedback is provided intermittently during robot execution allowing an online improvement of the robot’s behavior. In recent years, IIL has increasingly started to carve out its own space as a promising data-driven alternative for solving complex robotic tasks. The advantages of IIL are twofold, 1) it is data-efficient, as the human feedback guides the robot directly towards an improved behavior (in contrast with Reinforcement Learning (RL), where behaviors must be discovered by trial and error), and 2) it is robust, as the distribution mismatch between the teacher and learner trajectories is minimized by providing feedback directly over the learner’s trajectories (as opposed to offline IL methods such as Behavioral Cloning). Nevertheless, despite the opportunities that IIL presents, its terminology, structure, and applicability are not clear nor unified in the literature, slowing down its development and, therefore, the research of innovative formulations and discoveries. In this work, we attempt to facilitate research in IIL and lower entry barriers for new practitioners by providing a survey of the field that unifies and structures it. In addition, we aim to raise awareness of its potential, what has been accomplished and what are still open research questions. We organize the most relevant works in IIL in terms of human-robot interaction (i.e., types of feedback), interfaces (i.e., means of providing feedback), learning (i.e., models learned from feedback and function approximators), user experience (i.e., human perception about the learning process), applications, and benchmarks. Furthermore, we analyze similarities and differences between IIL and RL, providing a discussion on how the concepts offline, online, off-policy and on-policy learning should be transferred to IIL from the RL literature. We particularly focus on robotic applications in the real world and discuss their implications, limitations, and promising future areas of research.
@article{celemin2022interactive, title = {Interactive Imitation Learning in Robotics: {{A}} Survey}, author = {Celemin, Carlos and P\'erez-Dattari, Rodrigo and Chisari, Eugenio and Franzese, Giovanni and de Souza Rosa, Leandro and Prakash, Ravi and Ajanovi\'c, Zlatan and Ferraz, Marta and Valada, Abhinav and Kober, Jens}, year = {2022}, journal = {Foundations and Trends in Robotics}, volume = {10}, number = {1-2}, pages = {1--197}, issn = {1935-8253}, doi = {10.1561/2300000072}, }
PARTNR: Pick and Place Ambiguity Resolving by Trustworthy iNteractive leaRning

Jelle Luijkx, Zlatan Ajanovic, Laura Ferranti, and Jens Kober

In NeurIPS 2022 Workshop on Robot Learning, 2022

Abs arXiv Bib PDF Video Website

Several recent works show impressive results in mapping language-based human commands and image scene observations to direct robot executable policies (e.g., pick and place poses). However, these approaches do not consider the uncertainty of the trained policy and simply always execute actions suggested by the current policy as the most probable ones. This makes them vulnerable to domain shift and inefficient in the number of required demonstrations. We extend previous works and present the PARTNR algorithm that can detect ambiguities in the trained policy by analyzing multiple modalities in the pick and place poses using topological analysis. PARTNR employs an adaptive, sensitivity-based, gating function that decides if additional user demonstrations are required. User demonstrations are aggregated to the dataset and used for subsequent training. In this way, the policy can adapt promptly to domain shift and it can minimize the number of required demonstrations for a well-trained policy. The adaptive threshold enables to achieve the user-acceptable level of ambiguity to execute the policy autonomously and in turn, increase the trustworthiness of our system. We demonstrate the performance of PARTNR in a table-top pick and place task.
@inproceedings{luijkx2022partnr, title = {{{PARTNR}}: {{Pick}} and Place {{Ambiguity Resolving}} by {{Trustworthy iNteractive leaRning}}}, shorttitle = {{{PARTNR}}}, booktitle = {{{NeurIPS}} 2022 {{Workshop}} on {{Robot Learning}}}, author = {Luijkx, Jelle and Ajanovic, Zlatan and Ferranti, Laura and Kober, Jens}, year = {2022}, eprinttype = {arxiv}, eprintclass = {cs}, publisher = {{arXiv}}, keywords = {{68T05, 68T07, 68T40, 68T45, 68T50},Computer Science - Artificial Intelligence,Computer Science - Computation and Language,Computer Science - Human-Computer Interaction,Computer Science - Machine Learning,Computer Science - Robotics,I.2.10,I.2.6,I.2.7,I.2.9} }

2020

A Multi-Heuristic Search-Based Motion Planning for Autonomous Parking

Bhargav Adabala and Zlatan Ajanovic

In 30th International Conference on Automated Planning and Scheduling: Planning and Robotics Workshop, 2020

Abs Bib PDF

Planning is a crucial component of autonomous vehicle control. It is responsible for finding a collision-free sequence of states that take the vehicle towards its goal. In unstructured environments like parking lots or construction sites, due to the large search-space and kinodynamic constraints of the vehicle, real-time planning is challenging. Several state-ofthe-art solutions utilize heuristic search-based planning algorithms. However, they heavily rely on the quality of the single heuristic function used to guide the search, and they are not capable to achieve reasonable performance, resulting in unnecessary delays in the response of the vehicle. This work solves the planning problem by adopting a Multi-Heuristic Search approach, that enables the use of multiple heuristic functions and their advantages to capture different complexities of a given search space. Based on our knowledge, this approach was not used for this domain so far. For this purpose, multiple admissible and non-admissible heuristic functions are defined, original Multi-Heuristic A* Search was extended for bidirectional use and dealing with hybrid continuous-discrete search space and a mechanism for adapting scale of motion primitives is introduced. To demonstrate the advantage, Multi-Heuristic A* algorithm is benchmarked against a very popular heuristic search-based algorithm, Hybrid A*. The Multi-Heuristic A* algorithm outperformed Hybrid A* in terms of computation efficiency and motion plan (path) quality.
@inproceedings{adabala2020multi, title = {A Multi-Heuristic Search-Based Motion Planning for Autonomous Parking}, booktitle = {30th International Conference on Automated Planning and Scheduling: {{Planning}} and Robotics Workshop}, author = {Adabala, Bhargav and Ajanovic, Zlatan}, year = {2020}, }

Overview of Tools Supporting Planning for Automated Driving

Kailin Tong, Zlatan Ajanovic, and Georg Stettinger

In 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), 2020

DOI arXiv Bib PDF

@inproceedings{tong2020overview,
  doi = {10.1109/ITSC45102.2020.9294512},
  title = {Overview of Tools Supporting Planning for Automated Driving},
  author = {Tong, Kailin and Ajanovic, Zlatan and Stettinger, Georg},
  booktitle = {2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC)},
  pages = {1--8},
  year = {2020},
  organization = {IEEE}
}

Search-Based Motion Planning for Performance Autonomous Driving

Zlatan Ajanovic, Enrico Regolin, Georg Stettinger, Martin Horn, and Antonella Ferrara

In Advances in Dynamics of Vehicles on Roads and Tracks, Cham, 2020

Abs DOI arXiv Bib PDF Code

Driving on the limits of vehicle dynamics requires predictive planning of future vehicle states. In this work, a search-based motion planning is used to generate suitable reference trajectories of dynamic vehicle states with the goal to achieve the minimum lap time on slippery roads. The search-based approach enables to explicitly consider a nonlinear vehicle dynamics model as well as constraints on states and inputs so that even challenging scenarios can be achieved in a safe and optimal way. The algorithm performance is evaluated in simulated driving on a track with segments of different curvatures. Our code is available at https://git.io/JenvB.
@inproceedings{ajanovic2020search, title = {Search-{{Based Motion Planning}} for {{Performance Autonomous Driving}}}, booktitle = {Advances in {{Dynamics}} of {{Vehicles}} on {{Roads}} and {{Tracks}}}, author = {Ajanovic, Zlatan and Regolin, Enrico and Stettinger, Georg and Horn, Martin and Ferrara, Antonella}, editor = {Klomp, Matthijs and Bruzelius, Fredrik and Nielsen, Jens and Hillemyr, Angela}, year = {2020}, series = {Lecture {{Notes}} in {{Mechanical Engineering}}}, pages = {1144--1154}, publisher = {{Springer International Publishing}}, location = {{Cham}}, doi = {10.1007/978-3-030-38077-9_134}, isbn = {978-3-030-38077-9}, keywords = {Autonomous vehicles,Drifting,Motion planning,Trail-braking} }
Validating SuperHuman Automated Driving Performance

Zlatan Ajanovic, Matthijs Klomp, Bakir Lacevic, Barys Shyrokau, Paolo Pretto, Hassaan Islam, Georg Stettinger, and Martin Horn

In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2020

Abs DOI Bib PDF

Closed-loop validation of autonomous vehicles is an open problem, significantly influencing development and adoption of this technology. The main contribution of this paper is a novel approach to reproducible, scenario-based validation that decouples the problem into several sub-problems, while avoiding to brake the crucial couplings. First, a realistic scenario is generated from the real urban traffic. Second, human participants, drive in a virtual scenario (in a driving simulator), based on the real traffic. Third, human and automated driving trajectories are reproduced and compared in the real vehicle on an empty track without traffic. Thus, benefits of automation with respect to safety, efficiency and comfort can be clearly benchmarked in a reproducible manner. Presented approach is used to benchmark performance of SBOMP planner in one scenario and validate SuperHuman driving performance.
@inproceedings{ajanovic2020validating, title = {Validating {{SuperHuman Automated Driving Performance}}}, booktitle = {2020 {{IEEE International Conference}} on {{Systems}}, {{Man}}, and {{Cybernetics}} ({{SMC}})}, author = {Ajanovic, Zlatan and Klomp, Matthijs and Lacevic, Bakir and Shyrokau, Barys and Pretto, Paolo and Islam, Hassaan and Stettinger, Georg and Horn, Martin}, year = {2020}, pages = {3860--3867}, issn = {2577-1655}, doi = {10.1109/SMC42975.2020.9282822}, eventtitle = {2020 {{IEEE International Conference}} on {{Systems}}, {{Man}}, and {{Cybernetics}} ({{SMC}})}, keywords = {Accidents,automated driving,Benchmark testing,Couplings,lane change,multi-lane driving,planning,Safety,SuperHuman,Tools,traffic lights,Trajectory,urban driving,validation,Vehicles}, }

2019

PhD
Towards SuperHuman Autonomous Vehicles

Zlatan Ajanovic

Graz University of Technology, Dec 2019

Abs Bib HTML PDF

This thesis presents behavior and motion planning method that enables Autonomous Vehicles (AV) to achieve SuperHuman driving performance in terms of safety, efficiency and comfort. The developed method enables synergy of research in behavior and motion planning for Automated Driving, with research in eco-driving community, which target mainly complementary problem variations. Established approach in eco-driving is considering long planning horizons and multiple constraints (i.e. traffic lights, speed limits, etc.), but exclusively single lane driving. On the other hand, motion planning for Automated Driving considers multilane driving, but short planning horizons and decoupled (or hierarchical) solutions, focused on effectively reacting to the changing situations, and not on the long-term optimal behavior.As a result of the synergy, developed search-based optimal motion planning (SBOMP) solution enables optimal Automated Driving scalable to various challenging scenarios in urban, rural and highway environment. As a highlight, SBOMP enables, what is believed to be, the first demonstration of optimal multilane diving in dense traffic with traffic lights, while achieving SuperHuman driving performance. Even though, this scenario is pretty common in everyday driving, it was not tackled by any of these research communities before.The presented SBOMP framework is also extended to the third use-case, Performance Driving. By considering a more detailed vehicle model, SBOMP enables minimum lap-time driving on a slippery road, effectively entering and exiting drifting maneuvers and switching between right and left turns.The presented work is extensively tested in simulation, benchmarked with human driving behavior acquired in driving simulator study and in-vehicle testing on proving ground. The results show that in challenging urban driving scenario with traffic lights, AV outperforms even the best human drivers in terms of safety, efficiency and comfort. While human drivers violate traffic rules and even cause crashes, by using predictive planning, AV manages to drive smoothly through the traffic.Hopefully, this work contributes to the effort that Autonomous Vehicles become the first mass product of intelligent mobile robots in our society.
@phdthesis{ajanovic2019towards, title = {Towards SuperHuman Autonomous Vehicles}, keywords = {Autonomous Vehicles, Motion Planning, eco-driving, Automated Driving, Performance Driving, SuperHuman Autonomous Vehicles}, author = {Ajanovic, Zlatan}, year = {2019}, month = dec, day = {6}, language = {English}, school = {Graz University of Technology}, url = {https://graz.elsevierpure.com/en/publications/towards-superhuman-autonomous-vehicles} }
A Novel Approach to Model Exploration for Value Function Learning

Zlatan Ajanovic, Halil Beglerovic, and Bakir Lacevic

In RSS 2019 Workshop on Combining Learning and Reasoning, 2019

Abs arXiv Bib PDF

Planning and Learning are complementary approaches. Planning relies on deliberative reasoning about the current state and sequence of future reachable states to solve the problem. Learning, on the other hand, is focused on improving system performance based on experience or available data. Learning to improve the performance of planning based on experience in similar, previously solved problems, is ongoing research. One approach is to learn Value function (cost-to-go) which can be used as heuristics for speeding up search-based planning. Existing approaches in this direction use the results of the previous search for learning the heuristics. In this work, we present a search-inspired approach of systematic model exploration for the learning of the value function which does not stop when a plan is available but rather prolongs search such that not only resulting optimal path is used but also extended region around the optimal path. This, in turn, improves both the efficiency and robustness of successive planning. Additionally, the effect of losing admissibility by using ML heuristic is managed by bounding ML with other admissible heuristics.
@inproceedings{ajanovic2019novel, title = {A Novel Approach to Model Exploration for Value Function Learning}, booktitle = {{{RSS}} 2019 {{Workshop}} on {{Combining Learning}} and {{Reasoning}}}, author = {Ajanovic, Zlatan and Beglerovic, Halil and Lacevic, Bakir}, year = {2019}, eprinttype = {arxiv}, eprintclass = {cs}, keywords = {Computer Science - Artificial Intelligence,Computer Science - Machine Learning,Computer Science - Robotics} }

2018

Energy-Efficient Driving in Dynamic Environment: Globally Optimal MPC-like Motion Planning Framework

Zlatan Ajanović, Michael Stolz, and Martin Horn

In Advanced Microsystems for Automotive Applications 2017, Cham, 2018

Abs DOI Bib PDF

Predictive motion planning is a key for achieving energy-efficient driving, which is one of the major visions of automated driving nowadays. Motion planning is a challenging task, especially in the presence of other dynamic traffic participants. Two main issues have to be addressed. First, for globally optimal driving, the entire trip has to be considered at once. Second, the movement of other traffic participants is usually not known in advance. Both issues lead to increased computational effort. The length of the prediction horizon is usually large and the problem of unknown future movement of other traffic participants usually requires frequent replanning. This work proposes a novel motion planning approach for vehicles operating in dynamic environments. The above-mentioned problems are addressed by splitting the planning into a strategic planning part and situation-dependent replanning part. Strategic planning is done without considering other dynamic participants and is reused later in order to lower the computational effort during replanning phase.
@inproceedings{ajanovic2018energy, title = {Energy-{{Efficient Driving}} in {{Dynamic Environment}}: {{Globally Optimal MPC-like Motion Planning Framework}}}, shorttitle = {Energy-{{Efficient Driving}} in {{Dynamic Environment}}}, booktitle = {Advanced {{Microsystems}} for {{Automotive Applications}} 2017}, author = {Ajanovi\'c, Zlatan and Stolz, Michael and Horn, Martin}, editor = {Zach\"aus, Carolin and M\"uller, Beate and Meyer, Gereon}, year = {2018}, series = {Lecture {{Notes}} in {{Mobility}}}, pages = {111--122}, publisher = {{Springer International Publishing}}, location = {{Cham}}, doi = {10.1007/978-3-319-66972-4_10}, isbn = {978-3-319-66972-4}, keywords = {Dynamic environment,Eco-driving,Optimal speed trajectory,Real-time capability,Replanning} }
A Novel Model-Based Heuristic for Energy-Optimal Motion Planning for Automated Driving

Zlatan Ajanović, Michael Stolz, and Martin Horn

IFAC-PapersOnLine, 2018

Abs DOI arXiv Bib PDF

Predictive motion planning is the key to achieve energy-efficient driving, which is one of the main benefits of automated driving. Researchers have been studying the planning of velocity trajectories, a simpler form of motion planning, for over a decade now and many different methods are available. Dynamic programming has shown to be the most common choice due to its numerical background and ability to include nonlinear constraints and models. Although planning of an optimal trajectory is done in a systematic way, dynamic programming does not use any knowledge about the considered problem to guide the exploration and therefore explores all possible trajectories. A* is a search algorithm which enables using knowledge about the problem to guide the exploration to the most promising solutions first. Knowledge has to be represented in a form of a heuristic function, which gives an optimistic estimate of cost for transitioning to the final state, which is not a straightforward task. This paper presents a novel heuristics incorporating air drag and auxiliary power as well as operational costs of the vehicle, besides kinetic and potential energy and rolling resistance known in the literature. Furthermore, optimal cruising velocity, which depends on vehicle aerodynamic properties and auxiliary power, is derived. Results are compared for different variants of heuristic functions and dynamic programming as well.
@article{ajanovic2018novel, doi = {10.1016/j.ifacol.2018.07.042}, title = {A Novel Model-Based Heuristic for Energy-Optimal Motion Planning for Automated Driving}, author = {Ajanovi\'c, Zlatan and Stolz, Michael and Horn, Martin}, year = {2018}, journal = {IFAC-PapersOnLine}, volume = {51}, number = {9}, pages = {255--260}, publisher = {{Elsevier}} }
Safe Learning-Based Optimal Motion Planning for Automated Driving

Zlatan Ajanovic, Bakir Lacevic, Georg Stettinger, Daniel Watzenig, and Martin Horn

In ICML/IJCAI/AAMAS 2018 Workshop on Planning and Learning (PAL-18), 2018

Abs arXiv Bib PDF

This paper presents preliminary work on learning the search heuristic for the optimal motion planning for automated driving in urban traffic. Previous work considered search-based optimal motion planning framework (SBOMP) that utilized numerical or model-based heuristics that did not consider dynamic obstacles. Optimal solution was still guaranteed since dynamic obstacles can only increase the cost. However, significant variations in the search efficiency are observed depending whether dynamic obstacles are present or not. This paper introduces machine learning (ML) based heuristic that takes into account dynamic obstacles, thus adding to the performance consistency for achieving real-time implementation.
@inproceedings{ajanovic2018safe, title = {Safe Learning-Based Optimal Motion Planning for Automated Driving}, booktitle = {{{ICML}}/{{IJCAI}}/{{AAMAS}} 2018 {{Workshop}} on {{Planning}} and {{Learning}} ({{PAL-18}})}, author = {Ajanovic, Zlatan and Lacevic, Bakir and Stettinger, Georg and Watzenig, Daniel and Horn, Martin}, year = {2018}, eprinttype = {arxiv}, eprintclass = {cs, stat}, keywords = {Computer Science - Machine Learning,Computer Science - Robotics,Statistics - Machine Learning} }
Search-Based Optimal Motion Planning for Automated Driving

Zlatan Ajanovic, Bakir Lacevic, Barys Shyrokau, Michael Stolz, and Martin Horn

In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018

Abs DOI arXiv Bib PDF Video

This paper presents a framework for fast and robust motion planning designed to facilitate automated driving. The framework allows for real-time computation even for horizons of several hundred meters and thus enabling automated driving in urban conditions. This is achieved through several features. Firstly, a convenient geometrical representation of both the search space and driving constraints enables the use of classical path planning approach. Thus, a wide variety of constraints can be tackled simultaneously (other vehicles, traffic lights, etc.). Secondly, an exact cost-to-go map, obtained by solving a relaxed problem, is then used by A*-based algorithm with model predictive flavour in order to compute the optimal motion trajectory. The algorithm takes into account both distance and time horizons. The approach is validated within a simulation study with realistic traffic scenarios. We demonstrate the capability of the algorithm to devise plans both in fast and slow driving conditions, even when full stop is required.
@inproceedings{ajanovic2018search, title = {Search-{{Based Optimal Motion Planning}} for {{Automated Driving}}}, booktitle = {2018 {{IEEE}}/{{RSJ International Conference}} on {{Intelligent Robots}} and {{Systems}} ({{IROS}})}, author = {Ajanovic, Zlatan and Lacevic, Bakir and Shyrokau, Barys and Stolz, Michael and Horn, Martin}, year = {2018}, pages = {4523--4530}, issn = {2153-0866}, doi = {10.1109/IROS.2018.8593813}, eventtitle = {2018 {{IEEE}}/{{RSJ International Conference}} on {{Intelligent Robots}} and {{Systems}} ({{IROS}})}, keywords = {A* search,automated driving,Automation,Dynamics,lane change,motion planning,MPC,multi-lane driving,Planning,Roads,Search problems,traffic lights,Trajectory,Vehicle dynamics}, }

2017

Energy Efficient Driving in Dynamic Environment: Considering Other Traffic Participants and Overtaking Possibility

Zlatan Ajanović, Michael Stolz, and Martin Horn

In Comprehensive Energy Management – Eco Routing & Velocity Profiles, Cham, 2017

Abs DOI Bib PDF

This chapter studies energy efficient driving of (semi)autonomous electric vehicles operating in a dynamic environment with other traffic participants on a unidirectional, multi-lane road. This scenario is considered to be a so called hard problem, as constraints imposed are varying in time and space. Neglecting the constraints imposed from the surrounding traffic, the generation of an energy optimal speed trajectory may lead to bad results, with the risk of low driver acceptance when applied in a real driving environment. An existing approach satisfies constraints from surrounding traffic by modifying an existing unconstrained trajectory. In contrast to this, the proposed approach incorporates a leading vehicle’s motion as constraint in order to generate a new optimal speed trajectory in a global optimal sense. First simulation results show that energy optimal driving considering other vehicle participants is important. Even in simple setups significantly (8%) less energy is consumed at only 1.3% travelling time prolongation compared to the best constant speed driving strategy. Additionally, the proposed driving strategy is using 4.5% less energy and leads to 1.6% shorter travelling time compared to the existing overtaking approach. Using simulation studies, the proposed energy optimal driving strategy is analyzed in different scenarios.
@incollection{ajanovic2017energy, title = {Energy {{Efficient Driving}} in {{Dynamic Environment}}: {{Considering Other Traffic Participants}} and {{Overtaking Possibility}}}, shorttitle = {Energy {{Efficient Driving}} in {{Dynamic Environment}}}, booktitle = {Comprehensive {{Energy Management}} \textendash{} {{Eco Routing}} \& {{Velocity Profiles}}}, author = {Ajanovi\'c, Zlatan and Stolz, Michael and Horn, Martin}, editor = {Watzenig, Daniel and Brandst\"atter, Bernhard}, year = {2017}, series = {{{SpringerBriefs}} in {{Applied Sciences}} and {{Technology}}}, pages = {61--80}, publisher = {{Springer International Publishing}}, location = {{Cham}}, doi = {10.1007/978-3-319-53165-6_4}, isbn = {978-3-319-53165-6}, keywords = {Car following,Ecodriving,Optimal speed trajectory,Overtaking} }