•

•

•

Published December 16, 2025

ME 461 Project: Team 5

Our project is focused on enabling natural-language commands to control a robot via a large-language model (LLM).

IntermediateShowcase (no instructions)73

Things used in this project

Hardware components

Texas Instruments LAUNCHXL-F28379D C2000 Delfino LaunchPad

Software apps and online services

ChatGPT OpenAI

Story

PROJECT OVERVIEW:

Human-robot interaction (HRI) is shifting toward natural language interfaces powered by large-language models (LLMs). LLMs allow users to give intuitive, high-level commands instead of complex programming. Our team's objective is to improve user experience by simplifying communication with robots and making HRI more accessible for a broader range of users. Therefore, our project aims to enable robot control via natural-language commands by applying the ChatGPT OpenAI API.

There are two primary modes. The first mode is motion planning, where the user can provide a starting point, an ending point, and obstacles they want the robot to avoid. The second mode is custom, where the user can provide a specific path they want the robot to follow, such as a shape. After the user provides their commands and the LLM generates a path, the specific waypoints are parsed and stored in a CSV file. Following this, LabVIEW looks for the CSV file with the provided file name, and this data is sent over the robot's F28379D launchpad to execute the desired trajectory. Transmission Control Protocol (TCP) communication was enabled to support continuous input of user commands to the LLM, allowing for rendering of new paths and motion of the robot without having to restart the entire system. Furthermore, when the waypoint data is sent to the robot, it is done in groups of flag values and three points at a time to allow for any arbitrary number of total waypoints.

Once this information is received by the launchpad, the position control can be executed. The position control is achieved by first applying inverse kinematics to compute the desired steering angle. Next, the error between the desired steering angle and bearing (Figure 1) was computed. The robot was programmed to turn right or left until this error value was approximately 0.5 degrees. Several experiments were done to determine the ideal threshold value of 0.5 degrees. Our goal was to ensure that the angle error was not too small that the robot would overshoot and continuously turn, and that it was also not too large that the robot would not reach the desired ending point. After the specified angle error of 0.5 degrees was attained, the robot will move towards the (x, y) position of interest while checking its current location every 4 milliseconds. This control logic repeats until the robot has gone to all specified waypoints.

Fig 1. Robot Pose (ME 461 Lab 6 Manual)

Collision detection was implemented as well; if the robot was less than 200 millimeters away from an object or wall, it would stop its motion.

Video 1 provides additional detail on the sensors used to execute position and collision control: https://drive.google.com/file/d/1DWA-_gmSIKdq5NaS8PT7DvZ0-LxchyYt/view?usp=drive_link.

The complete process is summarized in Figure 2.

Fig 2. LLM-Based Path Planning Process

EXAMPLES:

Mode I: Motion Planning

An example of a prompt and the trajectory generated by the LLM (Figure 3) is below:

"Generate a 2D motion-planning trajectory. Map size: 3 × 3 meters. Start: (0, 0). Goal: (1.5, 1.5). Obstacles: obstacles = [(1, 1), (0.5, 0.5), ]. Using Greedy Search Path planning algorithm. Must start from positive X direction to the second point. Please generate exactly 6 waypoints (including start and end).Must be the points number I want. The trajectory must avoid all obstacles with ≥0.6 m safety distance. Points should be different. Output ONLY CSV: x, y (no header, no explanation)."

Fig 3. Motion Planning Trajectory Example

Video 2 demonstrates the robot following the above path: https://drive.google.com/file/d/1NM-Du00_HhI8mabA3atKgax7jdmfMSNa/view?usp=sharing.

Mode II: Custom

An example of a prompt and the trajectory generated by the LLM (Figure 4) is below:

"Generate exactly 18 waypoints such that: 1. The first waypoint must be exactly (0.00, 0.00). 2. The remaining waypoints must lie on the circle with center {(0, 0)} and radius {0.5}. 3. The second waypoint must be the point on the circle closest to the origin. 4. All remaining circle points must be evenly spaced along the circle. 5. Points should be different. Format all numbers to exactly two decimals. Output ONLY CSV: x, y (no header)."

Fig 4. Custom Trajectory Example

Video 3 demonstrates the robot following the above path: https://drive.google.com/file/d/1LdelW8QiVZEgtKQDTXVnD_8KetAHSvSP/view?usp=sharing.

from openai import OpenAI
import matplotlib.pyplot as plt

API_KEY = " " # add your ChatGPT OpenAI API key here
MODEL_NAME = "gpt-5.1"

client = OpenAI(api_key=API_KEY)

#parameters
task_type = "circle"
# task_type :
#   "motion_planning"
#   "circle"
#   "straight_line"
#   "custom"

num_waypoints = 18 #waypoints
start = (0, 0)
goal = (1.5, 1.5)
map_size = 3

# obstacles
obstacles = [
    (1, 1),
    (0.5, 0.5),
    
]

# Circle
circle_center = (1.5, 1.5)
circle_radius = 1


# =======================================
#   Prompt
# =======================================

def build_prompt():
    obstacle_str = ", ".join([f"({ox},{oy})" for ox, oy in obstacles])

    if task_type == "motion_planning":
        return f"""
Generate a 2D motion-planning trajectory.
Map size: {map_size}  {map_size} meters.
Start: {start}
Goal: {goal}
Obstacles: {obstacle_str}
Using Greedy Search Path planning algorithom
Must start from positive X direction to the second point.
Please generate exactly {num_waypoints} waypoints (including start and end).Must be the points number I want.
The trajectory must avoid all obstacles with 0.6 m safety distance.
Points should be different.
Output ONLY CSV: x,y (no header, no explanation).
"""

    elif task_type == "circle":
        return f"""
Generate exactly {num_waypoints} waypoints such that:
1. The first waypoint must be exactly (0.00,0.00).
2. The remaining waypoints must lie on the circle with center {(0,0)} and radius {0.5}.
3. The second waypoint must be the point on the circle closest to the origin.
4. All remaining circle points must be evenly spaced along the circle.
5. Points should be different.
Format all numbers to exactly two decimals.
Output ONLY CSV: x,y (no header).

"""

    elif task_type == "straight_line":
        return f"""
Generate a straight-line trajectory from {start} to {goal}.
Output exactly {num_waypoints} evenly spaced waypoints.
Output ONLY CSV: x,y (no header).
"""

    elif task_type == "custom":
        return f"""
Generate exactly {num_waypoints} waypoints such that:
1. The first waypoint must be exactly (0.00,0.00).
The car moves forward along the positive direction of the x-axis when starting
then turns left and moves forward 0.4m, then turns right and moves forward 0.2m
Then walk in a semi-circular arc with a radius of 0.5m.
Points should be different.
Output ONLY CSV: x,y (no header).
"""

    else:
        return "Invalid task type."


# ======================

prompt = build_prompt()

response = client.chat.completions.create(
    model=MODEL_NAME,
    messages=[
        {"role": "system", "content": "You output ONLY raw CSV data."},
        {"role": "user", "content": prompt},
    ],
    temperature=0.2
)

# parses waypoints and inputs them into a CSV file format

csv_text = response.choices[0].message.content.strip()

lines = []
for line in csv_text.splitlines():
    line = line.strip().replace("`", "")
    if line and not line.startswith("```"):
        lines.append(line)

csv_clean = "\n".join(lines)
print(csv_clean)

# ===== Save to CSV file =====
with open("C:/tao19_zj/py/trajectory.csv", "w") as f:
    f.write(csv_clean)

print("Saved to trajectory.csv")


points = []
for line in csv_clean.splitlines():
    x, y = line.split(',')
    points.append((float(x), float(y)))

# ======================
#Plot
# ======================

plt.figure(figsize=(5,5))
plt.xlim(-0.5, map_size)
plt.ylim(-0.5, map_size)
plt.grid(True)

xs, ys = zip(*points)
plt.plot(xs, ys, '-o', label="Trajectory")

plt.scatter(start[0], start[1], c='green', s=80, label="Start")
plt.scatter(goal[0], goal[1], c='red', s=80, label="Goal")

for (ox, oy) in obstacles:
    plt.scatter(ox, oy, marker='x', c='black', s=120, label="Obstacle")

plt.legend()
plt.title(f"LLM Trajectory: {task_type} ({num_waypoints} points)")
plt.show()

ME 461 Project: Team 5

Things used in this project

Hardware components

Software apps and online services

Story

PROJECT OVERVIEW:

EXAMPLES:

Code

LLM Python Code

LabVIEW

Code Composer Studio Files

Credits

Annika Srinivasan

Zhengyu Tao

Ohm Patel

Zijian S

Comments

Embed the widget on your own site

ME 461 Project: Team 5

ME 461 Project: Team 5

Things used in this project

Hardware components

Software apps and online services

Story

PROJECT OVERVIEW:

EXAMPLES:

Code

LLM Python Code

LabVIEW

Code Composer Studio Files

Credits

Annika Srinivasan

Zhengyu Tao

Ohm Patel

Zijian S

Comments

Related channels and tags