CS 530 Homework 2

With the fundamentals of AI agents firmly established, it is time to define your group projects. The goal of the project is to go through the planning phases involved in creating an agent, not in the business aspects. You do not need to have a business model. Agents that detect cybersecurity threats, deliver takeout, or prune your hedges are all equally valid, but you could also propose an impractical agent that hovers near a person and shoots down mosquitoes with a laser. You are not expected to build the agent.

So what will you be expected to do?

The final project should have several components:

• The goal of the agent, defined by a set of positive and negative rewards for different outcomes
• An estimate of the data required for the agent. This comes from:
  • Data, real or synthetic, that represents the agent’s sensed environment
  • A simulation or emulation of the agent interacting with that environment
• A set of requirements for the agent’s platform
  • This means sensors and hardware, and could include a list of real parts
• Perception algorithms:
  • Description of data collection or adaptation of an existing dataset (or both)
  • Data collection and testing scenarios
  • ML techniques used
• Agent Policy details
  • How does training take place? Or testing?
  • What planning and control algorithms are used?
  • Is there execution monitoring? What is it?

Example: Garden Bot

This is a brief summary of a completed project describing a gardening-bot.

• Rewards
  • Positive rewards for removing unwanted weeds and pruning desired plants
  • Negative rewards for removing desired plants, over-pruning, and under-pruning
• Data estimates derived from
  • A collection of robot-height videos from manicured and unmanicured areas
  • An example classifier that can identify areas in need of pruning
  • An example classifier to distinguish between weeds and desired plants
• Platform
  • A list of cameras, possibly a camera sensitive to UV for flower detection
  • A depth camera to aid in pruning
  • A magnetic sensor is used with electromagnetic boundary markers to define gardenbot’s range
  • A chassis, along with a power supply
  • A compute platform that either works within that chassis or remote compute that communicates wirelessly with GardenBot
  • Low, medium, and high cost estimates for different options. Don’t worry about practicality, the high could be absurd.
• Algorithms
  • Three trained systems
    • Detect traversable terrain
      • This one uses a pretrained model and clustering of features
    • Detection system for humans and other animals
      • Used a pretrained YOLO network
      • Possibly fine-tuned based upon perspective and scale
        • e.g. garden bot may only see a human hand rather than the entire human, but should not attempt to prune a hand
    • Unpruned vs Pruned classifier
      • Use features from a pretrained model and use an SVM as a classifier
        • This allows the user to give pictures to GardenBot’s existing dataset that identify pruned, unpruned, and overpruned shrubs
• Agent Policy
  • Pruning is trained and test in simulation
    • Real world images of plants are converted simulated skeletons using camera and depth information
    • Reinforcement learning is used to select the optimal cutting locations for different plants
      • The classifier of pruned, unpruned, and overpruned plants is used for the reward signal
  • Movement
    • A technique similar to Cognitive Mapping and Planning for Visual Navigation is used for mapping
    • Goals are defined by a version of the unpruned classifier that works at longer distances and a map that remembers how long it has been since an area has been explored
      • Reinforcement learning is then used to create an exploration policy
    • This will be modelled as episodic, with GardenBot patrolling as much garden as it can on a charge before returning to its charging station
      • After each episode, a certain amount of forgetting is applied to its maps, making it more likely to explore areas it hasn’t seen recently
  • Execution Monitoring
    • No pruning will be done with any animal, human, or limb in view
    • GarenBot will stop moving if any animal, human, or limb is directly in front of the agent

Note that each of the systems don’t need to be built, but you will need a mock-up of them. For example, you could demonstrate separate classifiers for pruned and unpruned bushes for real and simulated data, and reinforcement learning applied to the simulated data, but you don’t need to build the system to convert from real to simulated data. The simulated plants could be very simplified.

Homework 2 Deliverables

• Your group members
• A description of your agent that identifies key details
  • What are the agent goals?
  • A brief description of the agent
    • Is it a program? A drone? A ground robot?
  • What is the environment
    • Discrete, continuous, multi-agent, etc
• What kind of data will your agent require
  • This is basically asking you to decide on what sensors will be used
    • For GardenBot, it was mostly cameras
    • For an agent doing cyberthreat detection, it would be network monitoring tools
• A rough idea of how you will get data
  • Is there an existing dataset that you can use?
  • Will you collect real data?
    • For example, with a smartphone camera?
  • Will you simulate data?
    • For example, with American Truck Simulator for your autonomous driving agent?
• A rough idea of how you will validate an approach
  • Emulation: this means using real data but not actually interacting with the real world
    • Example: PilotNet Resimulator
  • Simulation
    • Example: Carla Driving Simulator

You won’t be penalized for changing your minds about some details later, you just need a rough outline.