Teaching Robots to Draw

Overview

Modern robotic arms have been trained to mimic the motions of the human arm to perform tasks humans would otherwise need to do. This includes industrial robots that were created to handle radioactive or biohazardous waste that would be dangerous for humans to touch, as well as robots built to manipulate objects in warehouses or factories. 

However, one of the more delicate operations humans perform with their fingers and arms—writing and drawing—has proven difficult for robots to replicate. Stefanie Tellex has demonstrated the invention of a robotic arm that can study just-written words or characters and then create a plan that will allow it to draw the same thing.

Market Opportunity

The ability to build a robotic arm that can write or draw in the physical realm would open the possibility for robots to take on a suite of new jobs. Such a machine could, for example, write on a sign or a post-it note to convey information such as that a particular hallway is closed.

A robotic arm that writes could also become a teaching aid by writing equations or drawing diagrams on a whiteboard. It could write the address on an envelope, or even learn to create art using a paintbrush or pen. All these examples require a robot that can wield a writing implement.

Innovation and Meaningful Advantages

Other attempts to teach robots to write have required the user to teach the machine the proper stroke order for all the marks required to draw a character or word, either by allowing the machine to watch the gestures that a human makes while writing or by telling the robot what the required marks are, and what order they must go it. However, this means a robot cannot simply “look” at an image and figure out what strokes are needed to reproduce it.   

Tellex’s approach creates a robotic system that does not need to be “told” the right way to draw a character. Instead, once the robot has been trained on a base dataset of writing demonstrations, it can look at a bitmapped image of one and learn to recreate it with a single continuous stroke, and without needed to watch a human write the character. In their demonstrations, her group has shown their robotic systems can draw characters on paper or on a whiteboard from a variety of languages, from English and French to Hindi and Chinese.

Collaboration Opportunity

We are seeking a licensing opportunity for this innovative technology.

Principal Investigator

Stefanie Tellex, PhD

Associate Professor of Computer Science

Associate Professor of Engineering

Brown University

Brown Tech ID #2584

stefanie_tellex@brown.edu

PI Website

IP Information

2022-07-12 US20220032468A1; published.

 

Contact

Brian Demers

Director of Business Development

Physical & Computational Sciences

Brian_Demers@brown.edu

Patent Information:
Category(s):
Robotics
For Information, Contact:
Brown Technology Innovations
350 Eddy Street - Box 1949
Providence, RI 02903
tech-innovations@brown.edu
401-863-7499
Inventors:
Stefanie Tellex
Atsunobu Kotani
Keywords:
© 2024. All Rights Reserved. Powered by Inteum