Dingding Zheng

I'm a graduate student at GRASP lab, University of Pennsylvania. I'm currently working on DCIST (a project focusing on multi-agent and heterogeneous systems) in Kumar Lab, which's a subgroup of GRASP.

My research interest lies in Multi-agent system, Micro Unmanned Aerial Vehicles (UAV), Robot Perception and Learning.

Email  /  CV  /  github  

profile photo
Research

I have a broad interest in different tracks of robotics such as: SWARM Intelligence, Human Robot Teaming, Optimal Control, Computer Vision, etc. Much of my work focuses on ensuring safe and agile performance of algorithms on real robotics platforms, as you can see from the following projects.

For now, I feel fortunate enough to be supervised by Prof. Vijay Kumar and Dr. Siddharth Mayya to work on multi-agent algorithms and their possible applications on quadrotor platforms.

The Distributed and Collaborative Intelligent Systems and Technology
(UPenn Kumar Lab Research)

Siddharth Mayya, Vijay Kumar, Dingding Zheng, Vinay Senthil
Summer 2020 -- Now
project page

"DCIST", led by UPenn, MIT, UC Berkeley, etc, is a project focusing on "Multi-robot and Heterogeneous Systems". It includes six tracks: Swarms, Networking, Intelligence, Autonomy, Resilience and Collaboration. We are developing algortithms which enable the Multi-robot systems to solve high-level tasks, such as rescue of hostages, exploration and monitoring. I'm currently using Graph Neural Networks to do imitation learning and efficient task allocation for the systems. Follow us to keep updated.




Projects



Playing Piano using a Robot
Dingding Zheng,
Spring 2023

In this project, I designed a robotic framework to play the piano with an anthropomorphic KUKA robotic arm-hand based on an music sheet input. I implemented a full-stack pipeline including offline collision-free trajectory planner, joint impedance controller and simulator. In drake environment, the 23-degree-of-freedom robot can play a classic song, Salut D’Amour.

Lane Lines Detection
Dingding Zheng,
Summer 2021

In this project, lane detection pipelines are implemented and tested on a real-world dataset.

For straight lines detection, images are converted to greyscale and Gaussian Blur method is applied to reduce noise. Then the blurred images are passed to cv2.Canny with max and min thresholds. After finding the edges and defining the interested region, cv2.HoughLinesP and cv2.addWeighted methods are used to draw the detected lines to images. However, several shortcomings still exist for this simple pipeline: 1. min and max values of Hough transform are predefined. This may harm the performance when we are trying to detect dashed lines. 2. the simple pipeline are not fit for "curve finding". Imagine when a lane is turning at an angle large enough to move the "lane end " from picture center, the detected lines may not align with reality.

To deal with these problems, an advanced pipleline is proposed. I computed the camera calibration matrix and distortion coefficients given a set of chessboard images. Following this, we can get undistorted images by using correction. Color transforms, gradients are used to create threshold binary images. Then, we can detect lane pixels and fit them to find the lane boundary. Finally, a video showing the lane boundaries and numerical estimation of lane curvature is generated.

Quadrotor Planning and Control (UPenn MEAM 620 Advanced Robotics)
Dingding Zheng, Shane Rozen-Levy, Huaiyu Chen
Spring 2020
project page/ report/ video / code

In this project, we designed algorithms to control a quadrotor. Controlled by a geometric non-linear controller, the quadrotor could follow a predefined trajectory and reached the goal without collision. We got the optimal trajectory by applying Down-sampling algorithm to the shortest path which was generated by A*. This lab focused on demonstrating the planning aspect of robot capabilities. During the lab, the CrazyFlie 2.0 was used for realistic demonstrations. A microcomputer is responsible for low-level control and estimation, while the onboard IMU provides feedback of angular velocities and accelerations. The attitude and thrust commands are sent to the quadrotor via the CrazyRadio after being computed in python.




F1tenth Racing (UPenn ESE 615 Autonomous Racing)
Shivangi Misra, Dingding Zheng, Weiyi Tang
Spring 2020
project page / report/ code

”F1tenth” is an open-source, small-scale racing car platform widely used for teaching and research in safe autonomy. Maneuvering a racing car to finish loops in minimum time has been studied for decades. However, it’s always computationally expensive and infeasible to solve this problem by real-time trajectory planning. In this project, we generated a velocity profile to find the maximum permissible speed of racing car on each waypoint on the path. Then, we use CMAES (Covariance Matrix Adaptation - Evolution Strategy) to generate the desired path for vehicle to track. The generated path has a relatively small curvature which allows the racing car to run at high, steady speed. Pure pursuit is used as the vehicle controller. To avoid obstacles, we implemented ODG-PFM (Obstacle-Dependent Gaussian Potential Field) and compared its performance VS. RRT*. In order to measure the performance of these two algorithms, we setup several testing maps and added noise to the environment.

Self-Assembling MOdular Robot for Extreme Shapeshifting
(UPenn ModLab Research)

Chao Liu, Mark Yim, Qian Lin, Hyun Kim, Zhaoxi Chen, Chang Liu, Dingding Zheng
Summer 2019, 2020
project page

SMORES-EP is a modular robot designed and built by the University of Pennsylvania, and used by researchers at UPenn and Cornell. Modular robots benefit systems for their excellent versatility and configurability. During the research, I helped the group to improve the controller for SMORES-EP and set up the testing environment. Besides, I worked on judging the similarity between different reconfigurations (topology).

Human & Robot agents Interaction (UPenn PRECIS Lab Research)
Dingding Zheng, Osbert Bastani
Fall 2019
code

In this project, we added car dynamics model into OpenAI multi-agent particle environment. Then, we implemented MPC controller to simulate the human decision making and trained robot car (blue agents) using MADDPG algorithm. The human agents (grey ones) are simulated using "Human Social Force Model".

Simultaneous Localization and Mapping (RGBD-SLAM)
Dingding Zheng
Spring 2020

This project shows an approach for mapping a scene while localizing the robot using a 2D laser scan. Here we integrated the IMU orientation and odometry information from a walking humanoid robot in order to build a 2D occupancy grid map of the walls and obstacles in the environment. Then additional camera and depth imagery from a Kinect sensor were integrated to build a textured map.


Orienation Tracking based Panorama Stitching using Unscented Kalman Filter
Dingding Zheng
Spring 2020

In this project, we implemented a kalman filter to track three dimensional orientation. Given IMU sensor readings from gyroscopes and accelerometers, we estimated the underlying 3D orientation by learning the appropriate model parameters from ground truth data given by a Vicon motion capture system. Finally, we generated real-time panoramic images from camera images using the 3D orientation filter.

Mask RCNN (UPenn CIS 680 Vision & Learning)
Dingding Zheng
Fall 2019
project page/ code

Mask RCNN is a a conceptually simple, flexible, and general framework for object instance segmentation.It extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Here we implemented it to do per-pixel object detection.

OpenAI gym racing car control using "Clipped PPO" (UPenn CIS 680 Final Project)
Dingding Zheng, Mengwei Zhang, Huize Huang
Fall 2020
report / code

In this project, we implemented "Clipped Proximal Policy Optimization (Clipped PPO)" algorithm and applied it in OpenAI racing-car v0.

GAN (UPenn CIS 680 Vision & Learning)
Huize Huang, Dingding Zheng, Mengwei Zhang
Fall 2019
project page/ code

Down-sampled the original images to get low-resolution images. Then implemented GAN to generate high-resolution images. We used MSE loss and Learned Perceptual Image Patch Similarity (LPIPS) metric to judge the model performance.

YOLO (UPenn CIS 680 Vision & Learning)
Dingding Zheng,
Fall 2019

Implemented YOLO to do object detection on street scene images.

Barrel Detection using Color Segmentation based on GMMs
Dingding Zheng
Spring 2019

We trained a GMM-based model to detect barrels in images and found the relative world coordinates of the barrel from images. Algorithms are implemented to learn the color model, segment the target color and finally localize the target object from images. We hand-labeled the training sets and then built a color classifier and a red barrel detector. Bounding boxes, distance to the barrels and the orientations are returned as the outputs of the model.

6DoF Pose Estimation (UPenn CIS 580 Machine Perception)
Dingding Zheng
Spring 2019
project page / code

In this project, we were given an image of oilcan to tain a heatmap-based neural network which estimates the location of the keypoints in that image. Each 2D heatmap corresponded to one keypoint and was responsible to localize this particular keypoint in the image. During training, we synthesized the heatmaps by identifying the location of each keypoint on the 2D image and placing a 2D Gaussian centered on this location on the corresponding heatmap. Then, we used the coordinates of detected keypoints to estimate the 6DoF pose of the object.

Logo Projection (UPenn CIS 580 Machine Perception)
Dingding Zheng
Spring 2019

Estimated the homography that maps the video images onto the logo points, then warped the sampled points according to the homography. Used this correspondence to project the "Penn Engineering" logo to the goal in a football match.

Scale Invariant Detection (UPenn CIS 580 Machine Perception)
Dingding Zheng
Spring 2019
project page / code

Approximated a Laplacian of Gaussian filter (LoG) by a Difference of Gaussians (DoG). This's a good filter for blob detection.

Acrobot (UPenn ESE 680 Reinforcement Learning)
Dingding Zheng
Fall 2019
code

Implemented Actor-Critic neural network to control OpenAI gym Acrobot v1.


Great thanks to Jon for his amazing website template!