Dingding Zheng

I'm a graduate student at GRASP lab, University of Pennsylvania. I'm currently working on DCIST (a project focusing on multi-agent and heterogeneous systems) in Kumar Lab, which's a subgroup of GRASP.

My research interest lies in Multi-agent system, Micro Unmanned Aerial Vehicles (UAV), Robot Perception and Learning.

Email  /  CV  /  github  

I have a broad interest in different tracks of robotics such as: SWARM Intelligence, Human Robot Teaming, Optimal Control, Computer Vision, etc. Much of my work focuses on ensuring safe and agile performance of algorithms on real robotics platforms, as you can see from the following projects.

For now, I feel fortunate enough to be supervised by Prof. Vijay Kumar and Dr. Siddharth Mayya to work on multi-agent algorithms and their possible applications on quadrotor platforms.

"DCIST", led by UPenn, MIT, UC Berkeley, etc, is a project focusing on "Multi-robot and Heterogeneous Systems". It includes six tracks: Swarms, Networking, Intelligence, Autonomy, Resilience and Collaboration. We are developing algortithms which enable the Multi-robot systems to solve high-level tasks, such as rescue of hostages, exploration and monitoring. I'm currently using Graph Neural Networks to do imitation learning and efficient task allocation for the systems. Follow us to keep updated.

In this project, I designed a robotic framework to play the piano with an anthropomorphic KUKA robotic arm-hand based on an music sheet input. I implemented a full-stack pipeline including offline collision-free trajectory planner, joint impedance controller and simulator. In drake environment, the 23-degree-of-freedom robot can play a classic song, Salut D’Amour.

In this project, lane detection pipelines are implemented and tested on a real-world dataset.

For straight lines detection, images are converted to greyscale and Gaussian Blur method is applied to reduce noise. Then the blurred images are passed to cv2.Canny with max and min thresholds. After finding the edges and defining the interested region, cv2.HoughLinesP and cv2.addWeighted methods are used to draw the detected lines to images. However, several shortcomings still exist for this simple pipeline: 1. min and max values of Hough transform are predefined. This may harm the performance when we are trying to detect dashed lines. 2. the simple pipeline are not fit for "curve finding". Imagine when a lane is turning at an angle large enough to move the "lane end " from picture center, the detected lines may not align with reality.

To deal with these problems, an advanced pipleline is proposed. I computed the camera calibration matrix and distortion coefficients given a set of chessboard images. Following this, we can get undistorted images by using correction. Color transforms, gradients are used to create threshold binary images. Then, we can detect lane pixels and fit them to find the lane boundary. Finally, a video showing the lane boundaries and numerical estimation of lane curvature is generated.

In this project, we designed algorithms to control a quadrotor. Controlled by a geometric non-linear controller, the quadrotor could follow a predefined trajectory and reached the goal without collision. We got the optimal trajectory by applying Down-sampling algorithm to the shortest path which was generated by A*. This lab focused on demonstrating the planning aspect of robot capabilities. During the lab, the CrazyFlie 2.0 was used for realistic demonstrations. A microcomputer is responsible for low-level control and estimation, while the onboard IMU provides feedback of angular velocities and accelerations. The attitude and thrust commands are sent to the quadrotor via the CrazyRadio after being computed in python.

”F1tenth” is an open-source, small-scale racing car platform widely used for teaching and research in safe autonomy. Maneuvering a racing car to finish loops in minimum time has been studied for decades. However, it’s always computationally expensive and infeasible to solve this problem by real-time trajectory planning. In this project, we generated a velocity profile to find the maximum permissible speed of racing car on each waypoint on the path. Then, we use CMAES (Covariance Matrix Adaptation - Evolution Strategy) to generate the desired path for vehicle to track. The generated path has a relatively small curvature which allows the racing car to run at high, steady speed. Pure pursuit is used as the vehicle controller. To avoid obstacles, we implemented ODG-PFM (Obstacle-Dependent Gaussian Potential Field) and compared its performance VS. RRT*. In order to measure the performance of these two algorithms, we setup several testing maps and added noise to the environment.

SMORES-EP is a modular robot designed and built by the University of Pennsylvania, and used by researchers at UPenn and Cornell. Modular robots benefit systems for their excellent versatility and configurability. During the research, I helped the group to improve the controller for SMORES-EP and set up the testing environment. Besides, I worked on judging the similarity between different reconfigurations (topology).

In this project, we added car dynamics model into OpenAI multi-agent particle environment. Then, we implemented MPC controller to simulate the human decision making and trained robot car (blue agents) using MADDPG algorithm. The human agents (grey ones) are simulated using "Human Social Force Model".

This project shows an approach for mapping a scene while localizing the robot using a 2D laser scan. Here we integrated the IMU orientation and odometry information from a walking humanoid robot in order to build a 2D occupancy grid map of the walls and obstacles in the environment. Then additional camera and depth imagery from a Kinect sensor were integrated to build a textured map.

In this project, we implemented a kalman filter to track three dimensional orientation. Given IMU sensor readings from gyroscopes and accelerometers, we estimated the underlying 3D orientation by learning the appropriate model parameters from ground truth data given by a Vicon motion capture system. Finally, we generated real-time panoramic images from camera images using the 3D orientation filter.

Mask RCNN is a a conceptually simple, flexible, and general framework for object instance segmentation.It extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition. Here we implemented it to do per-pixel object detection.

In this project, we implemented "Clipped Proximal Policy Optimization (Clipped PPO)" algorithm and applied it in OpenAI racing-car v0.

Down-sampled the original images to get low-resolution images. Then implemented GAN to generate high-resolution images. We used MSE loss and Learned Perceptual Image Patch Similarity (LPIPS) metric to judge the model performance.

Implemented YOLO to do object detection on street scene images.

We trained a GMM-based model to detect barrels in images and found the relative world coordinates of the barrel from images. Algorithms are implemented to learn the color model, segment the target color and finally localize the target object from images. We hand-labeled the training sets and then built a color classifier and a red barrel detector. Bounding boxes, distance to the barrels and the orientations are returned as the outputs of the model.

In this project, we were given an image of oilcan to tain a heatmap-based neural network which estimates the location of the keypoints in that image. Each 2D heatmap corresponded to one keypoint and was responsible to localize this particular keypoint in the image. During training, we synthesized the heatmaps by identifying the location of each keypoint on the 2D image and placing a 2D Gaussian centered on this location on the corresponding heatmap. Then, we used the coordinates of detected keypoints to estimate the 6DoF pose of the object.

Estimated the homography that maps the video images onto the logo points, then warped the sampled points according to the homography. Used this correspondence to project the "Penn Engineering" logo to the goal in a football match.

Approximated a Laplacian of Gaussian filter (LoG) by a Difference of Gaussians (DoG). This's a good filter for blob detection.

Implemented Actor-Critic neural network to control OpenAI gym Acrobot v1.

Great thanks to Jon for his amazing website template!