Undulating fin and tail propulsion—used by marine animals across many size scales—represents an efficient locomotion method, both on and underwater. It is silent and enables a variety of maneuvers. Robotic swimmers can achieve such life-like locomotion using soft actuators but typically measure several tens of centimeters, preventing applications that benefit from smaller scales. Here we present the design, material approaches, and fabrication processes of soft mm-scale electrohydraulic actuators. Our actuators use zipping of metalized polymer films to displace a dielectric fluid, which generates the bending of fins, creating a periodic undulation. With these actuators, we demonstrate flatworm-inspired robots that locomote both on or under water.