Because of their complementing roles and cooperative skills, heterogeneous unmanned aerial vehicle (UAV) swarms have become a serious threat to air defense systems. The heterogeneous UVA swarms often exhibit a non-convex shape and multi-skewed measurements, which is attributable to variations in the size and spatial distribution of its internal sub-targets. Most existing group target tracking relies upon the hypothesis of convex target shapes and uniform measurement distributions. However, these assumptions could no longer be valid in the real scenario, which may deteriorate the tracking performance. To address the problem of tracking a UAV swarm with non-convex contours and non-uniformly distributed measurements, an expectation maximization (EM)-based adaptive tracking for a heterogeneous UAV swarm with multi-skewed measurements is proposed. Firstly, a multi-skewed measurements noise model is established, and the number of the heterogeneous subgroup targets and the measurements noise parameters are calculated by using an online EM approach. Following the formulation of a joint probability density function for each heterogeneous subgroup target’s kinematic state, shape, and skewed noise parameters, online state and shape estimation is accomplished via a variational inference approach. Finally, the shapes of multiple subgroup targets are fused to acquire the extended shape of the heterogeneous UAV swarm. Simulation results demonstrate that the proposed algorithm significantly outperforms existing approaches, including random matrix model (RMM)-based, random hypersurface model (RHM)-based, multi-ellipsoidal model (MEM)-based, and skew-normal variational Bayes algorithms, in both kinematic state and shape estimation accuracy.