Ultrasonic vibration assisted machining provides effective manufacturing solutions for difficult-to-machine materials such as alloy materials, brittle materials and composite materials, which is widely used in aerospace, defense, electronics and other high-technology fields. The complex structure, high degree of specialization and poor reliability of ultrasonic vibration assisted machining device restrict the promotion of ultrasonic processing technology. In order to popularize the application scope of ultrasonic vibration assisted machining, an accessory ultrasonic vibration working table is designed based on the principle of ultrasonic energy transmission, which can be conveniently installed in the machining center to provide ultrasonic vibration assisted machining for the workpiece. First, two types of materials were applied to structural design of the table, the working frequency and vibration form were determined through the modal analysis, and the working stability was presented by harmonic response analysis. Second, the working table structure optimization was carried out by the method of multi-objective optimization. The quality of the working table was reduced under the condition of guaranteeing invariable total deformation, in order to reduce the loss of ultrasonic energy and improve the working reliability. Finally, in order to make the working table more suitable for the actual machining needs, the material of the working table was selected and the table size was adjusted by comparing the results of finite element analysis before and after optimization. The results of analysis indicate that 45 # steel vibration working table has better vibration stability and smaller vibration amplitude. The overall mass of the vibration table was reduced by 27%, the working frequency was reduced by 11% through multi-objective optimization. The bandwidth difference of resonance frequency between the two workbenches is quite small.