Corrupted data on COMM3 shouldn't affect COMM1 on the PLC unless the PLC is locked within a dead loop controlled by GOTO statement (in order to rectify the VFD communication failure issue), then the CPU does not spend time servicing the serial port and that could lead to failure on COMM1.
Are you using stepper motor? What's the stepping rate? When stepper motor is in action it actually would send out a XON/XOFF handshake via COMM1 which is normally connected to the PC. This inform the TRiLOGI that the stepper motor is in action and to reduce the communication loading on the PLC in order reduce the effect on stepper motor output waveform at high stepping rate. This may however confuse the MODBUS master and cause the communication to drop.
If that the case, it would be better to connect the PC/TLServer to the PLC via COMM1 and connect the HMI to the PLC via COMM3.
As for the VFD, this is more difficult to solve on a T100MD+ PLC since you have both comm ports already controlled by two masters where the PLC could not tell them to temporarily stop sending commands (except to actually switch out their communication by relay controlled by a digital output) so that the PLC can send data to the VFD. The easiest solution would be to use the T100MX-3224R+ PLC insead. This model has a second RS232 port which can be connected to the VFD up to 9600 bps, 8,n, 1.