Non-UK citizens wishing to join the UK armed forces travel to the UK at their own expense and we do not normally return them to their countries of origin at public expense when they are discharged. It is therefore very much a personal decision, in line with Home Office policy, as to whether non-UK citizens return to their country of origin on discharge. An individual will normally be granted 28 days upon discharge to allow time for an application for further leave to remain to be lodged, or depart from the UK. Once an application is lodged they are able to remain in the UK legally while the application is processed.
The only exception to the rules is for Gurkhas, who are enlisted in Nepal and brought to the UK at public expense, and are now discharged in the UK and then returned to Nepal (if they wish) at public expense.
These policies have been agreed with the Home Office in order to ensure that they are not abused. They balance recognition of service in the British Armed forces towards naturalisation with preventing its use as an improper method of entry into the UK.