Yes, a foreign citizen can start a business in Australia. If the business is run as a company structure, due to the minimum requirements for directors, there will need to be at least one Australian resident involved as a director of the company.
If you are a foreign citizen wanting to register a company and do not meet the minimum director requirements, get in contact with us and see how we may be able to help. See: https://www.patricia.com.au/other-services/australian-resident-director.