Is either one or the other quite suitable? If you said I never went to Australia when I was a child but after getting a job I went there last year, this means, quite simply, that you haven’t been to Australia. If you used instead I did not go, this may be interpreted with another context, for example if you didn’t go but were taken against your use, or maybe there was an opportunity you didn’t take, but now you did after getting a job.