A migration agent’s review of ChatGPT: Is it a reliable and accurate resource for Australian immigration advice? [Part 2]
In Part 1, we discussed the strengths and limitations of ChatGPT in providing immigration information. In this article, we will ask it more specific questions, test its ability, and explore how it can be used more effectively as an immigration resource.
It’s surprising what critical detail it can get right, but it’s advisable to still check official sources and undertake further research
In another chat, we asked ChatGPT about how to apply for a Subclass 189 visa. Although it provided helpful steps, it missed a crucial one - the skills assessment. This is concerning because it is a necessary step in the visa process. An immigration expert would have advised doing this step before submitting the EOI to reduce the risk of running out of time after the invitation is issued or visa refusal.
While it is possible to submit the EOI without a skills assessment result, doing so would expose the applicant to unnecessary risk. It is a reminder that users must still conduct their own research or seek professional immigration assistance to avoid wasting time and effort during their visa preparation process.
What is the biggest thing to look out for when using ChatGPT for Australian immigration information?
As mentioned earlier, ChatGPT’s knowledge cut-off is set to September 2021. Despite this, it can still provide answers that appear current. In the below example, we see it claiming that the information is valid as of March 2023. This can be dangerous for users who are not aware of the knowledge limitation, and another reason why users should not completely rely on ChatGPT without verifying its information.
We reminded ChatGPT what it said earlier about its knowledge cut off date, only for it to apologise for the confusion and confirm that its previous statement may not be accurate.
Being a responsible program, it became a lot more careful in its subsequent answers, reminding us that its information is valid only as of September 2021.
Does the user need to have assumed knowledge of Australian immigration to use ChatGPT?
ChatGPT can be helpful for users who are not familiar with Australian immigration requirements. Its answers can provide broad guidance to start immigration research. However, users should be aware that ChatGPT can only offer general immigration information, and as we have pointed out in the previous article, its information cannot replace professional advice from a registered migration agent. It is important for users to conduct further independent research as well.
ChatGPT is a useful additional resource for those seeking information on Australian immigration requirements in a conversational manner. However, as the questions become more specific, ChatGPT may require users to have a certain level of assumed knowledge. Take for example the below.
A search of the state’s regional migration website indicates that Tasmania does not currently offer nominations for offshore Subclass 190 visa applicants. This example highlights that ChatGPT requires the user to possess knowledge of the particular visa and the state nomination process to know that this answer is incorrect. In this case, it would be best for the user to seek the advice of a registered migration agent, or conduct further research to find accurate information before starting on any application.
Can ChatGPT assess visa eligibility and provide a points assessment for skilled migration?
Based on the previous answers, we have seen how ChatGPT can provide general information on visa eligibility based on the set requirements. However, it cannot provide tailored advice based on an individual’s specific circumstances. Additionally when asked to do a points assessment, it gave an incorrect calculation. We tested ChatGPT’s ability by asking the following questions.
A brief check of the above circumstances against the points calculator indicates a passing score of 65 and therefore ChatGPT’s answer is correct. However, when asked specifically how many points this person can claim, ChatGPT got a few things wrong.
For a start, the skilled employment breakdown is incorrect. Applicants can claim 5 points if they have 3 - 5 years of experience and 10 points for 5 - 8 years of experience.
The 20 points attributed to language assume that the applicant has Proficient English, whereas a UK passport holder would be deemed to hold Competent English and therefore score 0.
Based on the premise provided and some basic assumptions, it should have calculated 65 points but ChatGPT calculated a score of 60. Most importantly, this shows that ChatGPT cannot act like a migration professional and provide personalised immigration advice by asking the applicant questions to understand their circumstances.
This does not mean that ChatGPT cannot be prompted to ask questions. When instructed to ask the user questions with the goal of visa eligibility assessment and providing migration advice, it does so but eventually reverted to providing generic migration information after two Q&A prompts.
What’s the verdict?
From our perspective, ChatGPT can be a good starting point for those researching visa options to migrate to Australia. ChatGPT analyses the available immigration information and presents it in a simple manner. While it is not 100% accurate, it can provide generic guidance to help users conduct their own research to understand their options further.
However, ChatGPT’s knowledge cutoff is limited to September 2021, so it would not have the most up-to-date information. Alarmingly, it can sometimes confidently claim to provide answers that are based on current information. As visa requirements and processes can change frequently, it is important to exercise caution when using ChatGPT and verify the information through independent research, or seek help from a qualified migration agent.
Due to its conversational manner, it can be easy to fall into the trap of accepting ChatGPT’s answers as migration advice and this could be risky. Users should be aware that while ChatGPT can appear to provide immigration information well, it does not have the ability to make an assessment and provide personalised migration advice.
Immigration regulations can be complex and what works for one individual may not apply to another. ChatGPT is not designed to take into account the user’s individual circumstances and assess it against specific immigration regulations. As a result, relying solely on ChatGPT’s information and guidance could lead to inaccurate advice and potential problems. Due to this limitation, it is still advisable to seek assistance from a migration professional to avoid any costly missteps in the application process, especially for those with complex immigration history or situations.
The effectiveness of ChatGPT depends on the prompts given and how the user interacts with it. Our experience showed that its answers were detailed but not always thorough. Due to the way we have interacted with it, ChatGPT often reminds us to verify its claims through further research or seek professional assistance. In a separate conversation, ChatGPT does not provide such caveats. Since ChatGPT’s answers are based on the previous questions asked, different users may receive different answers with potentially varying degrees of accuracy and confidence, depending on their interaction with ChatGPT.
For now at least, ChatGPT cannot replace the knowledge and experience of a migration agent who has submitted multiple applications and has expertise in immigration procedures. ChatGPT may be useful in helping applicants prepare their immigration application, but it is still recommended to have a professional review the information before submitting, to avoid any negative consequences.
What’s next with ChatGPT-4 and Australian immigration?
At the time of writing, ChatGPT-4 is released but is not yet available to the public. It claims to have been trained on a larger and more diverse dataset than ChatGPT-3 and has more advanced natural language processing capabilities. In theory, ChatGPT-4 could provide better immigration information than ChatGPT-3 by sourcing from a wider range of information and processing it more effectively.
ChatGPT-4’s ability to perform human-like tasks such as reading comprehension, logical reasoning, and critical thinking has advanced significantly. This is evidenced by its success in passing complicated exams with high scores, including ranking in the top 10% of Bar Exam applicants.
We will know more about ChatGPT-4’s impact on Australian immigration once it is released to the public. But while for now AI learning models should not be relied upon as a substitute for professional advice from a human expert, one thing is clear, the gap between human experts and AI is getting closer.
Disclaimer: The contents of this article is based on the writer’s personal opinion and experience of using ChatGPT-3, not ChatGPT-4. The information in this article is intended to serve as a general guide only and should not be relied upon as legal or professional advice. It is advisable to seek the guidance of a registered migration agent for tailored advice that is specific to your individual circumstances. We do not accept any liability for any reliance on the information contained on this website.