DaTa TaDa

Team Name

DaTa TaDa

Team Members

Project Description

Australia has a rich ecosystem of government data fragmented across federal, state, and departmental levels. This information is a national asset, but with its breadth and diversity, it can sometimes be challenging for users to know where to start.

Our project introduces Data Navigator AI, an intelligent platform that makes it easier to discover, understand, and connect datasets. Users can simply enter natural language queries or apply intuitive filters (such as categories or time periods) to quickly locate relevant information.

Key features:
• AI-powered search that understands user intent.
• Unified dataset view (purpose, custodian, collection date, access method).
• Visual mapping of how datasets complement each other.

By simplifying access and enriching the user experience, our solution helps Australians gain insights more efficiently, while showcasing the tremendous value of existing government data holdings.

Data Story

Australia’s open government data environment is extensive and growing but fragmented, with contributions from agencies such as ABS, BOM, Education, and many others. This richness presents an exciting opportunity to empower decision-makers, researchers, and communities.
1. Diverse sources – Data is available from many agencies, each offering valuable perspectives.
2. Different access formats – Some datasets are downloadable, others available via APIs or portals, giving users multiple ways to engage.
3. Varied metadata – Datasets are described in different ways, which creates an opportunity to standardise and simplify discovery.
4. Complementary insights – Many datasets can be used together (e.g., demographics with climate or education), opening new avenues for analysis.
5. Potential for time savings – A unified discovery experience means users can focus more on deriving insights than on searching.

Our story:
• A public sector can ask “Show me some population projections related data please” and instantly find relevant datasets.
• A researcher can explore how education enrolment connects with employment outcomes, with relationship mapping.
• Community groups can leverage accessible open data to support planning and advocacy.
• Although data.gov.au is very comprehensive, with data from different departments, a lot of data within each of these departments still remains unlisted and difficult to find.

This story is about turning a diverse and rich ecosystem of datasets into a more connected and user-friendly experience, making Australia’s open data even more impactful.

Method:
• A combination of API calls and a built-in script fetching web pages containing data.
• The script applies a filter on a given website sitemap.xml to retrieve the web addresses containing keywords related to data.
• Then, a webcrawler visits this filtered list to confirm and return information about the data files (e.g. xlsx) contained within.
• We made sure that the webcrawler is compliant with the target website's robots.txt.
• The output is a JSON file combining the data from the API and the webcrawler.

Future Potential:
Because of hardware limitations for our local AI, there is the potential to include more datatsets into the system.