---
title: "Import Data v3.2.1.2"
canonical: "https://servicedesk.maxgeo.com/space/DS/2185167486/Import%20Data%20v3.2.1.2"
format: markdown
---
> Macro (toc)

The Import Data menu option allows the user to load data into the database. The import data menu allows you to import CSV, Excel, Assay Files as well as creating and editing import layouts.

![Maxgeo Workflow - Step 152.jpeg](media://56323329-195c-4c65-a2d4-e88a01a63150)


## Importing Large Data Files

We’ve introduced several improvements to make importing large datasets more efficient and reliable:

- **Improved Large Dataset Handling**  
You can now import files much larger than 3,500 rows without the application becoming unresponsive.
- **Optimized Backend Processing**  
The `ImportData.js` module has been refactored to support **chunked data transmission**, reducing memory usage and improving overall performance.
- **Asynchronous Import with Hangfire**  
Large imports are processed asynchronously using **Hangfire**, ensuring better fault tolerance and a more responsive experience during heavy operations.
- **Sequential Job Execution for Data Integrity**  
Import jobs are executed sequentially **per user and per database**, maintaining data integrity—especially for parent-child table relationships.


In **User Settings**, tick the "**Always Use Job Service For Import**" checkbox to run import jobs through the **Hangfire** "**Job Service**", meaning they will run in the background. **Notifications** will be available in the **Notifications** section in DataShed5. Email notifications will also be sent to the Users Inbox.

![Notifications - Step 0 (1).png](media://ea01f350-1cd3-4b19-a3a6-98a0b4d77b77)

[Notifications User Guide](https://maxgeo.atlassian.net/wiki/spaces/DS/pages/2185169655)


## Import Data Header

To make setting up imports easier and more intuitive, the **Import Layout** selection has been simplified into two dropdowns.

- **Updated Control Name:**
  - The original “**Import Layout**” dropdown is now called **“Layout Type”**.
  - This first dropdown lets you choose a layout category, such as:
    - Any CSV
    - Any Excel
    - Assay File
    - XRF File
    - Control Points (MDS5 only)

Once a category is selected, a second dropdown can be used to let you pick a specific layout from that group.

![Maxgeo Workflow - Step 152 (1).jpeg](media://781b48c9-35c8-4eb2-9984-ea55173d8895)

## Importing CSV Files

- To import a CSV file first you need to select the import Layout “**Any CSV File**”  
and then click "**New Layout**". Or you can choose a predefined Layout.
- Then click "**Load File**".
- This will open the Import Layout editor.
- Select the table to import into. Then select the CSV file to import.
- Choose a DataSet for the data. This can be left blank if the import file contains this data.
- If you choose a Default DataSet, you will need to confirm the choice.
- Select a Date Format mask. For CSV files "**DD/MM/YYYY**" is the usual preferred format.

> ℹ️ **Please Note:** If you do not select a date format, DataShed will use your browser locale configuration to determine the date format for importing.

![Maxgeo Workflow - Step 154.jpeg](media://21a41e0e-4daa-4652-ab1d-97c936fd6f72)

By selecting the “**Append Data Source to Buffer Table**” check box above the buffer table, upon import we will use the file name as your data source. You will notice, once the options have been checked, an info icon will appear next to the Data_Source ‘To Field’ explaining the action.

> ℹ️ **Please Note:** If you select this option and have data in your source file under the Data_Source column, this data will be ignored and populated with the name of the import file.

![Import Data - CSV Files - Step 13.png](media://acb8a6dd-67b1-4bf7-ba58-a08b046e7322)

In the Import Layout table, we will automatically try to match '**From**’ and ‘**To**’ fields if the names match. These are displayed in the screen grab above. Fields that are not matched will be displayed in a peach color.

> ℹ️ **Please Note: **Not all fields need to be mapped for you to save the layout or load data. If you do not map a field and attempt to load the data, we will only display the fields you have mapped in the buffer table.

![Import Data - CSV Files - Step 14.png](media://90d7d3c9-dcf4-43cf-9cab-9f445de11f57)

### Editing a CSV Import Layout

The System Import Layout screen allows you to add, delete and map ‘**To**’ and ‘**From**’ fields.

- To add fields, click the ‘**+**’ icon on the top left-hand side of the screen. This allows you to map a single ‘From’ field to multiple ‘To’ fields.
- Select the "**From**" field
- Then select the "**To**" field.
- To delete a field, select the field using the check box on the left-hand side,  
then select the ‘**Delete**’ trashcan icon at the top left. As shown in the image below.

![Import Data - CSV Files - Step 15.webp](media://ee837d30-6ec1-4daf-baaf-e5d44279eddd)

### Saving the Import Layout

Once you have edited your layout you can then either save the layout or import data using the buttons at the bottom of the Import Layout Editor.  
Clicking the ‘**Save Layout’** option will allow you to enter a name to save the layout for future use. Enter a name for the layout and this will now be available for you under the Import Layout drop down. Saving the layout will remember the destination table and the mapping configurations.  
After you have saved your layout or without saving the layout, you can proceed to click the ‘**Load**’ button. If you have any unmapped fields in your layout, a warning message will appear. If you do not require the unmapped fields and want to import data with the unmapped fields, click ‘**Yes**’. Otherwise click ‘**No**’ and map the remaining fields.

> ℹ️ **Please Note:** Any fields not mapped will not be displayed in the buffer table or able to import with the remainder of your mapped fields.

Clicking '**Yes**' will load the data to the buffer table for review prior to importing.  
The data will be loaded to the Import Buffer.


![Import Data - CSV Files - Step 22.webp](media://28677e75-65bc-4f83-9829-9d51ed4bc153)


## Importing Excel Files

- To import an Excel file first you need to select the import Layout “**Any Excel Spreadsheet**”  
and then click "**New Layout**". Or you can choose a predefined Layout.
- Then click "**Load File**". This will open the Import Layout editor.
- Select the table to import into. Then select the Excel file to import.
- Then select the "**Worksheet**".
- Choose a DataSet for the data. This can be left blank if the import file contains this data.  
If you choose a default DataSet, you will need to confirm the choice.
- Select a Date Format mask. For Excel files "**MM/DD/YYYY**" is the usual preferred format, no matter how dates are displayed in the Excel file.

![Maxgeo Workflow - Step 156.jpeg](media://2f368b74-c884-4a0c-b246-4637c2ceca02)

By selecting the “**Append Data Source to Buffer Table**” check box above the buffer table, upon import we will use the file name as your data source. You will notice, once the options have been checked, an info icon will appear next to the Data_Source ‘**To Field**’ explaining the action.


> ℹ️ **Please Note:** If you select this option and have data in your source file under the Data_Source column, this data will be ignored and populated with the name of the import file.

![Import Data - Excel Files - Step 13.png](media://39230819-7d91-4873-925e-2b8a6d75ce0b)

### Editing an Excel Import Layout

The System Import Layout screen allows you to add, delete and map ‘**To**’ and ‘**From**’ fields.

- To add fields, click the ‘**+**’ icon on the top left-hand side of the screen. This allows you to map a single ‘**From**’ field to multiple ‘**To’** fields.
- Select the ‘**From**' field
- Then select the '**To**’ field.
- To delete a field, select the field using the check box on the left-hand side,  
then select the ‘**Delete**’ trashcan icon at the top left. As shown in the image below.

![Import Data - Excel Files - Step 15.webp](media://d58ccb9f-7bd2-414b-b79f-1303190bc64a)

> 📝 Saving the Import Layout is the same as for CSV files above.


## Import Buffer Table

For both CSV and Excel file formats, once the data has been loaded into the buffer table, you can Edit, Delete, Update, Export and Import data that is within the buffer table.

Simply click into the cells to edit the data within the grid. Once you have edited the data, press enter or click out of the cell to save the updates you made.

> ℹ️ **Please Note**: If you delete any data that is a required field, a warning will appear.

To delete a row or multiple rows from the grid, simply select the row using the left-hand check box and click the ‘**Delete**’ trashcan icon on the top left-hand side of the grid. Selecting the check box in the top left-hand corner of the grid will select or deselect all rows.

The Import Buffer Table screen is divided into two tabs – **Append Buffer to Database** and **Merge Buffer to Database**.

![image](media://f340b484-0de7-46cc-8992-832029763086)

The **Append Buffer to Database** tab contains records that are new, these will be appended to the database using a SQL Bulk Insert command on clicking the **Append Data** button. 

The **Merge Buffer to Database** tab contains records that already exist, these records will be updated in the database using a SQL Update command on clicking the **Merge Data** button.

On both tabs, if a data error is encountered, a message will be displayed where you can choose either:

1. [Validate](https://maxgeo.atlassian.net/wiki/spaces/DS/pages/2185167486/Import+Data+v3.2.1.2#Assay-Import-Process) and display errors
  1. This will append records row by row. Valid records will be appended or merged while erroneous records will remain in the Import Buffer where they can be edited and corrected. An alert icon will be displayed, which will display the error for the record when clicked.
2. Return to the Import Buffer to manually correct the data.

To Export data from the grid, select the '**Export**’ icon on the top right-hand side of the screen. You are then able to choose between exporting all data from the grid or selected rows to Excel. 

![image](media://ae29cfc2-c3f0-406e-94bc-c618290254bf)

Once you've made necessary edits to the data in the grids, initiate the data import process by selecting the '**Append Data**' button located in the top-left corner. The system will attempt to load all data residing in the buffer table. 

> ℹ️ **Please Note: **Only data passing the [validation](https://maxgeo.atlassian.net/wiki/spaces/DS/pages/2185167486/Import+Data+v3.2.1.2#Assay-Import-Process) checks will be successfully loaded into the database. Any data failing the validation process will be retained in the buffer table for subsequent remediation.

![image](media://6e45fd8e-c578-4847-9bdb-aa9d09010556)

As mentioned above, if there are any errors with the data you are importing, a red error notification will appear in the top right corner as well as a warning triangle will appear on the left-hand side of the record. As shown in the screen grab below. 

![image](media://fe9bfdec-8c08-43f8-ad29-5e9651ff1923)

You can view the details of the error by clicking on the red triangle on the left-hand side of the record. This will display the error messages as shown in the image below.

![image](media://43990415-3ab7-4006-ac80-17be7272dedf)

Click ‘**OK**’ to exit the error message and fix any issues with the data in the buffer table. Once you have fixed the data, you can proceed to click the ‘**Append Data**’ or ‘**Merge Data**’ button again to reload the data. If successful a green notification will appear in the top right of the screen and the data will be loaded to the specified table and the buffer table will be cleared.

![image](media://9814a88e-0215-4f72-a490-8eed787c2fbb)

## Find and Replace

- A “**Find & Replace**” option has been added to the right-click menu on all columns within buffer tables (e.g. Import > Buffers). This feature is available only to users with **SysAdmin** or **UserPlus** roles.
- A third right-click option – “**Find & Replace**” – now appears alongside existing Fix and Unfix options on buffer table columns.
- The dialog title dynamically reflects the column name on which the action is performed for clarity.
  

A right click on a field name in the Import Buffer table, will display a menu with three items, the bottom of which is "**Find and Replace**".  
Click "**Find and Replace**" to open the below window.

> ℹ️ **Note:** The window is titled with the Field name that the operation will effect.

- Click the "**Enter value to replace with...**" field.
- Enter the value to replace it with, then click "**Apply**".
- A confirmation message will display. Click "**OK**" to continue.  
A success message is displayed.

![Maxgeo Workflow - Step 19.jpeg](media://b3b1b9bc-de04-45c2-bd1e-4a06aa296a4f)


## Import Layout Configurations

### Buffer Modifications

**A detailed guide on Buffer Modifications can be found ****[here](https://maxgeo.atlassian.net/wiki/spaces/DS/pages/1437270032)****.**

To access Buffer Modifications, click "**Import Layout Configurations**"  
A new "**Layout Type**" dropdown list has been added to the top left of the screen, this allows you to select a Layout Type.  
Layouts of the selected type are displayed in the list below.

Select the Layout to use.  
Buffer Modifications now support **Update To**, **SQL Delete** and **SQL Update** Actions.

![Import Buffer - Updates - Step 11.webp](media://492cd85d-1711-4d64-8129-ab114422b5cc)

Selected **UPDATE functions** and selected **SQL UPDATE statements** are supported in DataShed5 when using buffer modifications on a saved layout.

- The SQL Update tool has 3 uses:
  - Set one field equal to another field in the buffer table or a constant.
  - Use the **Where** clause to filter records to apply the **SQL Update** to.
  - Use the **CASE WHEN** statement to apply various values to replace.

> ⚠️ **Limits:** The SQL Update can’t be used in combination with Functions from Update To


**Syntax examples**  
**Set a field’s value using the Where clause**  
Program_Type = ‘Grade Control’ Where Prospect = ‘Nswazwi’  
Program_Type = Comments Where Comments = ‘Grade Control’  
**Case When**  
Core_Diameter = CASE WHEN Hole_Type = 'DD' THEN 'HQ' WHEN Hole_Type = 'DDH' THEN 'PQ' ELSE 'NR' END

When using constant values in functions, the text/dates need to be enclosed in double **“ “** or single** ' '** quotes.  
Example: ISNULL(Orig_Grid_ID, 'MGA94_51')  
or DATEDIFF(day, "01/01/2025", "24/04/2025")

![Import Buffer - Updates - Step 16.png](media://b4a258a9-728e-4418-8ebb-852a98f6af73)

When the Buffer Modifications have been configured, click "**Save**". Return to the Import Data screen.

Select your **Layout Type**, **Import Layout** and then click "**Load File**".  
Select your import file. Apply a Date Format. Then click "**Load**".

![Import Buffer - Updates - Step 18.webp](media://834db976-f461-46d5-811f-24b6979aeeb0)

The Import Buffer will load containing the Buffer Modifications as below.


![Maxgeo Workflow - Step 20.jpeg](media://99c8ac08-e50e-4aaa-9a00-6a74ca30d282)

Here is the raw CSV file used for the import.

![Import Buffer - Updates - Step 22.png](media://efdda9e7-130b-4b5f-aa8d-d274731b5adb)


## Importing Assay Files

Assays can be imported under the import menu item. First you need to configure your assay layout then you import the assay file and fix any potential errors with the loading.

### Assay Import Process

When DataShed initiates the assay import process, the following process occurs:

- **Data Validation:** Assays are loaded into an incoming buffer table, enabling checks to ensure data integrity. DataShed5 scrutinizes the lab method, element, unit, and lab libraries to validate data against these standards. The Generic Method is updated based on the Original Lab Method, and the Element and Repeat fields are populated from the Lab Element library.
- **Sample ID Matching:** Incoming Sample IDs are compared with those in each Sample table.
- **Result Calculation:** DataShed5 calculates the sysResult from the Assay Result. Text fields are converted to corresponding substitution values, and the text equivalent is entered into the sysAssayStatus field.
- **Automatic Record Merging:** The system automatically merges records that have successfully passed validation. The assays are subsequently ranked post the merge process.
- **Handling Unsuccessful Records:** To review unsuccessful records, navigate to the Assay Manager. Unmerged data is listed in the Unmerged Assay Results section. Upon resolving the unmerged data, click the **Process Batch** button. DataShed5 conducts checks and attempts to merge the data seamlessly.

### Import Layout Configurations

To configure your Assay Import Layout, first you need to click on the ‘**Import Data**’ menu item and then ‘**Import Layout Configuration**’ button in the top right corner.

![Maxgeo Workflow - Step 21 (2).jpeg](media://31f22af0-5120-44f8-8319-bc0809236a06)


To make setting up imports easier and more intuitive, the Import Layout Configurations selection has been simplified by adding a "**Layout Type**" dropdown.

This will only display Layout Configurations for the selected Layout Type.

In this case, select "**Assay File**".

![Maxgeo Workflow - Step 23.jpeg](media://c34214a7-fb4f-4745-bee1-d608a753af33)

The import layouts page shows the existing import layouts on the left, you can search, delete (trashcan icon) and edit existing import layouts. A new layout can be added by clicking “**Add New Layout**” button at the top left of the screen.

When the “**Add New Layout**” button is clicked, the Import Layout page is opened for the user to enter the details of the Import Layout. During this process the existing Import Layout list is disabled until the new layout page is saved or closed.

![Maxgeo Workflow - Step 24.jpeg](media://fee13106-9a54-4ac8-afd4-08c764ad1dd5)

Once you have entered the details of the Import Layout, click the ‘**Save**’ button to save the import layout or ‘**Close**’ to cancel the creation. Once the new layout is saved, a successful message is displayed at the top right and the created layout should be selected by default.

![image](media://7c406b52-5552-496c-80e6-3471d3d6e1cd)

### Select Import Layout Type

Next you will need to select the ‘**Import File Type**’ from the dropdown list. For example, “SGS CSV Format” is selected for '**Import File Type**’. The selected file type is saved to the database on selection. The save successful message is displayed on selection.

![image](media://13d6e5a0-9bac-4409-b6ee-2110677e790f)

### Import Layout List

The ‘**Import file type list**’ can be managed by clicking on the ‘**Import file type list**’ icon. When you hover the mouse over the icon, the “**Import file type list**” tooltip is displayed. (As per below screen grab)

![image](media://e40a5827-1eaf-4210-b03a-726b8fe90f2f)

Clicking “**Import file type list**” opens the file type list pop up which allows you to:

- Add a new file type using the **+** icon and updating the information for the new file type. File Type and Description Fields are required.
- Edit existing file types by either editing directly into the cell or using the provided drop downs.
- Delete/bulk delete file types by either selecting one or multiple rows using the checkboxes on the lefthand side and selecting the ‘**Delete**’ trashcan icon at the top left.

> ℹ️ **Please Note:** You are required to click the “Save” button to save the details to the database. Closing the popup without saving may lose changes you have made.

> ℹ️ **Please Note:** You cannot have a duplicate File Name or Description; a warning will appear if you attempt to save a supplicate record.

![image-20250701-051432.png](media://2e901c3b-a6a2-4ecb-9b68-3b8ed11c2953)

Once you have successfully updated the Import File Types and click “Save” you will receive a successful message as per below. 

![image-20250701-051652.png](media://dbf13c69-d337-4685-9c98-720dfbb9a4dd)

### Assay File Format Editor

Next you will need to select the 'Assay file format editor’ icon.

![BufferMods7.jpg](media://e220cc50-eca6-41f3-9df6-0f71e155edfd)

> ℹ️ **Please Note:** The Assay file format editor doesn’t appear until the destination table under Buffer Modifications has been selected.

Once you have clicked the ‘**File icon**’ (Assay file format editor), the Assay File Format Editor page will be opened in the same page.

The Assay File Format Editor displays the selected file type description as Scheme Name. 

You can now upload the **.csv** and **.sif** file types by clicking the “**Select File**” button. 

> ℹ️ **Please Note:** DataShed5 only supports these two file types.

![BufferMods8.jpg](media://4bbcacf9-dd5c-4ab4-be70-6ed645633cf0)


Once you have selected your file, the Layout Definition panel can be seen under the file editor panel. The layout definition can be defined for each file type. 

In the Layout Definition, go to the Data Item drop down list and select a Data Item.

For each comma delimited field, click the mouse cursor at the start of each comma delimited section. 

![BufferMods8a.jpg](media://668db992-596c-4ec5-be5c-e1e97821031f)


When selecting another Data Item, the value for the previous selected Data Item will be saved automatically. Once the layout definition has saved successfully, a message is displayed.

![BufferMods9.jpg](media://24b17803-a2e8-4b52-b9bb-d01db9193802)


### Import Substitution

To manage substitutions, click the “**Substitution**” button. The import substitution list popup window will be open. From there you can add, edit or delete substitutions. 

Click the **+** icon button to add a new substitution. The substitution set is pre-populated with the selected file type. (For example, SGS_CSV)

The Symbol and AssayStatus fields are required fields. So, leaving these fields empty, the following error message will be prompted.

![image-20250701-052235.png](media://60e6380f-5871-40f2-81c7-a1ba80858783)

The Symbol and other fields are free text fields, but the AssayStatus value can be selected from the dropdown list.

> ℹ️ **Please Note: **You cannot duplicate the substitution set and symbol.

![image-20250701-052315.png](media://7fcdd39e-c8f2-442f-8f9a-05ee5c8309d8)

Once the new record is added to the list, make sure you click the “**Save**” button to ensure that the data is saved to the database. 

> ℹ️ **Please Note:** Clicking the “**Close**” button may result in losing the pending changes so the system will always show the confirmation dialog for unsaved changes if there are any pending changes.

![image-20250701-052349.png](media://7b94bbcc-da94-4172-acb4-ee69f63b1715)

Clicking the “**Save**” button will save the new substitution record to the database and display the saved successful message.

![image-20250701-052511.png](media://a79b4ec5-cfb2-4383-aa2a-542b0d0a0c89)

Editing existing substitution records is the same as adding a new substitution records. Go to the field that needs to be edited, update the value. Then click the “**Save**” button to ensure that changes are saved to the database.

To delete a substitution you can either single delete or bulk delete by selecting the substitution record(s) and clicking the ‘**trachcan icon**’. The selected substitution record(s) will be deleted from the list.

![image-20250701-053133.png](media://e1919fb5-f0e0-4f4b-b453-2fe62eca03f4)

Then click the “**Save**” button to ensure that the records are removed from the database.

![image-20250701-053157.png](media://4b370b72-cd3b-460c-87a6-89a1b3fbeb45)


### Import Validation

Once you have finished updating your layout definition and substitutions you can validate the layout by clicking the ‘**Validate**’ button. The system will go through the saved data items for the selected file and display the data items in the result pop up (as the below diagram).

The progress bar shows the validation progress. The “**Copy to clipboard**” button allows you to copy the result to clipboard.

![image-20250702-054334.png](media://68d286d1-74ce-4194-ad62-048f00358bb3)


### Buffer Modifications

The Buffer Modifications panel opens when you click the ‘**Buffer modifications**’ button (the checklist icon). When you mouse over the checklist icon button, you will notice a “**Buffer modifications**” tooltip is displayed (as below diagram).

![BufferMods.jpg](media://309f71ce-4c8e-421d-99dc-e436c9b6eac4)


The buffer modifications section has a “Destination Table Name” dropdown. Here you can choose which table you would like the file to be imported to.

> ℹ️ **Please Note:** Once you have selected your destination table, please click save to save the selection to the database.

![BufferMods2.jpg](media://dcf4a8bf-a274-4217-9cfe-9495748ff4a0)


Click the **+** icon button to add a new field modification. The “FieldDB” column and the “Action” columns have dropdown lookup lists to select from, which is based on the selected import file type.

> ℹ️ **Please Note**: there is no validation check for any of the fields.

![BufferMods3.jpg](media://be77b29e-19c3-4ee0-bf13-350bed4d9768)


Once you have added the new buffer fields modifications, you need to click “**Save**” to ensure that the record(s) are saved to the database. A saved successfully message is displayed on clicking “**Save**” button.

![BufferMods4.jpg](media://ef7a3a2a-f576-4c48-8a23-072ea3972e4d)


To edit a modification, simply select a field, edit it as required and click ‘**Save’** to save your changes.

To delete a modification simply select one or more records and click the ‘**bin icon**’. These records will be removed from the list. Once again, click '**Save**’ to save you changes.

![BufferMods5.jpg](media://3a23d3b6-e5bf-4f3e-9afe-2924e87f8440)


### Check Config:

Next, you will need to check your configuration. Once you click the ‘**Check config button**’ (the check mark icon) The “**Check Config**” panel will open.

![BufferMods6.jpg](media://5365afc4-9b98-4f7d-9bff-2e8ae53c70fe)


The ‘**Check Config**’ section has two boxes, one is for the config description, the other is for any errors which may be found.

There are three main buttons ‘**Check Config**’, ‘**Check Config for All**’ and ‘**Save to File**’.

- The '**Check Config’** button checks the configuration for the selected import layout.
- The ‘**Check Config for All**’ button checks the configuration for all import layouts in the system.
- The ‘**Save to File**’ button exports and downloads the diagnostic result in a ‘.dsd’ file.

Once you select either the ‘**Check Config**’ or the ‘**Check Config for All**’ buttons, you will notice the 'Check config’ boxes will be populated with successful checks and errors as per below.

![image-20250702-055903.png](media://a1570463-974f-4abd-9a74-cab618523580)

Clicking the “**Save to File**” button will export and download the **DataShed Diagnostics File.dsd** file. The file can be opened in notepad. 


### Lab Checks and Standards

It is recommended that the labs report the checks and standards that they use for quality control and include the results in the results file. Most labs have a prefix or suffix which identifies lab standards and lab repeats.  
The stored procedures will process standards and lab QC samples by creating a unique SampleID, if necessary and populating the appropriate sample tables with SampleIDs.  
The procedure references a library table, **tblSYSIncomingDataMasks**, which must be configured to suit the client, with the Labcodes, QC_Categories and StandardIDs that are relevant to the client.

![LabFile.png](media://cf845aa8-6dd2-452e-918c-253c6d16fd43)

**The Process has 3 steps:**  
· **Setting up Incoming Data Masks:**  
This process identifies which records are lab QC results, and configures how the samples are to be processed.  
· **Updating libraries: Standards Reference and QC Category**  
Adds the relevant records to the library tables so that the samples can be inserted into the standard sample table or the sample QC table.  
· **Running the SP Data Driver**  
Runs the process to update the sample ids in the incoming assay table and populate the relevant sample tables with the sample ID.

### Setting up the Incoming Data Masks

The table **tblSYSIncomingDataMasks** is populated with the identifiers for the various QC data supplied by the lab. This information is generally supplied at the end of the lab file. The following notes detail how to populate this table.

![Import Data - Assay Files - Step 13.png](media://819e7e5e-7141-41dd-9b35-0d6868589a41)

The **%** is the wild card for this stored procedure.  
**Identifier: **This uses wildcards to isolate the key identifier which will indicate that the sample is a lab dup or standard.

> 📝 The identifier needs to include **all** the characters preceding the actual SampleID being repeated else the mask will not work. In the example above where dups are denoted by CH:, creating a mask like CH% will not work as the colon will be added to the SampleID when the data is processed. The mask needs to include the colon (:) and the same is true for the standards in this case.
> 📝 
> 📝 ST:% is the only acceptable identifier. Neither ST% nor S% will work.
> 📝 
> 📝 CH:% is the only acceptable identifier. Neither CH% nor C% will work.

**Lab Code: **The code of the lab as stated in the database library the table **tblSYSAssLab**.  
**QC Category: **The QC_Category of the lab check as stated in the table **tblLIBQCCategory**. Do not populate for “identifiers” that relate to standards. 

**Is Standard:** Flag if it is a standard (0 = no, -1 = yes). If this is false then the stored procedure will assume that it is a QC sample.  
**StandardID: **Populate with the Standard ID for “identifiers” that related to standards. Leave blank for “identifiers” that are QCsamples. Not essential to populate.  
**Append batch no**: Flag (0 = no, -1 = yes) to append the Batch_No to the SampleID. If the standards in the lab file don’t have the Batch_No attached, then set flag to yes. This will prevent duplication of standard SampleIDs, which must be avoided. Only relates to standards and must be set to no (0) for QC Samples.  
**Add Lab to BLANK**: Flag (0 = no, -1 = yes) to append the lab code to the StandardID where the StandardID =Blank. This enables the analysis (in QAQCR) of the Blanks from different labs on separate charts, rather than grouping all of the blanks from all labs together. This flag will add a new record to **tblSYSQCReference** where necessary (e.g., Blank_ALS_VAN, or Blank_SGS_VAN). Relates only to standard “BLANK”  
**STD nonUnique**: Flag (0 = no (SampleID is unique), -1 = yes (SampleID is not unique). If the standard SampleIDs in the lab file are not unique, then this flag generates a unique SampleID by adding the SourceRowNumber to the SampleID. This flag is typically set to no (0) for standard SampleIDs such as “ST01:  
ST02:” It is generally necessary to set this flag to yes (-1) for standard SampleIDs such as “ST:” without a unique number, or where SampleIDs = StandardID and do not have any prefix (such as “Blank”, “Sarm7B”), and are repeated in the laboratory file. Relates only to standard identifiers


## Importing pXRF Data

This guide offers a clear, step-by-step process for importing pXRF sample and assay data from Excel and CSV files. It covers layout creation, data review, and appending for both drillhole and surface samples. Designed to streamline data input, the guide helps users improve accuracy and efficiency in managing geochemical data for their projects.

### Import pXRF DH Sample Data

- Click "**Import Data**"
- Select the Import File format type - in this case an Excel file
- Click "**+ New Layout**" to create a new Import Layout.
- Click "**Load File**" to open the File Loader modal window.
- Select "**XRF DH Samples**" for DH Samples.
  - For Point Samples, select "XRF Surface Samples".
- Select the file and worksheet to import.
- Select a default DataSet (if required) and a Date Format.
- Match the fields to import and delete unrequired fields.
- Click "**Save Layout**" to save the Layout.
- Enter a Layout name and click "**Ok**".
- Click "**Load**" to load the import file to the Import Buffer.
- Review the data in the Import Buffer. When satisfied the data is correct, click "**Append Data**".
- A message "**Data has been appended successfully**." is displayed for a successful file import. The Import Buffer will be empty.

![Maxgeo Workflow - Step 26.jpeg](media://ae452b94-bea1-4141-9122-68077d8c1c21)


### Viewing the Imported Samples

- Click "**View Data**"
- Expand the **XRF Management** module.
- Select "**XRF DH Samples**"
- Apply a filter to see the imported samples.

![Maxgeo Workflow - Step 27.jpeg](media://fdf83f1f-882e-437e-9e9a-18a1ebc726d3)


### Importing pXRF Assay Results

- To import a pXRF file, select **pXRF File** from the drop down list.
- Then select "**+ New Layout**" from the Select Import Layout drop down.
- Click "**Load File**" to open the pXRF Import Layout screen.
- On the pXRF Import Layout screen, select the pXRF file to be imported.

> ℹ️ **NOTE:** pXRF files must be in CSV format and the Element Columns must be the right most columns.  
> ℹ️ Select the **Date Format, Units, Lab Code** and **Lab Method** from the drop down lists.

![Maxgeo Workflow - Step 29.jpeg](media://ec68a563-22e2-4ad9-a888-8b8c672d3dae)

### Field Matching

In the Import Layout table, we will automatically try to match 'From’ and ‘To’ fields if the names match. These are displayed below. Required fields that are not matched will be displayed in a blue color.

> ℹ️ **Please Note:** Not all fields need to be mapped for you to save the layout or load data. If you do not map a field and attempt to load the data, only the fields you have mapped will be displayed in the buffer table.

For matching the Elements, only the first pXRF Elements needs to be matched. The importer will automatically identify following Elements.  
Saving the layout is a required step before loading the data. To save the layout, click "**Save Layout**"  
then enter a valid Layout Name. If you click the "**Load**" button without saving the layout, a warning will display in the top right of the screen and the Save Layout screen will display.

After entering a name, click "**Save**".  
A "Saved layout successfully." message will be displayed.  
When the Layout Configuration is complete and the Layout saved, click "**Load**". The file will be imported as per the Layout Configuration.

> ℹ️ **NOTE:** Any unmatched fields will not be imported.

![Importing pXRF Sample and Assay Data - Step 28.png](media://8373ecc4-0178-417d-99c5-83194bca4fb2)


The Import Data dialog shows the import progress and will notify any errors in red. The example below shows a successful import.

Click "**Close**" to return to DataShed. The loaded pXRF data will be available in Assay Manager.

![Import Data - pXRF File - Step 15.png](media://e9458830-8acd-4042-86a0-d336f15a9f74)

### Viewing the imported pXRF Assays

- Click "**View Data**".
- Select the "**XRF Management**" module.
- Open the "**XRF DH Results**" table.
  - For Point samples, open the "**VW XRF Results**" table.
- Set an appropriate filter.

![Maxgeo Workflow - Step 27.jpeg](media://fdf83f1f-882e-437e-9e9a-18a1ebc726d3)


## Import Control Points

To import Coordinate Control Points, you need to select the import Layout ‘**Control Points**’ and click ‘**Load File**’.

 This will open the Import Layout editor. From this window you have the option to select the file to import. Once you have selected your CSV file, the Import Layout table will be displayed as per below screen grab.

> ℹ️ **Please Note:** The file type must be CSV and the format must be as indicated in the red rectangle below.

![image-20250702-061841.png](media://4dbf2c56-ecfc-4fe3-8958-061d61bb6099)

In the Import Layout table, we will automatically try to match 'From’ and ‘To’ fields if the names match. These are displayed in the screen grab above. Fields that are not matched will be displayed in a peach color.

After the fields have been correctly mapped, click the “**Load**” button to load the Control Points data to the Import Buffer.

### Import Buffer Table

Control Points data will be loaded to the “**SRS Control Points**” table located in the Database Management Module.

![Maxgeo Workflow - Step 30.jpeg](media://4711c23e-7434-4fbd-b06e-a40427203224)

### Add and Edit Coordinates

Coordinates are maintained in the “**DH Coordinates Normalised**” table located in the Drilling Management Module.

Click the **+** button to add a new record. If a transformation has been defined for the Grid ID, transformation records will be automatically generated when the new record is saved. The system prioritises the measured source coordinates for transformations based on the highest **Confidence Level** and best priority **Survey Method**.

Additionally, a Lat/Long transformation is automatically added to the collar record and is always kept up to date as the coordinates in the **DH Coordinates** table change over time.

If a new coordinate record with higher accuracy is entered — like one surveyed via DGPS — the system will regenerate the transformation records using the best coordinates available, and the collar Lat/Long fields will update to reflect the more accurate values.