---
title: "v3.0.3 Import Data"
canonical: "https://servicedesk.maxgeo.com/space/DS/1412104991/v3.0.3%20Import%20Data"
format: markdown
---
> Macro (toc)

The Import Data menu option allows the user to load data into the database. The import data menu allows you to import CSV, Excel, Assay Files as well as creating and editing import layouts. 

![image](media://c98e22fb-082d-4660-85d3-fc5583f95c4e)

## > Macro (anchor)

Importing CSV Files

> Macro (anchor)

> Macro (anchor)

To import a CSV file first you need to select the import Layout ‘Any CSV File’ and click ‘Load’

![image](media://3496e55e-50c2-4643-93d9-65b977686bdd)

This will open the Import Layout editor. From this window you have the option to select the Database Table, DataSet, File and Date Format. Once you have selected your Database Table and CSV File (Mandatory Fields), the Import Layout table will be displayed as per below screen grab.

**Please Note:** If you do not select a date format, we will use your browser locale configuration to determine the date format we will import to. 

![image](media://35e71a56-2bcb-47e4-b4ee-97e8ff7a43c0)

In the Import Layout table, we will automatically try match 'From’ and ‘To’ fields if the names match. These are displayed in the screen grab above. Fields that are not matched will be displayed in a peach color.

**Please Note:** Not all fields need to be mapped in order for you to save the layout or load data. If you do not map a field and attempt to load the data, we will only display the fields you have mapped in the buffer table.

## > Macro (anchor)

> Macro (anchor)

Editing a CSV Import Layout:

The System Import Layout screen allows you to add, delete and map ‘To’ and ‘From’ Fields. 

To add fields, click the ‘+’ Icon on the top left-hand side of the screen. This allows you to map a single ‘From’ field to multiple ‘To’ fields. 

![image](media://b92a114c-1cf2-4449-b0f9-2922559a1577)

To delete a field, select the field using the check box on the left-hand side and select the ‘Delete’ trashcan Icon at the top left. As shown in the screengrab below.

![image](media://67655988-59ba-4ddb-9d8d-6a9a46d750cd)

To map a > Macro (anchor)

‘From’ field to a ‘To’ Field, you can use the drop downs from both the ‘From’ field to a ‘To’ Field ad displayed in the image below. 

![image](media://557c4ca5-31c0-42ab-9b3d-4b24b373c1af)

By selecting the ‘Append Data Source to Buffer Table’ check box above the buffer table, upon import we will use the file name as you data source. You will notice, once the options have been checked, a info Icon will appear next to the Data_Source ‘To Field’ explaining the action. 

**Please Note:** If you select this option and have data in your source file under the Data_Source column, this data will be ignored and populated with the name of the import file. 

![image](media://dd54a140-2cdc-4985-a981-61f2e744718c)

## > Macro (anchor)

Saving A CSV Layout and Loading Data:

Once you have edited your layout you can then either save the layout or import data using the buttons at the bottom of the Import Layout Editor. 

![image](media://b5099170-ab5f-4cc7-9bd4-2ee4c53f80f0)

Clicking the Save Layout option will allow you to enter a name to save the layout for future use. Click ok, once you have entered a name for the layout and this will now be available for you under the Import Layout drop down. Saving the layout will remember the table and the mapping configurations you edited. 

![image](media://3c928b06-dd56-4194-9853-8cc2639f6b76)

After you have saved your layout or without saving the layout you can proceed to click the ‘Load’ button. If you have any unmapped fields in your layout, a warning message will appear. If you do not require to import data with the unmapped fields, click ‘Yes’. Otherwise click ‘No’ and map the remaining fields. 

**Please Note:** Any fields not mapped will not be displayed in the buffer table or able to import with the remainder of your mapped fields. 

Clicking ‘Yes” will load the data to the buffer table for review prior to importing. 

![image](media://8bfdac2c-ff0a-4d2a-85bf-2fa6feccea70)

## > Macro (anchor)

Importing Excel Files

To import a Excel file first you need to select the import Layout ‘Any Excel Spreadsheet’ and click ‘Load’

![image](media://1b5d3892-d4ab-4c2b-8650-2fe8d5551366)

This will open the Import Layout editor. From this window you have the option to select the Database Table, DataSet, File and Date Format. Once you have selected your Database Table and Excel Spreadsheet and WorkSheet Name (Mandatory Fields), the import layout table will be displayed as per below screen grab.

**Please Note:** If you do not select a date format, we will use your browser locale configuration to determine the date format we will import to. 

**Please Note:** DataShed5 only supports the .xlsx format. We do not support .xls


![image](media://27613037-d544-424c-8d87-9d699027f17a)

In the Import Layout table, we will automatically try match 'From’ and ‘To’ fields if the names match. These are displayed in the screen grab above. Fields that are not matched will be displayed in a peach color.

**Please Note:** Not all fields need to be mapped in order for you to save the layout or load data. If you do not map a field and attempt to load the data, we will only display the fields you have mapped in the buffer table.

## > Macro (anchor)

Editing and Excel Import Layout:

The Import Layout screen allows you to add, delete and map ‘To’ and ‘From’ Fields. 

To add fields, click the ‘+’ Icon on the top left-hand side of the screen. This allows you to map a single ‘From’ field to multiple ‘To’ fields. 

![image](media://d47ff932-8487-429a-a129-07eedd6a53af)

To delete a field, select the field using the check box on the left-hand side and select the ‘Delete’ trashcan Icon at the top left. As shown in the screengrab below.

![image](media://c42187a2-0682-43f3-9108-c52b224cf9b2)

To map a ‘From’ field to a ‘To’ Field, you can use the drop downs from both the ‘From’ field to a ‘To’ Field ad displayed in the image below. 

![image](media://a4e39261-20e1-4a6c-80b4-0ab539a2e12b)

By selecting the ‘Append Data Source to Buffer Table’ check box above the buffer table, upon import we will use the file name as you data source. You will notice, once the options have been checked, a info Icon will appear next to the Data_Source ‘To Field’ explaining the action. 

**Please Note:** If you select this option and have data in your source file under the Data_Source column, this data will be ignored and populated with the name of the import file. 

![image](media://6346b566-6167-46fb-9331-6f58a8b649d1)

## > Macro (anchor)

Saving An Excel Layout and Loading Data:

Once you have edited your layout you can then either save the layout or import data using the buttons at the bottom of the Import Layout Editor. 

![image](media://32ef9bbe-c8cf-4fcc-8311-300af8cbcb4c)

Clicking the Save Layout option will allow you to enter a name to save the layout for future use. Click ok, once you have entered a name for the layout and this will now be available for you under the Import Layout drop down. Saving the layout will remember the table and the mapping configurations you edited. 

![image](media://b6cfcfaa-6aa3-4f4d-8ab6-b9f0b3d2ddbb)

After you have saved your layout or without saving the layout you can proceed to click the ‘Load’ button. If you have any unmapped fields in your layout, a warning message will appear. If you do not require to import data with the unmapped fields, click ‘Yes’. Otherwise click ‘No’ and map the remaining fields. 

**Please Note:** Any fields not mapped will not be displayed in the buffer table or able to import with the remainder of your mapped fields. 

Clicking ‘Yes” will load the data to the buffer table for review prior to importing. 

## > Macro (anchor)

Buffer Table:

For both CSV and Excel file formats, once the data has been loaded into the buffer table, you can Edit, Delete, Update, Export and Import data that is within the buffer table. 

Simply click into the cells to edit the data within the grid. Once you have edited the data, press enter or click out of the cell to save the updates you made. Once you have saved the record you can use your arrow keys to toggle around the table.

**Please Note**: If you delete any data that is a required field a warning will appear as shown below. 

![image](media://106d68ac-ca35-41ba-a51c-70d110b95dcd)

To delete a row or multiple rows from the grid, simply, select the row using the left-hand check box and click the ‘Delete’ trash icon on the top left-hand side. Selecting the check box in the top left-hand corner of the gris will select or deselect all rows.

![image](media://b22c536a-7cea-437b-816d-1da8bd617797)

To Export data from the grid, select the 'Export’ Icon on the top right-hand side of the screen. You are then able to choose between exporting all data from the grid or selected rows to Excel. 

![image](media://67045f9f-9078-4d32-8c81-38171acde5ab)

Once you've made necessary edits to the data in the grids, initiate the data import process by selecting the 'Import Data' button located in the top-left corner. The system will make an attempt to load all data residing in the buffer table. 

**Please Note: **Only data passing the validation checks will be successfully loaded into the database. Any data failing the validation process will be retained in the buffer table for subsequent remediation.

![image](media://ff630ab0-b813-4c3d-8d3c-ac354ef85bd6)

As mentioned above, if there are any errors with the data you are importing, a red error notification will appear in the top right corner as well as a warning triangle will appear on the left hand side of the record. As shown in the screen grab below. 

![image](media://2eda30c5-5f06-42a5-976f-ed9ece215f66)

You can view the details of the error by clicking on the red triangle on the left-hand side of the record. This will display the error messages as shown in the image below.

![image](media://d6a6dd9c-e5dd-4f0a-a9a9-9f7bba1ad1ef)

Click ‘ok’ to exit the error message and fix any issues with the data in the buffer table. Once you have fixed the data, you can proceed to click the ‘Import Data’ button again to reload the data. If successful a green notification will appear in the top right of the screen and the data will be loaded to the specified table and the buffer table will be cleared.

![image](media://915ddd5d-6213-41a3-9dfe-87d3811d658f)

## > Macro (anchor)

Editing All Import layouts.

The selected layout can be edited by clicking the edit pencil icon next to layout name. The layout page also displays selected lab code and description.

When clicking the ‘edit icon’, the layout details are shown in edit/modify mode. The layout name can’t be edited therefore it’s disabled.

The other details can be edited in edit/modify mode. Once you have finished editing the layout, click the “Save” button to save the changes to the database.


![image](media://d7c3686f-03cd-41c2-b719-6a68a71fd13d)

## > Macro (anchor)

Deleting a layout.

The import layout can be deleted using the ‘bin icon’ available next to each import layout.

Click the ‘bin icon’ next to import layout will prompt the confirmation message in the dialog box (As below diagram)

![image](media://a0f824b3-e11c-44da-9d30-6b8eb2abca93)

After clicking “Yes” the import layout will be deleted from the list and from the database. A successful message will be displayed. If you select “No”, then the system will close the dialog box and the import layout will not be deleted.



![image](media://52fd0d4b-fc3f-4f94-9598-7224883e6487)

## > Macro (anchor)

Importing Assay Files

> Macro (anchor)

Assays can be imported under the import menu item. First you need to configure your assay layout then you import the assay file and fix any potential errors with the loading.

### > Macro (anchor)

Assay Import Process


When DataShed initiates the assay import process, the following process occurs:


1. **Data Validation:** Assays are loaded into an incoming buffer table, enabling checks to ensure data integrity. DataShed5 scrutinizes the lab method, element, unit, and lab libraries to validate data against these standards. The Generic Method is updated based on the Original Lab Method, and the Element and Repeat fields are populated from the Lab Element library.
2. **Sample ID Matching:** Incoming Sample IDs are compared with those in each Sample table.
3. **Result Calculation:** DataShed5 calculates the sysResult from the Assay Result. Text fields are converted to corresponding substitution values, and the text equivalent is entered into the sysAssayStatus field.
4. **Automatic Record Merging:** The system automatically merges records that have successfully passed validation. The assays are subsequently ranked post the merge process.
5. **Handling Unsuccessful Records:** To review unsuccessful records, navigate to the Assay Manager. Unmerged data is listed in the Unmerged Assay Results section. Upon resolving the unmerged data, click the Process Batch button. DataShed5 conducts checks and attempts to merge the data seamlessly.

data.  

### > Macro (anchor)

Import Layout Configurations

 To configure your Assay Import Layout, first you need to click on the ‘Import Data’ menu item and then ‘Import Layout Configuration’ button in the top right corner. 

![image](media://d17cb6c8-4079-4054-9812-3d7f214644c4)

The import layouts page shows the existing import layouts on the left, you can search, delete (Bin Icon) and edit existing import layouts. A new layout can be added by clicking “Add New Layout” button at the top left of the screen. 

When the “Add New Layout” button is clicked, the Import Layout page is opened for the user to enter the details of the Import Layout. During this process the existing Import Layout list is disabled until the new layout page is saved or closed.


![image](media://4bb35463-2b03-400e-8082-b29cb21e1243)


The user can choose or change the Layout Name, Description, Laboratory, Post Import Action, Process Scheme and Normalised checkbox. The only required field is the Layout Name. 

![image](media://0b9ad210-e95e-4239-a248-54b5b4002954)




Once you have entered the details of the Import Layout, click ‘Save’ button to save the import layout or ‘Close’ to cancel the creation. Once the new layout is saved, a successful message is displayed at the top right, and the created layout should be selected by default.

![image](media://03758caa-4403-4743-88d4-8fe369c5865d)

### > Macro (anchor)

> Macro (anchor)

> Macro (anchor)

> Macro (anchor)

> Macro (anchor)

Select Import File Type

Next you will need to select the ‘Import File Type’ from the dropdown list. For example, “SGS CSV Format” is selected for 'Import File Type’. The selected file type is saved to the database on selection. The save successful message is displayed on selection.

![image](media://92f626fb-27a6-4b3e-9675-eabf04aeb3b9)

### > Macro (anchor)

Import File List

The ‘Import file type list’ can be managed by click on the ‘Import file type list’ icon. When you hover the mouse over on the icon, the “Import file type list” tooltip is displayed. (As below screen grab)

![image](media://2ff57344-cf4b-4a36-b3a0-d97d6d33e0c7)

Clicking “Import file type list” opens the file type list pop up which allows you to:

- Add a new file type using the + Icon and updating the information for the new file type. File Type and Description Fields are required.
- Edit existing file types by either editing directly into the cell or using the provided drop downs.
- Delete/bulk delete file types by either selecting one or multiple rows using the checkboxes on the lefthand side and selecting the ‘Delete’ trach icon at the top left.

**Please Note:** You are required to click the “Save” button to save the details to the database. Closing the popup without saving may lose changes you have made.

**Please Note:** You cannot have a duplicate File Name or Description; a warning will appear if you attempt to save a supplicate record. 


![image](media://8e9fec1f-c5b1-4483-a242-fcce728739fc)

Once you have successfully updated the Import File Types and click save you will receive a successful message as per below. 

![image](media://81b36e78-ab72-4476-a93a-b1464985583e)

### > Macro (anchor)

Assay File Format Editor

Next you will need to select the 'Assay file format editor’ icon.

![image](media://8e81d5e6-47df-4eb0-9fb7-899636cf395a)

Once you have clicked the ‘File icon’ (Assay file format editor), the Assay File Format Editor page will be opened in the same page.

The Assay File Format Editor displays the selected file type description as Scheme Name. 

You can now upload the **.csv** and **.sif** file types by clicking “Select File” button. 

**Please Note:** DataShed5 only supports these two file types.


![image](media://148ca7a9-eabc-4ae1-bfcb-e4ed6c6b26c6)

Once you have selected your file, The Layout Definition panel can be seen under the file editor panel. The layout definition can be defined for each file type. 

In the Layout Definition go to the Data Item drop down list and select a Data Item.

For each comma delimited field, click the mouse cursor at the start of each comma delimited section. 

![image](media://6b7a1401-3730-4435-87b8-fc223256c2e3)

When selecting another Data Item, the value for previous selected Data Item will be saved automatically. Once the layout definition has saved successfully a message is displayed.


![image](media://b508831e-c8e0-402f-bfc6-83a808af6511)

### > Macro (anchor)

Import Substitution

To manage substitutions, click the “Substitution” button. The import substitution list popup window will be open. From your you can add, edit or delete substitutions. 

Click + icon button to add a new substitution. The substitution set is pre-populated with the selected file type. (For example, SGS_CSV)

The Symbol and AssayStatus fields are required fields. So, leaving these fields empty, the following error message will be prompted.

![image](media://01899743-229b-4e57-b54f-ea29f978cfe1)

The Symbol and other fields are free text fields, but AssayStatus value can be selected from the dropdown list.

Please Note: You cannot duplicate the substitution set and symbol.

![image](media://7e66c085-af34-49f8-99a4-e3698251b082)


Once the new record is added to the list, make sure you click the “Save” button to ensure that the data is saved to the database. 

**Please Note:** Clicking the “Close” button may result in losing the pending changes so the system will always show the confirmation dialog for unsaved changes if there is any pending changes.

![image](media://65ec0f19-1ac1-4b31-bc6b-4c2266a5edff)

Clicking “Save” button will save the new substitution record to the database and display the saved successful message.

![image](media://33f283ab-a64e-4d01-b64c-7bc46795a08f)

Editing existing substitution records is same as adding a new substitution records. Go to the field that needs to be edited, update the value. Then click the “Save” button to ensure that changes are saved to the database.

To delete a substitution you can either single delete and bulk delete by selecting the substitution record(s) and clicking the ‘Bin icon’. The selected substitution record(s) will be deleted from the list.

![image](media://fb9ce741-094f-4d4d-b43a-503d38ff1a8d)

Then click the “Save” button to ensure that the records are removed from the database.

![image](media://6e54d6b6-8c88-4d60-914e-77daf9178af4)

### > Macro (anchor)

Import Validation

Once you have finished updating your layout definition and substitutions you can validate the layout by clicking ‘Validate’ button. The system will go through the saved data items for the selected file and display the data items in the result pop up (As below diagram).

The progress bar shows the validation progress. The “Copy to clipboard” button allows you to copy the result to clipboard.



![image](media://6d3f8eb3-c632-4d51-8071-fd1d7f42f685)

### > Macro (anchor)

Buffer Modifications

The Buffer Modifications panel open when the you click the ‘Buffer modifications’ button (The checklist icon). When you mouse over on the checklist icon button, you will notice a “Buffer modifications” tooltip is displayed. (As below diagram)

![image](media://5d377063-171a-428f-b006-6e1c6b8c5d82)

The buffer modifications section has a Destination Table Name dropdown. Here you can choose which table you would like the file to be imported to. 

**Please Note:** Once you have selected your destination table, please click save to save the selection to the database.

![image](media://1efc4af2-e309-448e-ba3f-07e5b0253bd6)

Click plus icon button to add a new field modification. The FieldDB column and Action columns have dropdown lookup lists to select from which is based on the selected import file type.

**Please Note**: there is no validation check for any of the fields.


![image](media://5a2daadf-3931-4b00-af29-55445dc330a6)

Once you have added new buffer fields modifications you need to click “Save” to ensure that the record(s) are saved to the database. A saved successfully message is displayed on clicking “Save” button.

![image](media://72960446-098b-40f5-be5d-4b4e6f2f0fe3)

To edit a modification, simply select a field, edit it as required and click ‘Save’ to save your changes.

To delete a modification simply select one or more records and click the ‘Bin Icon’. These records will be removed from the list. Once again, click 'Save’ to save you changes. 

![image](media://aa2f5398-99d8-4e6e-83f2-7f980b763f62)

### > Macro (anchor)

Check Config:

Next, you will need to check your configuration. Once you click the ‘Check config button’ (The check mark icon) The Check Config panel will open. 

![image](media://05962d18-3786-4680-9203-de52766d3f68)

The ‘Check Config’ section has two boxes, one is for the config description, and one is for any errors which may be found.

There are three main buttons ‘Check Config’, ‘Check Config for All’, ‘Save to File’.

- The 'Check Config’ button checks the configuration for selected import layout.
- The ‘Check Config for All’ button checks the configuration for all import layouts in the system.
- The ‘Save to File’ button export and download the diagnostic result in .dsd file.

Once you select either the 'Check Config’ or the ‘Check Config for All’ buttons, you will notice the Check config boxes will be populated with successful checks and errors as per below. 

![image](media://0208f8d0-ed61-4a08-84e1-7eb7797f5fca)

Clicking the “Save to File” button will export and download the **DataShed Diagnostics File.dsd** file. The file can be opened in notepad.

### > Macro (anchor)

Assay File Import

Once you have finished configuring your assay file, you can now proceed to import the file. First, select the import data menu item and select ‘Assay File’ from the ‘Import Type’ drop down and click the ‘Load’ button. This will open the > Macro (anchor)

Import Data – Assay File window.

![image](media://f6511ddf-64a0-4c26-8882-d6db128bb699)

From the** **Import Data – Assay File window you can select the import layout you wish to use. 

Next, click the ‘Select Assay File” Button and choose the file from your local machine.

![image](media://41bec9a2-e768-46ed-ad1a-8038707bd42c)


Next, click the ‘Load’ button. The system will attempt to load the file and display the results in the Description and Error fields. If any errors are generated, these will need to be fixed prior to attempting to load the file again.

![image](media://7e431784-9a15-4206-8f01-2b4c7788e12e)

Once the processing is completed successfully, it prompts the successful message in the top right corner in green and displays no error messages. 

![image](media://0beec8b5-8343-4c25-80cc-9e0571848c19)

You are then able to select the ‘Go To Assay Manager’ button which will take you directly to the Assay Manager Dashboard to review your assay records and information. For more information on this, please go to the Assay Manager section of this User Guide.