Share this article
Latest news
With KB5043178 to Release Preview Channel, Microsoft advises Windows 11 users to plug in when the battery is low
Copilot in Outlook will generate personalized themes for you to customize the app
Microsoft will raise the price of its 365 Suite to include AI capabilities
Death Stranding Director’s Cut is now Xbox X|S at a huge discount
Outlook will let users create custom account icons so they can tell their accounts apart easier
LLMs need new prompting techniques to understand structured data
The large language models have trouble using data from tables
3 min. read
Published onMarch 11, 2024
published onMarch 11, 2024
Share this article
Read our disclosure page to find out how can you help Windows Report sustain the editorial teamRead more
Large Language Models (LLMs) are facing challenges when they have to deal with data from tables. Furthermore, it is unknown if they comprehend it or not. Thus,a research teamtries to verify the capability of AI to understand structured data by using a new benchmark system. Furthermore, they want to discover prompt techniques to improve the understanding of the LLMs.
Can LLMs be trained on structured data?
You can train LLMs on structured data. However, it is still being determined to which degree the LLM can learn and understand it. Thus, researchers are trying to figure out the best prompt techniques to teach LLMs how to handle data from tables and test which tables work best.
Researchers created a new benchmark called Structural Understanding Capabilities (SUC) to test the capability of LLMs to understand data from tables. Furthermore, to gather results, they used SUC and various prompting techniques.
To verify the efficiency of the structured data prompting techniques, researchers used the benchmark on both GPT-3.5 and GPT-4. Furthermore, they figured out that the results differ based on table format, content order, and partition marks. In addition, they used HTML tables, comma-separated values, and tab-separated values for the table format.
Results
HTML tables are the most efficient for the LLMs between all of the formats. Also, according to their research, the highest accuracy across seven tasks is only 65.43%. As a result, LLMs are far from perfect when it comes to structured data, and they need much more improvement. However, it is possible to enhance them with the right prompting techniques.
Researchers used a combination of self-augmented prompting with structured data to improve the LLM’s understanding of tables. Furthermore, they did this in three steps. The first step was to ask the AI to analyze a table, the second step was for the AI to generate a description based on the data, and the last step was to create a description using the previous information.
In a nutshell, LLMs are not yet ready to properly understand structured data. However, according to the research, you can train LLMs to understand data from tables using various input factors and self-augmented prompting. Furthermore, researchers will continue to study efficient ways to make LLMs understand tables better by integrating structural information to improve the LLMs’ performance with different structured data types.
This researchcould also helpvisual language models that use LLMsto improve prompt learning.
What are your thoughts? Is this research going to help LLMs? Let us know in the comments.
More about the topics:AI,artificial intelligence
Sebastian Filipoiu
Sebastian is a content writer with a desire to learn everything new about AI and gaming. So, he spends his time writing prompts on various LLMs to understand them better. Additionally, Sebastian has experience fixing performance-related problems in video games and knows his way around Windows. Also, he is interested in anything related to quantum technology and becomes a research freak when he wants to learn more.
User forum
0 messages
Sort by:LatestOldestMost Votes
Comment*
Name*
Email*
Commenting as.Not you?
Save information for future comments
Comment
Δ
Sebastian Filipoiu
Sebastian is a content writer with a desire to learn everything new about AI and gaming.
