Relevancy of a word to a Human Subject Field

    Every word we understand has a meaning to us in one or many subjects. The relevancy of a word to a subject may differ from person to person, according to their subject interest. But we are going to describe this for all the people generally.
This is basically about mapping the relationship between…

                                                Word ==> Subject

    As there are many subjects we are interested on we have to represent this in 1:N One to many database. But for the efficiency we have a second thought to use a flat file where there is a matrix.

    Why we are going to use a matrix is that we are going to map the Relevancy of a Word to a Subject. Actually the arrow in the above diagram will contain a numerical value for that. We put a Weight for the relevancy.

    For an example the word “Car” will be 100% relevant to the subject “Automobile” and 0% relevant to “Art” or “Education” and the Word “Pizza” will be 100% relevant for the subject “Food” and 0% for “Science”

N.B. This document is a prerequisite to go to the level 3

How are we going to do this

We have to go in step-by-step approach. Then we have to plan levels to complete one by one. After an each level we will be able to deliver a version of software that is usable. At the end we are going to make the software to learn itself new words and the relevancy to human subject fields, and able to categorize sentences and paragraphs of information to categories.

In the first level we are making something little more than a web browser. There will be a text field to enter keywords and a search button instead of the address bar. When the user enters keywords it will send search requests to Google or Yahoo or both. Then it will filter out the search results links on a page.

In the second level we are going to develop this to visit the resulted pages simultaneously and filter text information on the pages. It will separate the paragraphs and sentences and assign them to arrays to process further.

Before going to further levels we need to make another expert system, for the computer to understand the relevancy of a word to a specific Human Subject Field. We will discuss about the functionality of this module in MRM Dictionary document. For now, consider it as a “closed module that will work”. When we query for a word it will return an array of values telling what is that word mean to a human being. Read Relevancy of a Word to a Human Subject Field Document for further information.

By using the MRM dictionary we are going to make the third level version of the software; that will categorize the search results to subjects. It will query for each word in a sentence for the relevancy to subjects. The result will be a table of subjects and the percentage relevancy. We have to make the average and sort the subjects that the sentence is appropriate.

What are we going to do

    We are going to build software to refine text information found in the Internet for the user’s exact requirements. The software will produce a generated document with the required information refined from the Internet.

    When a user enters keywords and search, it will search for those keywords in existing search engines available on the Internet simultaneously. As the results are returned it will again request for the links in the search results. From those result pages, the software will filter out the human readable text. From those texts it will then categorize each sentence according to the relevance to the required subject field, to the interest of the user.

Introduction

    This is the official blog of A Multidimensional Relevance Matrix Method to Refine Web Content, a third-year student research project at the Sri Lanka Institute of IT. All rights reserved.