Website Quality Evaluation Methodology Universal Star : 1 st point – “ Content ”

The Internet continues to grow at a fast pace with over 1.5 billion websites in 2019 as compared with only one in 1991. The emergence of enormous websites of various complexities and types makes assessing the quality of these sites a vastly important, difficult and complicated task. With this concern, the current paper proposes a novel approach for website assessment by developing a new Website Quality Evaluation Methodology Universal Star (WQEMUS) with a theoretical and empirical basis. It became possible through the employment of the grounded theory methodology that enables relevant concepts to emerge from data. To improve the reliability and validity of the findings, an extensive literature review, in-depth and qualitative interviews, and a user evaluation survey were conducted and associated together. In this way, the study presents the results of the selection and categorization of generic quality attributes for WQEMUS with a three-tier structure, consisting of top-level quality criteria, sub-criteria and indicators. These quality dimensions are grounded on a combination of subjective and objective indicators. Consequently, WQEMUS becomes capable of estimating a wide range of different websites irrespective of domain affiliation and services they provide, including Web 3.0 sites.

Introduction. Any serious activity or business need a web presence to compete in today's fast growing digital age and a good website is the best tool to achieve assigned goals with a greater speed and ease. Every day, the amount of governmental and non-governmental organizations, companies and individuals, who are creating their branded websites, is growing up. There are over 1,5 billion websites on the world wide web by September 2019 as compared with 1 in 1991 [1], but only 56,45 % [2] are active and succeed to satisfy their users' requirements and needs. The main causes affecting the current unfavorable situation are the limited background and knowledge of designers, the blunt tolerance of browsers to endure even incorrect code [3] and the evolution of information and communications technology (ICT) which sets new and severe conditions for website development. Moreover, available websites in various domains are not only document-oriented, but also applicationoriented and user-driven. As a consequence, they are complex systems [4,5] with different types, very dynamic and application-and user-oriented. Therefore, today more than ever, high-quality sites are especially important.
Website quality is recognized as a key factor affecting the improvement or deterioration of website reputation, the increase or decrease of the number of online users and successful or unsuccessful customer retention [6][7][8][9][10]. A high quality makes websites relevant, quickly accessible, easily usable, beneficial and successful. The site should ensure accurate and credible information, attractive design and exterior view [11] to meet the continuing needs and assumptions of its users.
Typically, the website assessment process identifies existing problems and levels of website feasibility, as well as indicates problematic areas of the website that the developer should pay special attention. Having a proper website quality evaluation method (WQEM) with the possibility of its maximum usage at all stages of site construction can be extremely useful because such WQEMs help to identify existing problems and set tasks for a successful development of sites. In the development process, WQEMs enable developers to be on the alert and immediately respond to arisen problems. Next, after the website development process, WQEMs can estimate and benchmark the quality of the entire website. As for the process of website operation, WQEMs might be applied as a compass to indicate correct directions for the improvement of sites. For these various reasons, WQEMs assist web developers to remain concentrated on critical aspects to maintain the high quality of their sites.
Background information. Due to the proliferation of a huge number of sites of various complexities, which involve various systems and subsystems, applications, web technologies, languages and databases, website quality assessment remains a vibrant area of research and will require numerous experimental and analytical advances using well-designed WQEMs. Over many years a great number of various website assessment practices, schemes, models, frameworks, techniques, methods, metrics and methodologies were developed. According to our literature review, a number of metrics were built to measure specific sets of website quality features since the 1990s [12,13]. Many experts have offered numerous innovative approaches called WQEM, which is described by ISO/IEC25000-2005IEC as "defined set of characteristics and of relationships between them, which provides a framework for specifying quality requirements and evaluating quality". Some of them investigate websites based on subjective forms of the appraiser's personal privileges, while the others perform in the form of objective and statistical measurements. Consequently, they approach problems from different angles. During the 1950s, 1960s and 1970s, after many efforts, "user satisfaction" was introduced to WQEMs as an important aspect of software quality [14]. The earliest WQEMs checked picture size, background colors, audio files and content. These were substantial measurable attributes of software quality [15].
However, there is vagueness in the choice of WQEMs and no generally accepted agreement about the correctness of their identification, structure and classification. Some WQEMs are mainly geared towards estimating specific web services or types of websites such as electronic commerce, academic, tourist, cultural, enterprise environment, news and etc., while others check certain web quality parameters including content or design (e.g. [9,[16][17][18]). The majority of WQEMs are focused on ISO quality standards and particularly, on using it [19][20][21][22] as the root of their methods. As for the WQEMs that are designed for the assessment of all types of websites, they are either dealing with a limited number of quality attributes or outdated because of the rapid development of web technologies.
With these considerations in mind, the purpose of this paper is to propose a new theoretically and empirically based WQEM that can provide all-important assessment characteristics and encourage website quality improvement. Furthermore, newly introduced WQEM should be capable to reliably estimate a wide range of different sites regardless of their domain types. Thus, a comprehensive selection of relevant measurable quality features for all types of sites, a detailed definition of their importance and relationships, and properly placing them in a hierarchical structure are considered as strategic tasks for the engineering of a new WQEM.
Our critical literature review of existing relevant works in the area did not provide sufficient information about new breakthroughs in the development of holistic WQEMs over last decades. In this relation, six relatively exhaustive assessment methods with their standard characteristics will be considered in detail.
In 1992, the International Organization for Standardization (ISO) in cooperation with the International Electrotechnical Commission introduced a hierarchical model "ISO 9126 -Information Technology -Software Product Evaluation -Quality characteristics and guidelines". It consists of six major quality characteristics, each of which is very broad in nature. They are functionality, usability, efficiency, reliability, maintainability and portability. Further, these characteristics are refined into twenty-one internal quality sub-characteristics and twenty-seven external quality sub-characteristics. The main peculiarity of this quality model is that it can be applied to any kind of software products. Also, it is the most recognized and the fundamental model for many WQEMs, e.g. those created by [17,[23][24][25][26][27][28]. Then, it has been replaced by more extensive series of standards -ISO/IEC 25010:2011 that is called SQuaRE (Software product Quality Requirements and Evaluation), as demonstrated in fig. 1. Mich et al. [29] have created an approach called "2QCV3Q" for designing and assessing sites, which is independent of website goals and domains. 2QCV3Q was recently renamed to 7Loci and takes its name from the initials of Ciceronian Loci. It was developed for marketing purposes and was applied in a number of evaluation projects such as education, business, customer service [29,30] and mainly in tourism sites [3]. Table 1 summarizes the proposed 2QCV3Q model. Table 1 2QCV3Q or the 7-loci quality meta-model by Mich et al. [29] As reported in the above table, 2QCV3Q is based on seven main dimensions corresponding to six loci, namely: Quis (Identity), Quid (Content), Cur (Services), Ubi (Location), Quando (Management), Quomodo (Usability), Quibus Auxiliis (Feasibility).
After reviewing previous studies about different WQEMs, we come to know that Hasan and Abuelrub [31] presented another theoretical, all-purpose and more comprehensive framework for testing the quality of any sites and thereby, promoting the improvement of site design. They suggested four general dimensions such as content, design, organizational quality and user-friendliness. Subsequently, the framework is composed of three hierarchical levels, as shown in fig. 2. Fig. 2. Hierarchical structure of the quality items of the framework proposed by Hasan and Abuelrub [31] As a further example, a scientific paper by Polillo [32] has introduced a practical quality model for evaluating any kind of sites, including Web 2.0 sites. Accordingly, this methodological approach is structured into a set of nine measures that are further decomposed into thirty sub-characteristics, as listed in fig. 3. It is specifically designed to identify the needs of end-users and works in all the phases of the website development life cycle. Fig. 3 displays Polillo's dimensions, appointed for his web evaluation framework. Fig. 3. Screenshot of the constructed quality model by Polillo [32] The sub-characteristics for the Web 2.0 sites are written in italics A holistic, scalable and easy to apply Web Q-Model for evaluating websites of different domains, which is provided by Cimino and Micali [3], encompasses six dimensions that include interface communication (IC), content (CO), navigation (NA), management and accessibility (MA), interactivity (IN) and accessibility for people with disabilities (AD). It is aimed at helping web designers and managers to develop accurate sites and categorizes the quality factors into three levels with associated symbols such as "Basic"-Q, "Normal"-QQ and "Exciting"-QQQ to differentiate the features on the basis of their importance.
Last but not least, a three-factor framework that placed emphasis on relatively recent quality issues was evolved to evaluate multidimensional websites [33]. Rocha believes that any assessment research, organized in accordance with these three main criteria (content, service and technical qualities) would assure cross-sectional and detailed information of the global quality of sites ( fig. 4). The features of this method can be measured using a 3 or 5-point Likert scale (1completely disagree; 2disagree; 3do not agree or disagree; 4agree; 5completely agree). Fig. 4. Three main dimensions for website quality assessment by Rocha [33] In conclusion, several other WQEMs were not considered because of their predestination for specific types of websites and/or dealing with a limited number of quality factors. However, they helped to increase our understanding and interpretation of quality attributes and their relationships to properly formulate basic and new quality requirements for a novel WQEM.
Qualitative and quantitative research methodology approaches. Grounded Theory Method (GTM) as a flexible, heuristic and systematic way was adopted to address the assigned purposes of this paper. Strauss and Corbin acknowledged the importance of the multiplicity of perspectives and truth in the evolution of grounded theory [34][35][36][37]. Therefore, in order to cover such a multiplicity and diversity of quality elements we used literature review, interviews and survey techniques as well as other sources of information, including field notes, letters and books to collect data within the confines of GTM. However, collected data should be weighed in the terms of relevancy, quality and quantity [38, p. 16].
Literature review. In our grounded theory-based research, conducting a review of literature prior to data collection and analysis has provided not a theoretical background, but rather analytic underpinnings and strategies for the research. Accordingly, in the last decades, numerous studies in the context of website quality assessments have established website quality as a complex construct [39], which can be evaluated by defining measurable factors [7,8,40,41] or pervasive sets of features [42]. A number of scientists emphasized website measurement as multidimensional in nature [39,[43][44][45][46][47][48] and combined various website quality dimensions into one evaluating framework [49,50]. Furthermore, it is difficult to determine website quality attributes since they are impacted by issues related to culture, participants and even time [51] as well as refer to different levels of user expectations 1 [52]. In order to provide users with a quality site, the developer should first understand various aspects of quality, which are influenced by user expectations and preferences and then, reflect them in the design, construction and operation of sites [53].
Hence, in order to develop a new WQEM, a multi-dimensional approach has been adopted. An indepth and extensive analysis of a broad range of previous related studies on key quality factors for websites, evaluation metrics and various assessment methods with their specific quality attributes was performed to identify all necessary sets of top-level quality criteria, sub-criteria and indicators for our new methodology. These quality characteristics can be selected in accordance with their semantic orthogonality, measurability, possibility of automated assessment, attitude towards the site development process, and/or use in statistical or probabilistic analysis and etc. [9,25,29,[54][55][56][57]. Moreover, quality characteristics should be based on both theoretical and empirical knowledge to be sure of their importance and coverage of different aspects of successful websites.
In accordance with our research aims, all four coding categories of GTM such as open, axial, selective and theoretical were employed in the data analysis. Accordingly, existing repetitive item names were eliminated. Furthermore, similar items were merged and a wide set of quality items with an equivalent semantic meaning were categorized under top-level quality items. We focused on the broad characteristics of research units and subsequently, seven groups with twenty quality attributes were picked out based on the results of the coding processes. These seven groups are content, presentation, organization, ease of use, responsibility, functionality and security characteristics. In sum, an initial version of the methodology in the form of a requirement tree and a fairly broad list of representative quality criteria and sub-criteria was generated by studying literature sources.
In-depth interviews. Within the framework of the constructivist GTM, an in-depth interview research was carried out after finishing the literature review that helped establish a preliminary WQEM, i.e. a series of questions from the quality characteristics. Mills et al. [58, p. 9] stated that interview processes give the researcher an opportunity to raise ideas and mutually construct knowledge through engaging in discussions.
Interviewing should be open-ended, conversational and mutually shaped. In this way, it can ensure the required depth, richness and rigor of scientific research. In-depth interviews enable the inventor to ask for more details and explanation, delve more deeply into thematic issues [38] and contrast most important points and key details. Thirty-six international contributors from in-depth face to face interviews gave their voice and thoughts in our research directions, as characterized in table 2. Table 2 Background characteristics of eligible in-depth interview respondents Table 2 presents the distribution of interviewed women and men of all ages, educational levels and specialties with average daily internet consumptions. As planned, we were able to have a diverse and international sample of respondents, including 31,6 % specialists and experts in the field of Information Technology.
Next, the final numbers and countries of origin of the respondents are summarized in fig. 5. The process of data analysis in this in-depth interview research involved open, axial, selective and theoretical coding. After constantly comparing the collected data, codes, categories and memos between themselves, modern concepts of quality attributes for a new framework were conceptualized. As a result, the number of the most salient categories of quality items was five and not seven. These five categories comprise content, design, usability, reliability and organization. Moreover, seven new quality sub-items were merged from our data analysis. They are multilingualism, performance, compatibility, advertisement, intelligibility, searchability and evolvability. Besides, new relevant names have emerged instead of the previous ones such as up-to-date content, importance, multimedia, accuracy, reference, authorship, aesthetics, color, consistency, security, logos and brands, URL and interactivity.
It must be stressed that these various interpretations of the quality elements' meanings by in-depth interview participants have provided a broader picture of the topic and new perspectives. Therefore, they were conceptualized harmoniously to generate a grounded integration of concepts into the assessment theory and were also taken into account in determining indicators that comprise the third level dimensions of new Website Quality Evaluation Methodology Universal Star (WQEMUS).
Qualitative interviews (Part 1 of the results). This qualitative interview research, played an essential role in the formation of WQEMUS' attributes. In particular, Part 1 of the study results stimulates the perfection of the first two levels of WQEMUS. It also provides an opportunity to more confidently conduct upcoming series of closely interrelated studies. As a result of the data analysis using GTM's open, axial and theoretical coding, new codes and concepts such as mapping, privacy and traffic were derived. The uniqueness of content, design and other visual components of the site was confirmed to be a useful sub-criterion for the design criterion of WQEMUS. Further, the names of up-to-date content and evolvability have been replaced by currency and maintainability, respectively to make them shorter, more descriptive and common. More than this, the name of the core category -organization was renamed to "Reputation", which is considered to be more relevant for WQEMUS.
User evaluation and judgment survey. Following the previous in-depth and qualitative interviews, the author used the questionnaire technique with 162 respondents that took part in group-administered questionnaire survey 1 (GAQS 1). Accordingly, 31 (16,06 %) out of 193 participants of GAQS 1 left the proposed questions unanswered or answered only partially. It happened because they might not have enough time to answer a few more questions. Another reason might be that some respondents were in a hurry because they had scheduled meetings after the end of courses. Correspondingly, a total of 162 questionnaire responses were analyzed and all answers were pooled to produce reliable results in this user assessment and judgments survey research. Surely, the majority of answers had affirmative nature. That is why the amount of information provided was not complex.
The interview questions were based on the results of the previous in-depth and qualitative interview studies. The next step to produce reliable results in this semi-structured user evaluation and judgment survey was to analyze the responses of 162 questionnaire participants. Basically, the results of all four previous studiesthe literature review, in-depth and qualitative interviews and the present study were compared and integrated together. In this survey, an optimal mix of various selection and elimination procedures for agreeing on categories, recommended by GTM such as open, axial and theoretical coding was necessary to characterize quality measures and obtain the final list of the criteria and subcriteria of WQEMUS.
In fact, three new quality concepts such as compliance with usability and accessibility standards and taking antispyware actions were revealed, which were not mentioned in our previous in-depth and qualitative interview studies. Also, a new name "Identification" was given instead the "Logos and brands" sub-criterion. Notably, we were very fortunate that a new sub-criterion -antispyware was proposed at the very beginning of the survey since because of it we had to rearrange the questionnaire structure to be able to further intensify and carry out GAQS 1. Essentially, all important changes are taken into account.
Findings about three levels of WQEMUS: quality criteria, sub-criteria and indicators. The application of the above-mentioned GTM for conducting the literature review, in-depth interview and user evaluation and judgment survey gave the ground to concentrate around five top-level quality criteria for WQEMUS, which are exhaustive and used almost in all previous quality testing methods and techniques. Now, WQEMUS is able to adequately assess all kinds of websites and fig. 6 below summarizes the findings about it visually: The most often met website assessment dimensions in the scientific literature are considered as Web content [23,33,39,[59][60][61][62][63][64][65][66][67][68][69][70][71], Web design [39,50,61,65,[71][72][73][74][75] and usability [19,[76][77][78][79][80][81]. As for the names of the last two top criteria of the novel methodology, they were then identified by the participants of our research studies and the author as "Reliability" and "Reputation", because they better describe their corresponding sub-criteria. Later on, the author came up with the name of new WQEM, which is Website Quality Evaluation Methodology Universal Star (WQEMUS) by taking into account the formed five top-level quality criteria, groups or directions. "Universal", because the new WQEM is applicable to all kind of websites. The word "Star" was used in the name, since the new WQEM has five directions or points (the top-level quality criteria) like the star has (See fig. 6 and 7). Generally, all the listed top-level quality criteria establish the scope of WQEMUS and reflect the conceptual and holistic description of site quality.
The top-level quality criteria of WQEMUS unite sub-criteria. In turn, the sub-criteria that contribute to the improvement of the quality of sites are dependent on their top criteria, but should be independently measured during an estimation process using WQEMUS. All of the quality dimensions together are also interconnected and form a general idea and logic of WQEMUS. The developed hierarchical structure of WQEMUS with its classified quality criteria and sub-criteria is shown in fig. 7. Consequently, a rather large set of important quality factors for all kind of websites have been established. Fig. 7. Final sets of sub-criteria grouped into the content criterion of proposed WQEMUS Top quality criterion "Content". This criterion has been recognized as the most critical aspect of the website [82,83], the king dimension of websites [84] or a fundamental quality attribute that indicates the information architecture of a site [85]. It is associated with a distinctive quality, property [33], advantage and/or the trustfulness of presented information [86] about companies/organizations, site owners, provided goods and services, activities and others under the responsibility of content editors. Many respondents of our conducted studies replied that the main reason they come to the website is due to content and its diversity. Moreover, without content, there would be no views of website pages. Most of the surveyed users were goal oriented and usually look only for specific types of information while paying less attention to the other aspects of the website such as navigation, visual design and interactivity and others. Content reflects the ability of the site to meet users' request, responsiveness and trustworthiness [87,88]. Furthermore, worthy content takes a stand and considers issues of interest from different perspectives. It can be informative, useful or funny, but it always should encourage users that want more. Therefore, relevance, high quality, entirety and a good degree of content specialization are actually important.

Sub-criteria of "Content"
Currency. Currency refers to how far the website's content is modern, how often it is updated and whether the date of the last update is specified [43,50,56,59,64,[89][90][91][92][93][94][95][96][97][98][99][100]. Our indepth and qualitative interviews revealed that old content is boring and unbearable. An interesting statement "Why should I read old information?" has been given by some respondents. The final name was assigned to this criterion as currency after the qualitative interviews. Furthermore, every activity or business tends to change over time with new improvements, ideas, products, services, staff, innovations, news items and more. These changes need to be reflected not only for information portals, but on every site by involving continuous modification, transformation, correction and improvement to reflect changes in the environment, business etc. [101,102]. A frequently updated website increases its visit rate and avoids outdated information [103] since outdated information appears to be the primary causative factor of website failures and business losses [104]. Up-to-date content on the Internet is likely to be more relevant and useful for visitors as well as for search engines that are always looking for contemporary information. Modern search engine mechanisms have been established in accordance with the terms of real-time with using "Fresh Factor" in order to index online content.
Importance. The importance of content is related to the appropriate range, depth and scope of information [105], which in turn, is also exhaustive, complete, thorough [30,41,50,53,59,64,66,89,94,106] useful, comprehensive and audience-oriented. Furthermore, it was hard to choose a suitable name for this sub-criterion because names were very different even in the literature. After analyzing the data from in-depth interviews, we came to the conclusion to give this criterion the name of importance. Then this name was confirmed in further studies. The respondents of the current research proved that irrelevant or incomplete information will likely not be read by users and moreover, cause negative feelings to the site if they read it. Thence, specific, relevant, up-to-date and detailed information should be provided.
Written content is necessarily considered to be informative, meaningful and value added in the conformity with its audience [98] and should serve the needs of diverse users [30,64,84,95,98,107,108]. In addition, web content expect to be sufficient, understandable and appropriate [94,109] to the needs and requirements of all targeted groups of users, e.g. local or foreign citizens, travelers, researchers, clients, patients, students and others [61,65,71]. The other important features are content's accurateness and suitability to fit the task at hand [110]. A content-rich website [111] causes more traffic of visitors and increases the site's visibility across search engines. On the other hand, a complete content with the expected level of detail ensures users to come to adequate and reasonable decision-making on products or services [103]. Finally, the nature and volume of website usage are both essential detectors of success [65].
Multilingualism. A multi-language website allows visitors to interact with more than a single or official state language [89,92,98,108,[112][113][114][115][116]. In this connection, the content of sites should be provided at least with a minimum required number of languages in order to be understandable to everyone or support the execution of specific tasks on sites. Our findings from semi-structured in-depth and qualitative interviews as well as user evaluation and judgement survey revealed that it is very difficult for foreigners, who arrived in another country, but do not know its official language, to read and understand the proposed content in the local language. On the other hand, if a site wants to compete at an international level, then at least some of the international languages such as Arabic, English, German, Russian and/or others must be used. Here, as an international language, site owners usually choose the language that is understandable across the whole continent, mainland or group of countries in which the site is focused. However, we also revealed that specific or detailed information should be represented in local languages.
A multilanguage website attracts additional customers to business, more participants to projects and new visitors to content. Moreover, Fitzpatrick [89] and Krauss [108] believe that a multi-language support is claimed as the appropriateness of sites to different cultural backgrounds. In other words, multilingualism also satisfies the needs of clients, irrespective of their country of origin [31,113,117]. Finally, in most cases, it is essential to address a large number of visitors and it will thereby increase the significance of websites [118].
Multimedia. Multimedia directly affects to the website's aspect "look and feel". It is associated not only with text, but also non-text elements that provide more engaging contextual value of the site's content and purposes [30,59,64,66,84,98,100,108,[119][120][121][122][123][124][125]. Furthermore, non-text formats are worth more than a thousand words as far as they are visual and very effective for perception. In addition, our in-depth and qualitative interviews and user evaluation survey state that many people have difficulties in reading a long text. Content in the form of only text is boring and requires visitors to spend a lot of time and effort to master it. Therefore, integrating non-text formats with text when creating content makes it more easily understandable for a majority of the population, including people who are uneducated or have limited literacy skills. Additionally, multimedia increases users' desire and motivates them to pay more attention to the written content.
A number of scientists suggest that site visitors should get information they have requested in most commonly known types or formats [84,91,98]. Alternative text as a description could be also associated with all non-text information [56,98,126]. However, the application of multimedia components has to be effective without significantly affecting loading time [56,117,127]. It means that the amount of dedicated non-text information should not be too large, but in a reasonable amount and format as well as their size must also be maximally compressed and reduced to a minimum, since large-size multimedia slows down a page's rendering. As a result, it causes a deeply negative impression on users [56,128].
Accuracy. Presented content for a site ought to be free from spelling and grammar errors, accurate, precise and reliable [29, 41, 50, 53, 59, 84, 89-91, 93-95, 98, 121]. Grammar, spelling and punctuation mistakes that may change the meaning of content should be fixed or removed [129]. More importantly, correctly represented content, in turn, should not mislead users [94,113] and white spaces should be used necessarily to avoid overcrowded pages [128]. Further, web content has to be accurately and clearly worded as words intensify meanings. Accurate content and data, which are written in a systematic order, strengthen the reliability of trust in the website's founders [52,117]. The author explored that the respondents of the in-depth and qualitative interviews have been supporting both sides of this issue. The majority of participants have argued that spelling and grammar errors show the lack of essence, responsibility and seriousness of given content and professionalism of authors. As a result, the credibility and trust of users to such Internet content will be diminished as well as the image of the organization will continue to worsen dramatically. However, the remaining few participants, on the contrary, claimed that the presence of spelling and grammar errors is not problematic as users can still read online content, understand its meaning and benefit from it.
References. This sub-criterion involves stating references used from other information sources [59,64,130], adding sources of statements such as citations and direct links to other related sites [84,98,124] and data repositories in text. Thereto, various forms for repeated links, text and images [117,131] as well as conventional means for highlighting links and underlining words/labels, can be used. Following these, indicating resources such as source materials, references and citations increases the trustworthiness of web content [8]. Another critical issue is "how many outbound links, external outlinks or external links should one page have?" which involved a lot of debate and discussions in the world, particularly in SEO & marketing departments of mid-size and larger organizations. Well, the answer is not that simple. Fig. 8 reflects possible and optimal numbers. Here, it is worthy to note that emphasis should be on quality and not on quantity, since it is impossible to beat out sites that have 10,000 links. As shown in fig. 8, red section in the middle is "interquartile range" that covers everything from the 25th percentile (called Q1) to the 75th percentile (called Q3). So, the total range for external outlinks per web page was from 0 to 254. The Q1-Q3 range goes from 5 to 42. The median, dividing the top 50 % and bottom 50 %, was 19 2 . Our findings from the conducted literature review and indepth interviews exhibit that indicating different sources of information and links to other sites is not so important, but useful and additional plus point, which as a consequence, reduces significant time and efforts required to find more information on a topic of current interest.
Authority. References used to confirm the identification of resources increase the level of users' confidence and credibility as well as the reliability of sources. This can be done by providing information about authors, groups and individuals who are responsible for the content as well as website sponsors [30,64,89,114,119] and website managers [59,64,89,119]. Moreover, details about the qualifications, expertise levels [59,89] and legitimacy of authors for writing certain topics and responsible staff for adding or updating content on particular pages of the site should be also indicated. Furthermore, the physical addresses [92,98,132] and official e-mails of authors, site managers and/or organizations have to be available on sites [64,92,119]. There is an emphasis on a fair use of copyright sign [64] that is located mostly in the header section and the persistence of metadata and its components in necessary web pages [89,97]. The author's first three types of research confirm the fact that the Internet offers a huge amount of information today, but the question of guaranteeing their authenticity is the most essential and still remains an ongoing and challenging issue. Thus, there is nothing worse than the site that has no contact information. Authority is critical, especially for business sites as long as this sub-criterion requires the provision of additional information and key links to real authors or responsible staff members. By doing so, the site strengthens the customers/stakeholders' confidence that they are dealing with a real business or serious organizers rather than scammers. In sum, worthy websites cannot be made without authority.
Measurable indicators for the sub-criteria of "Content". As mentioned in the previous chapters, the content criterion on the high-levels of WQEMUS is decomposed into the sub-criteria such as currency, importance, authority, multimedia, multilingualism, accuracy and references. In turn, the sub-criteria are split up into measurable indicators, as summarized in table 3. Table 3