DESIGN OF A WEB-BASED PAPER SUBMISSION AND REVIEWING SYSTEM (PSRS)

Conference planning, organization and administration are very tedious tasks. In most cases the conference programme committee has to convene several meetings where submitted papers (via emails in most cases) are downloaded, discussed and accepted or rejected for presentation at the conference. This paper presents the design of a web-based conference paper management system which facilitates easy and efficient review of technical submissions to conferences. Our proposed system stores authors’ information, abstracts, papers and reviewers’ comments. The process of assignment of papers to reviewers is done using a set of objective parameters to determine the most suitable reviewers for each article. The system also collates camera ready accepted papers to generate conference proceeding for the conference. This work will reduce the amount of paperwork and the need for several meetings by the programme committee thus making conference organization a pleasure. Also the effectiveness of conference organization and management will be substantially improved.


INTRODUCTION
Before the popularity of the Internet, the organization of conferences was mainly based on conventional paper mail by post. Authors of academic papers would send their abstracts to a conference secretary, who would duplicate and forward them to reviewers. Once the reviewers returned their comments also by post, then the final notifications would then be sent in the same way. The whole procedure took a relatively very long time and data had to be retyped again and again. This indeed was very inefficient and ineffective.
The advent of fax-machines increased the speed of communication in most developed countries. Nevertheless, this caused additional work in the duplication and distribution of information. The successful invention of Internet turned around the mode of communication in the academic community. Although the use of attachments to an email message is still used to transfer bits and bytes of data, several web-based interfaces for different kinds of collaboration have been developed with there different functionalities. In this paper we have designed a dynamic web-based paper submission and review system (PSRS) to enhance the conference planning and organizations.
In section 2 we give an overview of existing webbased paper submission and review systems, section 3 is a detailed description of the architecture of our PSRS. In section 4 we discuss the intended mode of its implementation and the conclusion is given in section 5.

2.O VERVIEW AND FEATURES OF SOME EXISTING WEB-BASED PEER REVIEW SYSTEMS
There are quite a lot of web-based systems for online management of peer review processes for scholarly journals and conference proceedings. They have specialized features which vary widely, but the more highly developed programs share many characteristics in common. Some of the popular ones include: AllenTack, Bench>Press, Editkit, ESPERE, CyberChair. Others include Manuscript Central, available at http://www.scholarone.com/products_manuscriptcentral.html, Rapid Review available at: http://www.rapidreview.com/, Sciforum.net available at: http://sciforum.ouc.edu.cn/sciforum/, GNU Eprints available at: http://software.eprints.org, and others as described by (Ann Rheum Dis 2002, Karim Valji 2003, Maryam R. Mohassel 2004, Mark Ware 2005, Irwin 2005, Irwin 2006, Nicola Di Mauro et al 2005, Borron 2006,) Brief descriptions of some of these tools are given as follows: • AllenTrack™: AllenTrack™ available at http://www.allentrack.net/ was developed to facilitate online manuscript submission, review, and associated correspondence. It is designed to facilitate all essential editorial office functions from data entry, data retrieval, correspondence, and reporting, to workflow control, manuscript file management, and database access. AllenTrack™ is available from any computer, any operating system, any platform, anywhere in the world with an Internet connection (Mike Fitzpatrick, 2006). AllenTrack™ is an implementation of EJPress®, a suite of software tools developed by eJournalPress.com to support journal publishing.
• Bench>Press™: Bench>Press™ available at http://benchpress.highwire.org/ is a complete manuscript submission, tracking, review, and publishing system developed by Stanford University Libraries' HighWire Press®. The Bench>Press™ system was designed by a developer with significant real-world experience in manuscript management for a scientific journal. Bench>Press™ is an Internet application and operates with standard browsers, although a Java Script-enabled browser is recommended. Adobe ® Acrobat® Reader® is also required.
• EdiKit SM: EdiKitSM available at http://www.bepress.com/ is an innovative Web-based system used to manage an article and its progress from submission to publication. EdiKitSM automates • every procedural step in producing scholarly journals, from submission, review, and final publication. Effective use of EdiKitSM eliminates delays inherent in such tasks as mailing manuscripts, reviews, managing correspondence among authors, reviewers and editors, and bundling articles into an issue. As EdiKitSM is Web-based, editors can work from any computer connected to the Internet without loading programs onto their hard disk. EdiKitSM simplifies important functions such as setting a publications layout, tracking correspondence, and publishing electronic content. EdiKitSM is also used to manage the editorial procedures associated with the production of conference proceedings, newsletters, working papers, as well as an electronic version of previously published print journals. CyberChair available at http://cyberchair.cs.utwente.nl is a web-based groupware application that supports the review process of technical contributions to conferences (R. Van de stadt. 1998). It facilitates abstract, full paper and camera-ready article submissions, paper assignment to reviewers by the programme chair after an initial bidding process in which each reviewer indicates papers of interest after the collation of abstract submissions. It also helps to identify conflicting reviews and offer means for the reviewers concerned to communicate (see each other's reviews) and resolve conflicts, it offers support to the programme committee in the selection of best papers. This is done by encouraging reviewers to champion the inclusion of reviewed papers they consider most appropriate for inclusion during distributed PC meetings.
There are many more existing web-based systems apart from the ones that have been mentioned so far, although many of them have not been exclusively reported in literature. A summary of existing web-based conference management systems was reported by (Snodgrass et al 1998). Generally, the existing web-based peer review systems can be summarized as having the following features and functionalities: 1.
Automated submission of papers (web upload of abstract, full article and camera-ready papers); 2.
Automated notifications (notifications of acceptance and rejection); 3.
Article assignment to reviewers; 4.
Time reminders or enforcement; 7.
Reviewer comment threads (reviewers seeing each other reviews);
One of our contributions in this work is to introduce a provision for the tracking of revisions to submitted articles, making it easier for journal reviewers and editors to see what modifications have been made from the final "proofs". This feature appears to be absent in most software tools. In this paper, we show the design of a system that takes this into account. Secondly, we have evolved a suitable model for the assignment of conference/journal articles to reviewers based on expertise ranking using a set of objective parameters. This is to reduce the level of subjectivity involved in the paper assignment process, especially when it is done discretionally by the managing editor. This enhances accuracy, by ensuring that particular papers are assigned to the most qualified core expert reviewers available. This is achieved by matching the subject classification of each paper to the expertise classification of reviewers using the ACM classification template available at http://www.acm.org/class/1998.

DESCRIPTION OF THE PSRS ARCHITECTURE
The architecture of our web-based PSRS is shown in figure 1. It consists of the following: i) Reviewer's Information Interface: This interface presents a web form to capture relevant information from the reviewer that gives insight into the reviewer's expertise. This includes such information as: areas of research interest, publication experiences, journal editorship, programme committee membership etc. The interface also provides for the uploading of the resume of each reviewer. The information supplied by reviewers is used to determine the suitability of a reviewer, which is determined based on a set minimum benchmark. ii) Reviewer's Information Analysis: This component of the PSRS architecture implements the evaluation of reviewers' profiles with the purpose of ranking reviewers as can be deduced from the information provided through the reviewers information interface. In order to do this, it uses a set of objective parameters which are given as follows:

DESIGN OF A WEB-BASED PAPER SUBMISSION AND REVIEWING SYSTEM (PSRS)
assess the performance of reviewers from time to time in order to determine their relative relevance to the peer review If reviewer is member of a PC in a related conference then w4 = 3.0 process. For example it is possible to have a reviewer with high expertise ranking but very low relevance when parameters like availability and promptness of reviews are considered. Therefore, the result of the performance evaluation can be used to alter the order of the reviewers' queue in the overall interest of the peer review process.
If reviewer is a journal editor or associate editor of a journal in related field then w5 = 3.0 A reviewer's rating can thus be evaluated using the function: The function RE T (i) denoting a reviewer's rating is used to rank reviewers and organize the list of reviewers into a priority queue with the highest rated reviewer on top of the queue. Therefore, a higher ranked reviewer is given first priority during the assigning of articles to reviewers. A prospective reviewer must have obtained a minimum RE T (i) score of 15 points, which is directly equivalent to having a minimum of three journal or book publications in the particular subject area as a lead author to be considered qualified as a reviewer. The assignment procedure is such that a higher-ranked reviewer has the maximum number of articles per reviewer satisfied before a lower-ranked reviewer is considered. This ensures that the most qualified reviewers are first considered, thereby minimizing incidences of having non-core experts reviewing articles. All of these assignments are done strictly from the objective point of view.

RE T (i) = ∑
iii) Reviewers' Queue: This is a database of reviewers' information. It is indexed according to ACM subject classification template available at http://www.acm.org/class/1998 . Individual reviewer's record contains expertise ranking scores i.e. RE T (i) through which a sorted list of reviewers based on relative ranking in a particular subject field classification can be obtained.

v) Administrative task component:
The administrative tasks associated with the peer review process are shown in figure 1 enclosed in dotted box. The first task is the paper submission which is handled by an abstract/paper submission interface where authors upload their papers on the web and a serial number and paper-id is automatically generated for each paper submission. The next stage is the assignment of papers to reviewers. Once a paper has been submitted, the paper is mapped to a particular ACM subject classification and assigned to the highest ranked available reviewer in the particular subject field on the reviewers' queue. Our design allows a maximum number of two reviewers per paper and a maximum of two papers per reviewer. Thereafter, the reviewers' comments are collated after they have been posted by the reviewers to the managing editor. In cases where the opinion of the two reviewers about an article differs (1-accept, 1-reject). The managing editor intervenes by re-assigning the paper to another expert reviewer in the same subject field in order to obtain an independent third opinion on the paper before taking a final decision. The collation of reviewers' comments is followed by the issuing of letters of notification to authors concerning the status of paper submissions which is either 'accept' or 'reject'. In either case, the reviewers' comments are also sent to the authors together with the instructions for the production of camera-ready final submissions in the case of accepted papers. The archive of original submissions and reviewer's suggested corrections are kept by the PSRS system and is used to validate the correctness of camera-ready submissions by authors before the process of final collation of all camera-ready papers for the production of the conference proceedings. The tracking of revisions ensures that authors' final submissions adhere to reviewer's recommendations. iv) Performance Tracking Component: This contains the performance records of reviewers based on parameters such as: promptness and punctuality of reviews, availability, utility, and other sundry contributions. These parameters are used to

MODE OF IMPLEMENTATION
The PSRS will be implemented using PHP, HTML scripting languages, IIS web server and MySQL database. The web interface for paper submissions and capturing of reviewer's information will be designed and implemented with HTML, complemented with PHP scripts to provide the necessary server-side functionalities for post and request processing. The reviewers web pages from where reviewers can download papers that have been assigned to them will also be created using HTML. The reviewer's information analysis function will be implemented as a COM (Component Object Model) component that encapsulates the implementation of the assignment of papers to reviewers based on expertise ranking on the reviewers' queue. The methods of the COM object interface will be invoked as a server side commands using PHP scripts. Mail service functionalities will also be provided to facilitate sending and receiving of mail request between reviewers and the managing editor. Reviewers will be able to send their comments and verdicts on reviewed papers for review through mails, letters of notifications to authors on the status (accept or reject) of their paper will also be sent through the mail server. The status is respectively associated with the MailAccept.tpl and the MailReject.tpl templates, while camera-ready versions of manuscripts are required for accepted papers. For example an instance of a default mail template for accepted papers is shown as follows:

CONCLUSION
The PSRS when implemented will greatly reduce the drudgery associated with the peer review process of conferences and journals articles, especially in settings where the operations and coordination of the peer review process is still manual. Also the crucial task of paper assignment to reviewers which is largely dependent on the subjective judgement of the managing editor (journal) or the programme committee chair (conference) in most cases can be executed objectively with minimum prejudice by generating expertise ranking scores for each reviewer. This also minimizes instances of allocation of papers to non-core expert reviewers in particular subject fields during the review process.
Secondly, the tracking of revisions made after acceptance ensures compliance of authors with reviewers' recommendation which leads to the production of quality conference proceedings.
Also, the provision of performance tracking mechanism to monitor the performance records of reviewers over time using important character attributes in way to building a reliable reviewers' queue is also a boost for the PSRS and a rare feature in many of the existing web-based paper submission and reviewing systems.