DESIGNING TRANSLATION TOOL: BETWEEN SIGN LANGUAGE TO SPOKEN TEXT ON KINECT TIME SERIES DATA USING DYNAMIC TIME WARPING
DOI:
https://doi.org/10.22441/sinergi.2018.2.004Keywords:
Human-Computer Interaction, Natural User Interface, Speech Recognition, 3D Animation, Sign LanguageAbstract
The gesture is one of the most natural and expressive methods for the hearing impaired. Most researchers, however, focus on either static gestures, postures or a small group of dynamic gestures due to the complexity of dynamic gestures. We propose the Kinect Translation Tool to recognize the user's gesture. As a result, the Kinect Translation Tool can be used for bilateral communication with the deaf community. Since real-time detection of a large number of dynamic gestures is taken into account, some efficient algorithms and models are required. The dynamic time warping algorithm is used here to detect and translate the gesture. Kinect Sign Language should translate sign language into written and spoken words. Conversely, people can reply directly with their spoken word, which is converted into literal text together with the animated 3D sign language gestures. The user study, which included several prototypes of the user interface, was carried out with the observation of ten participants who had to gesture and spell the phrases in American Sign Language (ASL). The speech recognition tests for simple phrases have therefore shown good results. The system also recognized the participant's gesture very well during the test. The study suggested that a natural user interface with Microsoft Kinect could be interpreted as a sign language translator for the hearing impaired.Downloads
Downloads
Published
How to Cite
Issue
Section
License
The copyright to this article is transferred to Universitas Mercu Buana (UMB) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to UMB. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The Copyright Transfer Agreement (CTA) Form is able to download in CTA Form
We declare that:
1. This paper has not been published in the same form elsewhere.
2. It will not be submitted anywhere else for publication prior to acceptance/rejection by this Journal.
3. Copyright permission is obtained for materials published elsewhere and which require this permission for reproduction.
Furthermore, I/We hereby transfer the unlimited rights of publication of the above-mentioned paper in whole to UMB. The copyright transfer covers the exclusive right to reproduce and distribute the article, including reprints, translations, photographic reproductions, microform, electronic form (offline, online) or any other reproductions of similar nature.
The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.
Retained Rights/Terms and Conditions
1. As an author you (or your employer or institution) may do the following:
- make copies (print or electronic) of the article for your own personal use, including for your own classroom teaching use;
- make copies and distribute such copies (including through e-mail) of the article to research colleagues, for personal use by such colleagues (but not commercially or systematically, e.g. via an e-mail list or list server);
- present the article at a meeting or conference and distribute copies of the article to the delegates attending such meeting;
- for your employer, if the article is a 'work for hire', made within the scope of your employment, your employer may use all or part of the information in the article for other intra-company use (e.g. training);
- retain patent and trademark rights and rights to any process, procedure, or article of manufacture described in the article;
- include the article in full or in part in a thesis or dissertation (provided that this is not to be published commercially);
- use the article or any part thereof in a printed compilation of your works, such as collected writings or lecture notes (subsequent to publication of the article in the journal); and prepare other derivative works, to extend the article into book-length Form, or to otherwise reuse portions or excerpts in other works, with full acknowledgement of its original publication in the journal;
- may reproduce or authorize others to reproduce the article, material extracted from the article, or derivative works for the Author's personal use or for company use, provided that the source and the copyright notice are indicated, the copies are not used in any way that implies UMB endorsement of a product or service of any employer, and the copies themselves are not offered for sale.
All copies, print or electronic, or other use of the paper or article must include the appropriate bibliographic citation for the article's publication in the journal.
2. Requests from third parties
Although authors are permitted to reuse all or portions of the article in other works, this does not include granting third-party requests for reprinting, republishing, or other types of reuse. Requests for all uses not included above, including the authorization of third parties to reproduce or otherwise use all or part of the article (including figures and tables), should be referred to UMB by going to our website at https://puslit.mercubuana.ac.id/
3. Author Online Use
- Personal Servers. Authors and/or their employers shall have the right to post the accepted version of articles pre-print version of the article, or revised personal version of the final text of the article (to reflect changes made in the peer review and editing process) on their own personal servers or the servers of their institutions or employers without permission from UMB, provided that the posted version includes a prominently displayed UMB copyright notice and, when published, a full citation to the original publication, including a link to the article abstract in the journal homepage. Authors shall not post the final, published versions of their papers;
- Classroom or Internal Training Use. An author is expressly permitted to post any portion of the accepted version of his/her own articles on the Author's personal website or the servers of the Author's institution or company in connection with the Author's teaching, training, or work responsibilities, provided that the appropriate copyright, credit, and reuse notices appear prominently with the posted material. Examples of permitted uses are lecture materials, course packs, e-reserves, conference presentations, or in-house training courses;
- Electronic Preprints. Before submitting an article to a SINERGI, authors frequently post their manuscripts to their own website, employer's site, or another server inviting constructive colleague comments. Upon submission of an article to SINERGI, an author is required to transfer copyright in the article to UMB, and the Author must update any previously posted version of the article with a prominently displayed UMB copyright notice. Upon publication of an article by the UMB, the Author must replace any previously posted electronic versions of the article with either (1) the complete citation to the work with a Digital Object Identifier (DOI) or a link to the article abstract in SINERGI journal homepage, or (2) the accepted version only (not the final, published version), including the UMB copyright notice and full citation, with a link to the final, published article in journal homepage.
4. Articles in Press (AiP) service
UMB may choose to publish an abstract or portions of the paper before we publish it in the journal. However, be sure to contact our Production department immediately if you do not want us to make any such prior publication for any reason, including disclosure of a patentable invention.
5. Author/Employer Rights
If you are employed and prepared the article on a subject within the scope of your employment, the copyright in the article belongs to your employer as a work-for-hire. In that case, UMB assumes that when you sign this Form, you are authorized to do so by your employer and that your employer has consented to the transfer of copyright, to the representation and warranty of publication rights, and all other terms and conditions of this Form. If such authorization and consent have not been given to you, an authorized representative of your employer should sign this Form as the Author.
6. UMB Copyright Ownership
It is the formal policy of UMB to own the copyrights to all copyrightable material in its technical publications and to the individual contributions contained therein to protect the interests of the UMB, its authors and their employers, and, at the same time, to facilitate the appropriate reuse of this material by others. UMB distributes its technical publications worldwide by various means, such as hard copy, microfiche, microfilm, and electronic media. It also abstracts and may translate its publications and articles contained therein for inclusion in various compendiums, collective works, databases and similar publications.
