{"id":82,"date":"2005-12-03T11:25:08","date_gmt":"2005-12-03T16:25:08","guid":{"rendered":"http:\/\/blogs.law.harvard.edu\/hoanga\/2005\/12\/03\/nlp-techniques-applied-to-vietnamese\/"},"modified":"2005-12-03T11:25:08","modified_gmt":"2005-12-03T16:25:08","slug":"nlp-techniques-applied-to-vietnamese","status":"publish","type":"post","link":"https:\/\/archive.blogs.harvard.edu\/hoanga\/2005\/12\/03\/nlp-techniques-applied-to-vietnamese\/","title":{"rendered":"NLP techniques applied to Vietnamese"},"content":{"rendered":"<p><a name='a585'><\/a><\/p>\n<p>\n I&#8217;ve been getting interested in computational linguistics lately and have<br \/>\n been trying to read up on the<br \/>\n <a href=\"http:\/\/nlp.stanford.edu\/fsnlp\/\">fundamentals<\/a>.  It&#8217;s been quite<br \/>\n a bit of probability which I&#8217;m not sure I really understand that well but<br \/>\n I imagine it&#8217;ll get easier to understand if I work with it a bit more.\n <\/p>\n<p>\n Most of the examples in the textbook are focused on English so I thought it<br \/>\n would be interesting to apply it to Vietnamese.  After looking around for<br \/>\n grammars for Vietnamese that I could use as a model for being able to tag<br \/>\n a Vietnamese corpus I stumbled across the website for the<br \/>\n <a href=\"http:\/\/www.vietlex.com\/research.htm\">Vietnam Lexicography Centre<\/a>.<br \/>\n It looks like a great resource for Vietnamese computational linguistics issues.<br \/>\n There&#8217;s a paper describing an effort for descring the tags for part-of-speech<br \/>\n (POS) tagging in Vietnamese titled <i>Lexical descriptions for Vietnamese<br \/>\n language processing<\/i>.  It&#8217;s a very good description of problems running<br \/>\n across computational linguists trying to tackle Vietnamese (hint: the number<br \/>\n is very low).\n <\/p>\n<p>\n Here&#8217;s some brief highlights:\n <\/p>\n<ul>\n<li>The classification methods which would be a necessary model to use for<br \/>\n POS tagging is still in discussion among Vietnamese linguists.<\/li>\n<li>There has been some other efforts but without an implementation available<br \/>\n it&#8217;s impossible to check the effectiveness of those efforts<\/li>\n<li>They are trying to adhere to some model called the MULTEXT model (which<br \/>\n itself is some effort to help standardize some parts of linguistics databases<br \/>\n so researchers can actually exchange their data easier)<\/li>\n<li>Few resources for tools for Vietnamese text analysis are in the public<br \/>\n research domain<\/li>\n<\/ul>\n<p>\n I think the authors of the paper make a good point.  Publicly available<br \/>\n resources for linguistic processing research are very important.  I think<br \/>\n the problem is magnified far more for languages that are not &#8216;economic<br \/>\n powerhouse&#8217; languages such as English, French, or Japanese.  What I mean<br \/>\n by &#8216;economic powerhouse&#8217; is that the cultures that use these languages have<br \/>\n a much more stable financial base to build research efforts and derive<br \/>\n some sort of financial benefit from tools that are built.  However, for<br \/>\n Vietnamese or Cambodian I&#8217;d argue that economic incentives for doing<br \/>\n nlp research in these languages is quite low.  In order to progress research<br \/>\n in these languages I feel it is even MORE necessary to have the tools and<br \/>\n as much of the data as possible available for public research usage.  Without<br \/>\n such efforts I get the feeling much of the research will not make progress<br \/>\n since people will be too busy building fundamental building blocks rather<br \/>\n than slowly stacking the building blocks on top of one another to find out<br \/>\n what works and what doesn&#8217;t.\n <\/p>\n<p>\n One last note, in order to become a competent Vietnamese linguist it seems<br \/>\n from a cursory inspection you&#8217;ll need a decent command of Vietnamese (not<br \/>\n surprising) and perhaps slightly more surprising is some literacy in<br \/>\n French to catch some research that hasn&#8217;t been translated to English yet<br \/>\n (if it ever will be).\n <\/p>\n<p>\n Resources I found:<br \/>\n <br \/>\n <a href=\"http:\/\/www.vietlex.com\/research\/ALRWS2004-Paper007.pdf\">Lexical<br \/>\n descriptions for Vietnamese language processing<\/a>(PDF)<br \/>\n <br \/>\n <a href=\"http:\/\/si.unm.edu\/linguistics\/viet\/viet\">Vietnamese linguistic and<br \/>\n Cultural Information<\/a> (A little light imo)<br \/>\n <br \/>\n <a href=\"http:\/\/en.wikipedia.org\/wiki\/Vietnamese_language\">Wikipedia article on the Vietnamese language<\/a><br \/>\n <br \/>\n <a href=\"http:\/\/www.arts.mcgill.ca\/programs\/linguistics\/projects\/Vietnamese\/\">Vietnamese Grammar Project<\/a>(Seems dead and definitely far from complete)<br \/>\n <br \/>\n <a href=\"http:\/\/www.anu.edu.au\/~u9907217\/languages\/AAlecture6.html\">Mon-khmer.com&#8217;s description of Vietnamese<\/a>\n <\/p>\n","protected":false},"excerpt":{"rendered":"<p>I&#8217;ve been getting interested in computational linguistics lately and have been trying to read up on the fundamentals. It&#8217;s been quite a bit of probability which I&#8217;m not sure I really understand that well but I imagine it&#8217;ll get easier to understand if I work with it a bit more. Most of the examples in [&hellip;]<\/p>\n","protected":false},"author":704,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-82","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/archive.blogs.harvard.edu\/hoanga\/wp-json\/wp\/v2\/posts\/82","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/archive.blogs.harvard.edu\/hoanga\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/archive.blogs.harvard.edu\/hoanga\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/hoanga\/wp-json\/wp\/v2\/users\/704"}],"replies":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/hoanga\/wp-json\/wp\/v2\/comments?post=82"}],"version-history":[{"count":0,"href":"https:\/\/archive.blogs.harvard.edu\/hoanga\/wp-json\/wp\/v2\/posts\/82\/revisions"}],"wp:attachment":[{"href":"https:\/\/archive.blogs.harvard.edu\/hoanga\/wp-json\/wp\/v2\/media?parent=82"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/hoanga\/wp-json\/wp\/v2\/categories?post=82"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/hoanga\/wp-json\/wp\/v2\/tags?post=82"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}