Skip to content
GitLab
Commit 148c0605 authored by Aleš Horák's avatar Aleš Horák
Browse files

init

parent 50070129
Expand all Hide whitespace changes
Inline Side-by-side
README.md
View file @ 148c0605
# Dialog Presentation Nlp
Developed at [NLP Centre](https://nlp.fi.muni.cz/en), [FI MU](https://www.fi.muni.cz/index.html.en) for [Karel Pepper](https://nlp.fi.muni.cz/trac/pepper)
This is a dialog application, in both Czech and English, that shows a presentation about NLP lab with images on the tablet.
Start with "Show presentation." / "Ukaž prezentaci."
## Installation
* [make and install](https://nlp.fi.muni.cz/trac/pepper/wiki/InstallPkg) the package as usual for the Pepper robot
dialog_presentation_nlp.pml 0 → 100644
View file @ 148c0605
<?xml version="1.0" encoding="UTF-8" ?>
<Package name="dialog_presentation_nlp" format_version="4">
<Manifest src="manifest.xml" />
<BehaviorDescriptions/>
<Dialogs>
<Dialog name="dlg_presentation_nlp" src="dlg_presentation_nlp/dlg_presentation_nlp.dlg" />
</Dialogs>
<Resources>
<File name="style" src="html/css/style.css" />
<File name="index" src="html/index.html" />
<File name="icon" src="icon.png" />
<File name="icon" src="html/cover.jpg" />
<File name="icon" src="html/inf-lg-text-cze-rgb.png" />
<File name="icon" src="html/logo_NLPlab.png" />
<File name="icon" src="html/ptakopysk.png" />
<File name="icon" src="html/tree.png" />
<File name="icon" src="html/corpora.png" />
<File name="icon" src="html/krabice.jpg" />
<File name="icon" src="html/vl_anim.gif" />
<File name="jquery-1.11.0.min" src="html/js/jquery-1.11.0.min.js" />
<File name="main" src="html/js/main.js" />
<File name="robotutils" src="html/js/robotutils.js" />
</Resources>
<Topics>
<Topic name="dlg_presentation_nlp_czc" src="dlg_presentation_nlp/dlg_presentation_nlp_czc.top" topicName="dlg_presentation_nlp" language="cs_CZ" />
<Topic name="dlg_presentation_nlp_enu" src="dlg_presentation_nlp/dlg_presentation_nlp_enu.top" topicName="dlg_presentation_nlp" language="en_US" />
</Topics>
<IgnoredPaths>
<Path src=".metadata" />
</IgnoredPaths>
</Package>
dlg_presentation_nlp/dlg_presentation_nlp.dlg 0 → 100644
View file @ 148c0605
multilanguage:dlg_presentation_nlp
enu:dlg_presentation_nlp_enu.top
czc:dlg_presentation_nlp_czc.top
dlg_presentation_nlp/dlg_presentation_nlp_czc.top 0 → 100644
View file @ 148c0605
topic: ~dlg_presentation_nlp()
language: czc
u:([prezentace "Ukaž prezentaci" "Můžeš {nám} ukázat prezentaci" "Předveď prezentaci"])
^pCall(ALTabletService.showWebview("http://198.18.0.1/apps/dialog_presentation_nlp/"))
$presentation_nlp=1
Dobrý den.
^run(animations/Stand/Gestures/BowShort_2)
$presentation_nlp=2
Vítám vás v Laboratoři zpracování přirozeného jazyka.
$presentation_nlp=8
V prosinci jsem přijel do Brna, abych se tady naučil česky. Jde to pomalu.
$presentation_nlp=6
Tady v laboratoři vytvářejí nástroje pro češtinu, ale i další světové jazyky.
$presentation_nlp=3
Určitě z nich často používáte například
$presentation_nlp=7
kontrolu pravopisu,
^start(animations/Stand/Gestures/ShowTablet_3)
našeptávání v mobilech, strojový překlad,
$presentation_nlp=4
slovníky nebo jazykové hry.
^start(animations/Stand/Gestures/Hey_10)
$presentation_nlp=5
Děkuji za návštěvu a přijďte za mnou znovu.
^pCall(ALTabletService.showWebview("http://198.18.0.1/apps/dialog_presentation_nlp/"))
dlg_presentation_nlp/dlg_presentation_nlp_enu.top 0 → 100644
View file @ 148c0605
topic: ~dlg_presentation_nlp()
language: enu
u:(presentation "show presentation")
^pCall(ALTabletService.showWebview("http://198.18.0.1/apps/dialog_presentation_nlp/"))
$presentation_nlp=1
Hello, nice to meet you.
^run(animations/Stand/Gestures/BowShort_2)
$presentation_nlp=2
Welcome at the natural language processing laboratory.
$presentation_nlp=8
I came to Brno in December to learn Czech here. I am a slow learner.
$presentation_nlp=6
Here in the lab, researchers create computer tools for Czech and other world languages.
$presentation_nlp=3
You surely know, for instance,
$presentation_nlp=7
spell checking,
^start(animations/Stand/Gestures/ShowTablet_3)
auto-completion, machine translation,
$presentation_nlp=4
dictionaries or language games.
^start(animations/Stand/Gestures/Hey_10)
$presentation_nlp=5
Thank you for visiting us and come to see me again soon.
^pCall(ALTabletService.showWebview("http://198.18.0.1/apps/dialog_presentation_nlp/"))
html/css/style.css 0 → 100644
View file @ 148c0605
* {
-webkit-tap-highlight-color: rgba(0, 0, 0, 0);
}
body{
margin: 0px;
background-color: white;
font-size: 20px;
}
.tabletsized {
background-color: white;
width: 1280px;
height: 800px;
position: absolute;
font-size: 6rem
}
.centerimg {
margin: auto;
position: absolute;
left: 0;
right: 0;
top: 0;
bottom: 0;
}
.info {
font-size: 4rem
}
.speech {
font-size: 6rem
}
.centered {
position: absolute;
top: 50%;
left: 50%;
margin-right: -50%;
transform: translate(-50%, -50%);
text-align: center;
}
.levelbutton, .bigfriendlybutton {
border-radius: 2rem;
display: inline-block;
width: 30rem;
height: 10rem;
background-color: blue;
color: white;
text-align: center;
font-family: Arial, Helvetica, sans-serif;
font-size: 3rem;
font-style: normal;
font-weight: bold;
line-height: 5rem;
}
.levelbutton.clicked {
background-color: grey;
}
.levelbutton.clicked.highlighted {
background-color: darkblue;
}
#exit {
width: 15rem;
height: 6rem;
font-size: 4rem;
position: absolute;
top: 0rem;
right: 2rem;
background-color: grey;
color: white;
}
#biglabel {
font-size: 30rem;
color: white;
}
.active {
color: green;
}
.activestatus {
color: white;
}
.inactivestatus {
color: grey;
}
.activeconfident {
color: lightgreen;
}
.activenotconfident {
color: magenta;
}
.inactive {
color: red;
}
html/index.html 0 → 100644
View file @ 148c0605
<!DOCTYPE html>
<html>
<head>
<META http-equiv="Content-Type" content="text/html; charset=UTF-8">
<!-- needed to scale tablet display -->
<meta name="viewport" content="width=1280, user-scalable=no" />
<!-- application css -->
<link media="screen" href="css/style.css" rel="stylesheet" type="text/css">
</head>
<body class="unselectable">
<div class="tabletsized" >
<div id="noservice" class="" style="display:none">
Service ALDialogSubtitles not found :(
</div>
<div class="image" id="image1" style="display:none">
<img src="inf-lg-text-cze-rgb.png" class="centerimg">
</div>
<div class="image" id="image2" style="display:none">
<img src="logo_NLPlab.png" class="centerimg">
</div>
<div class="image" id="image3" style="display:none">
<img src="tree.png" class="centerimg">
</div>
<div class="image" id="image4" style="display:none">
<img src="ptakopysk.png" class="centerimg">
</div>
<div class="image" id="image5" style="display:none">
<img src="cover.jpg" class="centerimg">
</div>
<div class="image" id="image6" style="display:none">
<img src="corpora.png" class="centerimg">
</div>
<div class="image" id="image7" style="display:none">
<img src="vl_anim.gif" class="centerimg">
</div>
<div class="image" id="image8" style="display:none">
<img src="krabice.jpg" class="centerimg">
</div>
</div>
<!-- qimessaging & jquery stuff -->
<script src="js/jquery-1.11.0.min.js"></script>
<script src="js/robotutils.js"></script>
<script src="js/main.js"></script>
<!-- application script call -->
<script type="text/javascript">
$(document).ready(function(e){
var app = application();
})
</script>
</body>
</html>
html/js/jquery-1.11.0.min.js 0 → 100644
View file @ 148c0605
This diff is collapsed.
html/js/main.js 0 → 100644
View file @ 148c0605
var PRESENTATION_VARIABLE = "presentation_nlp";
function update_variable(stateJson) {
//console.log('update variable', stateJson)
//var state = JSON.parse(stateJson);
var num = stateJson;
if (num != undefined) {
$('.image').hide();
$('#image'+num).show();
}
}
function onFrontTactilTouched(value)
{
console.log("FrontTactilTouched event", value);
if (value) document.getElementById("touch").innerHTML = "touched";
else document.getElementById("touch").innerHTML = "Not touched";
}
var application = function(){
RobotUtils.onService(function (ALMemory) {
ALMemory.getData(PRESENTATION_VARIABLE).then(update_variable);
update_variable('1');
});
RobotUtils.subscribeToALMemoryEvent(PRESENTATION_VARIABLE, update_variable);
RobotUtils.subscribeToALMemoryEvent("FrontTactilTouched", onFrontTactilTouched);
//RobotUtils.onService(function (ALSpeechRecognition) {
// ALSpeechRecognition.subscribe('dialog_subtitles').then(function() {
// console.log("ALSpeechRecognition subscribed");
// });
//}, function() {
// console.log("Failed to subscribe to ALSpeechRecognition")
//});
////RobotUtils.subscribeToALMemoryEvent("FrontTactilTouched", onFrontTactilTouched);
//RobotUtils.subscribeToALMemoryEvent("SpeechDetected", onSpeechDetected);
////RobotUtils.subscribeToALMemoryEvent("ALSpeechRecognition/Status", onSpeechStatus);
//RobotUtils.subscribeToALMemoryEvent("Dialog/IsStarted", onSpeechStatus);
////RobotUtils.subscribeToALMemoryEvent("ALSpeechRecognition/ActiveListening", onSpeechStatus);
////RobotUtils.subscribeToALMemoryEvent("WordRecognized", onWordRecognized);
////RobotUtils.subscribeToALMemoryEvent("WordRecognizedAndGrammar", onWordRecognized);
////RobotUtils.subscribeToALMemoryEvent("Dialog/LastInput", onLastInput);
////RobotUtils.subscribeToALMemoryEvent("Dialog/Answered", onAnswered);
//RobotUtils.subscribeToALMemoryEvent("WordRecognizedAndGrammar", onLastInput);
//RobotUtils.subscribeToALMemoryEvent("ALTextToSpeech/CurrentSentence", onAnswered);
//RobotUtils.subscribeToALMemoryEvent("Dialog/SwitchLanguage", onLanguageTTS);
}
/*
var KEY_TABLETSETATE = "dialog_subtitles/TabletState";
function onFrontTactilTouched(value)
{
console.log("FrontTactilTouched event", value);
if (value) document.getElementById("touch").innerHTML = "touched";
else document.getElementById("touch").innerHTML = "Not touched";
}
function onSpeechDetected(value)
{
console.log("SpeechDetected event", value);
if (value) {
$("#speech").html("Speech detected");
$("#speech").removeClass("inactive");
$("#speech").addClass("active");
}
else {
$("#speech").html("Speech not detected");
$("#speech").removeClass("active");
$("#speech").addClass("inactive");
}
}
function onWordRecognized(value)
{
//console.log("WordRecognized event", value);
console.log("WordRecognizedAndGrammar event", value);
document.getElementById("word").innerHTML = value;
}
function onSpeechStatus(value)
{
console.log("ALSpeechRecognition/Status event", value);
document.getElementById("status").innerHTML = value;
}
var application = function(){
RobotUtils.onService(function (ALDialogSubtitles) {
$("#noservice").hide();
ALDialogSubtitles.get().then(function(level) {
// Find the button with the right level:
$(".levelbutton").each(function() {
var button = $(this);
if (button.data("level") == level) {
button.addClass("highlighted");
button.addClass("clicked");
}
});
// ... and show all buttons:
$("#buttons").show();
});
$(".levelbutton").click(function() {
// grey out the button, until we hear back that the click worked.
var button = $(this);
var level = button.data("level");
$(".levelbutton").removeClass("highlighted");
$(".levelbutton").removeClass("clicked");
button.addClass("clicked");
ALDialogSubtitles.set(level).then(function(){
button.addClass("highlighted");
});
})
}, function() {
console.log("Failed to get the service.")
$("#noservice").show();
});
RobotUtils.onService(function(ALTextToSpeech) {
// Bind button callbacks
$(".bleeper").click(function() {
ALTextToSpeech.say($(this).html());
});
});
RobotUtils.subscribeToALMemoryEvent("FrontTactilTouched", onFrontTactilTouched);
RobotUtils.subscribeToALMemoryEvent("SpeechDetected", onSpeechDetected);
RobotUtils.subscribeToALMemoryEvent("ALSpeechRecognition/Status", onSpeechStatus);
//RobotUtils.subscribeToALMemoryEvent("WordRecognized", onWordRecognized);
RobotUtils.subscribeToALMemoryEvent("WordRecognizedAndGrammar", onWordRecognized);
RobotUtils.onService(function (ALMemory) {
ALMemory.getData(KEY_TABLETSETATE).then(updateTabletState);
//updateTabletState('{"title": "TEST"}');
});
RobotUtils.subscribeToALMemoryEvent(KEY_TABLETSETATE, updateTabletState);
*/
html/js/main.js.subtitles 0 → 100644
View file @ 148c0605
var confidence_levels=Array();
var language = "English";
var last_speech_status = "Idle";
var subs_sp_detected = null
var subs_wrd_recognized = null
var subs_cur_sentence = null
var subs_sw_language = null
function onFrontTactilTouched(value)
{
console.log("FrontTactilTouched event", value);
if (value) document.getElementById("touch").innerHTML = "touched";
else document.getElementById("touch").innerHTML = "Not touched";
}
function onSpeechDetected(value)
{
console.log("SpeechDetected event", value);
if (value) {
$("#speech").html("💬");
}
else {
$("#speech").html("");
}
}
function onWordRecognized(value)
{
//console.log("WordRecognized event", value);
console.log("WordRecognizedAndGrammar event", value);
document.getElementById("word").innerHTML = value;
}
function setLanguage(grammar)
{
var new_language = '';
if (grammar.search("Czech") >= 0) new_language = "Czech";
else if (grammar == "") new_language = language;
else new_language = "English";
if (new_language != language) {
console.log("Changing language to", new_language);
language = new_language;
onSpeechStatus(last_speech_status);
}
}
function isConfident(word, confidence, grammar)
{
grammar = grammar.slice(0,3);
var threshold = confidence_levels[grammar]["Default"];
if (confidence >= threshold) {
console.log("confidence",confidence,"with grammar", grammar,">=",threshold);
return (true);
}
console.log("confidence",confidence,"with grammar", grammar,"<",threshold);
return (false);
}
function onLastInput(value)
{
//console.log("LastInput event", value);
console.log("WordRecognizedAndGrammar event", value);
var word = value[0];
var confidence = value[1];
var grammar = value[2];
setLanguage(grammar);
if (word) {
$("#word").html(word);
$("#word").removeClass("inactive");
if (isConfident(word,confidence,grammar)) {
$("#word").removeClass("activenotconfident");
$("#word").addClass("activeconfident");
}
else {
$("#word").removeClass("activeconfident");
$("#word").addClass("activenotconfident");
}
}
else {
$("#word").html("-");
$("#word").removeClass("activeconfident");
$("#word").removeClass("activenotconfident");
$("#word").addClass("inactive");
}
}
function onAnswered(value)
{
console.log("ALTextToSpeech/CurrentSentence event", value);
if (value) {
$("#answer").html(value);
$("#answer").removeClass("inactive");
$("#answer").addClass("active");
}
//else {
// $("#answer").html("-");
// $("#answer").removeClass("active");
// $("#answer").addClass("inactive");
//}
}
function onLanguageTTS(value)
{
console.log("ALTextToSpeech/languageTTS event", value);
if (value) setLanguage(value.charAt(0).toUpperCase() + value.slice(1));
}
function onSpeechStatus(value)
{
//console.log("ALSpeechRecognition/Status event", value,typeof(value));
console.log("Dialog/IsStarted event", value, typeof(value));
if (value == "1" || value == true || value == "ListenOn") {
last_speech_status = value;
$("#status").addClass("activestatus");
if (language == "Czech") $("#status").html("Mluvte prosím!");
else $("#status").html("Speak to me!");
subs_sp_detected = RobotUtils.subscribeToALMemoryEvent("SpeechDetected", onSpeechDetected);
subs_wrd_recognized = RobotUtils.subscribeToALMemoryEvent("WordRecognizedAndGrammar", onLastInput);
subs_cur_sentence = RobotUtils.subscribeToALMemoryEvent("ALTextToSpeech/CurrentSentence", onAnswered);
subs_sw_language = RobotUtils.subscribeToALMemoryEvent("Dialog/SwitchLanguage", onLanguageTTS);
}
else if (value == "0" || value == false || value == "ListenOff" || value == "Idle" || value == "Stop") {
last_speech_status = value;
$("#status").addClass("inactivestatus");
if (language == "Czech") $("#status").html("čekejte prosím");
else $("#status").html("wait please");
if (subs_sp_detected) subs_sp_detected.unsubscribe()
if (subs_wrd_recognized) subs_wrd_recognized.unsubscribe()
if (subs_cur_sentence) subs_cur_sentence.unsubscribe()
if (subs_sw_language) subs_sw_language.unsubscribe()
}
else if (value == "SpeechDetected" || value == "EndOfProcess") {
// no change
}
}
var application = function(){
RobotUtils.onService(function (ALDialogSubtitles) {
$("#noservice").hide();
$("#exit").show();
$("#exit").click(function() {
ALDialogSubtitles.show_tablet(0);
})
}, function() {
console.log("Failed to get the service.")
$("#noservice").show();
});
RobotUtils.onService(function (ALDialog) {
ALDialog.getAllConfidenceThresholds().then(function(levels) {
confidence_levels = levels;
console.log("ALDialog confidence thresholds:", confidence_levels);
console.log("BNF confidence threshold:", confidence_levels["BNF"]["Default"]);
});
}, function() {
console.log("Failed to get ALDialog confidence thresholds.")
});
//RobotUtils.onService(function (ALSpeechRecognition) {
// ALSpeechRecognition.subscribe('dialog_subtitles').then(function() {
// console.log("ALSpeechRecognition subscribed");
// });
//}, function() {
// console.log("Failed to subscribe to ALSpeechRecognition")
//});
////RobotUtils.subscribeToALMemoryEvent("FrontTactilTouched", onFrontTactilTouched);
//RobotUtils.subscribeToALMemoryEvent("SpeechDetected", onSpeechDetected);
////RobotUtils.subscribeToALMemoryEvent("ALSpeechRecognition/Status", onSpeechStatus);
RobotUtils.subscribeToALMemoryEvent("Dialog/IsStarted", onSpeechStatus);
////RobotUtils.subscribeToALMemoryEvent("ALSpeechRecognition/ActiveListening", onSpeechStatus);
////RobotUtils.subscribeToALMemoryEvent("WordRecognized", onWordRecognized);
////RobotUtils.subscribeToALMemoryEvent("WordRecognizedAndGrammar", onWordRecognized);
////RobotUtils.subscribeToALMemoryEvent("Dialog/LastInput", onLastInput);
////RobotUtils.subscribeToALMemoryEvent("Dialog/Answered", onAnswered);
//RobotUtils.subscribeToALMemoryEvent("WordRecognizedAndGrammar", onLastInput);
//RobotUtils.subscribeToALMemoryEvent("ALTextToSpeech/CurrentSentence", onAnswered);
//RobotUtils.subscribeToALMemoryEvent("Dialog/SwitchLanguage", onLanguageTTS);
}
/*
var KEY_TABLETSETATE = "dialog_subtitles/TabletState";
function updateTabletState(stateJson) {
var state = JSON.parse(stateJson);
if (state.title) {
$("#biglabel").show();
$("#biglabel").html(state.title);
} else {
$("#biglabel").hide();
}
}
function onFrontTactilTouched(value)
{
console.log("FrontTactilTouched event", value);
if (value) document.getElementById("touch").innerHTML = "touched";
else document.getElementById("touch").innerHTML = "Not touched";
}
function onSpeechDetected(value)
{
console.log("SpeechDetected event", value);
if (value) {
$("#speech").html("Speech detected");
$("#speech").removeClass("inactive");
$("#speech").addClass("active");
}
else {
$("#speech").html("Speech not detected");
$("#speech").removeClass("active");
$("#speech").addClass("inactive");
}
}
function onWordRecognized(value)
{
//console.log("WordRecognized event", value);
console.log("WordRecognizedAndGrammar event", value);
document.getElementById("word").innerHTML = value;
}
function onSpeechStatus(value)
{
console.log("ALSpeechRecognition/Status event", value);
document.getElementById("status").innerHTML = value;