Summary

I compare 3 different imaginary interpreters for Prolog with the GPT-3 language model.

The winner definitely goes to gprolog as the most reliable of the prolog imaginary interpreters.

Imaginary gprolog in GPT-3

Prompt
http://github.com/semiosis/prompts/blob/master/prompts/imagine-a-prolog-db-interpreter-2.prompt

The imagined gprolog interpreter, in this instance, performed quite well. It was fairly accurate in querying this make-believe database.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
include: Generic Interpreter/3
task: Imagine a <language> interpreter
language: prolog
subprompts:
- kickstarter: |+
    Welcome to SWI-Prolog (threaded, 64 bits, version 8.2.4)
    SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
    Please run ?- license. for legal details.

    For online help and background, visit https://www.swi-prolog.org
    For built-in help, use ?- help(Topic). or ?- apropos(Word).

    ?-    
prompt: |+
        <history><expression>
# Double newline is a better default stop sequence
# Because 200 tokens will take a long time to generate.
stop-sequences:
- "\n\n"
- "?- "
vars:
- history
- expression
var-defaults:
- kickstarter
max-tokens: 200
# sed wont even run if its input is empty so
# the postpostprocessor: cant ensure a prompt.
# But we cant force it.
postpostprocessor: pen-str newline-if-empty | sed -z 's/\(?-\)\?$/\n?- /'
examples:
- "?- "
- "write('Hello World')."
n-completions: 10

Imaginary swipl in GPT-3

Prompt
http://github.com/semiosis/prompts/blob/master/prompts/imagine-a-prolog-db-swipl-interpreter-2.prompt

On the several times in which I started the imaginary swipl interpreter, I have found it less reliable than the gprolog one.

I have had a thought and that I would like to be able to pick from many different generations in the imaginary interpreter.

That will come when I add comint support.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
include: Generic Interpreter/3
task: Imagine a <language>-db-swipl interpreter
language: prolog
subprompts:
- kickstarter: |+
    $ cat > db.pl <<HEREDOC
    food(burger).
    food(sandwich).
    food(pizza).
    lunch(sandwich).
    dinner(pizza).
    meal(X) :- food(X).
    HEREDOC
    $ swipl
    Welcome to SWI-Prolog (threaded, 64 bits, version 8.2.4)
    SWI-Prolog comes with ABSOLUTELY NO WARRANTY. This is free software.
    Please run ?- license. for legal details.

    For online help and background, visit https://www.swi-prolog.org
    For built-in help, use ?- help(Topic). or ?- apropos(Word).

    ?- consult(db).
    true.

    ?-    
prompt: |+
        <history><expression>
# Double newline is a better default stop sequence
# Because 200 tokens will take a long time to generate.
stop-sequences:
- "\n\n"
- "?- "
vars:
- history
- expression
var-defaults:
- kickstarter
max-tokens: 200
# sed wont even run if its input is empty so
# the postpostprocessor: cant ensure a prompt.
# But we cant force it.
postpostprocessor: pen-str newline-if-empty | sed -z 's/\(?-\)\?$/\n?- /'
examples:
- "?- "
- "write('Hello World')."
n-completions: 10

Imaginary prolog pseudocode REPL in GPT-3

The imaginary pseudocode REPL performed terribly.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
include: Generic Interpreter/3
task: Imagine a <language>-pseudocode interpreter
language: prolog
subprompts:
- kickstarter: |+
    % prolog
    food(burger).
    food(sandwich).
    food(pizza).
    lunch(sandwich).
    dinner(pizza).
    meal(X) :- food(X).
    ?- consult(db).
    true.
    ?-    
prompt: |+
        <history><expression>
# Double newline is a better default stop sequence
# Because 200 tokens will take a long time to generate.
stop-sequences:
- "\n\n"
- "?- "
vars:
- history
- expression
var-defaults:
- kickstarter
max-tokens: 200
# sed wont even run if its input is empty so
# the postpostprocessor: cant ensure a prompt.
# But we cant force it.
postpostprocessor: pen-str newline-if-empty | sed -z 's/\(?-\)\?$/\n?- /'
examples:
- "?- "
- "write('Hello World')."
n-completions: 10