This is an introduction to the imaginary
interpreter built into Pen.el.
The reason I am building REPLs and emacs modes
based around imaginary interpreters (which are
not at all deterministic or logically sound)
is that I believe they will be very powerful
and useful in the future.
It’s important to develop a language-agnostic
harness for working with arbitrarily many
imagined languages.
Solving the problems of utilising current LMs
will make it easier down the line to utilise
more advanced ones.
Steps to run
On your host machine, or from within the docker container’s shell
Run ii python, or any desired language as the first argument
To see what interpreters are available:
1
penl | grep -P -- "imagine.*-interpreter" | head -n 3
An interpreter prompt is selected depending on if a specialised prompt exists for the chosen language.
The generic ๐i prompts is comprised of:
kickstarter: A subprompt declaring that the following are examples of the specified language.
Consecutive Input/Output pairs.
Language-specific ๐i prompts are comprised of:
kickstarter: A subprompt which is typically a banner.
postpostprocessor: A script that takes the interpreter history and generates a new “user prompt”.
An interpreter prompt accepts as a first argument the history of the interaction with the interpreter.
This way, the interpreter has an imaginary state as you work with it.
ii script
Accepts a language as an argument
Docker bash-host interop:
๐i communicates with arbitrary LMs via the
bash (docker-host) interop of Pen.el.
rlwrap is used to keep a history of the inputs specific to each interpreter.
A REPL that keeps builds a transcript and passes that as input to the interpreter prompt.
include: Generic Interpreter/3task: Imagine a <language> interpreterlanguage: pythonsubprompts:
- kickstarter: |+ Python 3.8.5 (default, Jan 27 2021, 15:41:15)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.21.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]:prompt: |+ <history><expression>
<:pp>Outuser-prompt: "^In \\[[0-9]*\\]: "# Unfortunately, we can't generate the next In# prompt because we need to match on it with stop-sequences.# So the user prompt must be reconstructed manually.stop-sequences:
- "In ["# Create a user prompt with the number incremented.# This is like postprocessor but happens even later.# It is used in special circumstances when the prompt history is also required.postpostprocessor: pen-s python-gen-next-user-promptvars:
- history
- expressionvar-defaults:
- kickstarterexamples:
- "In [1]: "
- "5 + 5"
Using the bash interop
-p ensures that the entire prompt along
with the generated output is returned.
-u ensures that the cache is updated and a
new generation is returned.
Withholding the first argument
By supplying an empty string as the first
argument, the history is not passed to the
interpreter prompt. Instead, the prompt
function will use the kickstarter subprompt,
as it has been supplied as the default value
for the first variable.
Python 3.8.5 (default, Jan 27 2021, 15:41:15)
Type 'copyright', 'credits' or 'license'for more information
IPython 7.21.0 -- An enhanced Interactive Python. Type '?'for help.
In [1]: 5 + 5
Out[1]: 10
1
penf -p -u imagine-a-python-interpreter/2 "In [3]: ""5 + 5"
This will use a generic prompt which does any
language, but the name of the language has
been specified as Rubylang.
When using ๐i, if a specialised interpreter
prompt for a given language can be found then
that will be used.
However, since no interpreter can be found for
‘Rubylang’, the generic interpreter is used.
However, the generic interpreter can still
imagine the ‘Ruby’ language to a degree, but
is far less accurate.
The beauty of this, of course, is that we
don’t need to prime the interpreter with a
banner or terminal history, and we don’t need
to know what the prompt for Ruby looks like.
And a demo of a more catered imaginary interpreter
include: Generic Interpreter/3task: Imagine a <language> interpreterlanguage: rubysubprompts:
- kickstarter: |+ $ ruby -v
ruby 2.7.0p0 (2019-12-25 revision 647ee6f091) [x86_64-linux-gnu]
$ irb
2.7.0 :001 > RUBY_VERSION
=> "2.7.0"
2.7.0 :002 ># Because the prompt generates from at the end of the user expression# The generation will contain a starting newline.# Want to trim that with =๐i=, but not in the prompt function because a better harness could be made.prompt: |+<history><expression>user-prompt: "^2.7.0 :[0-9]* > "# Unfortunately, we can't generate the next In# prompt because we need to match on it with stop-sequences.# So the user prompt must be reconstructed manually.stop-sequences:
# This isn't ideal but ruby-gen-next-user-prompt needs to see a full prompt# So it can generate the next user prompt.
- " > "postpostprocessor: pen-str ruby-gen-next-user-promptvars:
- history
- expressionvar-defaults:
- kickstartern-completions: 10examples:
- "2.7.0 :002 > "
- "puts \"Hi\""
Support scripts
These are used as post-processors in the prompt and in ๐i.