GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — GitHub - shawntan/SUT: Repository for Sparse Universal Transformers

BranchesTags
Open more actions menu

Latest commit

 

History

3 Commits
3 Commits

Folders and files

NameName
Last commit message
Last commit date
scripts
scripts
 
 
slurm
slurm
 
 
sut_layer
sut_layer
 
 
.gitignore
.gitignore
 
 
README.md
README.md
 
 
__init__.py
__init__.py
 
 
halting.py
halting.py
 
 
sut_loss.py
sut_loss.py
 
 
sut_transformer.py
sut_transformer.py
 
 
transformer_base.py
transformer_base.py
 
 
transformer_config.py
transformer_config.py
 
 
transformer_decoder.py
transformer_decoder.py
 
 
transformer_encoder.py
transformer_encoder.py
 
 

Repository files navigation

Sparse Universal Transformers (SUT)

Implementation for WMT'14 En-De for SUT.

Link to paper


Steps to run:

  1. Clone and follow the preprocessing steps in Transformer-Clinic

    export TRANSFORMER_CLINIC_ROOT=/path/to/Transformer-Clinic
  2. Initialise evironment variables for experiment (base or big):

    source scripts/params_de_base.sh
    export EXP_NAME=sut_base
    # Submit job to cluster (32 compute nodes)
    sbatch slurm/launch.slurm
  3. Generate and calculate BLEU:

    scripts/eval_wmt14ende.sh checkpoints/sut_base

About

Repository for Sparse Universal Transformers

Resources

Readme
Activity

Stars

20 stars

Watchers

1 watching

Forks

1 fork
Report repository

Releases

Packages

Contributors

Languages

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.