GeezerVison

GeezerVison is showing wikitech.wikimedia.org — Help:Cloud Services introduction - Wikitech

(Redirected from Help:Cloud Services Introduction)

A Poster showing some Wikimedia Cloud services statistics and services — Poster-format overview

Wikimedia Cloud Services (WMCS) provides tools, services, and support for technical collaborators who want to contribute to Wikimedia software projects. Use Cloud Services to host your software tools for the Wikimedia movement, without charge.

What can you do with Cloud Services?

Host tools on Wikimedia servers

Tools and bots make it easier to edit and maintain Wikimedia projects. For developers who support Wikimedia projects by developing tools and bots, Toolforge provides the following features:

Free, reliable, and scalable shared hosting, including web servers, databases and other data storage
A distributed job processing system
Support for multiple users to collaboratively maintain and manage tools

To use Toolforge you need:

Some programming knowledge
An understanding of Unix command line

To get started, visit Help:Toolforge. Or, learn more about creating bots.

Run scripts and visualize data

PAWS is a Jupyter notebook installation hosted by Wikimedia. PAWS notebooks can be used for creating tutorials, running live code, creating data visualizations, running basic bots, and more.

A single PAWS notebook is maintained by a single user, but they can be downloaded and forked by other users. To use PAWS you need only a Wikimedia login and a web browser. Knowledge of Python is helpful, but not required.

Administer servers for software development

Open source software projects help the Wikimedia movement by improving core infrastructure (like MediaWiki), powering research and analytics, and supporting Wikimedia operations and software development. For advanced projects that aren't viable in the Toolforge environment, Cloud VPS (Virtual Private Server) provides the following features:

Free cloud computing environment, powered by OpenStack
Collaboratively-owned collections of virtual private servers, storage, firewall, and HTTPS proxy resources to projects
Access to a variety of data services
Freedom to install packages not provided by Debian or the Wikimedia Foundation

To use Cloud VPS, you need:

An open source project that isn't viable in the Toolforge environment, or can't be accomplished using other WMCS offerings
One or more active project maintainers who meet basic requirements
Advanced programming knowledge
Advanced experience with Unix command line
The ability to administer your own servers and manage your project's applications, data, runtime, middleware, and operating systems

To get started, visit Help:Cloud VPS.

What is the difference between Cloud VPS and Toolforge?

Cloud VPS is an Infrastructure as a service (IaaS) solution. It provides virtual machines, storage, firewall, and HTTPS proxy resources to projects. The members of each individual project are responsible for managing applications, data, runtime, middleware, and operating systems themselves. Cloud VPS projects use the domain wmcloud.org (some older projects use the legacy domain wmflabs.org).

Toolforge is a Platform as a service (PaaS) solution. It provides web servers, databases, and a distributed job processing system as managed services for tool maintainers. Toolforge tools use the domain toolforge.org.

Access databases and data dumps

What is a wiki replica? What's in the dumps? Learn the basics of Wikimedia open data and how to access all available datasets at Research:Data.

Access wiki databases for tool development

Tools and software hosted on Toolforge and Cloud VPS can directly access public data dumps and production wiki replicas.

Learn about accessing wiki replica databases from a tool account.
Learn about accessing dumps through shared storage services for Cloud VPS and Toolforge.

Query wiki replicas and dumps in a browser

PAWS provides a Jupyter notebook environment you can use to query wiki replicas and dumps, create interactive graphs, and use APIs for analysis.
Superset and Quarry are web interfaces for querying live replica SQL databases of public Wikimedia wikis.

To use Superset or Quarry you need only a Wikimedia login and a web browser, but you should have a basic understanding of SQL. Learn about SQL queries.

Get Exclusive Wikimedia Enterprise API Access

The Wikimedia Enterprise APIs are read-only APIs providing high-volume access to Wikimedia project data. Learn more on the Enterprise APIs page.

Following the Wikimedia Enterprise Principle of Access for All, Wikimedia community members get exclusive access to the same endpoints and features as paid accounts. From within PAWS, Cloud VPS, or Toolforge, make your calls to any endpoint described in the Enterprise API documentation without passing an authorization header. More ways of accessing the Wikimedia Enterprise APIs can be found at metawiki:Wikimedia Enterprise#Access.

Which service is right for you?


Activity / Needs	PAWS	Superset	Toolforge	Cloud VPS
Activity / Needs	Data as a service	Data as a service	Platform as a service	Infrastructure as a service
Write scripts and visualize data	✔
Write queries against wiki replica databases	✔	✔	✔	✔ via Toolforge
Access on wiki database dump files	✔		✔
Write and run bots	✔		✔	✔ if not viable on Toolforge
Run web services			✔	✔ if not viable on Toolforge
Build tools to improve Wikimedia projects			✔	✔ if not viable on Toolforge
Schedule or run continuous jobs			✔	✔ if not viable on Toolforge
Work with co-maintainers and co-admins			✔	✔
Need your own web domain				✔
Install apt packages not provided by Debian or the Wikimedia Foundation				✔
Administer your own virtual server				✔
Retrieve Wikimedia project data with high QPS using Wikimedia Enterprise APIs	✔		✔	✔
Platform / Environment	web browser	web browser	terminal	terminal
User knowledge	curious—advanced	curious—advanced	intermediate—advanced	advanced

Before you start

To use Cloud Services products, you must first create a Wikimedia account and a developer account.

Learn more

🎬 Video: Wikimedia Cloud Services introduction (2019)
📣 Slides: An introduction to Cloud Services presentation (2019)

Communication and support

Support and administration of the WMCS resources is provided by Wikimedia Foundation staff and Wikimedia movement volunteers. Please reach out with questions and join the conversation:

Discuss and receive general support

Chat in real time in the IRC channel #wikimedia-cloud ^connect or the bridged Telegram group
Discuss via email after you have subscribed to the cloud@ mailing list

Stay aware of critical changes and plans

Subscribe to the cloud-announce@ mailing list (all messages are also mirrored to the cloud@ list)
Read the News wiki page

Track work tasks and report bugs

Use a subproject of the #Cloud-Services Phabricator project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself

Read stories and WMCS blog posts

Read posts about Cloud Services on the Wikimedia Technical Blog. (Older posts can be read from the Cloud Services Blog.)

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Help:Cloud_Services_introduction&oldid=2400370"

Categories: