{"id":759,"date":"2006-10-29T23:09:45","date_gmt":"2006-10-30T03:09:45","guid":{"rendered":"http:\/\/blogs.law.harvard.edu\/metasj\/2006\/10\/29\/wikipedia-content-sometimes-available\/"},"modified":"2006-10-29T23:09:45","modified_gmt":"2006-10-30T03:09:45","slug":"wikipedia-content-sometimes-available","status":"publish","type":"post","link":"https:\/\/archive.blogs.harvard.edu\/sj\/2006\/10\/29\/wikipedia-content-sometimes-available\/","title":{"rendered":"Wikipedia content *sometimes* available"},"content":{"rendered":"<p><a name='a1325'><\/a><\/p>\n<p>The last available dump of the entire English Wikipedia is 10 weeks old.&nbsp; Many dumps have been attempted since then, all have failed.&nbsp; There is currently no way for an individual to sign up for incremental updates from the site, so the only option for updating one&#8217;s local dump\/mirror\/archive\/testing-ground is to retrieve the entire thing from download.wikimedia.org.<\/p>\n<p>XML dumps are not available from Wikimedia servers; you have to get a massive tarball &#8212; even though many people and most researchers will convert this to xml before doing anything further with it.&nbsp; Other useful but unavailable dumps include: randomized subsets of en:wp and other languages, a dump of all commons images, a dump of all images used on en:wp or other language wp&#8217;s, whether on commons or not, a dump of&nbsp;<span style=\"font-weight: bold;\">any media uploaded in the past 11 months<\/span> (latest dump <a href=\"http:\/\/download.wikimedia.org\/images\/special\/commons\/\">here<\/a>).&nbsp; Worth noting: the <a href=\"http:\/\/download.wikimedia.org\/images\">images\/<\/a> subdirectory on download.wikimedia.org isn&#8217;t officially linked to from <a href=\"http:\/\/download.wikimedia.org\">anywhere<\/a>.<\/p>\n<p>There are limits to use of information.&nbsp; One is free copyright, one is free format, but before all of those is discovery and access to any version at all.&nbsp; I hope these oversights are remedied soon.<\/p>\n<p><a href='http:\/\/download.wikimedia.org\/enwiki\/20060816\/'>Wikipedia content *sometimes* available &#8230;<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The last available dump of the entire English Wikipedia is 10 weeks old.&nbsp; Many dumps have been attempted since then, all have failed.&nbsp; There is currently no way for an individual to sign up for incremental updates from the site, so the only option for updating one&#8217;s local dump\/mirror\/archive\/testing-ground is to retrieve the entire thing [&hellip;]<\/p>\n","protected":false},"author":135,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[218],"tags":[],"class_list":["post-759","post","type-post","status-publish","format-standard","hentry","category-not-so-popular"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7iVvB-cf","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/posts\/759","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/users\/135"}],"replies":[{"embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/comments?post=759"}],"version-history":[{"count":0,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/posts\/759\/revisions"}],"wp:attachment":[{"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/media?parent=759"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/categories?post=759"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/archive.blogs.harvard.edu\/sj\/wp-json\/wp\/v2\/tags?post=759"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}