Backing up your knowledge base with the Zendesk API

Have more questions? Submit a request

78 Comments

  • Charles Nadeau

    Hi Jasper,


    Is it possible the restricted articles are set to another locale like en-uk instead of en-us, or some other locale? If so, you could modify the script to remove the locale filter, from:


    endpoint = zendesk + '/api/v2/help_center/{locale}/articles.json'.format(locale=language.lower())

    to:


    endpoint = zendesk + '/api/v2/help_center/articles.json'

     

    0
  • Jasper

    Hi Charles,


    The articles all have the local 'nl'.


    Nevertheless, I tried your solution to see if it would change anything. It didn't.


    This is an example of a URL of an article which is not included: /hc/nl/articles/208967385--Intern-Module-toevoegen-aan-praktijk 


     


     

    0
  • Charles Nadeau

    Hi Jasper,


    I'm not able to reproduce this on restricted content in my HC. The endpoint gets all articles regardless of access restriction or draft status. Maybe a pagination issue? You mention that the backup works partially for you. Does the number of files created total 30? That's the number of records per page returned by the API. Maybe the script is not fetching the next pages. Here are the two lines in the script that do that:


    while endpoint:
    ...
    endpoint = data['next_page']

    Another possibility is that you have a lot of articles and you're hitting the API rate limit. On the Essential plan, you can make up to 10 requests a minute, enough to back up 300 articles (10 page requests x 30 articles per page). On the Team plan, you can make 200 requests per minute, or enough for 6,000 articles.


    If these don't sound like the culprits, can post your code here (or on a file sharing site and share the link here) -- after stripping out your credentials? Thanks.

    0
  • Jasper

    Hi Charles,


    In the CSV file, I can see that 106 articles were backed up. So the limit should not be reached yet. 


    Therefore I'll post my code here. Thanks for your help!


    import os
    import datetime
    import csv

    import requests


    credentials = 'INSERT EMAIL HERE', 'INSERT PASSWORD HERE'
    session = requests.Session()
    session.auth = credentials

    zendesk = 'https://caresharingnetherlands.zendesk.com'
    language = 'nl'

    date = datetime.date.today()
    backup_path = os.path.join(str(date), language)
    if not os.path.exists(backup_path):
    os.makedirs(backup_path)

    log = []

    endpoint = zendesk + '/api/v2/help_center/{locale}/articles.json'.format(locale=language.lower())
    while endpoint:
    response = session.get(endpoint)
    if response.status_code != 200:
    print('Failed to retrieve articles with error {}'.format(response.status_code))
    exit()
    data = response.json()

    for article in data['articles']:
    title = '<h1>' + article['title'] + '</h1>'
    filename = '{id}.html'.format(id=article['id'])
    with open(os.path.join(backup_path, filename), mode='w', encoding='utf-8') as f:
    f.write(title + '\n' + article['body'])
    print('{title} copied!'.format(title=article['title']))

    log.append((filename, article['title'], article['author_id']))

    endpoint = data['next_page']

    with open(os.path.join(backup_path, '_log.csv'), mode='wt', encoding='utf-8') as f:
    writer = csv.writer(f)
    writer.writerow(('File', 'Title', 'Author ID'))
    for article in log:
    writer.writerow(article)

    0
  • Charles Nadeau

    Thanks, Jasper. I ran your version of the script with my Help Center and it successfully backed up all restricted content.


    In your case, only the articles from a specific section aren't being picked up, correct? That suggests there's something different with the section.


    If you can see the restricted articles in Help Center in a browser, and you use the same credentials to run the script that you use to sign in to Help Center, then the problem is not the access restrictions. The script "sees" the same articles you see in Help Center.


    Can you check the language setting of the category that contains the restricted section? (Browse to the category and click Edit Category on the toolbar.) The language of both the category and the section should be Dutch (nl).


     

    0
  • Charles Nadeau

    @Roland, @Adam, @David - 


    I added a section to the tutorial on restoring backed up articles. See Restoring articles above. Because it will necessarily overwrite Help Center content, please use it at your own risk.

    0
  • Russur

    Hi Charles,


     


    Your python script has helped me tremendously!! Thanks so much!


     


    Quick question, do you have some Python code that would allow me to go through a section, and edit Article attributes?


     


    It seems when zen desk updated the Help Center, it turned all of our articles to be able to be commented. I wanted the python script to go through all articles in a section, check if  disable comments is True, if not, then set it True.


    Would you have some skeleton python like that?


     


    Thanks,


     


     


    Russ

    0
  • Charles Nadeau

    Hi Russ,


    It could be fairly straightforward. Take a look at this Gist for one approach to the problem:


    https://gist.github.com/chucknado/8473cfbad848425e075ee4c0bfd66af4


    First, you get all the articles in the section.


    Second, you set up a for loop to check each one. If comments are already disabled, skip it. If not, fire off another API request to update the `comments_disabled` property.


     

    0
  • Russur

    Hi Charles,


     


    Thanks i will give this a try. 


     


    (I wasn't sure if i could in the inner loop do a put, while the outer loop was doing a get. I kinda thought so, but wasn't sure.)


     


    Thanks again!


     


    Russ


     

    0
  • Russur

    Hi Charles,


     


    Also along the lines of your restore script. I create my articles in Asciidoctor locally, and create html from that with the images also locally on my drive. When i have to create a new zen desk article i paste the html into the zen desk html window, but i still have to put the images in, one-by-one, manually into the article. Would it be possible to script the creation of a new article, with the images, into a zen desk article, all from the local html and image sources?


    (I'm just wondering if that could be done within the scope of the API?)


     


    Thanks,


     


     


    Russ

    0
  • Charles Nadeau

    You could use the Article Attachments API at https://developer.zendesk.com/rest_api/docs/help_center/article_attachments#create-article-attachment. 


    If you have a large knowledge base with lots of images, you might consider hosting the images on a separate server. You could then use a simple FTP client like Cyberduck to upload your images to the server. That's what the Docs team at Zendesk does to keep our sanity. We host thousands of images in several languages on an Amazon S3 file server from Amazon Web Services (I'm sure other options are available). Our articles link directly to them there.


    Charles


     

    0
  • Russur

    Hi Charles,


     


    I know that when i manually insert a graphic in Zen Desk Help Topics, under the covers it appears that a randomly named directory is created, and the graphic is placed there. Running the Article Attachments API, is the content_url that is returned, is that the new location of the graphic? If so, then, in the html , you could insert this url, and then the html and graphics would load seamlessly?


    So, if you could run a script to process all of your graphics as Article attachments, and then replace the local url, with the url the endpoint returns, that would work, right?


     


    btw, what does the Docs team do for their help? How have they streamlined their process, if you know?


     


    Thanks,


     


    Russ

    0
  • Charles Nadeau

    Hi Russ,


    The Zendesk Docs team doesn't use the API for managing images. We use it mostly for updating localized articles and moving content around from time to time. As explained above, because of the sheer number of images involved, we store them on a separate server and serve them from there. If you look at the src url of images in any of our articles, it'll look something like this:


    https://zen-marketing-documentation.s3.amazonaws.com/docs/en/monthly_usage_email_CSAT.png


    Off-loading the images to a dedicated file server is much easier to manage.


    Charles

    0
  • Tami Settergren

    Great article! As a tech writer, not a programmer, I really appreciate having such a useful script explained so clearly. It'll help me automate the updating of my translated articles.


    I've been trying to modify the restore script to create new articles in sections, but haven't made it work yet. I think the problem is with the endpoint, which I don't know how to modify to specify a section. Here's what I have so far (which gives an error 404):


    endpoint = '/api/v2/help_center/sections/{section}/articles/{id}/translations/{loc}.json'.format(id=article, section=section, loc=language.lower())


    Where {section} is defined as section = '115001738987' in the settings. I'm guessing about syntax of the endpoint...what am I doing wrong?


    An earlier comment refers to this article...:


    https://support.zendesk.com/hc/en-us/articles/203691406-Automating-your-first-localization-handoff-Help-Center-#post


    ...which might have my solution, but for some reason I'm not authorized to access that page.

    0
  • Charles Nadeau

    Thanks @Tami.


    Since you're creating articles, you'll want to use the Create Article endpoint:


    https://developer.zendesk.com/rest_api/docs/help_center/articles#create-article


    The translations endpoint in the restore script is generally used to update the content of an existing article in a specific language, or add a translation of an article.


    The correct url to the localization article is https://help.zendesk.com/hc/en-us/articles/229489108#post. I updated it in the comments above.


     


     

    1
  • Paul Lawrence

    Hi Charles,


    I have modified the script you provided in this article to do what I need it to do.


    I am having trouble retrieving the attachments. My initial test to retrieve attachments received this error for the response.status_code, "Failed to retrieve attachments with error 404".


    I did insert an IF statement for response.status_code == 404 to break so I can go to the next article, but now I fear I may be skipping attachments.


    Any suggestions?


    I can share the part of the script that is supposed to retrieve articles and their attachments.


    Paul


     

    0
  • Charles Nadeau

    Hi Paul,


    Unfortunately, I'm not very familiar with the Attachment API. A lot of people have trouble with it. It's on my task list to write a tutorial or article on how to use it.


    If you're trying to backup the images in your articles, another approach would be to scrape the articles for the images. See this comment in this thread:


    https://help.zendesk.com/hc/en-us/articles/229136947/comments/208775647


    Charles


     

    0
  • Alan Oehler

    Hi Charles,


    I was attempting to copy over images as you suggested in the comment at 


    https://help.zendesk.com/hc/en-us/articles/229136947/comments/208775647


    I'm getting "ModuleNotFoundError: No module named 'bs4'" when I run the script even though I used pip install to install the module and added the import statement documented in the article. Any tips about what might be wrong? 

    0
  • Alan Oehler

    Charles, problem solved. Not sure exactly what was wrong, but one of our developers suggested I try a virtual environment... so I did, and everything now works like a top!


     

    0
  • Nicole - Community Manager

    Glad to hear you got things sorted, Alan!

    0
  • Mike Maguire

    Hi Charles,

    I am pretty new to Python but was hoping you could help me out. I am getting stuck with having the date and language folders populate in my "backups" folder:[]. You'll see in the Terminal there's an error of "No such file or directory". Is this something you're familiar with?

    0
  • Joseph May

    Hi Mike-

    You are in the incorrect directory. Try

    $ cd Documents/Zendesk/backups

    then run it again.

    0
  • Mike Maguire

    Thanks for the response Joseph. I am still having a little bit of trouble. I submitted a Zendesk ticket and was hoping you could take a look!

    0
  • Charles Nadeau

    Hi Mike,

    Just FYI, I redacted the screenshot link in your comment. It showed sign-in credentials.

    Thanks.

    0
  • Joseph May

    Thanks Charles, I didn't even notice that.

    0
  • Mike Maguire

    Thank you both. I was able to backup all html files within their respective {locale} folders. As I'm reaching the bottom of the tutorial though, is there a way to import these to a new Zendesk account as that is the ultimate goal? I noticed this: "To create, it would need to be modified to use a different endpoint, as well as to specify a section and author for the article." Is there any documentation on this?

    0
  • Howie Paul

    Hi, 

    Long time listener, first time Python utiliser!

    Ive found this guide very handy and easy to follow, so thanks!

    Backup works lovely, but my purpose for this is so I can restore it to our sandbox.

    Youve provided a restore script, but only in the scenario where the files exist already. Can you help a guy out if they in fact dont exist?

     

    Thanks

    Howie

    0
  • Joseph May

    Hi Howie-

    The resource would have to be re-created in sandbox, as IDs would not carry over from your primary acct to sandbox. That said, I generally advise against using production data in a sandboxed environment. May I please ask what your goals are here?

    0
  • Howie Paul

    Hi joseph, thanks for your response.

     

    My goals for the use of the sandbox then:

    I have created a new theme for the Help Center which I do not want to make public, but I do want to sent it round to gauge peoples opinion. The best way I found to do this is by using the samdbox.

    Given the high level of customisation I employ I would need my documents imported too, in order to give the full feel of the new theme,

    Hope that makes sense.

    Howie

    0
  • Joseph May

    Hey Howie-

    Thanks for the explanation. The long and short of it is that all these resources will need to be recreated in the sandbox, generating their own unique IDs for Category, Section, Article, attachment, etc. Does this make sense? These values are not carried over from production.

    0

Please sign in to leave a comment.

Powered by Zendesk