[
  {
    "path": ".github/workflows/pythonpublish.yml",
    "content": "name: Upload Python Package\r\n\r\non:\r\n  release:\r\n    types: [created]\r\n\r\njobs:\r\n  deploy:\r\n    runs-on: ubuntu-latest\r\n    steps:\r\n    - uses: actions/checkout@v1\r\n    - name: Set up Python\r\n      uses: actions/setup-python@v1\r\n      with:\r\n        python-version: '3.x'\r\n    - name: Install dependencies\r\n      run: |\r\n        python -m pip install --upgrade pip\r\n        pip install setuptools wheel twine\r\n    - name: Build and publish\r\n      env:\r\n        TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}\r\n        TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}\r\n      run: |\r\n        python setup.py sdist bdist_wheel\r\n        twine upload dist/*\r\n"
  },
  {
    "path": ".gitignore",
    "content": "develop/\r\n.ipynb_checkpoints/\r\n.vscode/\r\n*egg-info/\r\n.idea/\r\nvenv2/\r\nbuild/\r\ndist/\r\n__pycache__/\r\ndata/"
  },
  {
    "path": "LICENSE",
    "content": "MIT License\r\n\r\nCopyright (c) 2021 tlyu0419\r\n\r\nPermission is hereby granted, free of charge, to any person obtaining a copy\r\nof this software and associated documentation files (the \"Software\"), to deal\r\nin the Software without restriction, including without limitation the rights\r\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\r\ncopies of the Software, and to permit persons to whom the Software is\r\nfurnished to do so, subject to the following conditions:\r\n\r\nThe above copyright notice and this permission notice shall be included in all\r\ncopies or substantial portions of the Software.\r\n\r\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\r\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\r\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\r\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\r\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\r\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\r\nSOFTWARE.\r\n"
  },
  {
    "path": "README.md",
    "content": "# Facebook_Crawler\n[![Downloads](https://pepy.tech/badge/facebook-crawler)](https://pepy.tech/project/facebook-crawler)\n[![Downloads](https://pepy.tech/badge/facebook-crawler/month)](https://pepy.tech/project/facebook-crawler)\n[![Downloads](https://pepy.tech/badge/facebook-crawler/week)](https://pepy.tech/project/facebook-crawler)\n\n## What's this?\n\nThis python package aims to help people who need to collect and analyze the public Fanspages or Groups data from Facebook with ease and efficiency. \n\nHere are the three big points of this project: \n1. Private: You don't need to log in to your account.\n2. Easy: Just key in the link of Fanspage or group and the target date, and it will work.\n3. Efficient: It collects the data through the requests package directly instead of opening another browser.\n\n\n這個 Python 套件旨在幫助使用者輕鬆且快速的收集 Facebook 公開粉絲頁和公開社團的資料，藉以進行後續的分析。\n\n以下是本專案的 3 個重點:\n1. 隱私: 不需要登入你個人的帳號密碼\n2. 簡單: 僅需輸入粉絲頁/社團的網址和停止的日期就可以開始執行程式\n3. 高效: 透過 requests 直接向伺服器請求資料，不需另外開啟一個新的瀏覽器\n\n## Quickstart\n### Install\n  ```pip\n  pip install -U facebook-crawler\n  ```\n\n### Usage\n- Facebook Fanspage \n  ```python\n  import facebook_crawler\n  pageurl= 'https://www.facebook.com/diudiu333'\n  facebook_crawler.Crawl_PagePosts(pageurl=pageurl, until_date='2021-01-01')\n  ```\n  ![quickstart_fanspage.png](https://raw.githubusercontent.com/TLYu0419/facebook_crawler/main/images/quickstart_fanspage.png)\n\n- Group\n  ```python\n  import facebook_crawler\n  groupurl = 'https://www.facebook.com/groups/pythontw'\n  facebook_crawler.Crawl_GroupPosts(groupurl, until_date='2021-01-01')\n  ```\n  ![quickstart_group.png](https://raw.githubusercontent.com/TLYu0419/facebook_crawler/main/images/quickstart_group.png)\n\n## FAQ\n- **How to get the comments or replies to the posts?**\n  > Please write an Email to me and tell me your project goal. Thanks!\n\n- **How can I find out the post's link through the data?**\n  > You can add the string 'https://www.facebook.com' in front of the POSTID, and it's just its post link. So, for example, if the POSTID is 123456789, and its link is 'https://www.facebook.com/12345679'.\n\n- **Can I directly collect the data in the specific time period?**\n  > No! This is the same as the behavior when we use Facebook. We need to collect the data from the newest posts to the older posts.\n\n## License\n[MIT License](https://github.com/TLYu0419/facebook_crawler/blob/main/LICENSE)\n\n## Contribution\n\n[![ecgo.png](https://raw.githubusercontent.com/TLYu0419/facebook_crawler/main/images/ecgo.png)](https://payment.ecpay.com.tw/QuickCollect/PayData?GcM4iJGUeCvhY%2fdFqqQ%2bFAyf3uA10KRo%2fqzP4DWtVcw%3d)\n\nA donation is not the limitation to utilizing this package, but it would be great to have your support. Either donate, star or fork are good methods to support me keep maintaining and developing this project.\n\nThanks to these donors' help, due to their kind help, this project could keep maintained and developed.\n\n**贊助不是使用這個套件的必要條件**，但如能獲得你的支持我將會非常感謝。不論是贊助、給予星星或分享都是很好的支持方式，幫助我繼續維護和開發這個專案\n\n由於這些捐助者的幫助，由於他們的慷慨的幫助，這個項目才得以持續維護和發展\n- Universities\n  - [Department of Social Work. The Chinese University of Hong Kong(香港中文大學社會工作學系)](https://web.swk.cuhk.edu.hk/zh-tw/)\n  - [Education, Graduate School of Curriculum and Instructional Communications Technology. National Taipei University.(國立台北教育大學課程與教學傳播科技研究所)](https://cict.ntue.edu.tw/?locale=zh_tw)\n  - [Department of Dusiness Administration. Chung Hua University.(中華大學企業管理學系)](https://ba.chu.edu.tw/?Lang=en)\n\n## Contact Info\n- Author: TENG-LIN YU\n- Email: tlyu0419@gmail.com\n- Facebook: https://www.facebook.com/tlyu0419\n- PYPI: https://pypi.org/project/facebook-crawler/\n- Github: https://github.com/TLYu0419/facebook_crawler\n\n## Log\n- 0.028: Modularized the crawler function.\n- 0.0.26\n  1. Auto changes the cookie after it's expired to keep crawling data without changing IP.\n"
  },
  {
    "path": "main.py",
    "content": "from paser import _parse_category, _parse_pagename, _parse_creation_time, _parse_pagetype, _parse_likes, _parse_docid, _parse_pageurl\r\nfrom paser import _parse_entryPoint, _parse_identifier, _parse_docid, _parse_composite_nojs, _parse_composite_graphql, _parse_relatedpages, _parse_pageinfo\r\nfrom requester import _get_homepage, _get_posts, _get_headers\r\nfrom utils import _init_request_vars\r\nfrom bs4 import BeautifulSoup\r\nimport os\r\nimport re\r\n\r\nimport json\r\nimport time\r\nimport tqdm\r\n\r\nimport pandas as pd\r\nimport pickle\r\n\r\nimport datetime\r\nimport warnings\r\nwarnings.filterwarnings(\"ignore\")\r\n\r\n\r\ndef Crawl_PagePosts(pageurl, until_date='2018-01-01', cursor=''):\r\n    # initial request variables\r\n    df, cursor, max_date, break_times = _init_request_vars(cursor)\r\n\r\n    # get headers\r\n    headers = _get_headers(pageurl)\r\n\r\n    # Get pageid, postid and entryPoint from homepage_response\r\n    homepage_response = _get_homepage(pageurl, headers)\r\n    entryPoint = _parse_entryPoint(homepage_response)\r\n    identifier = _parse_identifier(entryPoint, homepage_response)\r\n    docid = _parse_docid(entryPoint, homepage_response)\r\n\r\n    # Keep crawling post until reach the until_date\r\n    while max_date >= until_date:\r\n        try:\r\n            # Get posts by identifier, docid and entryPoint\r\n            resp = _get_posts(headers, identifier, entryPoint, docid, cursor)\r\n            if entryPoint == 'nojs':\r\n                ndf, max_date, cursor = _parse_composite_nojs(resp)\r\n                df.append(ndf)\r\n            else:\r\n                ndf, max_date, cursor = _parse_composite_graphql(resp)\r\n                df.append(ndf)\r\n            # Test\r\n            # print(ndf.shape[0])\r\n            break_times = 0\r\n        except:\r\n            # print(resp.json()[:3000])\r\n            try:\r\n                if resp.json()['data']['node']['timeline_feed_units']['page_info']['has_next_page'] == False:\r\n                    print('The posts of the page has run over!')\r\n                    break\r\n            except:\r\n                pass\r\n            print('Break Times {}: Something went wrong with this request. Sleep 20 seconds and send request again.'.format(\r\n                break_times))\r\n            print('REQUEST LOG >>  pageid: {}, docid: {}, cursor: {}'.format(\r\n                identifier, docid, cursor))\r\n            print('RESPONSE LOG: ', resp.text[:3000])\r\n            print('================================================')\r\n            break_times += 1\r\n\r\n            if break_times > 15:\r\n                print('Please check your target page/group has up to date.')\r\n                print('If so, you can ignore this break time message, if not, please change your Internet IP and run this crawler again.')\r\n                break\r\n\r\n            time.sleep(20)\r\n            # Get new headers\r\n            headers = _get_headers(pageurl)\r\n\r\n    # Concat all dataframes\r\n    df = pd.concat(df, ignore_index=True)\r\n    df['UPDATETIME'] = datetime.datetime.now().strftime(\"%Y-%m-%d %H:%M:%S\")\r\n    return df\r\n\r\n\r\ndef Crawl_GroupPosts(pageurl, until_date='2022-01-01'):\r\n    df = Crawl_PagePosts(pageurl, until_date)\r\n    return df\r\n\r\n\r\ndef Crawl_RelatedPages(seedpages, rounds):\r\n    # init\r\n    df = pd.DataFrame(data=[], columns=['SOURCE', 'TARGET', 'ROUND'])\r\n    pageurls = list(set(seedpages))\r\n    crawled_list = list(set(df['SOURCE']))\r\n    headers = _get_headers(pageurls[0])\r\n    for i in range(rounds):\r\n        print('Round {} started at: {}!'.format(\r\n            i, datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S')))\r\n        for pageurl in tqdm(pageurls):\r\n            if pageurl not in crawled_list:\r\n                try:\r\n                    homepage_response = _get_homepage(\r\n                        pageurl=pageurl, headers=headers)\r\n                    if 'Sorry, something went wrong.' not in homepage_response.text:\r\n                        entryPoint = _parse_entryPoint(homepage_response)\r\n                        identifier = _parse_identifier(\r\n                            entryPoint, homepage_response)\r\n                        relatedpages = _parse_relatedpages(\r\n                            homepage_response, entryPoint, identifier)\r\n                        ndf = pd.DataFrame({'SOURCE': homepage_response.url,\r\n                                            'TARGET': relatedpages,\r\n                                            'ROUND': i})\r\n                        df = pd.concat([df, ndf], ignore_index=True)\r\n                except:\r\n                    pass\r\n                    # print('ERROE: {}'.format(pageurl))\r\n        pageurls = list(set(df['TARGET']))\r\n        crawled_list = list(set(df['SOURCE']))\r\n    return df\r\n\r\n\r\ndef Crawl_PageInfo(pagenum, pageurl):\r\n    break_times = 0\r\n    global headers\r\n    while True:\r\n        try:\r\n            homepage_response = _get_homepage(pageurl, headers)\r\n            pageinfo = _parse_pageinfo(homepage_response)\r\n            with open('data/pageinfo/' + str(pagenum) + '.pickle', \"wb\") as fp:\r\n                pickle.dump(pageinfo, fp)\r\n            break\r\n        except:\r\n            break_times = break_times + 1\r\n            if break_times >= 5:\r\n                break\r\n            time.sleep(5)\r\n            headers = _get_headers(pageurl=pageurl)\r\n\r\n\r\nif __name__ == '__main__':\r\n\r\n    os.makedirs('data/', exist_ok=True)\r\n    # ===== fb_api_req_friendly_name: ProfileCometTimelineFeedRefetchQuery ====\r\n    pageurl = 'https://www.facebook.com/Gooaye'  # 股癌 Gooaye: 30.4萬追蹤,\r\n    pageurl = 'https://www.facebook.com/StockOldBull'  # 股海老牛: 16萬\r\n    pageurl = 'https://www.facebook.com/twherohan'\r\n    pageurl = 'https://www.facebook.com/diudiu333'\r\n    pageurl = 'https://www.facebook.com/chengwentsan'\r\n    pageurl = 'https://www.facebook.com/MaYingjeou'\r\n    pageurl = 'https://www.facebook.com/roberttawikofficial'\r\n    pageurl = 'https://www.facebook.com/NizamAbTitingan'\r\n    pageurl = 'https://www.facebook.com/joebiden'\r\n\r\n    # ==== fb_api_req_friendly_name: CometModernPageFeedPaginationQuery ====\r\n    pageurl = 'https://www.facebook.com/ebcmoney/'  # 東森財經: 81萬追蹤\r\n    pageurl = 'https://www.facebook.com/moneyweekly.tw/'  # 理財周刊: 36.3萬\r\n    pageurl = 'https://www.facebook.com/cmoneyapp/'  # CMoney 理財寶: 84.2萬\r\n    pageurl = 'https://www.facebook.com/emily0806'  # 艾蜜莉-自由之路: 20.9萬追蹤\r\n    pageurl = 'https://www.facebook.com/imoney889/'  # 林恩如-飆股女王: 10.2萬\r\n    pageurl = 'https://www.facebook.com/wealth1974/'  # 財訊: 17.5萬\r\n    pageurl = 'https://www.facebook.com/smart16888/'  # 郭莉芳理財講堂: 1.6萬\r\n    pageurl = 'https://www.facebook.com/smartmonthly/'  # Smart 智富月刊: 52.6萬\r\n    pageurl = 'https://www.facebook.com/ezmoney.tw/'  # 統一投信: 1.5萬\r\n    pageurl = 'https://www.facebook.com/MoneyMoneyMeg/'  # Money錢: 20.7萬\r\n    pageurl = 'https://www.facebook.com/imoneymagazine/'  # iMoney 智富雜誌: 38萬\r\n    pageurl = 'https://www.facebook.com/edigest/'  # 經濟一週 EDigest: 36.2萬\r\n    pageurl = 'https://www.facebook.com/BToday/'  # 今周刊:107萬\r\n    pageurl = 'https://www.facebook.com/GreenHornFans/'  # 綠角財經筆記: 25萬\r\n    pageurl = 'https://www.facebook.com/ec.ltn.tw/'  # 自由時報財經頻道 42,656人在追蹤\r\n    pageurl = 'https://www.facebook.com/MoneyDJ'  # MoneyDJ理財資訊 141,302人在追蹤\r\n    pageurl = 'https://www.facebook.com/YahooTWFinance/'  # Yahoo奇摩股市理財 149,624人在追蹤\r\n    pageurl = 'https://www.facebook.com/win3105'\r\n    pageurl = 'https://www.facebook.com/Diss%E7%BA%8F%E7%B6%BF-111182238148502/'\r\n\r\n    # fb_api_req_friendly_name: CometUFICommentsProviderQuery\r\n    pageurl = 'https://www.facebook.com/anuetw/'  # Anue鉅亨網財經新聞: 31.2萬追蹤\r\n    pageurl = 'https://www.facebook.com/wealtholic/'  # 投資癮 Wealtholic: 2.萬\r\n\r\n    # fb_api_req_friendly_name: PresenceStatusProviderSubscription_ContactProfilesQuery\r\n    # fb_api_req_friendly_name: GroupsCometFeedRegularStoriesPaginationQuery\r\n    pageurl = 'https://www.facebook.com/groups/pythontw'\r\n    pageurl = 'https://www.facebook.com/groups/corollacrossclub/'\r\n\r\n    df = Crawl_PagePosts(pageurl, until_date='2022-08-10')\r\n    # df = Crawl_RelatedPages(seedpages=pageurls, rounds=10)\r\n\r\n    df = pd.read_csv(\r\n        './data/relatedpages_edgetable.csv')[['SOURCE', 'TARGET', 'ROUND']]\r\n\r\n    headers = _get_headers(pageurl=pageurl)\r\n\r\n    for pagenum in tqdm(df['index']):\r\n        try:\r\n            Crawl_PageInfo(pagenum=pagenum, pageurl=df['pageurl'][pagenum])\r\n        except:\r\n            pass\r\n\r\n    homepage_response = _get_homepage(pageurl, headers)\r\n    pageinfo = _parse_pageinfo(homepage_response)\r\n\r\n    #\r\n    import pandas as pd\r\n    from main import Crawl_PagePosts\r\n    pageurl = 'https://www.facebook.com/hatendhu'\r\n    df = Crawl_PagePosts(pageurl, until_date='2014-11-01')\r\n    df\r\n    df.to_pickle('./data/20220926_hatendhu.pkl')\r\n"
  },
  {
    "path": "paser.py",
    "content": "import re\r\nimport json\r\nimport pandas as pd\r\nfrom bs4 import BeautifulSoup\r\nfrom utils import _extract_id, _init_request_vars\r\nimport datetime\r\nfrom requester import _get_pageabout, _get_pagetransparency, _get_homepage, _get_posts, _get_headers\r\nimport requests\r\n# Post-Paser\r\n\r\n\r\ndef _parse_edgelist(resp):\r\n    '''\r\n    Take edges from the response by graphql api\r\n    '''\r\n    edges = []\r\n    try:\r\n        edges = resp.json()['data']['node']['timeline_feed_units']['edges']\r\n    except:\r\n        for data in resp.text.split('\\r\\n', -1):\r\n            try:\r\n                edges.append(json.loads(data)[\r\n                             'data']['node']['timeline_list_feed_units']['edges'][0])\r\n            except:\r\n                edges.append(json.loads(data)['data'])\r\n    return edges\r\n\r\n\r\ndef _parse_edge(edge):\r\n    '''\r\n    Parse edge to take informations, such as post name, id, message..., etc. \r\n    '''\r\n    comet_sections = edge['node']['comet_sections']\r\n    # name\r\n    name = comet_sections['context_layout']['story']['comet_sections']['actor_photo']['story']['actors'][0]['name']\r\n\r\n    # creation_time\r\n    creation_time = comet_sections['context_layout']['story']['comet_sections']['metadata'][0]['story']['creation_time']\r\n\r\n    # message\r\n    try:\r\n        message = comet_sections['content']['story']['comet_sections']['message']['story']['message']['text']\r\n    except:\r\n        try:\r\n            message = comet_sections['content']['story']['comet_sections']['message_container']['story']['message']['text']\r\n        except:\r\n            message = comet_sections['content']['story']['comet_sections']['message_container']\r\n    # postid\r\n    postid = comet_sections['feedback']['story']['feedback_context'][\r\n        'feedback_target_with_context']['ufi_renderer']['feedback']['subscription_target_id']\r\n\r\n    # actorid\r\n    pageid = comet_sections['context_layout']['story']['comet_sections']['actor_photo']['story']['actors'][0]['id']\r\n\r\n    # comment_count\r\n    comment_count = comet_sections['feedback']['story']['feedback_context'][\r\n        'feedback_target_with_context']['ufi_renderer']['feedback']['comment_count']['total_count']\r\n\r\n    # reaction_count\r\n    reaction_count = comet_sections['feedback']['story']['feedback_context']['feedback_target_with_context'][\r\n        'ufi_renderer']['feedback']['comet_ufi_summary_and_actions_renderer']['feedback']['reaction_count']['count']\r\n\r\n    # share_count\r\n    share_count = comet_sections['feedback']['story']['feedback_context']['feedback_target_with_context'][\r\n        'ufi_renderer']['feedback']['comet_ufi_summary_and_actions_renderer']['feedback']['share_count']['count']\r\n\r\n    # toplevel_comment_count\r\n    toplevel_comment_count = comet_sections['feedback']['story']['feedback_context'][\r\n        'feedback_target_with_context']['ufi_renderer']['feedback']['toplevel_comment_count']['count']\r\n\r\n    # top_reactions\r\n    top_reactions = comet_sections['feedback']['story']['feedback_context']['feedback_target_with_context']['ufi_renderer'][\r\n        'feedback']['comet_ufi_summary_and_actions_renderer']['feedback']['cannot_see_top_custom_reactions']['top_reactions']['edges']\r\n\r\n    # comet_footer_renderer for link\r\n    try:\r\n        comet_footer_renderer = comet_sections['content']['story']['attachments'][0]['comet_footer_renderer']\r\n        # attachment_title\r\n        attachment_title = comet_footer_renderer['attachment']['title_with_entities']['text']\r\n        # attachment_description\r\n        attachment_description = comet_footer_renderer['attachment']['description']['text']\r\n    except:\r\n        attachment_title = ''\r\n        attachment_description = ''\r\n\r\n    # all_subattachments for photos\r\n    try:\r\n        try:\r\n            media = comet_sections['content']['story']['attachments'][0]['styles']['attachment']['all_subattachments']['nodes']\r\n            attachments_photos = ', '.join(\r\n                [image['media']['viewer_image']['uri'] for image in media])\r\n        except:\r\n            media = comet_sections['content']['story']['attachments'][0]['styles']['attachment']\r\n            attachments_photos = media['media']['photo_image']['uri']\r\n    except:\r\n        attachments_photos = ''\r\n\r\n    # cursor\r\n    cursor = edge['cursor']\r\n\r\n    # actor url\r\n    actor_url = comet_sections['context_layout']['story']['comet_sections']['actor_photo']['story']['actors'][0]['url']\r\n\r\n    # post url\r\n    post_url = comet_sections['content']['story']['wwwURL']\r\n\r\n    return [name, pageid, postid, creation_time, message, reaction_count, comment_count, toplevel_comment_count, share_count, top_reactions, attachment_title, attachment_description, attachments_photos, cursor, actor_url, post_url]\r\n\r\n\r\ndef _parse_domops(resp):\r\n    '''\r\n    Take name, data id, time , message and page link from domops\r\n    '''\r\n    data = re.sub(r'for \\(;;\\);', '', resp.text)\r\n    data = json.loads(data)\r\n    domops = data['domops'][0][3]['__html']\r\n    cursor = re.findall(\r\n        'timeline_cursor%22%3A%22(.*?)%22%2C%22timeline_section_cursor', domops)[0]\r\n    content_list = []\r\n    soup = BeautifulSoup(domops, 'lxml')\r\n\r\n    for content in soup.findAll('div', {'class': 'userContentWrapper'}):\r\n        # name\r\n        name = content.find('img')['aria-label']\r\n        # id\r\n        dataid = content.find('div', {'data-testid': 'story-subtitle'})['id']\r\n        # actorid\r\n        pageid = _extract_id(dataid, 0)\r\n        # postid\r\n        postid = _extract_id(dataid, 1)\r\n        # time\r\n        time = content.find('abbr')['data-utime']\r\n        # message\r\n        message = content.find('div', {'data-testid': 'post_message'})\r\n        if message == None:\r\n            message = ''\r\n        else:\r\n            if len(message.findAll('p')) >= 1:\r\n                message = ''.join(p.text for p in message.findAll('p'))\r\n            elif len(message.select('span > span')) >= 2:\r\n                message = message.find('span').text\r\n\r\n        # attachment_title\r\n        try:\r\n            attachment_title = content.find(\r\n                'a', {'data-lynx-mode': 'hover'})['aria-label']\r\n        except:\r\n            attachment_title = ''\r\n        # attachment_description\r\n        try:\r\n            attachment_description = content.find(\r\n                'a', {'data-lynx-mode': 'hover'}).text\r\n        except:\r\n            attachment_description = ''\r\n        # actor_url\r\n        actor_url = content.find('a')['href'].split('?')[0]\r\n\r\n        # post_url\r\n        post_url = 'https://www.facebook.com/' + postid\r\n        content_list.append([name, pageid, postid, time, message, attachment_title,\r\n                            attachment_description, cursor, actor_url, post_url])\r\n    return content_list, cursor\r\n\r\n\r\ndef _parse_jsmods(resp):\r\n    '''\r\n    Take postid, pageid, comment count , reaction count, sharecount, reactions and display_comments_count from jsmods\r\n    '''\r\n    data = re.sub(r'for \\(;;\\);', '', resp.text)\r\n    data = json.loads(data)\r\n    jsmods = data['jsmods']\r\n\r\n    requires_list = []\r\n    for requires in jsmods['pre_display_requires']:\r\n        try:\r\n            feedback = requires[3][1]['__bbox']['result']['data']['feedback']\r\n            # subscription_target_id ==> postid\r\n            subscription_target_id = feedback['subscription_target_id']\r\n            # owning_profile_id ==> pageid\r\n            owning_profile_id = feedback['owning_profile']['id']\r\n            # comment_count\r\n            comment_count = feedback['comment_count']['total_count']\r\n            # reaction_count\r\n            reaction_count = feedback['reaction_count']['count']\r\n            # share_count\r\n            share_count = feedback['share_count']['count']\r\n            # top_reactions\r\n            top_reactions = feedback['top_reactions']['edges']\r\n            # display_comments_count\r\n            display_comments_count = feedback['display_comments_count']['count']\r\n\r\n            # append data to list\r\n            requires_list.append([subscription_target_id, owning_profile_id, comment_count,\r\n                                 reaction_count, share_count, top_reactions, display_comments_count])\r\n        except:\r\n            pass\r\n\r\n    # reactions--video posts\r\n    for requires in jsmods['require']:\r\n        try:\r\n            # entidentifier ==> postid\r\n            entidentifier = requires[3][2]['feedbacktarget']['entidentifier']\r\n            # pageid\r\n            actorid = requires[3][2]['feedbacktarget']['actorid']\r\n            # comment count\r\n            commentcount = requires[3][2]['feedbacktarget']['commentcount']\r\n            # reaction count\r\n            likecount = requires[3][2]['feedbacktarget']['likecount']\r\n            # sharecount\r\n            sharecount = requires[3][2]['feedbacktarget']['sharecount']\r\n            # reactions\r\n            reactions = []\r\n            # display_comments_count\r\n            commentcount = requires[3][2]['feedbacktarget']['commentcount']\r\n\r\n            # append data to list\r\n            requires_list.append(\r\n                [entidentifier, actorid, commentcount, likecount, sharecount, reactions, commentcount])\r\n        except:\r\n            pass\r\n    return requires_list\r\n\r\n\r\ndef _parse_composite_graphql(resp):\r\n    edges = _parse_edgelist(resp)\r\n    df = []\r\n    for edge in edges:\r\n        try:\r\n            ndf = _parse_edge(edge)\r\n            df.append(ndf)\r\n        except:\r\n            pass\r\n    df = pd.DataFrame(df, columns=['NAME', 'PAGEID', 'POSTID', 'TIME', 'MESSAGE', 'REACTIONCOUNT', 'COMMENTCOUNT', 'DISPLAYCOMMENTCOUNT',\r\n                                   'SHARECOUNT', 'REACTIONS', 'ATTACHMENT_TITLE', 'ATTACHMENT_DESCRIPTION', 'ATTACHMENT_PHOTOS', 'CURSOR', 'ACTOR_URL', 'POST_URL'])\r\n    df = df[['NAME', 'PAGEID', 'POSTID', 'TIME', 'MESSAGE', 'ATTACHMENT_TITLE', 'ATTACHMENT_DESCRIPTION', 'ATTACHMENT_PHOTOS', 'REACTIONCOUNT',\r\n             'COMMENTCOUNT', 'DISPLAYCOMMENTCOUNT', 'SHARECOUNT', 'REACTIONS', 'CURSOR', 'ACTOR_URL', 'POST_URL']]\r\n    cursor = df['CURSOR'].to_list()[-1]\r\n    df['TIME'] = df['TIME'].apply(lambda x: datetime.datetime.fromtimestamp(\r\n        int(x)).strftime(\"%Y-%m-%d %H:%M:%S\"))\r\n    max_date = df['TIME'].max()\r\n    print('The maximum date of these posts is: {}, keep crawling...'.format(max_date))\r\n    return df, max_date, cursor\r\n\r\n\r\ndef _parse_composite_nojs(resp):\r\n    domops, cursor = _parse_domops(resp)\r\n    domops = pd.DataFrame(domops, columns=['NAME', 'PAGEID', 'POSTID', 'TIME', 'MESSAGE',\r\n                          'ATTACHMENT_TITLE', 'ATTACHMENT_DESCRIPTION', 'CURSOR', 'ACTOR_URL', 'POST_URL'])\r\n    domops['TIME'] = domops['TIME'].apply(\r\n        lambda x: datetime.datetime.fromtimestamp(int(x)).strftime(\"%Y-%m-%d %H:%M:%S\"))\r\n\r\n    jsmods = _parse_jsmods(resp)\r\n    jsmods = pd.DataFrame(jsmods, columns=[\r\n                          'POSTID', 'PAGEID', 'COMMENTCOUNT', 'REACTIONCOUNT', 'SHARECOUNT', 'REACTIONS', 'DISPLAYCOMMENTCOUNT'])\r\n\r\n    df = pd.merge(left=domops,\r\n                  right=jsmods,\r\n                  how='inner',\r\n                  on=['PAGEID', 'POSTID'])\r\n\r\n    df = df[['NAME', 'PAGEID', 'POSTID', 'TIME', 'MESSAGE', 'ATTACHMENT_TITLE', 'ATTACHMENT_DESCRIPTION',\r\n             'REACTIONCOUNT', 'COMMENTCOUNT', 'DISPLAYCOMMENTCOUNT', 'SHARECOUNT', 'REACTIONS', 'CURSOR',\r\n             'ACTOR_URL', 'POST_URL']]\r\n    max_date = df['TIME'].max()\r\n    print('The maximum date of these posts is: {}, keep crawling...'.format(max_date))\r\n    return df, max_date, cursor\r\n\r\n# Page paser\r\n\r\n\r\ndef _parse_pagetype(homepage_response):\r\n    if '/groups/' in homepage_response.url:\r\n        pagetype = 'Group'\r\n    else:\r\n        pagetype = 'Fanspage'\r\n    return pagetype\r\n\r\n\r\ndef _parse_pagename(homepage_response):\r\n    raw_json = homepage_response.text.encode('utf-8').decode('unicode_escape')\r\n    # pattern1\r\n    if len(re.findall(r'{\"page\":{\"name\":\"(.*?)\",', raw_json)) >= 1:\r\n        pagename = re.findall(r'{\"page\":{\"name\":\"(.*?)\",', raw_json)[0]\r\n        pagename = re.sub(r'\\s\\|\\sFacebook', '', pagename)\r\n        return pagename\r\n    # pattern2\r\n    if len(re.findall('\",\"name\":\"(.*?)\",\"', raw_json)) >= 1:\r\n        pagename = re.findall('\",\"name\":\"(.*?)\",\"', raw_json)[0]\r\n        pagename = re.sub(r'\\s\\|\\sFacebook', '', pagename)\r\n        return pagename\r\n\r\n\r\ndef _parse_entryPoint(homepage_response):\r\n    try:\r\n        entryPoint = re.findall(\r\n            '\"entryPoint\":{\"__dr\":\"(.*?)\"}}', homepage_response.text)[0]\r\n    except:\r\n        entryPoint = 'nojs'\r\n    return entryPoint\r\n\r\n\r\ndef _parse_identifier(entryPoint, homepage_response):\r\n    if entryPoint in ['ProfilePlusCometLoggedOutRouteRoot.entrypoint', 'CometGroupDiscussionRoot.entrypoint']:\r\n        # pattern 1\r\n        if len(re.findall('\"identifier\":\"{0,1}([0-9]{5,})\"{0,1},', homepage_response.text)) >= 1:\r\n            identifier = re.findall(\r\n                '\"identifier\":\"{0,1}([0-9]{5,})\"{0,1},', homepage_response.text)[0]\r\n\r\n        # pattern 2\r\n        elif len(re.findall('fb://profile/(.*?)\"', homepage_response.text)) >= 1:\r\n            identifier = re.findall(\r\n                'fb://profile/(.*?)\"', homepage_response.text)[0]\r\n\r\n        # pattern 3\r\n        elif len(re.findall('content=\"fb://group/([0-9]{1,})\" />', homepage_response.text)) >= 1:\r\n            identifier = re.findall(\r\n                'content=\"fb://group/([0-9]{1,})\" />', homepage_response.text)[0]\r\n\r\n    elif entryPoint in ['CometSinglePageHomeRoot.entrypoint', 'nojs']:\r\n        # pattern 1\r\n        if len(re.findall('\"pageID\":\"{0,1}([0-9]{5,})\"{0,1},', homepage_response.text)) >= 1:\r\n            identifier = re.findall(\r\n                '\"pageID\":\"{0,1}([0-9]{5,})\"{0,1},', homepage_response.text)[0]\r\n\r\n    return identifier\r\n\r\n\r\ndef _parse_docid(entryPoint, homepage_response):\r\n    soup = BeautifulSoup(homepage_response.text, 'lxml')\r\n    if entryPoint == 'nojs':\r\n        docid = 'NoDocid'\r\n    else:\r\n        for link in soup.findAll('link', {'rel': 'preload'}):\r\n            resp = requests.get(link['href'])\r\n            for line in resp.text.split('\\n', -1):\r\n                if 'ProfileCometTimelineFeedRefetchQuery_' in line:\r\n                    docid = re.findall('e.exports=\"([0-9]{1,})\"', line)[0]\r\n                    break\r\n\r\n                if 'CometModernPageFeedPaginationQuery_' in line:\r\n                    docid = re.findall('e.exports=\"([0-9]{1,})\"', line)[0]\r\n                    break\r\n\r\n                if 'CometUFICommentsProviderQuery_' in line:\r\n                    docid = re.findall('e.exports=\"([0-9]{1,})\"', line)[0]\r\n                    break\r\n\r\n                if 'GroupsCometFeedRegularStoriesPaginationQuery' in line:\r\n                    docid = re.findall('e.exports=\"([0-9]{1,})\"', line)[0]\r\n                    break\r\n            if 'docid' in locals():\r\n                break\r\n    return docid\r\n\r\n\r\ndef _parse_likes(homepage_response, entryPoint, headers):\r\n    if entryPoint in ['CometGroupDiscussionRoot.entrypoint']:\r\n        pageabout = _get_pageabout(homepage_response, entryPoint, headers)\r\n        members = re.findall(\r\n            ',\"group_total_members_info_text\":\"(.*?) total members\",\"', pageabout.text)[0]\r\n        members = re.sub(',', '', members)\r\n        return members\r\n    else:\r\n        # pattern 1\r\n        data = re.findall(\r\n            '\"page_likers\":{\"global_likers_count\":([0-9]{1,})},\"', homepage_response.text)\r\n        if len(data) >= 1:\r\n            likes = data[0]\r\n            return likes\r\n        # pattern 2\r\n        data = re.findall(\r\n            ' ([0-9]{0,},{0,}[0-9]{0,},{0,}[0-9]{0,},{0,}[0-9]{0,},{0,}[0-9]{0,},{0,}) likes', homepage_response.text)\r\n        if len(data) >= 1:\r\n            likes = data[0]\r\n            likes = re.sub(',', '', likes)\r\n            return likes\r\n\r\n\r\ndef _parse_creation_time(homepage_response, entryPoint, headers):\r\n    try:\r\n        if entryPoint in ['ProfilePlusCometLoggedOutRouteRoot.entrypoint']:\r\n            transparency_response = _get_pagetransparency(\r\n                homepage_response, entryPoint, headers)\r\n            transparency_info = re.findall(\r\n                '\"field_section_type\":\"transparency\",\"profile_fields\":{\"nodes\":\\[{\"title\":(.*?}),\"field_type\":\"creation_date\",', transparency_response.text)[0]\r\n            creation_time = json.loads(transparency_info)['text']\r\n\r\n        elif entryPoint in ['CometSinglePageHomeRoot.entrypoint']:\r\n            creation_time = re.findall(\r\n                ',\"page_creation_date\":{\"text\":\"Page created - (.*?)\"},', homepage_response.text)[0]\r\n\r\n        elif entryPoint in ['nojs']:\r\n            if len(re.findall('<span>Page created - (.*?)</span>', homepage_response.text)) >= 1:\r\n                creation_time = re.findall(\r\n                    '<span>Page created - (.*?)</span>', homepage_response.text)[0]\r\n            else:\r\n                creation_time = re.findall(\r\n                    ',\"foundingDate\":\"(.*?)\"}', homepage_response.text)[0][:10]\r\n\r\n        elif entryPoint in ['CometGroupDiscussionRoot.entrypoint']:\r\n            pageabout = _get_pageabout(homepage_response, entryPoint, headers)\r\n            creation_time = re.findall(\r\n                '\"group_history_summary\":{\"text\":\"Group created on (.*?)\"}},', pageabout.text)[0]\r\n\r\n        try:\r\n            creation_time = datetime.datetime.strptime(\r\n                creation_time, '%B %d, %Y')\r\n        except:\r\n            creation_time = creation_time + ', ' + datetime.datetime.now().year\r\n            creation_time = datetime.datetime.strptime(\r\n                creation_time, '%B %d, %Y')\r\n        creation_time = creation_time.strftime('%Y-%m-%d')\r\n    except:\r\n        creation_time = 'NotAvailable'\r\n    return creation_time\r\n\r\n\r\ndef _parse_category(homepage_response, entryPoint, headers):\r\n    pageabout = _get_pageabout(homepage_response, entryPoint, headers)\r\n    if entryPoint in ['ProfilePlusCometLoggedOutRouteRoot.entrypoint']:\r\n        if 'Page \\\\u00b7 Politician' in pageabout.text:\r\n            category = 'Politician'\r\n        if len(re.findall(r'\"text\":\"Page \\\\u00b7 (.*?)\"}', homepage_response.text)) >= 1:\r\n            category = re.findall(\r\n                r'\"text\":\"Page \\\\u00b7 (.*?)\"}', homepage_response.text)[0]\r\n        else:\r\n            soup = BeautifulSoup(pageabout.text)\r\n            for script in soup.findAll('script', {'type': 'application/ld+json'}):\r\n                if 'BreadcrumbList' in script.text:\r\n                    data = script.text.encode('utf-8').decode('unicode_escape')\r\n                    category = json.loads(data)['itemListElement']\r\n                    category = ' / '.join([cate['name'] for cate in category])\r\n    elif entryPoint in ['CometSinglePageHomeRoot.entrypoint', 'nojs']:\r\n        if len(re.findall('\",\"category_name\":\"(.*?)\",\"', homepage_response.text)) >= 1:\r\n            category = re.findall(\r\n                '\",\"category_name\":\"(.*?)\",\"', homepage_response.text)\r\n            category = ' / '.join([cate for cate in category])\r\n        else:\r\n            soup = BeautifulSoup(homepage_response.text)\r\n            if len(soup.findAll('span', {'itemprop': 'itemListElement'})) >= 1:\r\n                category = [span.text for span in soup.findAll(\r\n                    'span', {'itemprop': 'itemListElement'})]\r\n                category = ' / '.join(category)\r\n            else:\r\n                for script in soup.findAll('script', {'type': 'application/ld+json'}):\r\n                    if 'BreadcrumbList' in script.text:\r\n                        data = script.text.encode(\r\n                            'utf-8').decode('unicode_escape')\r\n                        category = json.loads(data)['itemListElement']\r\n                        category = ' / '.join([cate['name']\r\n                                              for cate in category])\r\n    elif entryPoint in ['PagesCometAdminSelfViewAboutContainerRoot.entrypoint']:\r\n        category = eval(re.findall(\r\n            '\"page_categories\":(.*?),\"addressEditable', homepage_response.text)[0])\r\n        category = ' / '.join([cate['text'] for cate in category])\r\n    elif entryPoint in ['CometGroupDiscussionRoot.entrypoint']:\r\n        category = 'Group'\r\n    try:\r\n        category = re.sub(r'\\\\/', '/', category)\r\n    except:\r\n        category = ''\r\n    return category\r\n\r\n\r\ndef _parse_pageurl(homepage_response):\r\n    pageurl = homepage_response.url\r\n    pageurl = re.sub('/$', '', pageurl)\r\n    return pageurl\r\n\r\n\r\ndef _parse_relatedpages(homepage_response, entryPoint, identifier):\r\n    relatedpages = []\r\n    if entryPoint in ['CometSinglePageHomeRoot.entrypoint']:\r\n        try:\r\n            data = re.findall(\r\n                r'\"related_pages\":\\[(.*?)\\],\"view_signature\"', homepage_response.text)[0]\r\n            data = re.sub('},{', '},,,,{', data)\r\n            for pages in data.split(',,,,', -1):\r\n                # print('id:', json.loads(pages)['id'])\r\n                # print('category_name:', json.loads(pages)['category_name'])\r\n                # print('name:', json.loads(pages)['name'])\r\n                url = json.loads(pages)['url']\r\n                url = url.split('?', -1)[0]\r\n                url = re.sub(r'/$', '', url)\r\n                # print('url:', url)\r\n                # print('========')\r\n                relatedpages.append(url)\r\n        except:\r\n            pass\r\n\r\n    elif entryPoint in ['nojs']:\r\n        soup = BeautifulSoup(homepage_response.text, 'lxml')\r\n        soup = soup.find(\r\n            'div', {'id': 'PageRelatedPagesSecondaryPagelet_{}'.format(identifier)})\r\n        for page in soup.select('ul > li > div'):\r\n            # print('name: ', page.find('img')['aria-label'])\r\n            url = page.find('a')['href']\r\n            url = url.split('?', -1)[0]\r\n            url = re.sub(r'/$', '', url)\r\n            # print('url:', url)\r\n            # print('===========')\r\n            relatedpages.append(url)\r\n\r\n    elif entryPoint in ['ProfilePlusCometLoggedOutRouteRoot.entrypoint', 'CometGroupDiscussionRoot.entrypoint']:\r\n        pass\r\n        # print('There\\'s no related pages recommend.')\r\n    return relatedpages\r\n\r\n\r\ndef _parse_pageinfo(homepage_response):\r\n    '''\r\n    Parse the homepage response to get the page information, including id, docid and api_name.\r\n    '''\r\n    # pagetype\r\n    pagetype = _parse_pagetype(homepage_response)\r\n\r\n    # pagename\r\n    pagename = _parse_pagename(homepage_response)\r\n\r\n    # entryPoint\r\n    entryPoint = _parse_entryPoint(homepage_response)\r\n\r\n    # identifier\r\n    identifier = _parse_identifier(entryPoint, homepage_response)\r\n\r\n    # docid\r\n    docid = _parse_docid(entryPoint, homepage_response)\r\n\r\n    # likes / members\r\n    likes = _parse_likes(homepage_response, entryPoint, headers)\r\n\r\n    # creation time\r\n    creation_time = _parse_creation_time(\r\n        homepage_response, entryPoint, headers)\r\n\r\n    # category\r\n    category = _parse_category(homepage_response, entryPoint, headers)\r\n\r\n    # pageurl\r\n    pageurl = _parse_pageurl(homepage_response)\r\n\r\n    return [pagetype, pagename, identifier, likes, creation_time, category, pageurl]\r\n\r\n\r\nif __name__ == '__main__':\r\n    # pageurls\r\n    pageurl = 'https://www.facebook.com/mohw.gov.tw'\r\n    pageurl = 'https://www.facebook.com/groups/pythontw'\r\n    pageurl = 'https://www.facebook.com/Gooaye'\r\n    pageurl = 'https://www.facebook.com/emily0806'\r\n    pageurl = 'https://www.facebook.com/anuetw/'\r\n    pageurl = 'https://www.facebook.com/wealtholic/'\r\n    pageurl = 'https://www.facebook.com/hatendhu'\r\n\r\n    headers = _get_headers(pageurl)\r\n    headers['Referer'] = 'https://www.facebook.com/hatendhu'\r\n    headers['Origin'] = 'https://www.facebook.com'\r\n    headers['Cookie'] = 'dpr=1.5; datr=rzIwY5yARwMzcR9H2GyqId_l'\r\n\r\n    homepage_response = _get_homepage(pageurl=pageurl, headers=headers)\r\n\r\n    entryPoint = _parse_entryPoint(homepage_response)\r\n    print(entryPoint)\r\n\r\n    identifier = _parse_identifier(entryPoint, homepage_response)\r\n\r\n    docid = _parse_docid(entryPoint, homepage_response)\r\n\r\n    df, cursor, max_date, break_times = _init_request_vars(cursor='')\r\n    cursor = 'AQHRlIMW9sczmHGnME47XeSdDNj6Jk9EcBOMlyxBdMNbZHM7dwd0rn8wsaxQxeXUsuhKVaMgVwPHb9YS9468INvb5yw2osoEmXd_sMXvj8rLhmBxeaJucMSPIDux_JuiHToC'\r\n    cursor = 'AQHRxSZTqUvlLpkXCnrOjdX0gZeyn-Q1cuJzn4SPJuZ5rkYi7nZFByE5pwy4AsBoUOtcmF28lNfXR_rqv7oO7545iURm_mx46aZLBDiYfPmgI2mjscHUTiVi5vv1vj5EXiF4'\r\n    resp = _get_posts(headers=headers, identifier=identifier,\r\n                      entryPoint=entryPoint, docid=docid, cursor=cursor)\r\n\r\n    # graphql\r\n    edges = _parse_edgelist(resp)\r\n    print(len(edges))\r\n    _parse_edge(edges[0])\r\n    edges[0].keys()\r\n    edges[0]['node'].keys()\r\n    edges[0]['node']['comet_sections'].keys()\r\n    edges[0]['node']['comet_sections']\r\n    df, max_date, cursor = _parse_composite_graphql(resp)\r\n    df\r\n    # nojs\r\n    content_list, cursor = _parse_domops(resp)\r\n\r\n    df, max_date, cursor = _parse_composite_nojs(resp)\r\n\r\n    # page paser\r\n\r\n    pagename = _parse_pagename(homepage_response).encode('utf-8').decode()\r\n    likes = _parse_likes(homepage_response, entryPoint, headers)\r\n    creation_time = _parse_creation_time(\r\n        homepage_response=homepage_response, entryPoint=entryPoint, headers=headers)\r\n    category = _parse_category(homepage_response, entryPoint, headers)\r\n    pageurl = _parse_pageurl(homepage_response)\r\n"
  },
  {
    "path": "requester.py",
    "content": "import re\r\nimport requests\r\nimport time\r\nfrom utils import _init_request_vars\r\n\r\n\r\ndef _get_headers(pageurl):\r\n    '''\r\n    Send a request to get cookieid as headers.\r\n    '''\r\n    pageurl = re.sub('www', 'm', pageurl)\r\n    resp = requests.get(pageurl)\r\n    headers = {'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',\r\n               'accept-language': 'en'}\r\n    headers['cookie'] = '; '.join(['{}={}'.format(cookieid, resp.cookies.get_dict()[\r\n                                  cookieid]) for cookieid in resp.cookies.get_dict()])\r\n    # headers['cookie'] = headers['cookie'] + '; locale=en_US'\r\n    return headers\r\n\r\n\r\ndef _get_homepage(pageurl, headers):\r\n    '''\r\n    Send a request to get the homepage response\r\n    '''\r\n    pageurl = re.sub('/$', '', pageurl)\r\n    timeout_cnt = 0\r\n    while True:\r\n        try:\r\n            homepage_response = requests.get(\r\n                pageurl, headers=headers, timeout=3)\r\n            return homepage_response\r\n        except:\r\n            time.sleep(5)\r\n            timeout_cnt = timeout_cnt + 1\r\n            if timeout_cnt > 20:\r\n                class homepage_response():\r\n                    text = 'Sorry, something went wrong.'\r\n                return homepage_response\r\n\r\n\r\ndef _get_pageabout(homepage_response, entryPoint, headers):\r\n    '''\r\n    Send a request to get the about page response\r\n    '''\r\n    pageurl = re.sub('/$', '', homepage_response.url)\r\n    pageabout = requests.get(pageurl + '/about', headers=headers)\r\n    return pageabout\r\n\r\n\r\ndef _get_pagetransparency(homepage_response, entryPoint, headers):\r\n    '''\r\n    Send a request to get the transparency page response\r\n    '''\r\n    pageurl = re.sub('/$', '', homepage_response.url)\r\n    if entryPoint in ['ProfilePlusCometLoggedOutRouteRoot.entrypoint']:\r\n        transparency_response = requests.get(\r\n            pageurl + '/about_profile_transparency', headers=headers)\r\n        return transparency_response\r\n\r\n\r\ndef _get_posts(headers, identifier, entryPoint, docid, cursor):\r\n    '''\r\n    Send a request to get new posts from fanspage/group.\r\n    '''\r\n    if entryPoint in ['nojs']:\r\n        params = {'page_id': identifier,\r\n                  'cursor': str({\"timeline_cursor\": cursor,\r\n                                 \"timeline_section_cursor\": '{}',\r\n                                 \"has_next_page\": 'true'}),\r\n                  'surface': 'www_pages_posts',\r\n                  'unit_count': 10,\r\n                  '__a': '1'}\r\n        resp = requests.get(url='https://www.facebook.com/pages_reaction_units/more/',\r\n                            params=params)\r\n\r\n    else:  # entryPoint in ['CometSinglePageHomeRoot.entrypoint', 'ProfilePlusCometLoggedOutRouteRoot.entrypoint', 'CometGroupDiscussionRoot.entrypoint']\r\n        data = {'variables': str({'cursor': cursor,\r\n                                  'id': identifier,\r\n                                  'count': 3}),\r\n                'doc_id': docid}\r\n        resp = requests.post(url='https://www.facebook.com/api/graphql/',\r\n                             data=data,\r\n                             headers=headers)\r\n    return resp\r\n\r\n\r\nif __name__ == '__main__':\r\n    pageurl = 'https://www.facebook.com/ec.ltn.tw/'\r\n    pageurl = 'https://www.facebook.com/Gooaye'\r\n    pageurl = 'https://www.facebook.com/groups/pythontw'\r\n    pageurl = 'https://www.facebook.com/hatendhu'\r\n    headers = _get_headers(pageurl)\r\n    homepage_response = _get_homepage(pageurl=pageurl, headers=headers)\r\n\r\n    df, cursor, max_date, break_times = _init_request_vars()\r\n    cursor = 'AQHRlIMW9sczmHGnME47XeSdDNj6Jk9EcBOMlyxBdMNbZHM7dwd0rn8wsaxQxeXUsuhKVaMgVwPHb9YS9468INvb5yw2osoEmXd_sMXvj8rLhmBxeaJucMSPIDux_JuiHToC'\r\n    cursor = 'AQHRixL5fPMA_nM-78jGg4LohG3M4a2-YQR6WSaWOTiqPRJ1dOGchYRzp1wdDtusNd-5FkCPXwByL_kZM2iyLIz1XHB8WIEzHYXTU3vQzviOI9GexNv__RPn1xnFJZddnjX3'\r\n\r\n    from paser import _parse_entryPoint, _parse_identifier, _parse_docid, _parse_composite_graphql\r\n    entryPoint = _parse_entryPoint(homepage_response)\r\n    identifier = _parse_identifier(entryPoint, homepage_response)\r\n    docid = _parse_docid(entryPoint, homepage_response)\r\n    df, cursor, max_date, break_times = _init_request_vars(cursor='')\r\n\r\n    resp = _get_posts(headers=headers, identifier=identifier,\r\n                      entryPoint=entryPoint, docid=docid, cursor=cursor)\r\n    ndf, max_date, cursor = _parse_composite_graphql(resp)\r\n    resp.json()\r\n    ndf\r\n    max_date\r\n    cursor\r\n    # print(len(resp.text))\r\n"
  },
  {
    "path": "requirements.txt",
    "content": "requests==2.24.0\r\nbs4==0.0.1\r\npandas==1.2.4         \r\nnumpy==1.20.3\r\ndicttoxml==1.7.4\r\nlxml==4.9.2"
  },
  {
    "path": "sample/20221013_Sample.ipynb",
    "content": "{\r\n \"cells\": [\r\n  {\r\n   \"cell_type\": \"code\",\r\n   \"execution_count\": 3,\r\n   \"id\": \"7946c9a9-0a27-4de2-a32e-8de574f8d7fb\",\r\n   \"metadata\": {\r\n    \"execution\": {\r\n     \"iopub.execute_input\": \"2022-10-13T14:01:50.402181Z\",\r\n     \"iopub.status.busy\": \"2022-10-13T14:01:50.402181Z\",\r\n     \"iopub.status.idle\": \"2022-10-13T14:02:32.391460Z\",\r\n     \"shell.execute_reply\": \"2022-10-13T14:02:32.391460Z\",\r\n     \"shell.execute_reply.started\": \"2022-10-13T14:01:50.402181Z\"\r\n    },\r\n    \"tags\": []\r\n   },\r\n   \"outputs\": [\r\n    {\r\n     \"name\": \"stdout\",\r\n     \"output_type\": \"stream\",\r\n     \"text\": [\r\n      \"The maximum date of these posts is: 2022-10-13 01:06:00, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-12 19:47:59, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-12 19:47:37, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-12 19:47:19, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-12 02:37:08, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-12 02:36:56, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-12 02:36:50, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-10 23:07:48, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-10 18:18:13, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-09 22:47:33, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-08 23:59:35, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-08 16:27:26, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-07 21:05:48, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-07 21:05:34, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-07 21:05:14, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-07 21:04:46, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-07 21:04:17, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-07 00:58:39, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-06 23:56:05, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-06 22:25:05, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-06 22:24:57, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-06 22:24:48, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-06 22:24:39, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-06 22:24:31, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-06 22:24:19, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-05 21:33:20, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-05 21:33:12, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-05 21:33:05, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-05 21:32:59, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-04 23:21:34, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-04 23:21:14, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-04 23:20:37, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-04 23:19:57, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-04 23:19:33, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-03 22:40:18, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-01 06:49:51, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-01 06:49:39, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-10-01 06:49:32, keep crawling...\\n\",\r\n      \"The maximum date of these posts is: 2022-09-30 00:28:37, keep crawling...\\n\"\r\n     ]\r\n    },\r\n    {\r\n     \"data\": {\r\n      \"text/html\": [\r\n       \"<div>\\n\",\r\n       \"<style scoped>\\n\",\r\n       \"    .dataframe tbody tr th:only-of-type {\\n\",\r\n       \"        vertical-align: middle;\\n\",\r\n       \"    }\\n\",\r\n       \"\\n\",\r\n       \"    .dataframe tbody tr th {\\n\",\r\n       \"        vertical-align: top;\\n\",\r\n       \"    }\\n\",\r\n       \"\\n\",\r\n       \"    .dataframe thead th {\\n\",\r\n       \"        text-align: right;\\n\",\r\n       \"    }\\n\",\r\n       \"</style>\\n\",\r\n       \"<table border=\\\"1\\\" class=\\\"dataframe\\\">\\n\",\r\n       \"  <thead>\\n\",\r\n       \"    <tr style=\\\"text-align: right;\\\">\\n\",\r\n       \"      <th></th>\\n\",\r\n       \"      <th>NAME</th>\\n\",\r\n       \"      <th>PAGEID</th>\\n\",\r\n       \"      <th>POSTID</th>\\n\",\r\n       \"      <th>TIME</th>\\n\",\r\n       \"      <th>MESSAGE</th>\\n\",\r\n       \"      <th>ATTACHMENT_TITLE</th>\\n\",\r\n       \"      <th>ATTACHMENT_DESCRIPTION</th>\\n\",\r\n       \"      <th>ATTACHMENT_PHOTOS</th>\\n\",\r\n       \"      <th>REACTIONCOUNT</th>\\n\",\r\n       \"      <th>COMMENTCOUNT</th>\\n\",\r\n       \"      <th>DISPLAYCOMMENTCOUNT</th>\\n\",\r\n       \"      <th>SHARECOUNT</th>\\n\",\r\n       \"      <th>REACTIONS</th>\\n\",\r\n       \"      <th>CURSOR</th>\\n\",\r\n       \"      <th>ACTOR_URL</th>\\n\",\r\n       \"      <th>POST_URL</th>\\n\",\r\n       \"      <th>UPDATETIME</th>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"  </thead>\\n\",\r\n       \"  <tbody>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>0</th>\\n\",\r\n       \"      <td>黑特東華 NDHU Hate</td>\\n\",\r\n       \"      <td>100064708507691</td>\\n\",\r\n       \"      <td>476738834493063</td>\\n\",\r\n       \"      <td>2022-10-13 01:06:00</td>\\n\",\r\n       \"      <td>#52635\\\\n\\\\n志學街一堆店家把東華學生當搖錢樹，大家應該聯合抵制一下，盤子店通通給他倒...</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>166</td>\\n\",\r\n       \"      <td>25</td>\\n\",\r\n       \"      <td>15</td>\\n\",\r\n       \"      <td>2</td>\\n\",\r\n       \"      <td>[{'reaction_count': 138, 'node': {'id': '16358...</td>\\n\",\r\n       \"      <td>AQHRWkzldCEuXabhI1tJPZnVEn7FKGxga7nPHhIBgGzMmB...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/people/%E9%BB%91%E7%8...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/permalink.php?story_f...</td>\\n\",\r\n       \"      <td>2022-10-13 22:02:32</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>1</th>\\n\",\r\n       \"      <td>黑特東華 NDHU Hate</td>\\n\",\r\n       \"      <td>100064708507691</td>\\n\",\r\n       \"      <td>476738777826402</td>\\n\",\r\n       \"      <td>2022-10-13 01:05:58</td>\\n\",\r\n       \"      <td>#52634\\\\n\\\\n電音社第二次社課\\\\n\\\\n提醒各位這禮拜日為電音社第二次上課，不管是沒有參...</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>https://scontent.ftpe10-1.fna.fbcdn.net/v/t39....</td>\\n\",\r\n       \"      <td>3</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>[{'reaction_count': 2, 'node': {'id': '1635855...</td>\\n\",\r\n       \"      <td>AQHRRN-x6Hvn-AZ35SOZ2CNtJpaM3_9yYj9j9dTx5LxCWy...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/people/%E9%BB%91%E7%8...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/permalink.php?story_f...</td>\\n\",\r\n       \"      <td>2022-10-13 22:02:32</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>2</th>\\n\",\r\n       \"      <td>黑特東華 NDHU Hate</td>\\n\",\r\n       \"      <td>100064708507691</td>\\n\",\r\n       \"      <td>476533761180237</td>\\n\",\r\n       \"      <td>2022-10-12 19:48:11</td>\\n\",\r\n       \"      <td>#52633\\\\n\\\\n10/12晚上7.左右在理工停車場靠學活那側撿到BKS1(安全帽藍芽耳機...</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>https://scontent.ftpe10-1.fna.fbcdn.net/v/t39....</td>\\n\",\r\n       \"      <td>3</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>[{'reaction_count': 3, 'node': {'id': '1635855...</td>\\n\",\r\n       \"      <td>AQHRD6AVoPeV1kHsvTjT0XPsC42w8-nrRawALu9lBRlhH5...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/people/%E9%BB%91%E7%8...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/permalink.php?story_f...</td>\\n\",\r\n       \"      <td>2022-10-13 22:02:32</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>3</th>\\n\",\r\n       \"      <td>黑特東華 NDHU Hate</td>\\n\",\r\n       \"      <td>100064708507691</td>\\n\",\r\n       \"      <td>476533567846923</td>\\n\",\r\n       \"      <td>2022-10-12 19:47:59</td>\\n\",\r\n       \"      <td>#52632\\\\n\\\\n#非黑特\\\\nThis is 頌啦！浪‧人‧鬆‧餅👏👏👏\\\\n東華巡迴場10...</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>https://scontent.ftpe10-1.fna.fbcdn.net/v/t39....</td>\\n\",\r\n       \"      <td>16</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>1</td>\\n\",\r\n       \"      <td>[{'reaction_count': 16, 'node': {'id': '163585...</td>\\n\",\r\n       \"      <td>AQHRMe5o8KJIMTVYawF1o8Vd0cJzDc1dF1eum3VLMIYTpL...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/people/%E9%BB%91%E7%8...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/permalink.php?story_f...</td>\\n\",\r\n       \"      <td>2022-10-13 22:02:32</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>4</th>\\n\",\r\n       \"      <td>黑特東華 NDHU Hate</td>\\n\",\r\n       \"      <td>100064708507691</td>\\n\",\r\n       \"      <td>476533317846948</td>\\n\",\r\n       \"      <td>2022-10-12 19:47:47</td>\\n\",\r\n       \"      <td>#52631\\\\n\\\\n想問下 禮拜一晚上有上拳擊課的學生 \\\\n因為連假所以補課 想問是什麼時候...</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>9</td>\\n\",\r\n       \"      <td>9</td>\\n\",\r\n       \"      <td>4</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>[{'reaction_count': 9, 'node': {'id': '1635855...</td>\\n\",\r\n       \"      <td>AQHRppObbMGsrzRdDW6mI7HAOUiedMntD77Xe_-pnyteE3...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/people/%E9%BB%91%E7%8...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/permalink.php?story_f...</td>\\n\",\r\n       \"      <td>2022-10-13 22:02:32</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>...</th>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>112</th>\\n\",\r\n       \"      <td>黑特東華 NDHU Hate</td>\\n\",\r\n       \"      <td>100064708507691</td>\\n\",\r\n       \"      <td>466889638811316</td>\\n\",\r\n       \"      <td>2022-10-01 06:49:29</td>\\n\",\r\n       \"      <td>#52524\\\\n\\\\n29號接近午夜時分在外環拉轉按喇叭，後來一路騎進學人宿舍往舊宿去、還繼續...</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>https://scontent.ftpe10-1.fna.fbcdn.net/v/t39....</td>\\n\",\r\n       \"      <td>47</td>\\n\",\r\n       \"      <td>5</td>\\n\",\r\n       \"      <td>2</td>\\n\",\r\n       \"      <td>1</td>\\n\",\r\n       \"      <td>[{'reaction_count': 34, 'node': {'id': '163585...</td>\\n\",\r\n       \"      <td>AQHRsblILurBMfS7TsLUVyW3GVjCkGIVSJLXMn3lb0_a0P...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/people/%E9%BB%91%E7%8...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/permalink.php?story_f...</td>\\n\",\r\n       \"      <td>2022-10-13 22:02:32</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>113</th>\\n\",\r\n       \"      <td>黑特東華 NDHU Hate</td>\\n\",\r\n       \"      <td>100064708507691</td>\\n\",\r\n       \"      <td>465849725581974</td>\\n\",\r\n       \"      <td>2022-09-30 00:28:39</td>\\n\",\r\n       \"      <td>#52523\\\\n\\\\n向晴裝有買車停在宿舍旁的車主可不可以管好自己的車，常常半夜一直逼逼逼逼讓...</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>6</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>[{'reaction_count': 6, 'node': {'id': '1635855...</td>\\n\",\r\n       \"      <td>AQHRXXwiAQgdKEmnvEo3mUXyzuH6f-wXNQDeyzxZr2kEky...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/people/%E9%BB%91%E7%8...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/permalink.php?story_f...</td>\\n\",\r\n       \"      <td>2022-10-13 22:02:32</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>114</th>\\n\",\r\n       \"      <td>黑特東華 NDHU Hate</td>\\n\",\r\n       \"      <td>100064708507691</td>\\n\",\r\n       \"      <td>465849705581976</td>\\n\",\r\n       \"      <td>2022-09-30 00:28:37</td>\\n\",\r\n       \"      <td>#52522\\\\n\\\\n宿舍都知道要公告說晚上十點過後要小聲要安靜\\\\n然後擷雲宿委還可以快十一點...</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>27</td>\\n\",\r\n       \"      <td>6</td>\\n\",\r\n       \"      <td>5</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>[{'reaction_count': 21, 'node': {'id': '163585...</td>\\n\",\r\n       \"      <td>AQHRB_nYO_BnR2qBBs4LXzM6CPeLfOlTMiCE3ZR6Ed9uZB...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/people/%E9%BB%91%E7%8...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/permalink.php?story_f...</td>\\n\",\r\n       \"      <td>2022-10-13 22:02:32</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>115</th>\\n\",\r\n       \"      <td>黑特東華 NDHU Hate</td>\\n\",\r\n       \"      <td>100064708507691</td>\\n\",\r\n       \"      <td>465849682248645</td>\\n\",\r\n       \"      <td>2022-09-30 00:28:35</td>\\n\",\r\n       \"      <td>#52521\\\\n\\\\n*非黑特*\\\\n誠徵*日領* 兼職、打工、工讀， 10/3(一)3位Pm1...</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>8</td>\\n\",\r\n       \"      <td>1</td>\\n\",\r\n       \"      <td>1</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>[{'reaction_count': 8, 'node': {'id': '1635855...</td>\\n\",\r\n       \"      <td>AQHRQ11A6xsDyh3EgTdpIoAWmxFJQkoSXUrlVoeHXSDylA...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/people/%E9%BB%91%E7%8...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/permalink.php?story_f...</td>\\n\",\r\n       \"      <td>2022-10-13 22:02:32</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>116</th>\\n\",\r\n       \"      <td>黑特東華 NDHU Hate</td>\\n\",\r\n       \"      <td>100064708507691</td>\\n\",\r\n       \"      <td>465849665581980</td>\\n\",\r\n       \"      <td>2022-09-30 00:28:33</td>\\n\",\r\n       \"      <td>#52520\\\\n\\\\n同學你的便當🍱留在統冠（志學）\\\\n由於一直等不到你來拿\\\\n先幫你冰起來\\\\...</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>https://scontent.ftpe10-1.fna.fbcdn.net/v/t39....</td>\\n\",\r\n       \"      <td>11</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>[{'reaction_count': 8, 'node': {'id': '1635855...</td>\\n\",\r\n       \"      <td>AQHR5AKAESdSG7AYj3DfxnH6Gb8piuyeh9hP-f4Y9IFOdv...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/people/%E9%BB%91%E7%8...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/permalink.php?story_f...</td>\\n\",\r\n       \"      <td>2022-10-13 22:02:32</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"  </tbody>\\n\",\r\n       \"</table>\\n\",\r\n       \"<p>117 rows × 17 columns</p>\\n\",\r\n       \"</div>\"\r\n      ],\r\n      \"text/plain\": [\r\n       \"               NAME           PAGEID           POSTID                 TIME  \\\\\\n\",\r\n       \"0    黑特東華 NDHU Hate  100064708507691  476738834493063  2022-10-13 01:06:00   \\n\",\r\n       \"1    黑特東華 NDHU Hate  100064708507691  476738777826402  2022-10-13 01:05:58   \\n\",\r\n       \"2    黑特東華 NDHU Hate  100064708507691  476533761180237  2022-10-12 19:48:11   \\n\",\r\n       \"3    黑特東華 NDHU Hate  100064708507691  476533567846923  2022-10-12 19:47:59   \\n\",\r\n       \"4    黑特東華 NDHU Hate  100064708507691  476533317846948  2022-10-12 19:47:47   \\n\",\r\n       \"..              ...              ...              ...                  ...   \\n\",\r\n       \"112  黑特東華 NDHU Hate  100064708507691  466889638811316  2022-10-01 06:49:29   \\n\",\r\n       \"113  黑特東華 NDHU Hate  100064708507691  465849725581974  2022-09-30 00:28:39   \\n\",\r\n       \"114  黑特東華 NDHU Hate  100064708507691  465849705581976  2022-09-30 00:28:37   \\n\",\r\n       \"115  黑特東華 NDHU Hate  100064708507691  465849682248645  2022-09-30 00:28:35   \\n\",\r\n       \"116  黑特東華 NDHU Hate  100064708507691  465849665581980  2022-09-30 00:28:33   \\n\",\r\n       \"\\n\",\r\n       \"                                               MESSAGE ATTACHMENT_TITLE  \\\\\\n\",\r\n       \"0    #52635\\\\n\\\\n志學街一堆店家把東華學生當搖錢樹，大家應該聯合抵制一下，盤子店通通給他倒...                    \\n\",\r\n       \"1    #52634\\\\n\\\\n電音社第二次社課\\\\n\\\\n提醒各位這禮拜日為電音社第二次上課，不管是沒有參...                    \\n\",\r\n       \"2    #52633\\\\n\\\\n10/12晚上7.左右在理工停車場靠學活那側撿到BKS1(安全帽藍芽耳機...                    \\n\",\r\n       \"3    #52632\\\\n\\\\n#非黑特\\\\nThis is 頌啦！浪‧人‧鬆‧餅👏👏👏\\\\n東華巡迴場10...                    \\n\",\r\n       \"4    #52631\\\\n\\\\n想問下 禮拜一晚上有上拳擊課的學生 \\\\n因為連假所以補課 想問是什麼時候...                    \\n\",\r\n       \"..                                                 ...              ...   \\n\",\r\n       \"112  #52524\\\\n\\\\n29號接近午夜時分在外環拉轉按喇叭，後來一路騎進學人宿舍往舊宿去、還繼續...                    \\n\",\r\n       \"113  #52523\\\\n\\\\n向晴裝有買車停在宿舍旁的車主可不可以管好自己的車，常常半夜一直逼逼逼逼讓...                    \\n\",\r\n       \"114  #52522\\\\n\\\\n宿舍都知道要公告說晚上十點過後要小聲要安靜\\\\n然後擷雲宿委還可以快十一點...                    \\n\",\r\n       \"115  #52521\\\\n\\\\n*非黑特*\\\\n誠徵*日領* 兼職、打工、工讀， 10/3(一)3位Pm1...                    \\n\",\r\n       \"116  #52520\\\\n\\\\n同學你的便當🍱留在統冠（志學）\\\\n由於一直等不到你來拿\\\\n先幫你冰起來\\\\...                    \\n\",\r\n       \"\\n\",\r\n       \"    ATTACHMENT_DESCRIPTION                                  ATTACHMENT_PHOTOS  \\\\\\n\",\r\n       \"0                                                                               \\n\",\r\n       \"1                           https://scontent.ftpe10-1.fna.fbcdn.net/v/t39....   \\n\",\r\n       \"2                           https://scontent.ftpe10-1.fna.fbcdn.net/v/t39....   \\n\",\r\n       \"3                           https://scontent.ftpe10-1.fna.fbcdn.net/v/t39....   \\n\",\r\n       \"4                                                                               \\n\",\r\n       \"..                     ...                                                ...   \\n\",\r\n       \"112                         https://scontent.ftpe10-1.fna.fbcdn.net/v/t39....   \\n\",\r\n       \"113                                                                             \\n\",\r\n       \"114                                                                             \\n\",\r\n       \"115                                                                             \\n\",\r\n       \"116                         https://scontent.ftpe10-1.fna.fbcdn.net/v/t39....   \\n\",\r\n       \"\\n\",\r\n       \"     REACTIONCOUNT  COMMENTCOUNT  DISPLAYCOMMENTCOUNT  SHARECOUNT  \\\\\\n\",\r\n       \"0              166            25                   15           2   \\n\",\r\n       \"1                3             0                    0           0   \\n\",\r\n       \"2                3             0                    0           0   \\n\",\r\n       \"3               16             0                    0           1   \\n\",\r\n       \"4                9             9                    4           0   \\n\",\r\n       \"..             ...           ...                  ...         ...   \\n\",\r\n       \"112             47             5                    2           1   \\n\",\r\n       \"113              6             0                    0           0   \\n\",\r\n       \"114             27             6                    5           0   \\n\",\r\n       \"115              8             1                    1           0   \\n\",\r\n       \"116             11             0                    0           0   \\n\",\r\n       \"\\n\",\r\n       \"                                             REACTIONS  \\\\\\n\",\r\n       \"0    [{'reaction_count': 138, 'node': {'id': '16358...   \\n\",\r\n       \"1    [{'reaction_count': 2, 'node': {'id': '1635855...   \\n\",\r\n       \"2    [{'reaction_count': 3, 'node': {'id': '1635855...   \\n\",\r\n       \"3    [{'reaction_count': 16, 'node': {'id': '163585...   \\n\",\r\n       \"4    [{'reaction_count': 9, 'node': {'id': '1635855...   \\n\",\r\n       \"..                                                 ...   \\n\",\r\n       \"112  [{'reaction_count': 34, 'node': {'id': '163585...   \\n\",\r\n       \"113  [{'reaction_count': 6, 'node': {'id': '1635855...   \\n\",\r\n       \"114  [{'reaction_count': 21, 'node': {'id': '163585...   \\n\",\r\n       \"115  [{'reaction_count': 8, 'node': {'id': '1635855...   \\n\",\r\n       \"116  [{'reaction_count': 8, 'node': {'id': '1635855...   \\n\",\r\n       \"\\n\",\r\n       \"                                                CURSOR  \\\\\\n\",\r\n       \"0    AQHRWkzldCEuXabhI1tJPZnVEn7FKGxga7nPHhIBgGzMmB...   \\n\",\r\n       \"1    AQHRRN-x6Hvn-AZ35SOZ2CNtJpaM3_9yYj9j9dTx5LxCWy...   \\n\",\r\n       \"2    AQHRD6AVoPeV1kHsvTjT0XPsC42w8-nrRawALu9lBRlhH5...   \\n\",\r\n       \"3    AQHRMe5o8KJIMTVYawF1o8Vd0cJzDc1dF1eum3VLMIYTpL...   \\n\",\r\n       \"4    AQHRppObbMGsrzRdDW6mI7HAOUiedMntD77Xe_-pnyteE3...   \\n\",\r\n       \"..                                                 ...   \\n\",\r\n       \"112  AQHRsblILurBMfS7TsLUVyW3GVjCkGIVSJLXMn3lb0_a0P...   \\n\",\r\n       \"113  AQHRXXwiAQgdKEmnvEo3mUXyzuH6f-wXNQDeyzxZr2kEky...   \\n\",\r\n       \"114  AQHRB_nYO_BnR2qBBs4LXzM6CPeLfOlTMiCE3ZR6Ed9uZB...   \\n\",\r\n       \"115  AQHRQ11A6xsDyh3EgTdpIoAWmxFJQkoSXUrlVoeHXSDylA...   \\n\",\r\n       \"116  AQHR5AKAESdSG7AYj3DfxnH6Gb8piuyeh9hP-f4Y9IFOdv...   \\n\",\r\n       \"\\n\",\r\n       \"                                             ACTOR_URL  \\\\\\n\",\r\n       \"0    https://www.facebook.com/people/%E9%BB%91%E7%8...   \\n\",\r\n       \"1    https://www.facebook.com/people/%E9%BB%91%E7%8...   \\n\",\r\n       \"2    https://www.facebook.com/people/%E9%BB%91%E7%8...   \\n\",\r\n       \"3    https://www.facebook.com/people/%E9%BB%91%E7%8...   \\n\",\r\n       \"4    https://www.facebook.com/people/%E9%BB%91%E7%8...   \\n\",\r\n       \"..                                                 ...   \\n\",\r\n       \"112  https://www.facebook.com/people/%E9%BB%91%E7%8...   \\n\",\r\n       \"113  https://www.facebook.com/people/%E9%BB%91%E7%8...   \\n\",\r\n       \"114  https://www.facebook.com/people/%E9%BB%91%E7%8...   \\n\",\r\n       \"115  https://www.facebook.com/people/%E9%BB%91%E7%8...   \\n\",\r\n       \"116  https://www.facebook.com/people/%E9%BB%91%E7%8...   \\n\",\r\n       \"\\n\",\r\n       \"                                              POST_URL           UPDATETIME  \\n\",\r\n       \"0    https://www.facebook.com/permalink.php?story_f...  2022-10-13 22:02:32  \\n\",\r\n       \"1    https://www.facebook.com/permalink.php?story_f...  2022-10-13 22:02:32  \\n\",\r\n       \"2    https://www.facebook.com/permalink.php?story_f...  2022-10-13 22:02:32  \\n\",\r\n       \"3    https://www.facebook.com/permalink.php?story_f...  2022-10-13 22:02:32  \\n\",\r\n       \"4    https://www.facebook.com/permalink.php?story_f...  2022-10-13 22:02:32  \\n\",\r\n       \"..                                                 ...                  ...  \\n\",\r\n       \"112  https://www.facebook.com/permalink.php?story_f...  2022-10-13 22:02:32  \\n\",\r\n       \"113  https://www.facebook.com/permalink.php?story_f...  2022-10-13 22:02:32  \\n\",\r\n       \"114  https://www.facebook.com/permalink.php?story_f...  2022-10-13 22:02:32  \\n\",\r\n       \"115  https://www.facebook.com/permalink.php?story_f...  2022-10-13 22:02:32  \\n\",\r\n       \"116  https://www.facebook.com/permalink.php?story_f...  2022-10-13 22:02:32  \\n\",\r\n       \"\\n\",\r\n       \"[117 rows x 17 columns]\"\r\n      ]\r\n     },\r\n     \"execution_count\": 3,\r\n     \"metadata\": {},\r\n     \"output_type\": \"execute_result\"\r\n    }\r\n   ],\r\n   \"source\": [\r\n    \"import pandas as pd\\n\",\r\n    \"from facebook_crawler import Crawl_PagePosts\\n\",\r\n    \"pageurl = 'https://www.facebook.com/hatendhu'\\n\",\r\n    \"df = Crawl_PagePosts(pageurl, until_date='2022-10-01')\\n\",\r\n    \"df\"\r\n   ]\r\n  },\r\n  {\r\n   \"cell_type\": \"code\",\r\n   \"execution_count\": 4,\r\n   \"id\": \"a7e18396-bfa1-4e9e-a507-162b2ae7b2d8\",\r\n   \"metadata\": {\r\n    \"execution\": {\r\n     \"iopub.execute_input\": \"2022-10-13T14:02:32.394376Z\",\r\n     \"iopub.status.busy\": \"2022-10-13T14:02:32.393376Z\",\r\n     \"iopub.status.idle\": \"2022-10-13T14:02:32.507986Z\",\r\n     \"shell.execute_reply\": \"2022-10-13T14:02:32.507986Z\",\r\n     \"shell.execute_reply.started\": \"2022-10-13T14:02:32.394376Z\"\r\n    }\r\n   },\r\n   \"outputs\": [],\r\n   \"source\": [\r\n    \"df.to_excel('./20221013_hatendhu.xlsx', index=False)\"\r\n   ]\r\n  },\r\n  {\r\n   \"cell_type\": \"code\",\r\n   \"execution_count\": null,\r\n   \"id\": \"63e56913-ba96-4a21-b548-25ce27095430\",\r\n   \"metadata\": {},\r\n   \"outputs\": [],\r\n   \"source\": []\r\n  }\r\n ],\r\n \"metadata\": {\r\n  \"kernelspec\": {\r\n   \"display_name\": \"Python 3 (ipykernel)\",\r\n   \"language\": \"python\",\r\n   \"name\": \"python3\"\r\n  },\r\n  \"language_info\": {\r\n   \"codemirror_mode\": {\r\n    \"name\": \"ipython\",\r\n    \"version\": 3\r\n   },\r\n   \"file_extension\": \".py\",\r\n   \"mimetype\": \"text/x-python\",\r\n   \"name\": \"python\",\r\n   \"nbconvert_exporter\": \"python\",\r\n   \"pygments_lexer\": \"ipython3\",\r\n   \"version\": \"3.10.5\"\r\n  }\r\n },\r\n \"nbformat\": 4,\r\n \"nbformat_minor\": 5\r\n}\r\n"
  },
  {
    "path": "sample/FansPages.ipynb",
    "content": "{\r\n \"cells\": [\r\n  {\r\n   \"cell_type\": \"code\",\r\n   \"execution_count\": 2,\r\n   \"id\": \"37db8006-7e78-4d00-bfd2-862997b366c9\",\r\n   \"metadata\": {},\r\n   \"outputs\": [\r\n    {\r\n     \"name\": \"stdout\",\r\n     \"output_type\": \"stream\",\r\n     \"text\": [\r\n      \"Collecting facebook_crawler\\n\",\r\n      \"  Using cached facebook_crawler-0.0.25-py3-none-any.whl (7.0 kB)\\n\",\r\n      \"Requirement already satisfied: pandas in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from facebook_crawler) (1.3.4)\\n\",\r\n      \"Requirement already satisfied: bs4 in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from facebook_crawler) (0.0.1)\\n\",\r\n      \"Requirement already satisfied: lxml in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from facebook_crawler) (4.6.4)\\n\",\r\n      \"Requirement already satisfied: requests in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from facebook_crawler) (2.26.0)\\n\",\r\n      \"Requirement already satisfied: numpy in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from facebook_crawler) (1.21.4)\\n\",\r\n      \"Requirement already satisfied: pytz>=2017.3 in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from pandas->facebook_crawler) (2021.3)\\n\",\r\n      \"Requirement already satisfied: python-dateutil>=2.7.3 in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from pandas->facebook_crawler) (2.8.2)\\n\",\r\n      \"Requirement already satisfied: beautifulsoup4 in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from bs4->facebook_crawler) (4.10.0)\\n\",\r\n      \"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from requests->facebook_crawler) (1.26.7)\\n\",\r\n      \"Requirement already satisfied: idna<4,>=2.5; python_version >= \\\"3\\\" in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from requests->facebook_crawler) (3.3)\\n\",\r\n      \"Requirement already satisfied: certifi>=2017.4.17 in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from requests->facebook_crawler) (2021.10.8)\\n\",\r\n      \"Requirement already satisfied: charset-normalizer~=2.0.0; python_version >= \\\"3\\\" in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from requests->facebook_crawler) (2.0.9)\\n\",\r\n      \"Requirement already satisfied: six>=1.5 in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas->facebook_crawler) (1.16.0)\\n\",\r\n      \"Requirement already satisfied: soupsieve>1.2 in /home/tlyu0419/github/envs/env/lib/python3.8/site-packages (from beautifulsoup4->bs4->facebook_crawler) (2.3.1)\\n\",\r\n      \"Installing collected packages: facebook-crawler\\n\",\r\n      \"Successfully installed facebook-crawler-0.0.25\\n\"\r\n     ]\r\n    }\r\n   ],\r\n   \"source\": [\r\n    \"!pip install facebook_crawler\"\r\n   ]\r\n  },\r\n  {\r\n   \"cell_type\": \"code\",\r\n   \"execution_count\": 3,\r\n   \"id\": \"f421b324-f6a8-479a-8b63-c80d6543e12f\",\r\n   \"metadata\": {},\r\n   \"outputs\": [\r\n    {\r\n     \"name\": \"stdout\",\r\n     \"output_type\": \"stream\",\r\n     \"text\": [\r\n      \"TimeStamp: 2021-12-03.\\n\",\r\n      \"TimeStamp: 2021-08-03.\\n\",\r\n      \"TimeStamp: 2021-01-16.\\n\",\r\n      \"TimeStamp: 2020-09-09.\\n\"\r\n     ]\r\n    },\r\n    {\r\n     \"data\": {\r\n      \"text/html\": [\r\n       \"<div>\\n\",\r\n       \"<style scoped>\\n\",\r\n       \"    .dataframe tbody tr th:only-of-type {\\n\",\r\n       \"        vertical-align: middle;\\n\",\r\n       \"    }\\n\",\r\n       \"\\n\",\r\n       \"    .dataframe tbody tr th {\\n\",\r\n       \"        vertical-align: top;\\n\",\r\n       \"    }\\n\",\r\n       \"\\n\",\r\n       \"    .dataframe thead th {\\n\",\r\n       \"        text-align: right;\\n\",\r\n       \"    }\\n\",\r\n       \"</style>\\n\",\r\n       \"<table border=\\\"1\\\" class=\\\"dataframe\\\">\\n\",\r\n       \"  <thead>\\n\",\r\n       \"    <tr style=\\\"text-align: right;\\\">\\n\",\r\n       \"      <th></th>\\n\",\r\n       \"      <th>NAME</th>\\n\",\r\n       \"      <th>TIME</th>\\n\",\r\n       \"      <th>MESSAGE</th>\\n\",\r\n       \"      <th>LINK</th>\\n\",\r\n       \"      <th>PAGEID</th>\\n\",\r\n       \"      <th>POSTID</th>\\n\",\r\n       \"      <th>COMMENTCOUNT</th>\\n\",\r\n       \"      <th>REACTIONCOUNT</th>\\n\",\r\n       \"      <th>SHARECOUNT</th>\\n\",\r\n       \"      <th>DISPLAYCOMMENTCOUNT</th>\\n\",\r\n       \"      <th>ANGER</th>\\n\",\r\n       \"      <th>HAHA</th>\\n\",\r\n       \"      <th>LIKE</th>\\n\",\r\n       \"      <th>LOVE</th>\\n\",\r\n       \"      <th>SORRY</th>\\n\",\r\n       \"      <th>SUPPORT</th>\\n\",\r\n       \"      <th>WOW</th>\\n\",\r\n       \"      <th>UPDATETIME</th>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"  </thead>\\n\",\r\n       \"  <tbody>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>0</th>\\n\",\r\n       \"      <td>丟丟妹</td>\\n\",\r\n       \"      <td>2021-12-03 02:36:02</td>\\n\",\r\n       \"      <td>怎麼分不清 到底是誰帶壞誰XD 丟丟妹 和 Alizabeth 娘娘 根本姐妹淘🤣 互相「曉...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/diudiu333</td>\\n\",\r\n       \"      <td>1723714034327589</td>\\n\",\r\n       \"      <td>4964611303571163</td>\\n\",\r\n       \"      <td>704.0</td>\\n\",\r\n       \"      <td>21797.0</td>\\n\",\r\n       \"      <td>546.0</td>\\n\",\r\n       \"      <td>572.0</td>\\n\",\r\n       \"      <td>6.0</td>\\n\",\r\n       \"      <td>5798.0</td>\\n\",\r\n       \"      <td>15841.0</td>\\n\",\r\n       \"      <td>108.0</td>\\n\",\r\n       \"      <td>6.0</td>\\n\",\r\n       \"      <td>17.0</td>\\n\",\r\n       \"      <td>21.0</td>\\n\",\r\n       \"      <td>2022-01-03 12:42:18</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>1</th>\\n\",\r\n       \"      <td>丟丟妹</td>\\n\",\r\n       \"      <td>2021-11-28 21:11:13</td>\\n\",\r\n       \"      <td>我喜歡這樣的陽光☀️ 開放留言 想要丟丟賣什麼？ 遇到瓶頸了⋯⋯⋯😢 ❤️ #留起來</td>\\n\",\r\n       \"      <td>https://www.facebook.com/diudiu333</td>\\n\",\r\n       \"      <td>1723714034327589</td>\\n\",\r\n       \"      <td>4949186545113639</td>\\n\",\r\n       \"      <td>1675.0</td>\\n\",\r\n       \"      <td>12674.0</td>\\n\",\r\n       \"      <td>22.0</td>\\n\",\r\n       \"      <td>1142.0</td>\\n\",\r\n       \"      <td>1.0</td>\\n\",\r\n       \"      <td>7.0</td>\\n\",\r\n       \"      <td>12510.0</td>\\n\",\r\n       \"      <td>125.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>25.0</td>\\n\",\r\n       \"      <td>6.0</td>\\n\",\r\n       \"      <td>2022-01-03 12:42:18</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>2</th>\\n\",\r\n       \"      <td>丟丟妹</td>\\n\",\r\n       \"      <td>2021-11-19 18:02:56</td>\\n\",\r\n       \"      <td>人客啊 客倌們😆歡迎留言+1 2A頂級大閘蟹 （含繩6兩） 特價3088免運 （6隻一箱）最...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/diudiu333</td>\\n\",\r\n       \"      <td>1723714034327589</td>\\n\",\r\n       \"      <td>4918431521522475</td>\\n\",\r\n       \"      <td>719.0</td>\\n\",\r\n       \"      <td>32242.0</td>\\n\",\r\n       \"      <td>65.0</td>\\n\",\r\n       \"      <td>543.0</td>\\n\",\r\n       \"      <td>1.0</td>\\n\",\r\n       \"      <td>31.0</td>\\n\",\r\n       \"      <td>31884.0</td>\\n\",\r\n       \"      <td>290.0</td>\\n\",\r\n       \"      <td>3.0</td>\\n\",\r\n       \"      <td>18.0</td>\\n\",\r\n       \"      <td>15.0</td>\\n\",\r\n       \"      <td>2022-01-03 12:42:18</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>3</th>\\n\",\r\n       \"      <td>丟丟妹</td>\\n\",\r\n       \"      <td>2021-11-12 22:35:08</td>\\n\",\r\n       \"      <td>丟丟妹 終於不用離婚🤣 這次直播半路認走 #乾哥哥：#郭丟丟！ 叫買能力根本 #隔空遺傳？！...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/diudiu333</td>\\n\",\r\n       \"      <td>1723714034327589</td>\\n\",\r\n       \"      <td>4894343770597917</td>\\n\",\r\n       \"      <td>477.0</td>\\n\",\r\n       \"      <td>18072.0</td>\\n\",\r\n       \"      <td>656.0</td>\\n\",\r\n       \"      <td>431.0</td>\\n\",\r\n       \"      <td>23.0</td>\\n\",\r\n       \"      <td>3495.0</td>\\n\",\r\n       \"      <td>14351.0</td>\\n\",\r\n       \"      <td>125.0</td>\\n\",\r\n       \"      <td>17.0</td>\\n\",\r\n       \"      <td>29.0</td>\\n\",\r\n       \"      <td>32.0</td>\\n\",\r\n       \"      <td>2022-01-03 12:42:18</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>4</th>\\n\",\r\n       \"      <td>丟丟妹</td>\\n\",\r\n       \"      <td>2021-11-10 21:00:52</td>\\n\",\r\n       \"      <td>各位親愛的帥哥美女們  由於訊息無限爆炸中 （已經快馬加鞭） 有留訊息就是 已接結單唷 造成...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/diudiu333</td>\\n\",\r\n       \"      <td>1723714034327589</td>\\n\",\r\n       \"      <td>4887230927975868</td>\\n\",\r\n       \"      <td>352.0</td>\\n\",\r\n       \"      <td>9598.0</td>\\n\",\r\n       \"      <td>17.0</td>\\n\",\r\n       \"      <td>220.0</td>\\n\",\r\n       \"      <td>1.0</td>\\n\",\r\n       \"      <td>19.0</td>\\n\",\r\n       \"      <td>9476.0</td>\\n\",\r\n       \"      <td>59.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>34.0</td>\\n\",\r\n       \"      <td>9.0</td>\\n\",\r\n       \"      <td>2022-01-03 12:42:18</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>...</th>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>75</th>\\n\",\r\n       \"      <td>丟丟妹</td>\\n\",\r\n       \"      <td>2020-05-03 16:52:40</td>\\n\",\r\n       \"      <td>猜猜我在哪裡  陳冠霖牛仔部落格 連靜雯joanne lien</td>\\n\",\r\n       \"      <td>https://www.facebook.com/diudiu333</td>\\n\",\r\n       \"      <td>1723714034327589</td>\\n\",\r\n       \"      <td>184909892573719</td>\\n\",\r\n       \"      <td>2921.0</td>\\n\",\r\n       \"      <td>12355.0</td>\\n\",\r\n       \"      <td>851.0</td>\\n\",\r\n       \"      <td>2921.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>2022-01-03 12:42:18</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>76</th>\\n\",\r\n       \"      <td>丟丟妹</td>\\n\",\r\n       \"      <td>2020-04-30 13:36:40</td>\\n\",\r\n       \"      <td>踢爆娃娃機...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/diudiu333</td>\\n\",\r\n       \"      <td>1723714034327589</td>\\n\",\r\n       \"      <td>3173673282664983</td>\\n\",\r\n       \"      <td>946.0</td>\\n\",\r\n       \"      <td>22357.0</td>\\n\",\r\n       \"      <td>1035.0</td>\\n\",\r\n       \"      <td>856.0</td>\\n\",\r\n       \"      <td>76.0</td>\\n\",\r\n       \"      <td>2661.0</td>\\n\",\r\n       \"      <td>19275.0</td>\\n\",\r\n       \"      <td>161.0</td>\\n\",\r\n       \"      <td>25.0</td>\\n\",\r\n       \"      <td>71.0</td>\\n\",\r\n       \"      <td>88.0</td>\\n\",\r\n       \"      <td>2022-01-03 12:42:18</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>77</th>\\n\",\r\n       \"      <td>丟丟妹</td>\\n\",\r\n       \"      <td>2020-04-29 21:51:13</td>\\n\",\r\n       \"      <td>今日直播暫停乙次 丟丟身體不舒服了😭  明天下午見 開播 要想我喲  嗚嗚😢</td>\\n\",\r\n       \"      <td>https://www.facebook.com/diudiu333</td>\\n\",\r\n       \"      <td>1723714034327589</td>\\n\",\r\n       \"      <td>3171969702835341</td>\\n\",\r\n       \"      <td>167.0</td>\\n\",\r\n       \"      <td>4123.0</td>\\n\",\r\n       \"      <td>8.0</td>\\n\",\r\n       \"      <td>164.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>3.0</td>\\n\",\r\n       \"      <td>3992.0</td>\\n\",\r\n       \"      <td>96.0</td>\\n\",\r\n       \"      <td>7.0</td>\\n\",\r\n       \"      <td>5.0</td>\\n\",\r\n       \"      <td>20.0</td>\\n\",\r\n       \"      <td>2022-01-03 12:42:18</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>78</th>\\n\",\r\n       \"      <td>丟丟妹</td>\\n\",\r\n       \"      <td>2020-04-17 19:46:50</td>\\n\",\r\n       \"      <td>帥哥美女今天星期五，丟妹偷懶去 明天下午二點 準時見❤️❤️😄有人會報到嗎 刷起來 😂</td>\\n\",\r\n       \"      <td>https://www.facebook.com/diudiu333</td>\\n\",\r\n       \"      <td>1723714034327589</td>\\n\",\r\n       \"      <td>3141849955847316</td>\\n\",\r\n       \"      <td>116.0</td>\\n\",\r\n       \"      <td>4804.0</td>\\n\",\r\n       \"      <td>20.0</td>\\n\",\r\n       \"      <td>114.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>6.0</td>\\n\",\r\n       \"      <td>4744.0</td>\\n\",\r\n       \"      <td>48.0</td>\\n\",\r\n       \"      <td>1.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>5.0</td>\\n\",\r\n       \"      <td>2022-01-03 12:42:18</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>79</th>\\n\",\r\n       \"      <td>丟丟妹</td>\\n\",\r\n       \"      <td>2020-04-15 20:48:08</td>\\n\",\r\n       \"      <td>丟丟 今天太累累了 😢 直播暫停乙次   王董想要替代丟丟當家  睡覺去 ～～ 可以 （到來...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/diudiu333</td>\\n\",\r\n       \"      <td>1723714034327589</td>\\n\",\r\n       \"      <td>3136987929666852</td>\\n\",\r\n       \"      <td>145.0</td>\\n\",\r\n       \"      <td>6943.0</td>\\n\",\r\n       \"      <td>9.0</td>\\n\",\r\n       \"      <td>131.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>23.0</td>\\n\",\r\n       \"      <td>6847.0</td>\\n\",\r\n       \"      <td>59.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>14.0</td>\\n\",\r\n       \"      <td>2022-01-03 12:42:18</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"  </tbody>\\n\",\r\n       \"</table>\\n\",\r\n       \"<p>80 rows × 18 columns</p>\\n\",\r\n       \"</div>\"\r\n      ],\r\n      \"text/plain\": [\r\n       \"   NAME                 TIME  \\\\\\n\",\r\n       \"0   丟丟妹  2021-12-03 02:36:02   \\n\",\r\n       \"1   丟丟妹  2021-11-28 21:11:13   \\n\",\r\n       \"2   丟丟妹  2021-11-19 18:02:56   \\n\",\r\n       \"3   丟丟妹  2021-11-12 22:35:08   \\n\",\r\n       \"4   丟丟妹  2021-11-10 21:00:52   \\n\",\r\n       \"..  ...                  ...   \\n\",\r\n       \"75  丟丟妹  2020-05-03 16:52:40   \\n\",\r\n       \"76  丟丟妹  2020-04-30 13:36:40   \\n\",\r\n       \"77  丟丟妹  2020-04-29 21:51:13   \\n\",\r\n       \"78  丟丟妹  2020-04-17 19:46:50   \\n\",\r\n       \"79  丟丟妹  2020-04-15 20:48:08   \\n\",\r\n       \"\\n\",\r\n       \"                                              MESSAGE  \\\\\\n\",\r\n       \"0   怎麼分不清 到底是誰帶壞誰XD 丟丟妹 和 Alizabeth 娘娘 根本姐妹淘🤣 互相「曉...   \\n\",\r\n       \"1          我喜歡這樣的陽光☀️ 開放留言 想要丟丟賣什麼？ 遇到瓶頸了⋯⋯⋯😢 ❤️ #留起來   \\n\",\r\n       \"2   人客啊 客倌們😆歡迎留言+1 2A頂級大閘蟹 （含繩6兩） 特價3088免運 （6隻一箱）最...   \\n\",\r\n       \"3   丟丟妹 終於不用離婚🤣 這次直播半路認走 #乾哥哥：#郭丟丟！ 叫買能力根本 #隔空遺傳？！...   \\n\",\r\n       \"4   各位親愛的帥哥美女們  由於訊息無限爆炸中 （已經快馬加鞭） 有留訊息就是 已接結單唷 造成...   \\n\",\r\n       \"..                                                ...   \\n\",\r\n       \"75                    猜猜我在哪裡  陳冠霖牛仔部落格 連靜雯joanne lien   \\n\",\r\n       \"76                                           踢爆娃娃機...   \\n\",\r\n       \"77             今日直播暫停乙次 丟丟身體不舒服了😭  明天下午見 開播 要想我喲  嗚嗚😢   \\n\",\r\n       \"78       帥哥美女今天星期五，丟妹偷懶去 明天下午二點 準時見❤️❤️😄有人會報到嗎 刷起來 😂   \\n\",\r\n       \"79  丟丟 今天太累累了 😢 直播暫停乙次   王董想要替代丟丟當家  睡覺去 ～～ 可以 （到來...   \\n\",\r\n       \"\\n\",\r\n       \"                                  LINK            PAGEID            POSTID  \\\\\\n\",\r\n       \"0   https://www.facebook.com/diudiu333  1723714034327589  4964611303571163   \\n\",\r\n       \"1   https://www.facebook.com/diudiu333  1723714034327589  4949186545113639   \\n\",\r\n       \"2   https://www.facebook.com/diudiu333  1723714034327589  4918431521522475   \\n\",\r\n       \"3   https://www.facebook.com/diudiu333  1723714034327589  4894343770597917   \\n\",\r\n       \"4   https://www.facebook.com/diudiu333  1723714034327589  4887230927975868   \\n\",\r\n       \"..                                 ...               ...               ...   \\n\",\r\n       \"75  https://www.facebook.com/diudiu333  1723714034327589   184909892573719   \\n\",\r\n       \"76  https://www.facebook.com/diudiu333  1723714034327589  3173673282664983   \\n\",\r\n       \"77  https://www.facebook.com/diudiu333  1723714034327589  3171969702835341   \\n\",\r\n       \"78  https://www.facebook.com/diudiu333  1723714034327589  3141849955847316   \\n\",\r\n       \"79  https://www.facebook.com/diudiu333  1723714034327589  3136987929666852   \\n\",\r\n       \"\\n\",\r\n       \"    COMMENTCOUNT  REACTIONCOUNT  SHARECOUNT  DISPLAYCOMMENTCOUNT  ANGER  \\\\\\n\",\r\n       \"0          704.0        21797.0       546.0                572.0    6.0   \\n\",\r\n       \"1         1675.0        12674.0        22.0               1142.0    1.0   \\n\",\r\n       \"2          719.0        32242.0        65.0                543.0    1.0   \\n\",\r\n       \"3          477.0        18072.0       656.0                431.0   23.0   \\n\",\r\n       \"4          352.0         9598.0        17.0                220.0    1.0   \\n\",\r\n       \"..           ...            ...         ...                  ...    ...   \\n\",\r\n       \"75        2921.0        12355.0       851.0               2921.0    0.0   \\n\",\r\n       \"76         946.0        22357.0      1035.0                856.0   76.0   \\n\",\r\n       \"77         167.0         4123.0         8.0                164.0    0.0   \\n\",\r\n       \"78         116.0         4804.0        20.0                114.0    0.0   \\n\",\r\n       \"79         145.0         6943.0         9.0                131.0    0.0   \\n\",\r\n       \"\\n\",\r\n       \"      HAHA     LIKE   LOVE  SORRY  SUPPORT   WOW           UPDATETIME  \\n\",\r\n       \"0   5798.0  15841.0  108.0    6.0     17.0  21.0  2022-01-03 12:42:18  \\n\",\r\n       \"1      7.0  12510.0  125.0    0.0     25.0   6.0  2022-01-03 12:42:18  \\n\",\r\n       \"2     31.0  31884.0  290.0    3.0     18.0  15.0  2022-01-03 12:42:18  \\n\",\r\n       \"3   3495.0  14351.0  125.0   17.0     29.0  32.0  2022-01-03 12:42:18  \\n\",\r\n       \"4     19.0   9476.0   59.0    0.0     34.0   9.0  2022-01-03 12:42:18  \\n\",\r\n       \"..     ...      ...    ...    ...      ...   ...                  ...  \\n\",\r\n       \"75     0.0      0.0    0.0    0.0      0.0   0.0  2022-01-03 12:42:18  \\n\",\r\n       \"76  2661.0  19275.0  161.0   25.0     71.0  88.0  2022-01-03 12:42:18  \\n\",\r\n       \"77     3.0   3992.0   96.0    7.0      5.0  20.0  2022-01-03 12:42:18  \\n\",\r\n       \"78     6.0   4744.0   48.0    1.0      0.0   5.0  2022-01-03 12:42:18  \\n\",\r\n       \"79    23.0   6847.0   59.0    0.0      0.0  14.0  2022-01-03 12:42:18  \\n\",\r\n       \"\\n\",\r\n       \"[80 rows x 18 columns]\"\r\n      ]\r\n     },\r\n     \"execution_count\": 3,\r\n     \"metadata\": {},\r\n     \"output_type\": \"execute_result\"\r\n    }\r\n   ],\r\n   \"source\": [\r\n    \"import facebook_crawler\\n\",\r\n    \"pageurl= 'https://www.facebook.com/diudiu333'\\n\",\r\n    \"facebook_crawler.Crawl_PagePosts(pageurl=pageurl, until_date='2021-01-01')\"\r\n   ]\r\n  },\r\n  {\r\n   \"cell_type\": \"code\",\r\n   \"execution_count\": 2,\r\n   \"id\": \"0045bec9-db09-43c2-afd7-13439ab0c19d\",\r\n   \"metadata\": {},\r\n   \"outputs\": [\r\n    {\r\n     \"name\": \"stdout\",\r\n     \"output_type\": \"stream\",\r\n     \"text\": [\r\n      \"TimeStamp: 2022-01-01.\\n\",\r\n      \"TimeStamp: 2021-06-04.\\n\",\r\n      \"TimeStamp: 2021-03-08.\\n\",\r\n      \"TimeStamp: 2020-12-11.\\n\",\r\n      \"TimeStamp: 2020-10-07.\\n\",\r\n      \"TimeStamp: 2020-07-19.\\n\",\r\n      \"TimeStamp: 2020-05-10.\\n\",\r\n      \"TimeStamp: 2020-02-14.\\n\",\r\n      \"TimeStamp: 2019-12-25.\\n\",\r\n      \"TimeStamp: 2019-11-16.\\n\",\r\n      \"TimeStamp: 2019-08-23.\\n\",\r\n      \"TimeStamp: 2019-06-06.\\n\",\r\n      \"TimeStamp: 2019-03-20.\\n\",\r\n      \"TimeStamp: 2018-12-27.\\n\",\r\n      \"TimeStamp: 2018-09-01.\\n\",\r\n      \"TimeStamp: 2017-12-29.\\n\",\r\n      \"TimeStamp: 2017-06-26.\\n\",\r\n      \"TimeStamp: 2016-12-11.\\n\",\r\n      \"TimeStamp: 2016-05-15.\\n\",\r\n      \"TimeStamp: 2016-04-05.\\n\",\r\n      \"TimeStamp: 2016-03-12.\\n\",\r\n      \"TimeStamp: 2016-02-06.\\n\",\r\n      \"TimeStamp: 2016-01-03.\\n\",\r\n      \"TimeStamp: 2015-11-28.\\n\"\r\n     ]\r\n    },\r\n    {\r\n     \"data\": {\r\n      \"text/html\": [\r\n       \"<div>\\n\",\r\n       \"<style scoped>\\n\",\r\n       \"    .dataframe tbody tr th:only-of-type {\\n\",\r\n       \"        vertical-align: middle;\\n\",\r\n       \"    }\\n\",\r\n       \"\\n\",\r\n       \"    .dataframe tbody tr th {\\n\",\r\n       \"        vertical-align: top;\\n\",\r\n       \"    }\\n\",\r\n       \"\\n\",\r\n       \"    .dataframe thead th {\\n\",\r\n       \"        text-align: right;\\n\",\r\n       \"    }\\n\",\r\n       \"</style>\\n\",\r\n       \"<table border=\\\"1\\\" class=\\\"dataframe\\\">\\n\",\r\n       \"  <thead>\\n\",\r\n       \"    <tr style=\\\"text-align: right;\\\">\\n\",\r\n       \"      <th></th>\\n\",\r\n       \"      <th>NAME</th>\\n\",\r\n       \"      <th>TIME</th>\\n\",\r\n       \"      <th>MESSAGE</th>\\n\",\r\n       \"      <th>LINK</th>\\n\",\r\n       \"      <th>PAGEID</th>\\n\",\r\n       \"      <th>POSTID</th>\\n\",\r\n       \"      <th>COMMENTCOUNT</th>\\n\",\r\n       \"      <th>REACTIONCOUNT</th>\\n\",\r\n       \"      <th>SHARECOUNT</th>\\n\",\r\n       \"      <th>DISPLAYCOMMENTCOUNT</th>\\n\",\r\n       \"      <th>ANGER</th>\\n\",\r\n       \"      <th>DOROTHY</th>\\n\",\r\n       \"      <th>HAHA</th>\\n\",\r\n       \"      <th>LIKE</th>\\n\",\r\n       \"      <th>LOVE</th>\\n\",\r\n       \"      <th>SORRY</th>\\n\",\r\n       \"      <th>SUPPORT</th>\\n\",\r\n       \"      <th>TOTO</th>\\n\",\r\n       \"      <th>WOW</th>\\n\",\r\n       \"      <th>UPDATETIME</th>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"  </thead>\\n\",\r\n       \"  <tbody>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>0</th>\\n\",\r\n       \"      <td>馬英九</td>\\n\",\r\n       \"      <td>2022-01-01 08:00:56</td>\\n\",\r\n       \"      <td>今天是中華民國一百一十一年元旦，清晨的氣溫很低，看著國旗在空中飄揚，心中交集著感謝與感慨之情...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/MaYingjeou/</td>\\n\",\r\n       \"      <td>118250504903757</td>\\n\",\r\n       \"      <td>4936286629766763</td>\\n\",\r\n       \"      <td>1243.0</td>\\n\",\r\n       \"      <td>38906.0</td>\\n\",\r\n       \"      <td>403.0</td>\\n\",\r\n       \"      <td>1022.0</td>\\n\",\r\n       \"      <td>5.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>22.0</td>\\n\",\r\n       \"      <td>38254.0</td>\\n\",\r\n       \"      <td>458.0</td>\\n\",\r\n       \"      <td>4.0</td>\\n\",\r\n       \"      <td>157.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>6.0</td>\\n\",\r\n       \"      <td>2022-01-05 23:16:30</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>1</th>\\n\",\r\n       \"      <td>馬英九</td>\\n\",\r\n       \"      <td>2021-12-21 17:05:14</td>\\n\",\r\n       \"      <td>冬至天氣冷颼颼，來一碗熱熱的燒湯圓 🤤 大家喜歡吃芝麻湯圓還是花生湯圓呢？ #冬至 #吃湯圓...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/MaYingjeou/</td>\\n\",\r\n       \"      <td>118250504903757</td>\\n\",\r\n       \"      <td>4893842230677870</td>\\n\",\r\n       \"      <td>1047.0</td>\\n\",\r\n       \"      <td>28432.0</td>\\n\",\r\n       \"      <td>179.0</td>\\n\",\r\n       \"      <td>844.0</td>\\n\",\r\n       \"      <td>15.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>64.0</td>\\n\",\r\n       \"      <td>27961.0</td>\\n\",\r\n       \"      <td>325.0</td>\\n\",\r\n       \"      <td>2.0</td>\\n\",\r\n       \"      <td>55.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>10.0</td>\\n\",\r\n       \"      <td>2022-01-05 23:16:30</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>2</th>\\n\",\r\n       \"      <td>馬英九</td>\\n\",\r\n       \"      <td>2021-12-17 17:00:00</td>\\n\",\r\n       \"      <td>公民投票是憲法賦予人民的權利 1218，穿得暖暖，出門投票！ #四個都同意人民最有利 #返鄉...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/MaYingjeou/</td>\\n\",\r\n       \"      <td>118250504903757</td>\\n\",\r\n       \"      <td>4874626962599397</td>\\n\",\r\n       \"      <td>1883.0</td>\\n\",\r\n       \"      <td>45037.0</td>\\n\",\r\n       \"      <td>519.0</td>\\n\",\r\n       \"      <td>1372.0</td>\\n\",\r\n       \"      <td>31.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>63.0</td>\\n\",\r\n       \"      <td>43941.0</td>\\n\",\r\n       \"      <td>336.0</td>\\n\",\r\n       \"      <td>16.0</td>\\n\",\r\n       \"      <td>634.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>16.0</td>\\n\",\r\n       \"      <td>2022-01-05 23:16:30</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>3</th>\\n\",\r\n       \"      <td>馬英九</td>\\n\",\r\n       \"      <td>2021-12-11 09:00:00</td>\\n\",\r\n       \"      <td>公投前最後一個周末，你收到投票通知單了嗎？ 萊劑不是保健品 公投重新綁大選 三接興建傷藻礁⋯...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/MaYingjeou/</td>\\n\",\r\n       \"      <td>118250504903757</td>\\n\",\r\n       \"      <td>4846491325412961</td>\\n\",\r\n       \"      <td>3254.0</td>\\n\",\r\n       \"      <td>39639.0</td>\\n\",\r\n       \"      <td>809.0</td>\\n\",\r\n       \"      <td>1742.0</td>\\n\",\r\n       \"      <td>60.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>78.0</td>\\n\",\r\n       \"      <td>38702.0</td>\\n\",\r\n       \"      <td>196.0</td>\\n\",\r\n       \"      <td>11.0</td>\\n\",\r\n       \"      <td>568.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>24.0</td>\\n\",\r\n       \"      <td>2022-01-05 23:16:30</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>4</th>\\n\",\r\n       \"      <td>馬英九</td>\\n\",\r\n       \"      <td>2021-11-14 13:52:11</td>\\n\",\r\n       \"      <td>鯉魚潭鐵三初體驗🏊‍♂️🚴‍♂️🏃‍♂️ 順利完賽💪💪💪 #2021年花蓮太平洋鐵人三項錦標...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/MaYingjeou/</td>\\n\",\r\n       \"      <td>118250504903757</td>\\n\",\r\n       \"      <td>4764707240258037</td>\\n\",\r\n       \"      <td>2409.0</td>\\n\",\r\n       \"      <td>73538.0</td>\\n\",\r\n       \"      <td>501.0</td>\\n\",\r\n       \"      <td>2017.0</td>\\n\",\r\n       \"      <td>15.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>41.0</td>\\n\",\r\n       \"      <td>72407.0</td>\\n\",\r\n       \"      <td>557.0</td>\\n\",\r\n       \"      <td>4.0</td>\\n\",\r\n       \"      <td>292.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>222.0</td>\\n\",\r\n       \"      <td>2022-01-05 23:16:30</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>...</th>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>475</th>\\n\",\r\n       \"      <td>馬英九</td>\\n\",\r\n       \"      <td>2015-10-19 18:10:33</td>\\n\",\r\n       \"      <td>「六堆忠義祠」源於清康熙60年，六堆團練協助平定朱一貴起事，清廷興建了「西勢忠義亭」奉祀在戰...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/MaYingjeou/</td>\\n\",\r\n       \"      <td>118250504903757</td>\\n\",\r\n       \"      <td>1050731538322311</td>\\n\",\r\n       \"      <td>1176.0</td>\\n\",\r\n       \"      <td>21803.0</td>\\n\",\r\n       \"      <td>316.0</td>\\n\",\r\n       \"      <td>705.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>21801.0</td>\\n\",\r\n       \"      <td>2.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>2022-01-05 23:16:30</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>476</th>\\n\",\r\n       \"      <td>馬英九</td>\\n\",\r\n       \"      <td>2015-10-12 18:07:34</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>https://www.facebook.com/MaYingjeou/</td>\\n\",\r\n       \"      <td>118250504903757</td>\\n\",\r\n       \"      <td>1047937641935034</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>2022-01-05 23:16:30</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>477</th>\\n\",\r\n       \"      <td>馬英九</td>\\n\",\r\n       \"      <td>2015-10-11 13:40:25</td>\\n\",\r\n       \"      <td></td>\\n\",\r\n       \"      <td>https://www.facebook.com/MaYingjeou/</td>\\n\",\r\n       \"      <td>118250504903757</td>\\n\",\r\n       \"      <td>1047435965318535</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>2022-01-05 23:16:30</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>478</th>\\n\",\r\n       \"      <td>馬英九</td>\\n\",\r\n       \"      <td>2015-10-10 14:08:46</td>\\n\",\r\n       \"      <td>今天是民國104年國慶日，讓我們一起祝中華民國生日快樂！ 今年的國慶日，還有特別的意義。今年...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/MaYingjeou/</td>\\n\",\r\n       \"      <td>118250504903757</td>\\n\",\r\n       \"      <td>1047031385358993</td>\\n\",\r\n       \"      <td>2219.0</td>\\n\",\r\n       \"      <td>36765.0</td>\\n\",\r\n       \"      <td>679.0</td>\\n\",\r\n       \"      <td>1357.0</td>\\n\",\r\n       \"      <td>3.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>1.0</td>\\n\",\r\n       \"      <td>36761.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>2022-01-05 23:16:30</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>479</th>\\n\",\r\n       \"      <td>馬英九</td>\\n\",\r\n       \"      <td>2015-10-09 19:58:31</td>\\n\",\r\n       \"      <td>故宮博物院於民國14年的國慶日成立，今年是它的90歲生日。故宮成立後幾經波折，對日抗戰爆發後...</td>\\n\",\r\n       \"      <td>https://www.facebook.com/MaYingjeou/</td>\\n\",\r\n       \"      <td>118250504903757</td>\\n\",\r\n       \"      <td>1046725805389551</td>\\n\",\r\n       \"      <td>993.0</td>\\n\",\r\n       \"      <td>14761.0</td>\\n\",\r\n       \"      <td>306.0</td>\\n\",\r\n       \"      <td>590.0</td>\\n\",\r\n       \"      <td>3.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>14757.0</td>\\n\",\r\n       \"      <td>1.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>NaN</td>\\n\",\r\n       \"      <td>0.0</td>\\n\",\r\n       \"      <td>2022-01-05 23:16:30</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"  </tbody>\\n\",\r\n       \"</table>\\n\",\r\n       \"<p>480 rows × 20 columns</p>\\n\",\r\n       \"</div>\"\r\n      ],\r\n      \"text/plain\": [\r\n       \"    NAME                 TIME  \\\\\\n\",\r\n       \"0    馬英九  2022-01-01 08:00:56   \\n\",\r\n       \"1    馬英九  2021-12-21 17:05:14   \\n\",\r\n       \"2    馬英九  2021-12-17 17:00:00   \\n\",\r\n       \"3    馬英九  2021-12-11 09:00:00   \\n\",\r\n       \"4    馬英九  2021-11-14 13:52:11   \\n\",\r\n       \"..   ...                  ...   \\n\",\r\n       \"475  馬英九  2015-10-19 18:10:33   \\n\",\r\n       \"476  馬英九  2015-10-12 18:07:34   \\n\",\r\n       \"477  馬英九  2015-10-11 13:40:25   \\n\",\r\n       \"478  馬英九  2015-10-10 14:08:46   \\n\",\r\n       \"479  馬英九  2015-10-09 19:58:31   \\n\",\r\n       \"\\n\",\r\n       \"                                               MESSAGE  \\\\\\n\",\r\n       \"0    今天是中華民國一百一十一年元旦，清晨的氣溫很低，看著國旗在空中飄揚，心中交集著感謝與感慨之情...   \\n\",\r\n       \"1    冬至天氣冷颼颼，來一碗熱熱的燒湯圓 🤤 大家喜歡吃芝麻湯圓還是花生湯圓呢？ #冬至 #吃湯圓...   \\n\",\r\n       \"2    公民投票是憲法賦予人民的權利 1218，穿得暖暖，出門投票！ #四個都同意人民最有利 #返鄉...   \\n\",\r\n       \"3    公投前最後一個周末，你收到投票通知單了嗎？ 萊劑不是保健品 公投重新綁大選 三接興建傷藻礁⋯...   \\n\",\r\n       \"4    鯉魚潭鐵三初體驗🏊‍♂️🚴‍♂️🏃‍♂️ 順利完賽💪💪💪 #2021年花蓮太平洋鐵人三項錦標...   \\n\",\r\n       \"..                                                 ...   \\n\",\r\n       \"475  「六堆忠義祠」源於清康熙60年，六堆團練協助平定朱一貴起事，清廷興建了「西勢忠義亭」奉祀在戰...   \\n\",\r\n       \"476                                                      \\n\",\r\n       \"477                                                      \\n\",\r\n       \"478  今天是民國104年國慶日，讓我們一起祝中華民國生日快樂！ 今年的國慶日，還有特別的意義。今年...   \\n\",\r\n       \"479  故宮博物院於民國14年的國慶日成立，今年是它的90歲生日。故宮成立後幾經波折，對日抗戰爆發後...   \\n\",\r\n       \"\\n\",\r\n       \"                                     LINK           PAGEID            POSTID  \\\\\\n\",\r\n       \"0    https://www.facebook.com/MaYingjeou/  118250504903757  4936286629766763   \\n\",\r\n       \"1    https://www.facebook.com/MaYingjeou/  118250504903757  4893842230677870   \\n\",\r\n       \"2    https://www.facebook.com/MaYingjeou/  118250504903757  4874626962599397   \\n\",\r\n       \"3    https://www.facebook.com/MaYingjeou/  118250504903757  4846491325412961   \\n\",\r\n       \"4    https://www.facebook.com/MaYingjeou/  118250504903757  4764707240258037   \\n\",\r\n       \"..                                    ...              ...               ...   \\n\",\r\n       \"475  https://www.facebook.com/MaYingjeou/  118250504903757  1050731538322311   \\n\",\r\n       \"476  https://www.facebook.com/MaYingjeou/  118250504903757  1047937641935034   \\n\",\r\n       \"477  https://www.facebook.com/MaYingjeou/  118250504903757  1047435965318535   \\n\",\r\n       \"478  https://www.facebook.com/MaYingjeou/  118250504903757  1047031385358993   \\n\",\r\n       \"479  https://www.facebook.com/MaYingjeou/  118250504903757  1046725805389551   \\n\",\r\n       \"\\n\",\r\n       \"     COMMENTCOUNT  REACTIONCOUNT  SHARECOUNT  DISPLAYCOMMENTCOUNT  ANGER  \\\\\\n\",\r\n       \"0          1243.0        38906.0       403.0               1022.0    5.0   \\n\",\r\n       \"1          1047.0        28432.0       179.0                844.0   15.0   \\n\",\r\n       \"2          1883.0        45037.0       519.0               1372.0   31.0   \\n\",\r\n       \"3          3254.0        39639.0       809.0               1742.0   60.0   \\n\",\r\n       \"4          2409.0        73538.0       501.0               2017.0   15.0   \\n\",\r\n       \"..            ...            ...         ...                  ...    ...   \\n\",\r\n       \"475        1176.0        21803.0       316.0                705.0    0.0   \\n\",\r\n       \"476           NaN            NaN         NaN                  NaN    0.0   \\n\",\r\n       \"477           NaN            NaN         NaN                  NaN    0.0   \\n\",\r\n       \"478        2219.0        36765.0       679.0               1357.0    3.0   \\n\",\r\n       \"479         993.0        14761.0       306.0                590.0    3.0   \\n\",\r\n       \"\\n\",\r\n       \"     DOROTHY  HAHA     LIKE   LOVE  SORRY  SUPPORT  TOTO    WOW  \\\\\\n\",\r\n       \"0        NaN  22.0  38254.0  458.0    4.0    157.0   NaN    6.0   \\n\",\r\n       \"1        NaN  64.0  27961.0  325.0    2.0     55.0   NaN   10.0   \\n\",\r\n       \"2        NaN  63.0  43941.0  336.0   16.0    634.0   NaN   16.0   \\n\",\r\n       \"3        NaN  78.0  38702.0  196.0   11.0    568.0   NaN   24.0   \\n\",\r\n       \"4        NaN  41.0  72407.0  557.0    4.0    292.0   NaN  222.0   \\n\",\r\n       \"..       ...   ...      ...    ...    ...      ...   ...    ...   \\n\",\r\n       \"475      NaN   0.0  21801.0    2.0    0.0      0.0   NaN    0.0   \\n\",\r\n       \"476      NaN   0.0      0.0    0.0    0.0      0.0   NaN    0.0   \\n\",\r\n       \"477      NaN   0.0      0.0    0.0    0.0      0.0   NaN    0.0   \\n\",\r\n       \"478      NaN   1.0  36761.0    0.0    0.0      0.0   NaN    0.0   \\n\",\r\n       \"479      NaN   0.0  14757.0    1.0    0.0      0.0   NaN    0.0   \\n\",\r\n       \"\\n\",\r\n       \"              UPDATETIME  \\n\",\r\n       \"0    2022-01-05 23:16:30  \\n\",\r\n       \"1    2022-01-05 23:16:30  \\n\",\r\n       \"2    2022-01-05 23:16:30  \\n\",\r\n       \"3    2022-01-05 23:16:30  \\n\",\r\n       \"4    2022-01-05 23:16:30  \\n\",\r\n       \"..                   ...  \\n\",\r\n       \"475  2022-01-05 23:16:30  \\n\",\r\n       \"476  2022-01-05 23:16:30  \\n\",\r\n       \"477  2022-01-05 23:16:30  \\n\",\r\n       \"478  2022-01-05 23:16:30  \\n\",\r\n       \"479  2022-01-05 23:16:30  \\n\",\r\n       \"\\n\",\r\n       \"[480 rows x 20 columns]\"\r\n      ]\r\n     },\r\n     \"execution_count\": 2,\r\n     \"metadata\": {},\r\n     \"output_type\": \"execute_result\"\r\n    }\r\n   ],\r\n   \"source\": [\r\n    \"import facebook_crawler\\n\",\r\n    \"pageurl= 'https://www.facebook.com/MaYingjeou'\\n\",\r\n    \"df = facebook_crawler.Crawl_PagePosts(pageurl=pageurl, until_date='2016-01-01')\\n\",\r\n    \"df\"\r\n   ]\r\n  },\r\n  {\r\n   \"cell_type\": \"code\",\r\n   \"execution_count\": null,\r\n   \"id\": \"5696cdfc-8f3a-47cd-a936-97112f077db4\",\r\n   \"metadata\": {},\r\n   \"outputs\": [],\r\n   \"source\": []\r\n  }\r\n ],\r\n \"metadata\": {\r\n  \"kernelspec\": {\r\n   \"display_name\": \"Python 3 (ipykernel)\",\r\n   \"language\": \"python\",\r\n   \"name\": \"python3\"\r\n  },\r\n  \"language_info\": {\r\n   \"codemirror_mode\": {\r\n    \"name\": \"ipython\",\r\n    \"version\": 3\r\n   },\r\n   \"file_extension\": \".py\",\r\n   \"mimetype\": \"text/x-python\",\r\n   \"name\": \"python\",\r\n   \"nbconvert_exporter\": \"python\",\r\n   \"pygments_lexer\": \"ipython3\",\r\n   \"version\": \"3.8.10\"\r\n  }\r\n },\r\n \"nbformat\": 4,\r\n \"nbformat_minor\": 5\r\n}\r\n"
  },
  {
    "path": "sample/Group.ipynb",
    "content": "{\r\n \"cells\": [\r\n  {\r\n   \"cell_type\": \"code\",\r\n   \"execution_count\": 2,\r\n   \"id\": \"f31b5034-e3d4-40f4-b52a-8e4addba342d\",\r\n   \"metadata\": {},\r\n   \"outputs\": [\r\n    {\r\n     \"name\": \"stdout\",\r\n     \"output_type\": \"stream\",\r\n     \"text\": [\r\n      \"TimeStamp: 2022-01-05.\\n\",\r\n      \"TimeStamp: 2022-01-02.\\n\",\r\n      \"TimeStamp: 2021-12-28.\\n\",\r\n      \"TimeStamp: 2021-12-26.\\n\",\r\n      \"TimeStamp: 2021-12-21.\\n\",\r\n      \"TimeStamp: 2021-12-17.\\n\",\r\n      \"TimeStamp: 2021-12-14.\\n\",\r\n      \"TimeStamp: 2021-12-12.\\n\",\r\n      \"TimeStamp: 2021-12-08.\\n\",\r\n      \"TimeStamp: 2021-12-07.\\n\",\r\n      \"TimeStamp: 2021-12-04.\\n\",\r\n      \"TimeStamp: 2021-11-30.\\n\",\r\n      \"TimeStamp: 2021-11-28.\\n\",\r\n      \"TimeStamp: 2021-11-25.\\n\",\r\n      \"TimeStamp: 2021-11-22.\\n\",\r\n      \"TimeStamp: 2021-11-19.\\n\",\r\n      \"TimeStamp: 2021-11-18.\\n\",\r\n      \"TimeStamp: 2021-11-15.\\n\",\r\n      \"TimeStamp: 2021-11-14.\\n\",\r\n      \"TimeStamp: 2021-11-11.\\n\",\r\n      \"TimeStamp: 2021-11-09.\\n\",\r\n      \"TimeStamp: 2021-11-07.\\n\",\r\n      \"TimeStamp: 2021-11-05.\\n\",\r\n      \"TimeStamp: 2021-11-03.\\n\",\r\n      \"TimeStamp: 2021-11-02.\\n\",\r\n      \"TimeStamp: 2021-10-31.\\n\",\r\n      \"TimeStamp: 2021-10-28.\\n\",\r\n      \"TimeStamp: 2021-10-26.\\n\",\r\n      \"TimeStamp: 2021-10-24.\\n\",\r\n      \"TimeStamp: 2021-10-22.\\n\",\r\n      \"TimeStamp: 2021-10-20.\\n\",\r\n      \"TimeStamp: 2021-10-18.\\n\",\r\n      \"TimeStamp: 2021-10-17.\\n\",\r\n      \"TimeStamp: 2021-10-15.\\n\",\r\n      \"TimeStamp: 2021-10-13.\\n\",\r\n      \"TimeStamp: 2021-10-11.\\n\",\r\n      \"TimeStamp: 2021-10-09.\\n\",\r\n      \"TimeStamp: 2021-10-07.\\n\",\r\n      \"TimeStamp: 2021-10-05.\\n\",\r\n      \"TimeStamp: 2021-10-03.\\n\",\r\n      \"TimeStamp: 2021-10-01.\\n\",\r\n      \"TimeStamp: 2021-09-30.\\n\",\r\n      \"TimeStamp: 2021-09-28.\\n\",\r\n      \"TimeStamp: 2021-09-25.\\n\",\r\n      \"TimeStamp: 2021-09-23.\\n\",\r\n      \"TimeStamp: 2021-09-21.\\n\",\r\n      \"TimeStamp: 2021-09-19.\\n\",\r\n      \"TimeStamp: 2021-09-18.\\n\",\r\n      \"TimeStamp: 2021-09-16.\\n\",\r\n      \"TimeStamp: 2021-09-15.\\n\",\r\n      \"TimeStamp: 2021-09-13.\\n\",\r\n      \"TimeStamp: 2021-09-11.\\n\",\r\n      \"TimeStamp: 2021-09-09.\\n\",\r\n      \"TimeStamp: 2021-09-08.\\n\",\r\n      \"TimeStamp: 2021-09-06.\\n\",\r\n      \"TimeStamp: 2021-09-05.\\n\",\r\n      \"TimeStamp: 2021-09-03.\\n\",\r\n      \"TimeStamp: 2021-09-02.\\n\",\r\n      \"TimeStamp: 2021-09-01.\\n\",\r\n      \"TimeStamp: 2021-08-29.\\n\",\r\n      \"TimeStamp: 2021-08-27.\\n\",\r\n      \"TimeStamp: 2021-08-25.\\n\",\r\n      \"TimeStamp: 2021-08-23.\\n\",\r\n      \"TimeStamp: 2021-08-22.\\n\",\r\n      \"TimeStamp: 2021-08-20.\\n\",\r\n      \"TimeStamp: 2021-08-18.\\n\",\r\n      \"TimeStamp: 2021-08-17.\\n\",\r\n      \"TimeStamp: 2021-08-15.\\n\",\r\n      \"TimeStamp: 2021-08-14.\\n\",\r\n      \"TimeStamp: 2021-08-12.\\n\",\r\n      \"TimeStamp: 2021-08-11.\\n\",\r\n      \"TimeStamp: 2021-08-10.\\n\",\r\n      \"TimeStamp: 2021-08-08.\\n\",\r\n      \"TimeStamp: 2021-08-06.\\n\",\r\n      \"TimeStamp: 2021-08-04.\\n\",\r\n      \"TimeStamp: 2021-08-02.\\n\",\r\n      \"TimeStamp: 2021-07-31.\\n\",\r\n      \"TimeStamp: 2021-07-29.\\n\",\r\n      \"TimeStamp: 2021-07-28.\\n\",\r\n      \"TimeStamp: 2021-07-26.\\n\",\r\n      \"TimeStamp: 2021-07-24.\\n\",\r\n      \"TimeStamp: 2021-07-22.\\n\",\r\n      \"TimeStamp: 2021-07-21.\\n\",\r\n      \"TimeStamp: 2021-07-19.\\n\",\r\n      \"TimeStamp: 2021-07-18.\\n\",\r\n      \"TimeStamp: 2021-07-17.\\n\",\r\n      \"TimeStamp: 2021-07-16.\\n\",\r\n      \"TimeStamp: 2021-07-14.\\n\",\r\n      \"TimeStamp: 2021-07-12.\\n\",\r\n      \"TimeStamp: 2021-07-10.\\n\",\r\n      \"TimeStamp: 2021-07-09.\\n\",\r\n      \"TimeStamp: 2021-07-07.\\n\",\r\n      \"TimeStamp: 2021-07-04.\\n\",\r\n      \"TimeStamp: 2021-07-03.\\n\",\r\n      \"TimeStamp: 2021-07-02.\\n\",\r\n      \"TimeStamp: 2021-06-30.\\n\",\r\n      \"TimeStamp: 2021-06-28.\\n\",\r\n      \"TimeStamp: 2021-06-26.\\n\",\r\n      \"TimeStamp: 2021-06-25.\\n\",\r\n      \"TimeStamp: 2021-06-22.\\n\",\r\n      \"TimeStamp: 2021-06-21.\\n\",\r\n      \"TimeStamp: 2021-06-19.\\n\",\r\n      \"TimeStamp: 2021-06-18.\\n\",\r\n      \"TimeStamp: 2021-06-16.\\n\",\r\n      \"TimeStamp: 2021-06-14.\\n\",\r\n      \"TimeStamp: 2021-06-12.\\n\",\r\n      \"TimeStamp: 2021-06-11.\\n\",\r\n      \"TimeStamp: 2021-06-10.\\n\",\r\n      \"TimeStamp: 2021-06-08.\\n\",\r\n      \"TimeStamp: 2021-06-06.\\n\",\r\n      \"TimeStamp: 2021-06-06.\\n\",\r\n      \"TimeStamp: 2021-06-04.\\n\",\r\n      \"TimeStamp: 2021-06-02.\\n\",\r\n      \"TimeStamp: 2021-05-31.\\n\",\r\n      \"TimeStamp: 2021-05-29.\\n\",\r\n      \"TimeStamp: 2021-05-26.\\n\",\r\n      \"TimeStamp: 2021-05-25.\\n\",\r\n      \"TimeStamp: 2021-05-23.\\n\",\r\n      \"TimeStamp: 2021-05-21.\\n\",\r\n      \"TimeStamp: 2021-05-19.\\n\",\r\n      \"TimeStamp: 2021-05-17.\\n\",\r\n      \"TimeStamp: 2021-05-13.\\n\",\r\n      \"TimeStamp: 2021-05-11.\\n\",\r\n      \"TimeStamp: 2021-05-09.\\n\",\r\n      \"TimeStamp: 2021-05-07.\\n\",\r\n      \"TimeStamp: 2021-05-05.\\n\",\r\n      \"TimeStamp: 2021-05-04.\\n\",\r\n      \"TimeStamp: 2021-05-03.\\n\",\r\n      \"TimeStamp: 2021-04-30.\\n\",\r\n      \"TimeStamp: 2021-04-29.\\n\",\r\n      \"TimeStamp: 2021-04-28.\\n\",\r\n      \"TimeStamp: 2021-04-27.\\n\",\r\n      \"TimeStamp: 2021-04-26.\\n\",\r\n      \"TimeStamp: 2021-04-24.\\n\",\r\n      \"TimeStamp: 2021-04-22.\\n\",\r\n      \"TimeStamp: 2021-04-20.\\n\",\r\n      \"TimeStamp: 2021-04-19.\\n\",\r\n      \"TimeStamp: 2021-04-17.\\n\",\r\n      \"TimeStamp: 2021-04-16.\\n\",\r\n      \"TimeStamp: 2021-04-14.\\n\",\r\n      \"TimeStamp: 2021-04-12.\\n\",\r\n      \"TimeStamp: 2021-04-11.\\n\",\r\n      \"TimeStamp: 2021-04-08.\\n\",\r\n      \"TimeStamp: 2021-04-07.\\n\",\r\n      \"TimeStamp: 2021-04-06.\\n\",\r\n      \"TimeStamp: 2021-04-04.\\n\",\r\n      \"TimeStamp: 2021-04-03.\\n\",\r\n      \"TimeStamp: 2021-04-01.\\n\",\r\n      \"TimeStamp: 2021-03-30.\\n\",\r\n      \"TimeStamp: 2021-03-29.\\n\",\r\n      \"TimeStamp: 2021-03-27.\\n\",\r\n      \"TimeStamp: 2021-03-26.\\n\",\r\n      \"TimeStamp: 2021-03-25.\\n\",\r\n      \"TimeStamp: 2021-03-24.\\n\",\r\n      \"TimeStamp: 2021-03-21.\\n\",\r\n      \"TimeStamp: 2021-03-19.\\n\",\r\n      \"TimeStamp: 2021-03-18.\\n\",\r\n      \"TimeStamp: 2021-03-16.\\n\",\r\n      \"TimeStamp: 2021-03-14.\\n\",\r\n      \"TimeStamp: 2021-03-12.\\n\",\r\n      \"TimeStamp: 2021-03-11.\\n\",\r\n      \"TimeStamp: 2021-03-09.\\n\",\r\n      \"TimeStamp: 2021-03-08.\\n\",\r\n      \"TimeStamp: 2021-03-06.\\n\",\r\n      \"TimeStamp: 2021-03-05.\\n\",\r\n      \"TimeStamp: 2021-03-04.\\n\",\r\n      \"TimeStamp: 2021-03-02.\\n\",\r\n      \"TimeStamp: 2021-02-28.\\n\",\r\n      \"TimeStamp: 2021-02-27.\\n\",\r\n      \"TimeStamp: 2021-02-25.\\n\",\r\n      \"TimeStamp: 2021-02-23.\\n\",\r\n      \"TimeStamp: 2021-02-20.\\n\",\r\n      \"TimeStamp: 2021-02-18.\\n\",\r\n      \"TimeStamp: 2021-02-16.\\n\",\r\n      \"TimeStamp: 2021-02-14.\\n\",\r\n      \"TimeStamp: 2021-02-11.\\n\",\r\n      \"TimeStamp: 2021-02-09.\\n\",\r\n      \"TimeStamp: 2021-02-07.\\n\",\r\n      \"TimeStamp: 2021-02-05.\\n\",\r\n      \"TimeStamp: 2021-02-03.\\n\",\r\n      \"TimeStamp: 2021-02-01.\\n\",\r\n      \"TimeStamp: 2021-01-31.\\n\",\r\n      \"TimeStamp: 2021-01-29.\\n\",\r\n      \"TimeStamp: 2021-01-27.\\n\",\r\n      \"TimeStamp: 2021-01-25.\\n\",\r\n      \"TimeStamp: 2021-01-23.\\n\",\r\n      \"TimeStamp: 2021-01-21.\\n\",\r\n      \"TimeStamp: 2021-01-20.\\n\",\r\n      \"TimeStamp: 2021-01-18.\\n\",\r\n      \"TimeStamp: 2021-01-15.\\n\",\r\n      \"TimeStamp: 2021-01-13.\\n\",\r\n      \"TimeStamp: 2021-01-11.\\n\",\r\n      \"TimeStamp: 2021-01-09.\\n\",\r\n      \"TimeStamp: 2021-01-07.\\n\",\r\n      \"TimeStamp: 2021-01-06.\\n\",\r\n      \"TimeStamp: 2021-01-03.\\n\",\r\n      \"TimeStamp: 2020-12-30.\\n\"\r\n     ]\r\n    },\r\n    {\r\n     \"data\": {\r\n      \"text/html\": [\r\n       \"<div>\\n\",\r\n       \"<style scoped>\\n\",\r\n       \"    .dataframe tbody tr th:only-of-type {\\n\",\r\n       \"        vertical-align: middle;\\n\",\r\n       \"    }\\n\",\r\n       \"\\n\",\r\n       \"    .dataframe tbody tr th {\\n\",\r\n       \"        vertical-align: top;\\n\",\r\n       \"    }\\n\",\r\n       \"\\n\",\r\n       \"    .dataframe thead th {\\n\",\r\n       \"        text-align: right;\\n\",\r\n       \"    }\\n\",\r\n       \"</style>\\n\",\r\n       \"<table border=\\\"1\\\" class=\\\"dataframe\\\">\\n\",\r\n       \"  <thead>\\n\",\r\n       \"    <tr style=\\\"text-align: right;\\\">\\n\",\r\n       \"      <th></th>\\n\",\r\n       \"      <th>ACTORID</th>\\n\",\r\n       \"      <th>NAME</th>\\n\",\r\n       \"      <th>GROUPID</th>\\n\",\r\n       \"      <th>POSTID</th>\\n\",\r\n       \"      <th>TIME</th>\\n\",\r\n       \"      <th>CONTENT</th>\\n\",\r\n       \"      <th>COMMENTCOUNT</th>\\n\",\r\n       \"      <th>SHARECOUNT</th>\\n\",\r\n       \"      <th>LIKECOUNT</th>\\n\",\r\n       \"      <th>UPDATETIME</th>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"  </thead>\\n\",\r\n       \"  <tbody>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>0</th>\\n\",\r\n       \"      <td>\\\"100008360976636\\\"</td>\\n\",\r\n       \"      <td>蔣忠祐</td>\\n\",\r\n       \"      <td>197223143437</td>\\n\",\r\n       \"      <td>10161825149923438</td>\\n\",\r\n       \"      <td>2022-01-04 19:28:20</td>\\n\",\r\n       \"      <td>小弟想問一下，因為本身只寫過python也只會ML其他的都不太熟悉。 如果我今天想要寫一個簡...</td>\\n\",\r\n       \"      <td>49</td>\\n\",\r\n       \"      <td>7</td>\\n\",\r\n       \"      <td>42</td>\\n\",\r\n       \"      <td>2022-01-05 22:47:28</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>1</th>\\n\",\r\n       \"      <td>1372450133</td>\\n\",\r\n       \"      <td>陳柏勳</td>\\n\",\r\n       \"      <td>197223143437</td>\\n\",\r\n       \"      <td>10161825076258438</td>\\n\",\r\n       \"      <td>2022-01-04 18:16:42</td>\\n\",\r\n       \"      <td>各位高手大大 小弟剛自學 python 我先在 Jupyter 寫一個檔案 並存檔為Hell...</td>\\n\",\r\n       \"      <td>14</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>10</td>\\n\",\r\n       \"      <td>2022-01-05 22:47:28</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>2</th>\\n\",\r\n       \"      <td>\\\"100029978773864\\\"</td>\\n\",\r\n       \"      <td>沈宗叡</td>\\n\",\r\n       \"      <td>197223143437</td>\\n\",\r\n       \"      <td>10161818603678438</td>\\n\",\r\n       \"      <td>2022-01-01 14:46:35</td>\\n\",\r\n       \"      <td>在練習網路上的題目： https://tioj.ck.tp.edu.tw/problems/...</td>\\n\",\r\n       \"      <td>15</td>\\n\",\r\n       \"      <td>3</td>\\n\",\r\n       \"      <td>3</td>\\n\",\r\n       \"      <td>2022-01-05 22:47:28</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>3</th>\\n\",\r\n       \"      <td>696780981</td>\\n\",\r\n       \"      <td>MaoYang Chien</td>\\n\",\r\n       \"      <td>197223143437</td>\\n\",\r\n       \"      <td>10161826718243438</td>\\n\",\r\n       \"      <td>2022-01-05 11:29:06</td>\\n\",\r\n       \"      <td>這個開源工具使用 python、flask 等技術開發 如果你用「專案管理」的方法來完成一堂...</td>\\n\",\r\n       \"      <td>0</td>\\n\",\r\n       \"      <td>9</td>\\n\",\r\n       \"      <td>24</td>\\n\",\r\n       \"      <td>2022-01-05 22:47:28</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>4</th>\\n\",\r\n       \"      <td>\\\"100006223006386\\\"</td>\\n\",\r\n       \"      <td>劉奕德</td>\\n\",\r\n       \"      <td>197223143437</td>\\n\",\r\n       \"      <td>10161816780833438</td>\\n\",\r\n       \"      <td>2021-12-31 23:20:43</td>\\n\",\r\n       \"      <td>想詢問版上各位大大 近期在做廠房的耗電量改善 想到python可以應用在建立模型及預測 請問...</td>\\n\",\r\n       \"      <td>37</td>\\n\",\r\n       \"      <td>2</td>\\n\",\r\n       \"      <td>29</td>\\n\",\r\n       \"      <td>2022-01-05 22:47:28</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>...</th>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"      <td>...</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>3935</th>\\n\",\r\n       \"      <td>\\\"105673814305452\\\"</td>\\n\",\r\n       \"      <td>用圖片高效學程式</td>\\n\",\r\n       \"      <td>105673814305452</td>\\n\",\r\n       \"      <td>10160837021273438</td>\\n\",\r\n       \"      <td>2020-12-27 13:00:34</td>\\n\",\r\n       \"      <td>【圖解演算法教學】Bubble Sort 大隊接力賽 這次首次嘗試以「動畫」形式，來演示Bu...</td>\\n\",\r\n       \"      <td>2</td>\\n\",\r\n       \"      <td>9</td>\\n\",\r\n       \"      <td>14</td>\\n\",\r\n       \"      <td>2022-01-05 22:47:28</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>3936</th>\\n\",\r\n       \"      <td>\\\"100051655785472\\\"</td>\\n\",\r\n       \"      <td>劉昶林</td>\\n\",\r\n       \"      <td>197223143437</td>\\n\",\r\n       \"      <td>10160814944163438</td>\\n\",\r\n       \"      <td>2020-12-19 23:56:39</td>\\n\",\r\n       \"      <td>嗨各位， 我現在有個問題，我想要將yolov4(darknet) 跟 Arduino做結合，...</td>\\n\",\r\n       \"      <td>34</td>\\n\",\r\n       \"      <td>13</td>\\n\",\r\n       \"      <td>72</td>\\n\",\r\n       \"      <td>2022-01-05 22:47:28</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>3937</th>\\n\",\r\n       \"      <td>\\\"100002220077374\\\"</td>\\n\",\r\n       \"      <td>Hugh LI</td>\\n\",\r\n       \"      <td>197223143437</td>\\n\",\r\n       \"      <td>10160840505808438</td>\\n\",\r\n       \"      <td>2020-12-28 20:01:25</td>\\n\",\r\n       \"      <td>想問一下各位一個蠻基礎的問題 (先撇開其他條件) c=s[a:b]  c[::-1] 為甚麼...</td>\\n\",\r\n       \"      <td>10</td>\\n\",\r\n       \"      <td>1</td>\\n\",\r\n       \"      <td>12</td>\\n\",\r\n       \"      <td>2022-01-05 22:47:28</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>3938</th>\\n\",\r\n       \"      <td>504805657</td>\\n\",\r\n       \"      <td>Roy Hsu</td>\\n\",\r\n       \"      <td>197223143437</td>\\n\",\r\n       \"      <td>10160837961533438</td>\\n\",\r\n       \"      <td>2020-12-27 22:27:50</td>\\n\",\r\n       \"      <td>我想實現 用python 去更換 資料夾的icon ( windows ) 網路上看到這個 ...</td>\\n\",\r\n       \"      <td>6</td>\\n\",\r\n       \"      <td>1</td>\\n\",\r\n       \"      <td>3</td>\\n\",\r\n       \"      <td>2022-01-05 22:47:28</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"    <tr>\\n\",\r\n       \"      <th>3939</th>\\n\",\r\n       \"      <td>\\\"100001849455014\\\"</td>\\n\",\r\n       \"      <td>Vivi Chen</td>\\n\",\r\n       \"      <td>197223143437</td>\\n\",\r\n       \"      <td>10160837910348438</td>\\n\",\r\n       \"      <td>2020-12-27 22:05:17</td>\\n\",\r\n       \"      <td>Test 的 SayHello 函式在初始化後, 就不用再輸入'Amy' 最終目的就是只要在...</td>\\n\",\r\n       \"      <td>6</td>\\n\",\r\n       \"      <td>1</td>\\n\",\r\n       \"      <td>9</td>\\n\",\r\n       \"      <td>2022-01-05 22:47:28</td>\\n\",\r\n       \"    </tr>\\n\",\r\n       \"  </tbody>\\n\",\r\n       \"</table>\\n\",\r\n       \"<p>3940 rows × 10 columns</p>\\n\",\r\n       \"</div>\"\r\n      ],\r\n      \"text/plain\": [\r\n       \"                ACTORID           NAME          GROUPID             POSTID  \\\\\\n\",\r\n       \"0     \\\"100008360976636\\\"            蔣忠祐     197223143437  10161825149923438   \\n\",\r\n       \"1            1372450133            陳柏勳     197223143437  10161825076258438   \\n\",\r\n       \"2     \\\"100029978773864\\\"            沈宗叡     197223143437  10161818603678438   \\n\",\r\n       \"3             696780981  MaoYang Chien     197223143437  10161826718243438   \\n\",\r\n       \"4     \\\"100006223006386\\\"            劉奕德     197223143437  10161816780833438   \\n\",\r\n       \"...                 ...            ...              ...                ...   \\n\",\r\n       \"3935  \\\"105673814305452\\\"       用圖片高效學程式  105673814305452  10160837021273438   \\n\",\r\n       \"3936  \\\"100051655785472\\\"            劉昶林     197223143437  10160814944163438   \\n\",\r\n       \"3937  \\\"100002220077374\\\"        Hugh LI     197223143437  10160840505808438   \\n\",\r\n       \"3938          504805657        Roy Hsu     197223143437  10160837961533438   \\n\",\r\n       \"3939  \\\"100001849455014\\\"      Vivi Chen     197223143437  10160837910348438   \\n\",\r\n       \"\\n\",\r\n       \"                     TIME                                            CONTENT  \\\\\\n\",\r\n       \"0     2022-01-04 19:28:20  小弟想問一下，因為本身只寫過python也只會ML其他的都不太熟悉。 如果我今天想要寫一個簡...   \\n\",\r\n       \"1     2022-01-04 18:16:42  各位高手大大 小弟剛自學 python 我先在 Jupyter 寫一個檔案 並存檔為Hell...   \\n\",\r\n       \"2     2022-01-01 14:46:35  在練習網路上的題目： https://tioj.ck.tp.edu.tw/problems/...   \\n\",\r\n       \"3     2022-01-05 11:29:06  這個開源工具使用 python、flask 等技術開發 如果你用「專案管理」的方法來完成一堂...   \\n\",\r\n       \"4     2021-12-31 23:20:43  想詢問版上各位大大 近期在做廠房的耗電量改善 想到python可以應用在建立模型及預測 請問...   \\n\",\r\n       \"...                   ...                                                ...   \\n\",\r\n       \"3935  2020-12-27 13:00:34  【圖解演算法教學】Bubble Sort 大隊接力賽 這次首次嘗試以「動畫」形式，來演示Bu...   \\n\",\r\n       \"3936  2020-12-19 23:56:39  嗨各位， 我現在有個問題，我想要將yolov4(darknet) 跟 Arduino做結合，...   \\n\",\r\n       \"3937  2020-12-28 20:01:25  想問一下各位一個蠻基礎的問題 (先撇開其他條件) c=s[a:b]  c[::-1] 為甚麼...   \\n\",\r\n       \"3938  2020-12-27 22:27:50  我想實現 用python 去更換 資料夾的icon ( windows ) 網路上看到這個 ...   \\n\",\r\n       \"3939  2020-12-27 22:05:17  Test 的 SayHello 函式在初始化後, 就不用再輸入'Amy' 最終目的就是只要在...   \\n\",\r\n       \"\\n\",\r\n       \"      COMMENTCOUNT  SHARECOUNT  LIKECOUNT           UPDATETIME  \\n\",\r\n       \"0               49           7         42  2022-01-05 22:47:28  \\n\",\r\n       \"1               14           0         10  2022-01-05 22:47:28  \\n\",\r\n       \"2               15           3          3  2022-01-05 22:47:28  \\n\",\r\n       \"3                0           9         24  2022-01-05 22:47:28  \\n\",\r\n       \"4               37           2         29  2022-01-05 22:47:28  \\n\",\r\n       \"...            ...         ...        ...                  ...  \\n\",\r\n       \"3935             2           9         14  2022-01-05 22:47:28  \\n\",\r\n       \"3936            34          13         72  2022-01-05 22:47:28  \\n\",\r\n       \"3937            10           1         12  2022-01-05 22:47:28  \\n\",\r\n       \"3938             6           1          3  2022-01-05 22:47:28  \\n\",\r\n       \"3939             6           1          9  2022-01-05 22:47:28  \\n\",\r\n       \"\\n\",\r\n       \"[3940 rows x 10 columns]\"\r\n      ]\r\n     },\r\n     \"execution_count\": 2,\r\n     \"metadata\": {},\r\n     \"output_type\": \"execute_result\"\r\n    }\r\n   ],\r\n   \"source\": [\r\n    \"import facebook_crawler\\n\",\r\n    \"groupurl = 'https://www.facebook.com/groups/pythontw'\\n\",\r\n    \"facebook_crawler.Crawl_GroupPosts(groupurl, until_date='2021-01-01')\"\r\n   ]\r\n  },\r\n  {\r\n   \"cell_type\": \"code\",\r\n   \"execution_count\": null,\r\n   \"id\": \"df297081-8a01-4a6c-9719-9b4dfa87410c\",\r\n   \"metadata\": {},\r\n   \"outputs\": [],\r\n   \"source\": []\r\n  }\r\n ],\r\n \"metadata\": {\r\n  \"kernelspec\": {\r\n   \"display_name\": \"Python 3 (ipykernel)\",\r\n   \"language\": \"python\",\r\n   \"name\": \"python3\"\r\n  },\r\n  \"language_info\": {\r\n   \"codemirror_mode\": {\r\n    \"name\": \"ipython\",\r\n    \"version\": 3\r\n   },\r\n   \"file_extension\": \".py\",\r\n   \"mimetype\": \"text/x-python\",\r\n   \"name\": \"python\",\r\n   \"nbconvert_exporter\": \"python\",\r\n   \"pygments_lexer\": \"ipython3\",\r\n   \"version\": \"3.8.10\"\r\n  }\r\n },\r\n \"nbformat\": 4,\r\n \"nbformat_minor\": 5\r\n}\r\n"
  },
  {
    "path": "setup.py",
    "content": "import setuptools\r\n\r\nwith open(\"README.md\", \"r\") as fh:\r\n  long_description = fh.read()\r\n\r\nsetuptools.setup(\r\n  name=\"facebook_crawler\",\r\n  version=\"0.0.28\",\r\n  author=\"TENG-LIN YU\",\r\n  author_email=\"tlyu0419@gmail.com\",\r\n  description=\"Facebook crawler package can help you crawl the posts on public fanspages and groups from Facebook.\",\r\n  long_description=long_description,\r\n  long_description_content_type=\"text/markdown\",\r\n  url=\"https://github.com/TLYu0419/facebook_crawler\",\r\n  packages=setuptools.find_packages(),\r\n  py_modules=['facebook_crawler'],\r\n  classifiers=[\r\n  \"Programming Language :: Python :: 3\",\r\n  \"License :: OSI Approved :: Apache Software License\",\r\n  \"Operating System :: OS Independent\",\r\n  ],\r\n  python_requires=\">=3.6\",\r\n  install_requires=[\r\n      \"requests\",\r\n      \"bs4\",\r\n      \"numpy\",\r\n      \"pandas\",\r\n      \"lxml\"\r\n    ],\r\n)"
  },
  {
    "path": "tests/test_facebook_crawler.py",
    "content": "import pytest\r\n\r\n\r\n@pytest.fixture\r\ndef pageurl():\r\n    return 'https://www.facebook.com/groups/pythontw'\r\n\r\n\r\n@pytest.fixture\r\ndef until_date():\r\n    from datetime import datetime\r\n    from datetime import timedelta\r\n    return str(datetime.now() - timedelta(days=1))\r\n\r\n\r\ndef test_Crawl_PagePosts(pageurl, until_date):\r\n    from main import Crawl_PagePosts\r\n    df = Crawl_PagePosts(pageurl, until_date=until_date)\r\n    assert df\r\n\r\n\r\ndef test_Crawl_GroupPosts(pageurl, until_date):\r\n    from main import Crawl_GroupPosts\r\n    df = Crawl_GroupPosts(pageurl, until_date=until_date)\r\n    assert df\r\n"
  },
  {
    "path": "tests/test_page_parser.py",
    "content": "import pytest\r\n\r\nfrom utils import _init_request_vars\r\n\r\n\r\n@pytest.fixture\r\ndef pageurl():\r\n    return 'https://www.facebook.com/groups/pythontw'\r\n\r\n\r\n@pytest.fixture\r\ndef headers(pageurl):\r\n    from requester import _get_headers\r\n    return _get_headers(pageurl)\r\n\r\n\r\n@pytest.fixture\r\ndef homepage_response(pageurl, headers):\r\n    from requester import _get_homepage\r\n    return _get_homepage(pageurl=pageurl, headers=headers)\r\n\r\n\r\n@pytest.fixture\r\ndef entryPoint(homepage_response):\r\n    from page_paser import _parse_entryPoint\r\n    return _parse_entryPoint(homepage_response)\r\n\r\n\r\n@pytest.fixture\r\ndef identifier(entryPoint, homepage_response):\r\n    from page_paser import _parse_identifier\r\n    return _parse_identifier(entryPoint, homepage_response)\r\n\r\n\r\n@pytest.fixture\r\ndef docid(entryPoint, homepage_response):\r\n    from page_paser import _parse_docid\r\n    return _parse_docid(entryPoint, homepage_response)\r\n\r\n\r\n@pytest.fixture\r\ndef cursor():\r\n    _, cursor, _, _ = _init_request_vars()\r\n    return cursor\r\n\r\n\r\ndef test_parse_pagetype(homepage_response):\r\n    from page_paser import _parse_pagetype\r\n    page_type = _parse_pagetype(homepage_response)\r\n    assert page_type.lower() in ['group', 'fanspage']\r\n\r\n\r\ndef test_parse_pagename(homepage_response):\r\n    from page_paser import _parse_pagename\r\n    page_name = _parse_pagename(homepage_response)\r\n    assert page_name\r\n\r\n\r\ndef test_parse_entryPoint(homepage_response):\r\n    from page_paser import _parse_entryPoint\r\n    entryPoint = _parse_entryPoint(homepage_response)\r\n    assert entryPoint\r\n\r\n\r\ndef test_parse_identifier(entryPoint, homepage_response):\r\n    from page_paser import _parse_identifier\r\n    identifier = _parse_identifier(entryPoint, homepage_response)\r\n    assert identifier\r\n\r\n\r\ndef test_parse_docid(entryPoint, homepage_response):\r\n    from page_paser import _parse_docid\r\n    docid = _parse_docid(entryPoint, homepage_response)\r\n    assert docid\r\n\r\n\r\ndef test_parse_likes(homepage_response, entryPoint, headers):\r\n    from page_paser import _parse_likes\r\n    likes = _parse_likes(homepage_response, entryPoint, headers)\r\n    assert likes\r\n\r\n\r\ndef test_parse_creation_time(homepage_response, entryPoint, headers):\r\n    from page_paser import _parse_creation_time\r\n    creation_time = _parse_creation_time(homepage_response, entryPoint, headers)\r\n    assert creation_time\r\n\r\n\r\ndef test_parse_category(homepage_response, entryPoint, headers):\r\n    from page_paser import _parse_category\r\n    category = _parse_category(homepage_response, entryPoint, headers)\r\n    assert category\r\n\r\n\r\ndef test_parse_pageurl(homepage_response):\r\n    from page_paser import _parse_pageurl\r\n    pageurl = _parse_pageurl(homepage_response)\r\n    assert pageurl\r\n"
  },
  {
    "path": "tests/test_post_parser.py",
    "content": "import pytest\r\n\r\nfrom post_paser import _parse_edgelist, _parse_edge, _parse_domops, _parse_jsmods, _parse_composite_graphql\r\nfrom utils import _init_request_vars\r\n\r\n\r\n@pytest.fixture\r\ndef pageurl():\r\n    return 'https://www.facebook.com/groups/pythontw'\r\n\r\n\r\n@pytest.fixture\r\ndef headers(pageurl):\r\n    from requester import _get_headers\r\n    return _get_headers(pageurl)\r\n\r\n\r\n@pytest.fixture\r\ndef homepage_response(pageurl, headers):\r\n    from requester import _get_homepage\r\n    return _get_homepage(pageurl=pageurl, headers=headers)\r\n\r\n\r\n@pytest.fixture\r\ndef entryPoint(homepage_response):\r\n    from page_paser import _parse_entryPoint\r\n    return _parse_entryPoint(homepage_response)\r\n\r\n\r\n@pytest.fixture\r\ndef identifier(entryPoint, homepage_response):\r\n    from page_paser import _parse_identifier\r\n    return _parse_identifier(entryPoint, homepage_response)\r\n\r\n\r\n@pytest.fixture\r\ndef docid(entryPoint, homepage_response):\r\n    from page_paser import _parse_docid\r\n    return _parse_docid(entryPoint, homepage_response)\r\n\r\n\r\n@pytest.fixture\r\ndef cursor():\r\n    _, cursor, _, _ = _init_request_vars()\r\n    return cursor\r\n\r\n\r\n@pytest.fixture\r\ndef posts_response(headers, identifier, entryPoint, docid, cursor):\r\n    from requester import _get_posts\r\n    return _get_posts(headers=headers,\r\n                      identifier=identifier,\r\n                      entryPoint=entryPoint,\r\n                      docid=docid,\r\n                      cursor=cursor)\r\n\r\n\r\n@pytest.fixture()\r\ndef edges(posts_response):\r\n    return _parse_edgelist(posts_response)\r\n\r\n\r\ndef test_parse_edgelist(posts_response):\r\n    edges = _parse_edgelist(posts_response)\r\n    assert len(edges) > 0\r\n\r\n\r\ndef test_parse_edge(edges):\r\n    for edge in edges:\r\n        result = _parse_edge(edge)\r\n        assert result\r\n\r\n\r\ndef test_parse_domops(posts_response):\r\n    content_list, cursor = _parse_domops(posts_response)\r\n    assert content_list is not None\r\n    assert cursor is not None\r\n\r\n\r\ndef test_parse_jsmods(posts_response):\r\n    _parse_jsmods(posts_response)\r\n\r\n\r\ndef test_parse_composite_graphql(posts_response):\r\n    df, max_date, cursor = _parse_composite_graphql(posts_response)\r\n    assert df is not None\r\n    assert max_date is not None\r\n    assert cursor is not None\r\n\r\n\r\ndef test_parse_composite_nojs(posts_response):\r\n    df, max_date, cursor = _parse_composite_graphql(posts_response)\r\n    assert df is not None\r\n    assert max_date is not None\r\n    assert cursor is not None\r\n"
  },
  {
    "path": "tests/test_requester.py",
    "content": "import pytest\r\n\r\nfrom utils import _init_request_vars\r\n\r\n\r\n@pytest.fixture\r\ndef pageurl():\r\n    return 'https://www.facebook.com/groups/pythontw'\r\n\r\n\r\n@pytest.fixture\r\ndef headers(pageurl):\r\n    from requester import _get_headers\r\n    return _get_headers(pageurl)\r\n\r\n\r\n@pytest.fixture\r\ndef homepage_response(pageurl, headers):\r\n    from requester import _get_homepage\r\n    return _get_homepage(pageurl=pageurl, headers=headers)\r\n\r\n\r\n@pytest.fixture\r\ndef entryPoint(homepage_response):\r\n    from page_paser import _parse_entryPoint\r\n    return _parse_entryPoint(homepage_response)\r\n\r\n\r\n@pytest.fixture\r\ndef identifier(entryPoint, homepage_response):\r\n    from page_paser import _parse_identifier\r\n    return _parse_identifier(entryPoint, homepage_response)\r\n\r\n\r\n@pytest.fixture\r\ndef docid(entryPoint, homepage_response):\r\n    from page_paser import _parse_docid\r\n    return _parse_docid(entryPoint, homepage_response)\r\n\r\n\r\n@pytest.fixture\r\ndef cursor():\r\n    _, cursor, _, _ = _init_request_vars()\r\n    return cursor\r\n\r\n\r\ndef test_get_homepage(pageurl, headers):\r\n    from requester import _get_homepage\r\n    homepage_response = _get_homepage(pageurl=pageurl, headers=headers)\r\n    assert homepage_response.status_code == 200\r\n    assert homepage_response.url == pageurl\r\n\r\n\r\ndef test_get_pageabout(homepage_response, entryPoint, headers):\r\n    from requester import _get_pageabout\r\n    pageabout = _get_pageabout(homepage_response, entryPoint, headers)\r\n    assert pageabout.status_code == 200\r\n\r\n\r\ndef test_get_posts(headers, identifier, entryPoint, docid, cursor):\r\n    from requester import _get_posts\r\n    resp = _get_posts(headers=headers,\r\n                      identifier=identifier,\r\n                      entryPoint=entryPoint,\r\n                      docid=docid,\r\n                      cursor=cursor)\r\n    assert resp.status_code == 200\r\n"
  },
  {
    "path": "tests/test_utils.py",
    "content": "import pytest\r\n\r\nfrom utils import _extract_id, _extract_reactions\r\n\r\n\r\n@pytest.fixture\r\ndef dataid():\r\n    return 'feed_subtitle_107207125979624;5762739400426340;;9'\r\n\r\n\r\ndef test_extract_pageid(dataid):\r\n    pageid = _extract_id(dataid, 0)\r\n    assert pageid == '107207125979624'\r\n\r\n\r\ndef test_extract_postid(dataid):\r\n    postid = _extract_id(dataid, 1)\r\n    assert postid == '5762739400426340'\r\n\r\n\r\n# def test_extract_reactions(reactions, reaction_type):\r\n#     reactions = _extract_reactions(reactions, reaction_type)\r\n#     assert reactions is not None\r\n"
  },
  {
    "path": "utils.py",
    "content": "import sqlite3\r\nimport re\r\nimport datetime\r\nimport requests\r\n\r\n\r\ndef _connect_db():\r\n    conn = sqlite3.connect('./facebook_crawler.db', check_same_thread=False)\r\n    return conn\r\n\r\n\r\ndef _extract_id(string, num):\r\n    '''\r\n    Extract page and post id from the feed_subtitle infroamtion\r\n    '''\r\n    try:\r\n        return re.findall('[0-9]{5,}', string)[num]\r\n    except:\r\n        print('ERROR from extract {}'.froamt(string))\r\n        return string\r\n\r\n\r\ndef _extract_reactions(reactions, reaction_type):\r\n    '''\r\n    Extract reaction_type from reactions.\r\n    Possible reaction_type's value will be one of ['LIKE', 'HAHA', 'WOW', 'LOVE', 'SUPPORT', 'SORRY', 'ANGER'] \r\n    '''\r\n    for reaction in reactions:\r\n        if reaction['node']['localized_name'].upper() == reaction_type.upper():\r\n            return reaction['reaction_count']\r\n    return 0\r\n\r\n\r\ndef _init_request_vars(cursor=''):\r\n    # init parameters\r\n    # cursor = ''\r\n    df = []\r\n    max_date = datetime.datetime.now().strftime('%Y-%m-%d')\r\n    break_times = 0\r\n    return df, cursor, max_date, break_times\r\n\r\n\r\ndef _download_images(link):\r\n    filename = link.split(r'?')[0].split(r'/')[-1]\r\n    resp = requests.get(link)\r\n    with open(f'data/photos/{filename}', 'wb') as f:\r\n        f.write(resp.content)\r\n\r\n\r\ndef parse_raw_json(string):\r\n    dic = {}\r\n    for line in string.split('\\n'):\r\n        if line != '':\r\n            values = line.split(': ', -1)\r\n            dic[values[0]]= values[1]\r\n    return dic\r\n\r\nif '__name__' == '__main__':\r\n\r\n    df, cursor, max_date, break_times = _init_request_vars(cursor='aa')\r\n    cursor\r\n"
  }
]