Skip to main content

Copy Folder Does Not Copy All Images

Comments

26 comments

  • cPRex Jurassic Moderator
    Hey there! We don't recommend using FileZilla as they have been reported for installing malware recently. It would be best to use a different FTP client for your security. Do you see any errors in the main cPanel log at /usr/local/cpanel/logs/error_log when you perform this work in File Manager? That's the first place I'd check to examine this behavior.
    0
  • PostAlmostAnything
    Is that the same error log which is accessible from the main cPanel dashboard? I ask because unless that log is accessible from cPanel either via the dashboard or File Manager I can't access it because I use a remote Linux server without a remote desktop. I tried using FileZilla anyway but the files which failed to copy were not visible in the directory listing from the server. I know the files are there because when I search for the file names using File Manager they're found at their original locations. What might cause files to not be copied and prevent FileZilla from displaying them? Also, please don't respond to people with /default/linux/location/file.ext and start responding with at least one link to how to access the same information from WHM/cPanel without having to use the terminal. If I wanted to use a command line I would not pay for cPanel.
    0
  • PostAlmostAnything
    Another problem, I just tried to access the home directory backup that I downloaded only to get a message from WinRAR calling the backup corrupt and when I dove into it there were entire sites missing. It is like your software began to download a backup to my local computer via the internet but stopped before it was finished.
    0
  • cPRex Jurassic Moderator
    I ask because unless that log is accessible from cPanel either via the dashboard or File Manager I can't access it because I use a remote Linux server without a remote desktop.

    I don't understand this portion of the reply. Why would the lack of a remote desktop tool keep you from being able to see a log file? Do you have full root access to the server, or only access to your specific cPanel account?
    0
  • PostAlmostAnything
    What does an error message from File Manager look like? If I were to copy an error log file and do a text search for error message from File Manager what would I look for?
    0
  • PostAlmostAnything
    I don't understand this portion of the reply. Why would the lack of a remote desktop tool keep you from being able to see a log file? Do you have full root access to the server, or only access to your specific cPanel account?

    I have full root access but I'm not terribly familiar with Linux. I started off in .Net working with Windows but needed a PBN so I rented a Linux VPS, installed cPanel, and have used that to run the PBN. I don't know how to access files on the system unless I can use the UI you guys provide, so I need to know how to access that information by first logging into cPanel or WHM then navigating to wherever you guys display that stuff.
    0
  • cPRex Jurassic Moderator
    Unfortunately there isn't going to be a way to see those logs through either interface - you would have to use the command line to view that information. There is the WHM >> Terminal option to use the command line through a browser, but you still need to be cautious as it is full root access directly to the server. The file you'll want to explore is /usr/local/cpanel/logs/error_log. If you can recreate the error in real-time in one browser tab, while watching Terminal in another, you can just run this command to see if anything comes up in that log: tail -f /usr/local/cpanel/logs/error_log
    That will allow the log to scroll in real-time while you watch for anything related to appear. I don't have any specific examples of what that could be - since this isn't normal behavior, I don't have a particular reference I can provide. If you aren't able to perform that process, please submit a ticket to our team along with replication steps so we can take a look at the machine.
    0
  • PostAlmostAnything
    The odd thing is I am staring at one of the missing images live on the web at its original location on the same server it just does not show up in that file on cPanel and does not copy with other via the File Manager copy tool.
    0
  • PostAlmostAnything
    I will try this. Thanks for the command. Please give users on this forum such commands along with initial responses. I always pull may hair out when someone says to check out /etc/whatever/noideahowto.access because I know to view it I will also have to figure out what to type into the terminal. Also, your quote reply feature is broken. Whenever I try to post a reply with the default quote it says there was an unknown error. Then I delete the quote and it works.
    0
  • PostAlmostAnything
    I issued that command but all I see in the terminal is some brief list of stuff that makes no sense with nothing that looks like an error message. I tried pasting it but your form won't allow me to submit the paste.
    0
  • PostAlmostAnything
    Got this message which I hope helps: [2023-01-20 21:40:08 +0000] info [cpsrvd] version 11.106.0.13 online [2023-01-20 21:51:27 +0000] info [whostmgrd] Successfully verified signature for cpanel (key types: release). cpsrvd 2f81788fe755d: PID 21676: The cPanel Server operation timed out at cpsrvd.pl line 658. [2023-01-20 22:03:07 +0000] info [cpaneld] Internal Server Error: "POST /cpsess4649843505/json-api/cpanel HTTP/1.1" 500 Error ID 2f81788fe755d [2023-01-20 22:03:40 +0000] warn [whostmgrd] (XID zs86eu) Documents are not permitted to contain null characters, or new lines.
    0
  • PostAlmostAnything
    Here is what I am doing. I used WP Migrate DB to change all the image locations for the new site to the old one. Oddly, the multisite install can fun the same images from the original URL structure. Now I am running the reGenerate Thumbnails Advanced plugin which should detect the images and regenerate thumbnails for them. My hope is that images in posts with URLs like /uploads/2022 will be saved in the new directory at /uploads/sites/4/2022. I had hoped that a string replace to the original URLs would be enough but Wordpress stores featured images in the database as 2022/month/image.jpg etc. so something else adds the /sites/4 part at some unknown point. If these images are saved properly I'll just do another string replace so that the images in the post content point to the regenerated thumbnails. I am still concerned about the copying issue though. I can't think of nor can anyone here seem capable of telling me how it is possible for cPanel not to copy every file. I also am concerned about the lack of feedback from the cPanel UI. For instance, I expected to be promoted with a notice that some files already exist in the directory and being asked if I want to overwrite them. I receive no feedback at all other than what your UI claims the new folder sizes are afterwards.
    0
  • cPRex Jurassic Moderator
    Thanks for the additional details. I'm not sure what is happening with that machine, so it's time to create a ticket with our support team. Can you do that, and then post the ticket number here so I can follow along?
    0
  • PostAlmostAnything
    Thanks for the additional details. I'm not sure what is happening with that machine, so it's time to create a ticket with our support team. Can you do that, and then post the ticket number here so I can follow along?

    Is it possible to create a ticket without having to pay a fee?
    0
  • cPRex Jurassic Moderator
    There is never a charge for our support services :D
    0
  • PostAlmostAnything
    There is never a charge for our support services :D

    How about you tell me how to use WHM to remove the 10000 file limit on FTP clients? I think this default setting is the problem because there are way more than 10,000 images in the folder. Right now every time I download the folder via FileZilla it only downloads 9,998 images. I have seen instructions for how to remove this stupid default limit using the Terminal, but I didn't buy cPanel to use a CLI. I bought it to make stuff easy, so please tell me which options to click in WHM to get where I need to go please Looks like there is no way to change this ridiculous limit without using the CLI. Typical of you guys, put some stupid default limit in place which shouldn't be there to begin with then make people jump through hoops to fix it. You should just add it to Tweak Settings or FTP Configuration as a binary checkbox so that people can get rid of the limit easily.
    0
  • cPRex Jurassic Moderator
    The setting you're referring to is called LimitRecursion, and it is put in place by the FTP clients on the machine. There isn't a way to adjust that file through the interface, so you'll have to use the command line to change that. You can find more details on that process here:
    0
  • cPRex Jurassic Moderator
    If you'd like to see additional features added into the cPanel interface, you can always use the link in my signature to submit a feature request and then I can bring that up with the team during our weekly meeting.
    0
  • PostAlmostAnything
    If you'd like to see additional features added into the cPanel interface, you can always use the link in my signature to submit a feature request and then I can bring that up with the team during our weekly meeting.

    So I have to use the CLI and some instructions from some random website just to view all files in a folder
    0
  • PostAlmostAnything
    Why don't you guys at least explain how to fix the recursion limit? You have a page with a brief explanation which would work for people like you who use a CLI all the time but not people like your customers who purchase your service to avoid such things. In it you say stuff like "Create the /var/cpanel/conf/pureftpd/local file with the touch command, if one does not exist." but don't tell people what a "touch command" is or how to create a file. How do I create a file with a touch command? OK, I have the WHM Terminal open and I entered cd /etc/ but I don't know how to actually open the pureftpd.conf file. I tried vi pureftpd.conf but all I got was a bunch of lines of nothing but ~ followed by a notice of a file being created. Then I checked that directory and found no files by that name but files with a pure-ftpd prefix including pure-ftpd.conf, so is your documentation incorrect? Must be a documentation error because I could open the right filename just fine. Then I tried to view the contents of /var/cpanel/conf/pureftpd/ using the ls command but all it said was "main main.cache root_password_disabled" so that is where I am now stuck. Also, you have a link to pureftpd.org but the search feature on that site is broken. When you type into the search box and click the search button or hit enter it does nothing, so I did a Google search for site:pureftpd.org limitrecursion which returned zero results. Finally, in addition to having the wrong file name for the .conf in your instructions, you also fail to specify the file extension of the /local file I am expected to create. You say to format it as YAML whatever that is.
    0
  • cPRex Jurassic Moderator
    While cPanel does help make administering a server easier, it doesn't make the skills obsolete. There will still be things that have to be done on the command line from time to time. What if a system package breaks that is managed by the operating system? What if you have a disk usage issue you need to track down and it turns out to be inodes? What if a MySQL database gets corrupted? There are many areas that aren't under the control or management of cPanel, and a system admin would need to get into the machine and check log files to determine the root cause. Our documentation doesn't go into detail about how to create a file, because you should know that before you attempt to do this. It is assumed there is at least some level of familiarity with Linux on the part of the end-user. Without these basic skills, there is really no reason to attempt to use the command line. With your experience level, I'm *extremely* hesitant to provide any additional commands as running the wrong thing on your server, or even making a typo on the command line, could take your sites offline. I'd recommend doing some research in how to use a text editor. Once you have that down, the very last blue note under "For more information about Pure-FTPd, visit the FTP Server Configuration | cPanel & WHM Documentation will take care of your issue. You may also want to bump the "8" value there as that is the number of subdirectories the FTP client can display. If you have more than 8 sublevels, you may not see/copy those as expected either. I would also recommend two additional things - it would be a good idea to start getting familiar with other file management tools, such as sFTP, as standard FTP is likely going away due to inherent security issues at some point in the near future, and a single directory with more than 10,000 files that aren't organized generally doesn't seem ideal.
    0
  • PostAlmostAnything
    I finally expanded the 10000 file limit to 500000 and started a FTP download of the folder in question. The image count is now at about 425,000 files. I have been using FTP to backup my home directories for some time now and it looks like those backups were missing a lot whenever over 10000 files were in a folder. I was forced to stop using your regular backup utility after the directories got too big to store backups on the server for download and the home directory backup feature started timing out.
    0
  • PostAlmostAnything
    I'm working on an article about this to warn other people because not only is this default setting inexcusable but you guys also fail to diagnose the problem when people like me post about the symptoms here. The first response to my post should have been to release a patch fixing the problem, but absent a patch the second response to my post should have been a notice that cPanel has settings which make it impossible to download all files by default in many cases and instructions for how to fix the problem. Instead all I got was what seems like general advice. One of the reasons for posting here is because staff members moderate this forum. Staff members should be aware of the constraints created by their product and be prepared to identify those constraints for their victims. I should not have been the one to diagnose this problem, but it seems the best advice I got was to create a support ticket due to the failure to copy not being normal server behavior. That is only half true, while it is not normal server behavior it is normal cPanel behavior when over 10,000 files exist in a folder. Someday someone will lose almost everything as a result of this. They will have a large site or large network featuring folders with far more than 10,000 files in them, reach a point where the normal backup tools from cPanel timeout, and start using FTP like I did with no reason to think of any reason why all files would not be downloaded.
    0
  • quietFinn
    It might have helped if you had told (at least approximately) how many files you have in a directory, instead of saying "a huge folder of images".
    0
  • PostAlmostAnything
    It might have helped if you had told (at least approximately) how many files you have in a directory, instead of saying "a huge folder of images".

    How is it possible to view the file count or total size of a folder in cPanel? So far it looks like all the folder sizes in File Manager are grossly inaccurate and unless you change the recursion limit it won't tell you how many files you have if the file count exceeds 10,000.
    0
  • cPRex Jurassic Moderator
    This post has verged into "agenda posting" territory. FTP limitations have been well-known for decades, and are configurable by the system administrator. I'm going to lock this thread now.
    0

Please sign in to leave a comment.